Stroboscopic Detection Method, Electronic Device and Storage Medium

Abstract

The present application discloses a stroboscopic detection method, an electronic device and a storage medium. The method includes: obtaining a sequence of image frames; wherein, the sequence of image frames includes consecutive long-exposure images and at least two short-exposure images; obtaining a ratio component by using the brightness of the long-exposure images and the brightness of the short-exposure images; obtaining the motion regions of each pixel point in the short-exposure images and the column vector of brightness means located in the non-motion regions based on the ratio component; and performing stroboscopic detection on the sequence of image frames based on the column vector of brightness means. Through this design method, the ratio vector can be confirmed based on different exposures to determine the stroboscopic stripe frequency, multiple frames of images are used for calculation to avoid the situations of stationary stroboscopic stripes and slow stroboscopic stripes, at the same time, the motion interference is removed by multiple frames and the results are compared with each other to increase the accuracy rate.

Description

Technical Field

[0001] This application relates to the technical field of signal processing, and particularly to a stroboscopic detection method, an electronic device, and a storage medium. Background Art

[0002] In video surveillance services, stroboscopic phenomena often occur in indoor or outdoor artificial light source scenarios, which is caused by the variation of the radiation energy of some artificial light sources with the alternating current cycle. Currently, most surveillance devices use CMOS (Complementary Metal Oxide Semiconductor) as sensors. Such sensors adopt a progressive exposure strategy. When the exposure duration is not an integer multiple of the artificial light source's light energy change cycle, the amount of light energy received by each row of pixels will be different, resulting in bright and dark stripes in the video. At the same time, if the interval between the starting exposure times of two adjacent frames is not an integer multiple of the artificial light source's light energy change cycle, the positions of the bright and dark stripes in the two adjacent frames will be different, making the stripes appear to be constantly scrolling. Therefore, in the surveillance device, the output time of each frame of image is fixed as an integer multiple of the exposure time cycle. For example, at a 100hz light source, the output time of each frame of image is 40ms, that is, 25 frames per second, to eliminate the phase difference of the exposure time of the same row in two adjacent frames and make the stripes stop scrolling. Currently, the mainstream stroboscopic detection scheme is based on scrolling stripes. By using the difference image of the stripes in the front and rear frames, most of the background information of the image is removed while the stroboscopic features of the image are retained, so as to determine whether there is a stroboscopic phenomenon. However, this type of method is not suitable for the situation where the stripes scroll slowly or even do not move. Therefore, there is an urgent need for a new stroboscopic detection method to solve the above problems. Summary of the Invention

[0003] The main technical problem to be solved by this application is to provide a stroboscopic detection method, an electronic device, and a storage medium, which can increase the credibility of the detection result by comparing two frequency domain information with each other.

[0004] To solve the above technical problem, a technical solution adopted by this application is: to provide a stroboscopic detection method, including: obtaining an image frame sequence; wherein, the image frame sequence includes consecutive long-exposure images and at least two short-exposure images; obtaining a ratio component by using the brightness of the long-exposure image and the brightness of the short-exposure image; obtaining the motion area of each pixel point in the short-exposure image and the column vector of the brightness mean value located in the non-motion area based on the ratio component; and performing stroboscopic detection on the image frame sequence based on the column vector of the brightness mean value.

[0005] Wherein, the number of short-exposure images in the image frame sequence is two, and the ratio component is the ratio of the brightness of the long-exposure image to the brightness of the two short-exposure images respectively.

[0006] Among them, the step of obtaining the motion region of each pixel point in the short-exposure image and the column vector of brightness means located in the non-motion region based on the ratio component includes: obtaining a first difference between the brightnesses of the same pixel point in two short-exposure images and the square value of the first difference, and obtaining a marking value of the motion region of the pixel point based on the relationship between the square value and the motion threshold; obtaining a second difference between 1 and the marking value, obtaining a first ratio between the sum of the products of the ratio component and the second difference and the sum of the second differences, and using the first ratio as the column vector of brightness means; wherein, the column vector of brightness means in the non-motion region is not 0.

[0007] Among them, the step of obtaining the marking value of the motion region of the pixel point based on the relationship between the square value and the motion threshold includes: in response to the square value being greater than the motion threshold, setting the marking value of the motion region of the pixel point to 1; and / or, in response to the square value being less than or equal to the motion threshold, setting the marking value of the motion region of the pixel point to 0.

[0008] Among them, before the step of performing stroboscopic detection on the image frame sequence based on the column vector of brightness means, it includes: obtaining the brightness frequency-domain feature corresponding to the short-exposure image based on the column vector of brightness means; and obtaining the amplitude of the brightness frequency-domain feature and the phase of the brightness frequency-domain feature corresponding to the short-exposure image by using the brightness frequency-domain feature.

[0009] Among them, the brightness frequency-domain feature includes a real-part feature and an imaginary-part feature; the step of obtaining the amplitude of the brightness frequency-domain feature and the phase of the brightness frequency-domain feature corresponding to the short-exposure image by using the brightness frequency-domain feature includes: obtaining a first product between half of the height of the column vector of brightness means and the sum of the squares of the real-part feature and the imaginary-part feature, and obtaining a second ratio between the imaginary-part feature and the real-part feature, and using the first product as the amplitude of the brightness frequency-domain feature and using the arctangent function value of the second ratio as the phase of the brightness frequency-domain feature.

[0010] Among them, the amplitude values of the luminance frequency domain features include a plurality of position points from a first position point to a second position point; the step of performing stroboscopic detection on the image frame sequence based on the luminance mean column vector includes: obtaining two maximum values of the amplitude values of the luminance frequency domain features and the positions where the maximum values of the amplitude values of the luminance frequency domain features are located among the first position point to the second position point; in response to the two positions being the same, or in response to the absolute value of the difference between the two maximum values of the amplitude values of the luminance frequency domain features being less than or equal to a first frequency domain threshold, determining whether both of the two maximum values of the amplitude values of the luminance frequency domain features are less than a second frequency domain threshold; if not, determining whether the absolute value of the difference between the luminance frequency domain phase values corresponding to the two positions is greater than a third frequency domain threshold; if not, obtaining a predicted stroboscopic frequency of the image frame sequence based on the height of the luminance mean column vector, and performing stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency.

[0011] Among them, the step of obtaining a predicted stroboscopic frequency of the image frame sequence based on the height of the luminance mean column vector, and performing stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency includes: obtaining a second product of the difference between the start exposure times of the upper and lower rows of the sensor and the stroboscopic stripe period, and taking the reciprocal of the second product as the predicted stroboscopic frequency; wherein, the stroboscopic stripe period is the number of pixels from the starting point of the stroboscopic stripe to the next starting point, and the stroboscopic stripe period is a third ratio of the height of the luminance mean column vector to the position point; in response to the predicted stroboscopic frequency being greater than a first preset value and less than a second preset value, or in response to the predicted stroboscopic frequency being greater than a third preset value and less than a fourth preset value, determining that there is stroboscopy in the image frame sequence; wherein, the first preset value, the second preset value, the third preset value, and the fourth preset value are all related to a fourth frequency domain threshold.

[0012] Among them, after the step of performing stroboscopic detection on the image frame sequence based on the luminance mean column vector, it includes: in response to there being stroboscopy in the image frame sequence, adjusting the exposure time and the image gain value of the image frame sequence.

[0013] To solve the above technical problems, another technical solution adopted by this application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is configured to execute the program instructions to implement the stroboscopic detection method mentioned in any of the above embodiments.

[0014] To solve the above technical problems, yet another technical solution adopted by this application is: to provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is used to implement the stroboscopic detection method mentioned in any of the above embodiments.

[0015] Differing from the prior art, the beneficial effects of the present application are as follows: The stroboscopic detection method provided by the present application includes: acquiring an image frame sequence; wherein, the image frame sequence includes consecutive long-exposure images and at least two short-exposure images; then obtaining a ratio component by using the brightness of the long-exposure image and the brightness of the short-exposure images; further obtaining the motion regions of each pixel point in the short-exposure images and the column vector of brightness means located in the non-motion regions based on the ratio component; and finally performing stroboscopic detection on the image frame sequence based on the column vector of brightness means. Through this design, the ratio vector can be confirmed based on different exposures to determine the stroboscopic stripe frequency, and multiple frames of images are used for calculation to avoid the situations of static stroboscopic stripes and slow stroboscopic stripes. At the same time, motion interference is removed by multiple frames and the results are compared with each other to increase the accuracy rate. Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0017] Figure 1 is a schematic flowchart of an embodiment of the stroboscopic detection method of the present application;

[0018] Figure 2 is a schematic diagram of a long-exposure image and a short-exposure image;

[0019] Figure 3 is Figure 1 a schematic flowchart of an embodiment of step S3 in

[0020] Figure 4 is Figure 1 a schematic flowchart of an embodiment before step S1 in

[0021] Figure 5 is Figure 1 a schematic flowchart of an embodiment of step S4 in

[0022] Figure 6 is Figure 5 a schematic flowchart of an embodiment of step S37 in

[0023] Figure 7 is a schematic structural diagram of an embodiment of the stroboscopic detection system of the present application;

[0024] Figure 8 is a schematic framework diagram of an embodiment of the electronic device of the present application;

[0025] Figure 9It is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. Specific Embodiments

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0027] The current stroboscopic detection methods mainly include the following: (1) Calculate the average of the row mean vectors and difference vectors of adjacent frame images as the waveform floating center of the row difference vector of the current frame image. Identify the vectors in the difference vector that are equal to the floating center as position vectors, and determine the distances between adjacent points in the position vectors and compare them with the corresponding distances in the predetermined stroboscopic frequency set. Select the most suitable predetermined stroboscopic frequency according to the proportion of each predetermined stroboscopic frequency in the N-frame image sequence in the predetermined stroboscopic frequency set; (2) Statistically calculate the row sum of the luminance components of two adjacent frame images in the video frame sequence, that is, calculate the stroboscopic feature components and the corresponding peak function and valley function to determine whether there is stroboscopic flickering of the video frame caused by a light source in the current frame image; (3) Calculate the luminance column vector and the intra-frame differential luminance column vector for a single image, and obtain its flag bit information, that is, determine the stroboscopic statistical value by calculating the filtered differential luminance column vector and the carrier wave, so as to determine whether there is stroboscopic in the image; (4) For the case of slow rolling stroboscopic, collect three frames of images, obtain the differential column vectors for two adjacent frames of images, and use the obtained differential column vectors to determine the long sequence stroboscopic feature values, and calculate respectively whether there is stroboscopic. However, these methods all have defects. For example, in the first and second methods, the stroboscopic stripe frequency is determined based on the differences of the difference vectors of adjacent rows, and it cannot be used in scenarios where the stripes of adjacent frame images are basically unchanged and the difference vectors cannot be distinguished; the third method calculates using a single frame. Although it takes into account the case of static stroboscopic stripes, the stroboscopic stripes exist in various forms. Therefore, calculating using a single frame may lead to calculation errors and misjudgments in complex scenarios; the fourth method calculates using three frames. Although it takes into account the case of slow stroboscopic stripes, assuming that the stripes are static at this time, the differential vectors of the three frames of images still cannot obtain effective stroboscopic feature information.

[0028] Please refer to Figure 1 and Figure 2 , Figure 1 It is a schematic flowchart of an embodiment of the stroboscopic detection method of the present application. Figure 2 It is a schematic diagram of a long-exposure image and a short-exposure image. The stroboscopic detection method includes:

[0029] S1: Obtain an image frame sequence.

[0030] Specifically, the image frame sequence includes consecutive long-exposure images and at least two short-exposure images. Specifically, in this embodiment, the number of short-exposure images in the image frame sequence is two. Every fixed time interval, an image frame sequence of a long-exposure image, a first short-exposure image, and a second short-exposure image is obtained. Specifically, the image frame sequence can be obtained from an image signal processing unit, and only the luminance channel (i.e., the luminance of each image) needs to be obtained. For example, in the wide dynamic range mode of a monitoring device, long-exposure and multiple short-exposure images can be continuously obtained. At this time, the long-exposure and short-exposure images obtained are Bayer array images, and the format of the Bayer array image is the raw format. At this time, any pixel type value among the R value, Gr value, Gb value, or b value of each smallest unit of the Bayer array image can be taken to form the luminance images of the long and short exposures. Another example is that in the normal linear mode of a monitoring device, the exposure time can be adjusted first to obtain 1 long-exposure image, and then the exposure time is adjusted again to obtain the remaining two short-exposure images. At this time, the image format obtained is the yuv format, and only the luminance channel (y channel) needs to be taken as the luminance image. At this time, the luminance of the long-exposure image obtained is relatively large, and the luminance of the two short-exposure images is relatively small. There are displacements of moving objects among the three images. Currently, there are mainly two fixed periods for artificial light sources, namely the 50hz or 60hz alternating current periods, and the light intensity frequencies corresponding to the artificial light sources are 100hz or 120hz. The exposure time set for the long-exposure image needs to be an integer multiple of the reciprocal of the light intensity frequency. Since the alternating current frequencies in different countries have been determined, the exposure time of the long-exposure image can be determined. For example, 1 / 100hz = 0.01s. The exposure times set for the first short-exposure image and the second short-exposure image need to be equal and less than the integer multiple of the reciprocal of the light intensity frequency. At this time, there is no stroboscopic phenomenon in the long-exposure image, and there is a stroboscopic phenomenon in the first short-exposure image and the second short-exposure image. As Figure 2 shown, Figure 2 in which a is the long-exposure image without stroboscopic stripes, b is the first short-exposure image with stroboscopic stripes, and c is the second short-exposure image with stroboscopic stripes. Of course, in other embodiments, the number of short-exposure images in the image frame sequence can be more than two, and this application does not limit it here.

[0031] S2: Obtain a ratio component using the luminance of the long-exposure image and the luminance of the short-exposure image.

[0032] Specifically, the ratio components are obtained using the brightness of the long-exposure image and the brightness of the short-exposure images. Since there are two short-exposure images here, two corresponding ratio components will be obtained. Specifically, in this embodiment, the first ratio component D1 is obtained using the brightness L1 of the long-exposure image and the brightness S1 of the first short-exposure image, and the second ratio component D2 is obtained using the brightness L1 of the long-exposure image and the brightness S2 of the second short-exposure image. The ratio components are the ratios of the brightness of the long-exposure image to the brightness of the two short-exposure images respectively, and the specific calculation formula is:

[0033] D1 = L1 / (S1 + eps)

[0034] D2 = L1 / (S2 + eps)

[0035] Where eps is a very small value, whose function is to prevent the divisor from being zero and has no actual physical meaning. Since the first long-exposure image has no stroboscopic stripes, while the two short-exposure images have stroboscopic stripes, and at the same time the three images are slightly different due to inter-frame motion, the two ratio images mainly contain stripe information and some abnormal pixels caused by motion. Based on different exposures to confirm the ratio vector, thereby determining the stroboscopic stripe frequency, which can be used in scenarios where the stripes of adjacent frame images are basically unchanged and the difference vector cannot be distinguished.

[0036] S3: Obtain the motion region of each pixel point in the short-exposure image and the column vector of the brightness mean value located in the non-motion region based on the ratio components.

[0037] Specifically, in this embodiment, the number of short-exposure images in the image frame sequence is two. Please refer to Figure 3 , Figure 3 which Figure 1 is the flow schematic diagram of an implementation manner of step S3 in

[0038] S10: Obtain the first difference between the brightnesses of the same pixel point in the two short-exposure images and the squared value of the first difference, and obtain the marking value of the motion region of the pixel point based on the relationship between the squared value and the motion threshold.

[0039] Specifically, the short-exposure image includes multiple pixel points. The first difference between the brightness S1(i, j) of the pixel point (i, j) in the first short-exposure image and the brightness S2(i, j) of this pixel point in the second short-exposure image is S1(i, j) - S2(i, j), and its squared value is (S1(i, j) - S2(i, j))2. Specifically, in this embodiment, the step of obtaining the marking value of the motion region of the pixel point based on the relationship between the squared value and the motion threshold in step S10 includes performing an image binarization operation on the result of the squared value using the motion threshold move_th1. Specifically, when the squared value (S1(i, j) - S2(i, j))2 When it is greater than the motion threshold move_th1, it indicates that there is motion at the position of the pixel point (i, j), and the marking value move(i, j) of the motion area of this pixel point is set to 1; when the squared value is less than or equal to the motion threshold move_th1, it indicates that there is no motion at the position of the pixel point (i, j), and the marking value move(i, j) of the motion area of this pixel point is set to 0. Specifically, it can be expressed as:

[0040] move(i,j) = 1 if (S1(i,j) - S2(i,j)) 2 > move_th1

[0041] move(i,j) = 0 if (S1(i,j) - S2(i,j)) 2 ≤ move_th1

[0042] Among them, move_th1 is the motion threshold, which can be set artificially according to the actual situation, and move(i, j) is the marking value of the motion area of the pixel point (i, j). In this way, it can be obtained whether there is motion at the position of a pixel point in the short-exposure image. If there is motion here, it needs to be removed to prevent interference with the next operation.

[0043] S11: Obtain the second difference between 1 and the marking value, obtain the first ratio between the sum of the products of the ratio component and the second difference and the sum of the second differences, and use the first ratio as the column vector of the brightness mean.

[0044] Specifically, the column vector of the brightness mean of the non-motion area is not 0. Since there are two short-exposure images, the corresponding brightness mean vectors include the first brightness mean vector D1_line(i) and the second brightness mean vector D2_line(i). Their specific calculation formulas are:

[0045]

[0046] Among them, D1(i, j) is the first ratio component of the pixel point (i, j) in the first short-exposure image, D2(i, j) is the second ratio component of the pixel point (i, j) in the second short-exposure image, and D1_line(i) and D2_line(i) represent the first column vector of the brightness mean and the second column vector of the brightness mean respectively. Combining the above text, taking the first short-exposure image as an example, when there is motion at the position of the pixel point (i, j), the marking value move(i, j )If it is equal to 1, then D1_line(i) = 0. That is to say, the position of this pixel point in the first short-exposure image is removed, which can prevent interference with the next operation. Still in the first short-exposure image, when there is no motion at the position of the pixel point (i, j), that is, the position of this pixel point is a non-moving area, the marking value of the moving area of this pixel point is move(i, j ) If it is equal to 0, then D1_line(i) ≠ 0. In this embodiment, the luminance mean column vector mentioned here refers to the luminance mean column vector of the non-moving area. In addition, the situation of the second short-exposure image is similar to that of the first short-exposure image, which will not be elaborated here. Using multiple frames for calculation takes into account the situations of static stroboscopic stripes and slow stroboscopic stripes. At the same time, motion interference is removed from multiple frames, and the results are compared with each other to increase the accuracy rate.

[0047] S4: Perform stroboscopic detection on the image frame sequence based on the luminance mean column vector.

[0048] Specifically, in this embodiment, please refer to Figure 4 , Figure 4 is Figure 1 a schematic flowchart of a previous embodiment before step S1 in

[0049] S20: Obtain the luminance frequency-domain feature corresponding to the short-exposure image based on the luminance mean column vector.

[0050] Specifically, calculate the Fourier frequency-domain features of the first luminance mean column vector D1_line(i) and the second luminance mean column vector D2_line(i). The specific calculation formula is as follows:

[0051] D1_f = fft(D1_line(i))

[0052] D2_f = fft(D2_line(i))

[0053] Among them, fft is the fast Fourier transform algorithm, which is a known algorithm and will not be elaborated in this application. D1_f is the first luminance frequency-domain feature, and D2_f is the second luminance frequency-domain feature.

[0054] S21: Obtain the amplitude and phase of the luminance frequency-domain feature corresponding to the short-exposure image by using the luminance frequency-domain feature.

[0055] Specifically, the luminance frequency-domain features include real-part features and imaginary-part features. In this embodiment, step S21 specifically includes: obtaining a first product between half of the height of the column vector of the luminance mean and the sum of the squares of the real-part features and the imaginary-part features, and obtaining a second ratio between the imaginary-part features and the real-part features, and taking the first product as the amplitude of the luminance frequency-domain features, and taking the arctangent function value of the second ratio as the phase of the luminance frequency-domain features. The specific calculation formula is as follows:

[0056] D1_f_abs(i) = 2 / height * (D1_f_re(i) 2 + D1_f_im(i) 2 )

[0057] D2_f_abs(i) = 2 / height * (D2_f_re(i) 2 + D2_f_im(i) 2 )

[0058] D1_f_angle(i) = arctan(D1_f_im(i) / D1_f_re(i))

[0059] D2_f_angle(i) = arctan(D2_f_im(i) / D2_f_re(i))

[0060] Wherein, D1_f_abs is the amplitude of the first luminance frequency-domain feature, D2_f_abs is the amplitude of the second luminance frequency-domain feature, D1_f_angle is the phase of the first luminance frequency-domain feature, D2_f_angle is the phase of the second luminance frequency-domain feature, height is the height of the column vector, D1_f_re and D1_f_im are respectively the real-part feature and the imaginary-part feature of the first luminance frequency-domain feature D1_f, D2_f_re and D2_f_im are respectively the real-part feature and the imaginary-part feature of the second luminance frequency-domain feature D2_f, and arctan is the arctangent function.

[0061] Specifically, in this embodiment, the amplitude of the luminance frequency-domain features includes multiple position points from the first position point to the second position point. Among them, the first position point is the second point in the amplitude of the luminance frequency-domain features, and the second position point is the height / 2-th point in the amplitude of the luminance frequency-domain features. Please refer to Figure 5 , Figure 5 is Figure 1 a schematic flowchart of an implementation manner of step S4 in

[0062] S30: Obtain two maximum values of the amplitude of the luminance frequency-domain features and the positions where the maximum values of the amplitude of the luminance frequency-domain features are located among the first position point to the second position point.

[0063] Specifically, in the first luminance frequency domain feature amplitude, find the maximum value D1_max of the first luminance frequency domain feature amplitude and its position P1 from the 2nd point to the height / 2-th point. In the second luminance frequency domain feature amplitude, find the maximum value D2_max of the second luminance frequency domain feature amplitude and its position P2 from the 2nd point to the height / 2-th point. Among them, in the luminance frequency domain feature amplitude, the first point is the frequency domain DC component, and the points from the height / 2 + 1-th point to the height-th point are symmetric frequency domain features, which do not participate in the subsequent calculations.

[0064] S31: Determine whether the two positions are the same, or determine whether the absolute value of the difference between the maximum values of the two luminance frequency domain feature amplitudes is less than or equal to the first frequency domain threshold.

[0065] Specifically, determine whether the two positions P1 and P2 are the same, or determine whether the absolute value of D1_max - D2_max is less than or equal to the first frequency domain threshold f_th1.

[0066] S32: If so, determine whether the maximum values of the two luminance frequency domain feature amplitudes are both less than the second frequency domain threshold.

[0067] Specifically, if it is determined that the two positions P1 and P2 are the same, or it is determined that the absolute value of D1_max - D2_max is less than or equal to the first frequency domain threshold f_th1, it means that there is no difference or a small difference in the results of the first luminance mean column vector D1_line(i) and the second luminance mean column vector D2_line(i). At this time, determine whether the maximum value D1_max of the first luminance frequency domain feature amplitude and the maximum value D2_max of the second luminance frequency domain feature amplitude are both less than the second frequency domain threshold f_th2. Among them, the first frequency domain threshold f_th1 and the second frequency domain threshold f_th2 can be set artificially according to the actual situation and are not limited in this application.

[0068] S33: Otherwise, return to step S1.

[0069] Specifically, if it is determined that the two positions P1 and P2 are not the same, or it is determined that the absolute value of D1_max - D2_max is greater than the first frequency domain threshold f_th1, it means that there is a large difference in the results of the first luminance mean column vector D1_line(i) and the second luminance mean column vector D2_line(i), which may be caused by a moving object or other external changes, and it is necessary to return to step S1 and re-measure.

[0070] S34: If so, determine that there is no stroboscopic in the image frame sequence.

[0071] Specifically, if the maximum value D1_max of the first luminance frequency-domain feature amplitude and the maximum value D2_max of the second luminance frequency-domain feature amplitude are both less than the second frequency-domain threshold f_th2, it is considered that there is no stroboscopic phenomenon in the scene, and the process ends.

[0072] S35: If not, then determine whether the absolute value of the difference between the phases of the luminance frequency-domain features corresponding to the two positions is greater than the third frequency-domain threshold.

[0073] Specifically, if at least one of the maximum value D1_max of the first luminance frequency-domain feature amplitude and the maximum value D2_max of the second luminance frequency-domain feature amplitude is less than the second frequency-domain threshold f_th2, then find the first luminance frequency-domain feature phase D1_f_angle(P1) and the second luminance frequency-domain feature phase D2_f_angle(P2) corresponding to the two positions P1 and P2, and determine whether the absolute value of the difference between the first luminance frequency-domain feature phase D1_f_angle(P1) and the second luminance frequency-domain feature phase D2_f_angle(P2) is greater than the third frequency-domain threshold f_th3. Among them, the third frequency-domain threshold f_th3 can be artificially set according to the actual situation and is not limited in this application.

[0074] S36: If so, return to step S1.

[0075] Specifically, when the absolute value of the difference between them is greater than the third frequency-domain threshold f_th3, it indicates that there is a large difference in the results of the first luminance mean column vector D1_line(i) and the second luminance mean column vector D2_line(i) at this time, which may also be caused by a moving object or other external changes. At this time, it is necessary to return to step 1 and re-measure.

[0076] S37: If not, then obtain the predicted stroboscopic frequency of the image frame sequence based on the height of the luminance mean column vector, and perform stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency.

[0077] Specifically, when the absolute value of the difference between them is less than or equal to the third frequency-domain threshold f_th3, there is no difference or a small difference in the results of the first luminance mean column vector D1_line(i) and the second luminance mean column vector D2_line(i). Then, obtain the predicted stroboscopic frequency of the image frame sequence based on the height of the luminance mean column vector, and perform stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency. Please refer to Figure 6 , Figure 6 Yes Figure 5 The flowchart of an embodiment of step S37 in

[0078] S370: Obtain the second product of the difference between the starting exposure times of the upper and lower rows of the sensor and the stroboscopic stripe period, and use the reciprocal of the second product as the predicted stroboscopic frequency.

[0079] Specifically, the stroboscopic stripe period is the number of pixels from the starting point of the stroboscopic stripe to the next starting point, and the stroboscopic stripe period is the third ratio of the height of the column vector of the brightness mean value to the position point. Specifically, in this embodiment, the calculation formula for predicting the stroboscopic frequency is:

[0080] predict_f = 1 / (line_time * height / P1)

[0081] Where line_time is the difference between the starting exposure times of the upper and lower rows of the CMOS sensor, and this parameter is a preset parameter and is not limited here. height / P1 is the number of pixels from the starting point of the stroboscopic stripe to the next starting point, that is, the stroboscopic stripe period. Of course, the above sensor can also be other sensors, and this application is not limited here.

[0082] S371: Determine whether the predicted stroboscopic frequency is greater than the first preset value and less than the second preset value, or determine whether the predicted stroboscopic frequency is greater than the third preset value and less than the fourth preset value.

[0083] Specifically, the first preset value, the second preset value, the third preset value, and the fourth preset value are all related to the fourth frequency domain threshold f_th4. Among them, the fourth frequency domain threshold f_th4 can be set artificially according to the actual situation and is not limited in this application.

[0084] S372: If so, determine that there is stroboscopic in the image frame sequence.

[0085] S373: Otherwise, determine that there is no stroboscopic in the image frame sequence.

[0086] In this embodiment, when the predicted stroboscopic frequency perdict_f is close to the light intensity frequencies of artificial light sources generated by the two main alternating currents of 100hz or 120hz, that is:

[0087] 100hz - f_th4 < predict_f < 100hz + f_th4

[0088] or 120hz - f_th4 < predict_f < 120hz + f_th4

[0089] Among them, the first preset value is 100 hz - f_th4, the second preset value is 100 hz + f_th4, the third preset value is 120 hz - f_th4, and the fourth preset value is 120 hz + f_th4. When the predicted stroboscopic frequency perdict_f is close to 100 hz, it is considered that there is 100 hz stroboscopic in the current scene. When the predicted stroboscopic frequency perdict_f is close to 120 hz, it is considered that there is 120 hz stroboscopic in the current scene. When the predicted stroboscopic frequency perdict_f is not close to 100 hz nor close to 120 hz, it is considered that there is no stroboscopic phenomenon in the current scene.

[0090] Specifically, in this embodiment, after step S4, it includes: in response to the existence of stroboscopic in the image frame sequence, adjusting the exposure time and image gain value of the image frame sequence. Specifically, when it is considered that there is 100 hz stroboscopic, the brightness is adjusted to an integer multiple of the reciprocal of 100 hz stroboscopic, such as 10 ms, etc.; when it is considered that there is 120 hz stroboscopic, the brightness is adjusted to an integer multiple of the reciprocal of 120 hz stroboscopic, such as 8.33 ms, etc. At this time, increasing the exposure time will cause the image brightness to increase, and it is necessary to reduce the gain value to keep the image brightness basically consistent. The formula is as follows:

[0091] gain_result = gain * shutter / shutter_result

[0092] Among them, gain is the image gain value before adjusting the exposure, shutter is the exposure time value before adjusting the exposure, shutter_result is the exposure time value after adjusting the exposure, and gain_result is the new gain value after adjusting the exposure. By adjusting the exposure time and image gain value, the image brightness can be increased and the probability of stroboscopic in the image can be reduced.

[0093] Through this design method, the ratio vector can be confirmed based on different exposures to determine the flicker fringe frequency, and multiple frames of images are used for calculation to avoid the situation of static stroboscopic fringes and slow stroboscopic fringes. At the same time, multiple frames are used to remove motion interference and the results are compared with each other to increase the accuracy rate.

[0094] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of an embodiment of the stroboscopic detection system of the present application. The stroboscopic detection system specifically includes:

[0095] An acquisition module 10, configured to acquire an image frame sequence; among them, the image frame sequence includes consecutive long-exposure images and at least two short-exposure images.

[0096] A ratio component module 12, coupled to the acquisition module 10, configured to obtain a ratio component by using the brightness of the long-exposure image and the brightness of the short-exposure image.

[0097] The column vector module 14 is coupled to the ratio component module 12 and is configured to obtain, based on the ratio component, the motion regions of each pixel point in the short-exposure image and the column vector of the brightness means located in the non-motion regions.

[0098] The detection module 16 is coupled to the column vector module 14 and is configured to perform stroboscopic detection on the image frame sequence based on the column vector of the brightness means.

[0099] Please refer to Figure 8 , Figure 8 , which is a schematic diagram of the framework of an embodiment of the electronic device of the present application. The electronic device includes a memory 20 and a processor 22 that are coupled to each other. Specifically, in this embodiment, program instructions are stored in the memory 20, and the processor 22 is configured to execute the program instructions to implement the stroboscopic detection method mentioned in any of the above embodiments.

[0100] Specifically, the processor 22 can also be referred to as a CPU (Central Processing Unit). The processor 22 may be an integrated circuit chip with signal processing capabilities. The processor 22 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Additionally, the processor 22 can be implemented jointly by multiple integrated circuit chips.

[0101] Please refer to Figure 9 , Figure 9It is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 30 stores a computer program 300 that can be read by a computer. The computer program 300 can be executed by a processor to implement the stroboscopic detection method mentioned in any of the above embodiments. Among them, the computer program 300 can be stored in the computer-readable storage medium 30 in the form of a software product, including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The computer-readable storage medium 30 with a storage function can be various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0102] In summary, different from the prior art, the stroboscopic detection method provided by the present application includes: obtaining an image frame sequence; wherein, the image frame sequence includes consecutive long-exposure images and at least two short-exposure images; then obtaining a ratio component by using the brightness of the long-exposure images and the brightness of the short-exposure images; further obtaining the motion region of each pixel point in the short-exposure images and the column vector of the brightness means located in the non-motion region based on the ratio component; and finally performing stroboscopic detection on the image frame sequence based on the column vector of the brightness means. Through this design method, the ratio vector can be confirmed based on different exposures to determine the flicker stripe frequency, multiple frames of images are used for calculation to avoid the situation of stationary stroboscopic stripes and slow stroboscopic stripes, and at the same time, multiple frames are used to remove motion interference and compare the results with each other to increase the accuracy rate.

[0103] The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be included in the patent protection scope of the present application by the same token.

Claims

1. A stroboscopic detection method, characterized in that, Including: Obtain a sequence of image frames; wherein, the sequence of image frames includes consecutive long-exposure images and at least two short-exposure images; Obtain a ratio component using the brightness of the long-exposure image and the brightness of the short-exposure images; Based on the ratio component, obtain the motion regions of each pixel point in the short-exposure images and the column vector of the brightness mean values in the non-motion regions; Based on the column vector of the brightness mean values, perform stroboscopic detection on the sequence of image frames.

2. The stroboscopic detection method according to claim 1, wherein The number of short-exposure images in the sequence of image frames is two, and the ratio component is the ratio of the brightness of the long-exposure image to the brightness of the two short-exposure images respectively.

3. The stroboscopic detection method according to claim 1 or 2, characterized in that, The step of obtaining the motion regions of each pixel point in the short-exposure images and the column vector of the brightness mean values in the non-motion regions based on the ratio component includes: Obtain the first difference between the brightnesses of the same pixel point in the two short-exposure images and the square value of the first difference, and based on the relationship between the square value and the motion threshold, obtain the marking value of the motion region of the pixel point; Obtain the second difference between 1 and the marking value, obtain the first ratio between the sum of the products of the ratio component and the second difference and the sum of the second differences, and use the first ratio as the column vector of the brightness mean values; wherein, the column vector of the brightness mean values in the non-motion regions is not 0.

4. The stroboscopic detection method according to claim 3, wherein The step of obtaining the marking value of the motion region of the pixel point based on the relationship between the square value and the motion threshold includes: In response to the square value being greater than the motion threshold, set the marking value of the motion region of the pixel point to 1; and / or, In response to the square value being less than or equal to the motion threshold, set the marking value of the motion region of the pixel point to 0.

5. The stroboscopic detection method according to claim 1, characterized in that, Before the step of performing stroboscopic detection on the sequence of image frames based on the column vector of the brightness mean values, it includes: Based on the column vector of the brightness mean values, obtain the brightness frequency domain characteristics corresponding to the short-exposure images; Use the brightness frequency domain characteristics to obtain the amplitude of the brightness frequency domain characteristics and the phase of the brightness frequency domain characteristics corresponding to the short-exposure images.

6. The stroboscopic detection method according to claim 5, wherein The brightness frequency domain characteristics include real part characteristics and imaginary part characteristics; The step of using the brightness frequency domain characteristics to obtain the amplitude of the brightness frequency domain characteristics and the phase of the brightness frequency domain characteristics corresponding to the short-exposure images includes: Obtain the first product between half of the height of the column vector of the brightness mean values and the sum of the squares of the real part characteristics and the imaginary part characteristics, and obtain the second ratio between the imaginary part characteristics and the real part characteristics, and use the first product as the amplitude of the brightness frequency domain characteristics and use the arctangent function value of the second ratio as the phase of the brightness frequency domain characteristics.

7. The stroboscopic detection method according to claim 6, wherein The amplitude of the brightness frequency domain characteristics includes multiple position points from the first position point to the second position point; the step of performing stroboscopic detection on the sequence of image frames based on the column vector of the brightness mean values includes: Obtain two maximum values of the amplitude of the brightness frequency domain characteristics and the positions where the maximum values of the amplitude of the brightness frequency domain characteristics are located among the first position point to the second position point; In response to the two positions being consistent, or in response to the absolute value of the difference between the maximum values of the two luminance frequency-domain feature amplitudes being less than or equal to a first frequency-domain threshold, determine whether the maximum values of the two luminance frequency-domain feature amplitudes are both less than a second frequency-domain threshold; If not, then determine whether the absolute value of the difference between the phases of the luminance frequency-domain features corresponding to the two positions is greater than a third frequency-domain threshold; If not, then obtain the predicted stroboscopic frequency of the image frame sequence based on the height of the luminance mean column vector, and perform stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency.

8. The stroboscopic detection method according to claim 7, characterized in that, The step of obtaining the predicted stroboscopic frequency of the image frame sequence based on the height of the luminance mean column vector and performing stroboscopic detection on the image frame sequence according to the predicted stroboscopic frequency includes: Obtain a second product of the difference between the start exposure times of the upper and lower rows of the sensor and the stroboscopic stripe period, and use the reciprocal of the second product as the predicted stroboscopic frequency; wherein, the stroboscopic stripe period is the number of pixels from the start point of the stroboscopic stripe to the next start point, and the stroboscopic stripe period is the third ratio of the height of the luminance mean column vector to the position point; In response to the predicted stroboscopic frequency being greater than a first preset value and less than a second preset value, or in response to the predicted stroboscopic frequency being greater than a third preset value and less than a fourth preset value, determine that there is stroboscopy in the image frame sequence; wherein, the first preset value, the second preset value, the third preset value, and the fourth preset value are all related to a fourth frequency-domain threshold.

9. The stroboscopic detection method according to claim 1, characterized in that After the step of performing stroboscopic detection on the image frame sequence based on the luminance mean column vector, it includes: In response to there being stroboscopy in the image frame sequence, adjust the exposure time and image gain value of the image frame sequence.

10. An electronic device, characterized in that, It includes a memory and a processor that are mutually coupled, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the stroboscopic detection method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is used to implement the stroboscopic detection method according to any one of claims 1 to 9.

Images