Image stroboscopic detection method and device, storage medium, and terminal

Abstract

An image stroboscopic detection method and device, storage medium, and terminal, the method comprising: providing at least three consecutive frames of images; extracting a brightness column vector of each frame of the image, each brightness vector in the brightness column vector is used to indicate the brightness value of the minimum unit of each row in the image; based on the brightness column vectors of the at least three frames of images, determining a stroboscopic statistical value, the stroboscopic statistical value is used to indicate the number of stroboscopic times of each of the brightness vectors; at least based on the stroboscopic statistical value, determining whether the at least three frames of images have stroboscopic. The present invention can effectively reduce the computational complexity.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to an image stroboscopic detection method and device, a storage medium, and a terminal. Background Art

[0002] Since the energy of AC power changes periodically over time, when the light source works under this type of power, if the exposure information of the shooting device does not match the frequency of the light source, there will be changes in brightness and darkness when shooting images due to the difference in light intensity during the exposure process. In addition, since the exposure strategy of complementary metal oxide semiconductor (CMOS) components is line-by-line exposure and there is a time difference between lines, the changes in brightness and darkness will roll, which is the fundamental cause of stroboscopic images.

[0003] The currently commonly used image stroboscopic detection method processes the difference image of two adjacent frames during the shooting process, which can better retain the stroboscopic characteristics in the image on the basis of effectively removing the image background information.

[0004] However, the computational complexity of the existing technology is high.

[0005] After research, the inventors of the present invention discovered that, in the prior art, it is necessary to utilize discrete Fourier transform to calculate one-dimensional discrete Fourier transform coefficients corresponding to 100 Hz and 120 Hz for all pixels in each row of the difference image in units of rows, and to determine from the frequency domain results which frequency is dominant to guide the adjustment of the current exposure information. That is, the processing involves conversion operations from the pixel domain to the frequency domain, resulting in high computational complexity. Summary of the invention

[0006] The technical problem solved by the present invention is to provide an image stroboscopic detection method and device, a storage medium, and a terminal, which can effectively reduce the computational complexity.

[0007] To solve the above technical problems, an embodiment of the present invention provides an image flicker detection method, comprising the following steps: providing at least three consecutive frames of images; extracting a brightness column vector of each frame of the image, wherein each brightness vector in the brightness column vector is used to indicate the brightness value of the minimum unit of each row in the image; based on the brightness column vectors of the at least three frames of images, determining a flicker statistic value, wherein the flicker statistic value is used to indicate the number of flickers of each of the brightness vectors; and determining whether flicker exists in the at least three frames of images at least based on the flicker statistic value.

[0008] Optionally, the at least three frames of images are all Bayer array images; extracting the brightness column vector of each frame of image includes: extracting the R value, Gr value, Gb value and B value of each minimum unit in each frame of image; determining the brightness vector matrix of each frame of image according to the R value, Gr value, Gb value and B value; and determining the brightness column vector of each frame of image based on the brightness vector matrix.

[0009] Optionally, the following formula is used to determine the brightness vector matrix of each frame of the image according to the R value, Gr value, Gb value and B value:

[0010] Y=(77×R+75×(Gr+Gb)+29×B)>>10

[0011] Among them, Y is used to represent the brightness vector matrix, R is used to represent the R value matrix, Gr is used to represent the Gr value matrix, Gb is used to represent the Gb value matrix, and B is used to represent the B value matrix.

[0012] Optionally, based on the brightness vector matrix, determining the brightness column vector of each frame image includes: downsampling each frame image to obtain a sampled image of a preset size, wherein the preset size includes a preset number of rows and a preset number of columns; and summing the brightness vectors of each row of the sampled image to generate a brightness column vector of the preset number of columns.

[0013] Optionally, the following formula is used to perform a summation operation on the brightness vector of each row of the sampled image:

[0014]

[0015] Among them, Y(p,q) is used to represent the brightness vector of the pth row and qth column of the sampled image, Y_acc(p) is used to represent the brightness vector of the pth row of the sampled image, and width is used to represent the number of minimum units in each minimum unit of each row in the sampled image.

[0016] Optionally, based on the brightness column vectors of at least three frames of images, determining the flicker statistics includes: subtracting the brightness column vectors of each two adjacent frames of the at least three frames of images to obtain at least a first 2-frame differential brightness column vector diff1 and a second 2-frame differential brightness column vector diff2; standardizing the first 2-frame differential brightness column vector diff1 and the second 2-frame differential brightness column vector diff2 to obtain a first standardized 2-frame differential brightness column vector and a second standardized 2-frame differential brightness column vector; determining flag information; and determining the flicker statistics based on at least the flag information.

[0017] Optionally, the standardization processing of the first two-frame differential brightness column vector diff1 and the second two-frame differential brightness column vector diff2 includes: determining the average value diff_mean, the maximum value diff_max and the minimum value diff_min of the differential brightness column vector of each frame; and using the following formula to standardize the first two-frame differential brightness column vector diff1 and the second two-frame differential brightness column vector diff2:

[0018] max_sq_range=max(diff_max2-diff_min2,diff_max1-diff_min1)

[0019]

[0020]

[0021] Among them, max_sq_range is used to indicate the maximum sequence range, Diff_nm1 is used to indicate the first standardized 2-frame differential brightness column vector after standardization, Diff_nm2 is used to indicate the second standardized 2-frame differential brightness column vector after standardization, diff_mean1 is used to indicate the average value of the first differential brightness column vector, diff_mean2 is used to indicate the average value of the second differential brightness column vector, diff_max1 is used to indicate the maximum value of the first differential brightness column vector, diff_max2 is used to indicate the maximum value of the second differential brightness column vector, diff_min 1 is used to indicate the minimum value of the first differential brightness column vector, and diff_min2 is used to indicate the minimum value of the second differential brightness column vector.

[0022] 8. Optionally, use the following formula to determine the flag information:

[0023]

[0024] Among them, n is used to represent the number of brightness vectors in the brightness column vector, diff_sign(i) is used to represent the flag information of the i-th brightness vector, Diff_nm1(i) is used to represent the i-th brightness vector of the first standardized processing of 2-frame differential brightness column vectors, and Diff_nm2(i) is used to represent the i-th brightness vector of the second standardized processing of 2-frame differential brightness column vectors.

[0025] Optionally, the first two-frame differential brightness column vector diff1 adopts a first filtered differential brightness column vector Diff_ft1; and the second two-frame differential brightness column vector diff2 adopts a second filtered differential brightness column vector Diff_ft2.

[0026] Optionally, each filtered differential brightness column vector Diff_ft is determined using the following formula:

[0027]

[0028] Among them, Diff_ft(j) is used to represent the j-th filtered differential brightness column vector, filt_len is used to represent the filtering window length, and Diff_nm(k+j) is used to represent the k+j-th brightness vector of each standardized processing of the 2-frame differential brightness column vector.

[0029] Optionally, the following formula is used to determine the flag information:

[0030]

[0031] Among them, diff_ft_sign(i) is used to represent the filter flag information of the i-th brightness vector, n is used to represent the number of brightness vectors in the brightness column vector, Diff_ft1 is used to represent the first filtered differential brightness column vector, and Diff_ft2 is used to represent the second filtered differential brightness column vector.

[0032] Optionally, determining the flicker statistics value at least based on the flag information includes: determining one or more points where Diff_ft1(i) and Diff_ft2(i) intersect, recorded as filter transition points; taking each filter transition point as the origin, using each preset search window in the preset search window group, searching for the number of times the value is 0 and the number of times the value is 1 in diff_ft_sign(i) in the first direction and the second direction, respectively, recorded as the number of times the value is 0 in the first direction, the number of times the value is 0 in the second direction, the number of times the value is 1 in the first direction, and the number of times the value is 0 in the second direction, respectively; determining the flicker statistics value based on the number of times the value is 0 in the first direction, the number of times the value is 0 in the second direction, the number of times the value is 1 in the first direction, and the number of times the value is 0 in the second direction obtained from each filter transition point and each preset search window; wherein the first direction and the second direction are different, and the first direction and the second direction are selected from the direction in which the sequence number of the brightness vector in the brightness column vector increases or decreases.

[0033] Optionally, determining the flicker statistical value according to the number of times the first direction takes 0, the number of times the second direction takes 0, the number of times the first direction takes 1, and the number of times the second direction takes 0 obtained from each filter transition point and each preset search window includes: for each preset search window of each filter transition point, if |pos_l-peak_dist| and |neg_r-peak_dist| are both ≤peak_dist / x or |neg_l-peak_dist| and |pos_r-peak_dist| are both ≤peak_dist / x, determining that the count of the flicker statistical value flicker_val is increased by a preset flicker weight value; traversing each preset search window of each filter transition point until the traversal is completed or the statistical flicker statistical value flicker_val reaches a preset upper limit value; wherein pos_l is used to indicate the number of times the first direction takes 1, pos_r is used to indicate the number of times the second direction takes 1, neg_l is used to indicate the number of times the first direction takes 0, pos_r is used to indicate the number of times the second direction takes 0, peak_dist is used to indicate the window length of each preset search window, and x is used to indicate a preset integer value.

[0034] Optionally, the initial value of the stroboscopic statistical value is 0, and the preset stroboscopic weight value is 1.

[0035] Optionally, the number of the images is at least five frames, and the first filtered differential brightness column vector Diff_ft1 and the second filtered differential brightness column vector Diff_ft2 are obtained based on the first three frames of images; before determining whether the image has flicker based at least on the flicker statistics, the image flicker detection method further includes: determining a third filtered differential brightness column vector Diff_ft1' and a fourth filtered differential brightness column vector Diff_ft2' based on the third to fifth frames of images; performing a modulo operation on Diff_ft1 and Diff_ft2 and then cascading them to generate a first cascade brightness column vector of length 2n, and performing a modulo operation on Diff_ft1 and Diff_ft2; f_ft1' and Diff_ft2' are cascaded after a modulo operation to generate a second cascaded brightness column vector with a length of 2n; the first cascaded brightness column vector and the second cascaded brightness column vector are cascaded to obtain a cascaded column vector diff_long with a length of 4n; a part or multiple parts of the brightness vector in the cascaded column vector with a length of 4n are selected, m values ​​are uniformly sampled for each part in turn, and a subtraction operation is performed between each adjacent two values ​​to obtain m-1 gradient values; according to the m-1 gradient values ​​of each part, a weight value of the stroboscopic statistical value is determined; wherein n is used to represent the number of brightness vectors in the brightness column vector.

[0036] Optionally, selecting a part of the brightness vectors in the concatenated column vectors having a length of 4n includes: in the concatenated column vectors diff_long having a length of 4n, selecting brightness vectors using the following conditions, which are recorded as a brightness vector set:

[0037] diff_still_ft(s+t)≥diff_still_ft(s)&&

[0038] diff_still_ft(st)≥diff_still_ft(s)&&

[0039] diff_still_ft≤300(t=1,2,...,T)

[0040] Determine the serial number difference between adjacent brightness vectors in the brightness vector set, and if the serial number difference is between 40 and 150, select diff_long between the serial numbers to which the serial number difference belongs as a part of the brightness vector in the cascade column vector with a length of 4n; traverse the brightness vector set to obtain a part or multiple parts of the brightness vector in the cascade column vector with a length of 4n; wherein s is used to represent the sth brightness vector traversed, T is a preset positive integer value, and t is a positive integer selected from the range from 1 to T.

[0041] Optionally, determining the weight value of the stroboscopic statistical value according to the m-1 gradient values ​​of each part includes: counting the number of negative gradient values ​​among the m-1 gradient values ​​of each part; determining the weight value of the stroboscopic statistical value according to the number of negative gradient values; wherein, the greater the number of negative gradient values, the greater the weight value of the stroboscopic statistical value.

[0042] Optionally, determining whether the image has flicker at least according to the flicker statistical value includes: determining a flicker characteristic quantization value Flk_val according to the flicker statistical value and a weight value of the flicker statistical value by using the following formula;

[0043] Flk_val=flicker_val×WEIGHT(flicker_long, thrd_n) / 4

[0044]

[0045] At least according to the quantified value of the flicker feature, determine whether the image has flicker; wherein Flk_val is used to represent the quantified value of the flicker feature, thrd_n is used to represent a preset flicker threshold, the WEIGHT(x, y) function is used to indicate the weight value of the flicker statistical value, and x and y are used to represent input variables of the WEIGHT(x, y) function; the larger the quantified value of the flicker feature, the greater the probability that the image has flicker.

[0046] Optionally, determining whether the image has stroboscopic characteristics at least according to the stroboscopic characteristic quantization value Flk_val includes: determining a long sequence stroboscopic characteristic value Flk_v_val according to the stroboscopic characteristic quantization value Flk_val using the following formula:

[0047]

[0048] According to the long sequence stroboscopic characteristic value Flk_v_val being greater than a preset long sequence stroboscopic characteristic threshold, it is determined that stroboscopic exists in the image; wherein Flk_v_val is used to indicate the long sequence stroboscopic characteristic value, and thrd_v is used to indicate the preset stroboscopic threshold.

[0049] To solve the above technical problems, an embodiment of the present invention provides an image flicker detection device, comprising: a providing module, used to provide at least three consecutive frames of images; an extraction module, used to extract the brightness column vector of each frame of the image, each brightness vector in the brightness column vector is used to indicate the brightness value of the minimum unit of each row in the image; a flicker statistics value determination module, used to determine the flicker statistics value based on the brightness column vectors of the at least three frames of images, the flicker statistics value is used to indicate the number of flickers of each of the brightness vectors; a flicker determination module, used to determine whether the at least three frames of images have flicker based on at least the flicker statistics value.

[0050] In order to solve the above technical problem, an embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above image stroboscopic detection method are executed.

[0051] In order to solve the above technical problem, an embodiment of the present invention provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of the above image stroboscopic detection method when running the computer program.

[0052] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0053] In an embodiment of the present invention, by setting the brightness column vector of each frame of image to be extracted, determining the flicker statistics based on the brightness column vectors of the at least three frames of image, and setting the flicker statistics to indicate the number of flickers of each of the brightness vectors, it is possible to perform calculations only within the pixel domain to determine whether the image has flicker. Compared with the prior art, which requires a conversion operation from the pixel domain to the frequency domain during the processing, resulting in higher computational complexity, the solution of the embodiment of the present invention can effectively reduce the computational complexity.

[0054] Furthermore, in an embodiment of the present invention, a brightness vector matrix is ​​determined according to R value, Gr value, Gb value and B value, and then a brightness column vector of each frame of image is determined, which can improve the accuracy of determining the brightness column vector of each frame of image.

[0055] Furthermore, by normalizing the first 2-frame differential brightness column vector diff1 and the second 2-frame differential brightness column vector diff2 to obtain a first normalized 2-frame differential brightness column vector and a second normalized 2-frame differential brightness column vector, the accuracy of determining the flag information can be improved.

[0056] Furthermore, by setting the first two-frame differential brightness column vector diff1 to adopt the first filtered differential brightness column vector Diff_ft1, and the second two-frame differential brightness column vector diff2 to adopt the second filtered differential brightness column vector Diff_ft2, waveform noise can be eliminated by filtering, thereby further improving the accuracy of determining the filter flag information.

[0057] Furthermore, by cascading the brightness column vectors and then determining the weight value of the flicker statistics, flicker can be detected based on images with more frames and longer brightness column vectors, and the weight value can be used to effectively adjust the flicker detection, thereby effectively improving the accuracy of detection.

[0058] Further, the long sequence stroboscopic characteristic value Flk_v_val is determined, and then based on the long sequence stroboscopic characteristic value Flk_v_val being greater than the preset long sequence stroboscopic characteristic threshold, it is determined that stroboscopic exists, and a brightness column vector with a longer length can be further used to detect stroboscopic, thereby further effectively improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flow chart of an image stroboscopic detection method in an embodiment of the present invention;

[0060] Figure 2 is a schematic diagram of a Bayer array image in an embodiment of the present invention;

[0061] Figure 3 yes Figure 1A flowchart of a specific implementation of step S13;

[0062] Figure 4 is a partial flow chart of another image stroboscopic detection method in an embodiment of the present invention;

[0063] Figure 5 It is a structural schematic diagram of an image stroboscopic detection device in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] With the rapid iteration of consumer and industrial electronic devices, cameras have become a standard feature of many embedded devices. The imaging element of current mainstream embedded camera devices is a combination of complementary metal oxide semiconductor (CMOS) and rolling shutter. Such elements are easily affected by stroboscopic light when working under AC power lighting conditions, resulting in a sharp decrease in the quality of preview images.

[0065] As mentioned above, stroboscopic images may appear in the captured images. The currently used image stroboscopic detection method processes the difference image of two adjacent frames during the shooting process, which can effectively remove the image background information and better retain the stroboscopic features in the image. However, the computational complexity of the existing technology is relatively high.

[0066] Specifically, in an existing specific implementation, the processing object of the currently commonly used image stroboscopic detection method is the difference image of two adjacent frames during the shooting process, so that the stroboscopic characteristics in the image can be better retained on the basis of effectively removing the image background information. Its mainstream means is to use discrete Fourier transform, calculate the one-dimensional discrete Fourier transform coefficients corresponding to 100Hz and 120Hz for all pixels in each row of the difference image in units of rows, and determine which frequency is dominant from the frequency domain results to guide the adjustment of the current exposure information.

[0067] After research, the inventors of the present invention discovered that, in the prior art, it is necessary to utilize discrete Fourier transform to calculate one-dimensional discrete Fourier transform coefficients corresponding to 100 Hz and 120 Hz for all pixels in each row of the difference image in units of rows, and to determine from the frequency domain results which frequency is dominant to guide the adjustment of the current exposure information. That is, the processing involves conversion operations from the pixel domain to the frequency domain, resulting in high computational complexity.

[0068] More specifically, the currently commonly used image stroboscopic detection methods can guarantee a relatively ideal detection accuracy for ideal stroboscopic phenomena. The definition of ideal stroboscopic here refers to the light and dark intervals in the preview image being clearly visible, the number of stripes being between 3 and 5, and the light and dark stripes rolling at an appropriate rate. However, due to the differences in the properties of the imaging sensor (Sensor) (such as whether it is a binning output / different resolution sizes, etc.), the stroboscopic phenomena of different models of camera equipment vary greatly, especially the stroboscopic phenomena that are stationary or extremely slow in rolling. In visual perception, there is no sign of "flickering", but a pure superposition of image background information and light and dark stripes. When encountering such stroboscopic phenomena, the performance of the commonly used image stroboscopic detection methods will no longer meet expectations, and the frequency domain algorithms represented by discrete Fourier transform require that each pixel point be mapped to the frequency domain, resulting in a high computational complexity of such algorithms.

[0069] In an embodiment of the present invention, by setting the brightness column vector of each frame of image to be extracted, determining the flicker statistics based on the brightness column vectors of the at least three frames of image, and setting the flicker statistics to indicate the number of flickers of each of the brightness vectors, it is possible to perform calculations only within the pixel domain to determine whether the image has flicker. Compared with the prior art, which requires a conversion operation from the pixel domain to the frequency domain during the processing, resulting in higher computational complexity, the solution of the embodiment of the present invention can effectively reduce the computational complexity.

[0070] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0071] Reference Figure 1 , Figure 1 1 is a flow chart of an image stroboscopic detection method in an embodiment of the present invention. The image stroboscopic detection method may include steps S11 to S14:

[0072] Step S11: providing at least three consecutive frames of images;

[0073] Step S12: extracting the brightness column vector of each frame of the image, wherein each brightness vector in the brightness column vector is used to indicate the brightness value of the minimum unit of each row in the image;

[0074] Step S13: determining a stroboscopic statistical value based on the brightness column vectors of the at least three frames of images, wherein the stroboscopic statistical value is used to indicate the number of stroboscopic times of each of the brightness vectors;

[0075] Step S14: determining whether there is flicker in the at least three frames of images at least according to the flicker statistics.

[0076] In a specific implementation of step S11, at least three consecutive frames of images are provided.

[0077] Furthermore, the at least three frames of images may all be Bayer array images.

[0078] Reference Figure 2 , Figure 2 is a schematic diagram of a Bayer array image in an embodiment of the present invention. The Bayer array image may include multiple minimum cells, for example Figure 2 The area encircled by the middle circle.

[0079] Each Cell may include a blue pixel (B), a green-blue pixel (Gb), a green-red pixel (Gr), and a red pixel (R), and each Cell may have an R value, a Gr value, a Gb value, and a B value.

[0080] In the specific implementation of step S12, the brightness column vector of each frame of the image may be extracted, and each brightness vector in the brightness column vector is used to indicate the brightness value of the minimum unit of each row in the image.

[0081] Furthermore, the step of extracting the brightness column vector of each frame of image may include: extracting the R value, Gr value, Gb value and B value of each minimum unit in each frame of image; determining the brightness vector matrix of each frame of image according to the R value, Gr value, Gb value and B value; and determining the brightness column vector of each frame of image based on the brightness vector matrix.

[0082] Furthermore, the following formula may be used to determine the brightness vector matrix of each frame of the image according to the R value, Gr value, Gb value, and B value:

[0083] Y=(77×R+75×(Gr+Gb)+29×B)>>10

[0084] Among them, Y is used to represent the brightness vector matrix, R is used to represent the R value matrix, Gr is used to represent the Gr value matrix, Gb is used to represent the Gb value matrix, and B is used to represent the B value matrix.

[0085] It should be pointed out that by setting the right shift to 10 bits (>>10), the number of bits of the brightness value can be reduced in a controlled manner, thereby reducing the impact of different colors on the brightness value, thereby improving the accuracy of strobe detection in subsequent operations.

[0086] In the embodiment of the present invention, the brightness vector matrix is ​​determined according to the R value, the Gr value, the Gb value and the B value, and then the brightness column vector of each frame image is determined, which can improve the accuracy of determining the brightness column vector of each frame image.

[0087] It should be pointed out that the image processing format of the present invention is not limited to the Bayer array image, and can be applied to all kinds of digital image domains that can count brightness information, wherein the digital image domain includes but is not limited to the original image (RAW) domain, the YCrCb color encoding (YUV) domain, and the full color (RGB) domain.

[0088] Furthermore, based on the brightness vector matrix, determining the brightness column vector of each frame image includes: downsampling each frame image to obtain a sampled image of a preset size, wherein the preset size includes a preset number of rows and a preset number of columns; and summing the brightness vectors of each row of the sampled image to generate a brightness column vector of the preset number of columns.

[0089] Furthermore, the following formula may be used to perform a summation operation on the brightness vector of each row of the sampled image:

[0090]

[0091] Among them, Y(p,q) is used to represent the brightness vector of the pth row and qth column of the sampled image, Y_acc(p) is used to represent the brightness vector of the pth row of the sampled image, and width is used to represent the number of minimum units in each minimum unit of each row in the sampled image.

[0092] In a specific implementation of the embodiment of the present invention, the corresponding sampling coefficient can be calculated according to the ratio of the input image size to the preset resolution, and the current image can be downsampled to the preset size. Taking the preset size of 640 (pixels) × 480 (pixels) as an example, the sampled images are added one by one in the row direction to generate a column vector with a height of 480. That is, in the above formula, the maximum value of j is 480.

[0093] In a specific implementation of step S13, the stroboscopic statistical value may be determined based on the brightness column vectors of at least three frames of images.

[0094] Reference Figure 3 , Figure 3 yes Figure 1 The step of determining the stroboscopic statistical value may include steps S31 to S34, and each step is described below.

[0095] In step S31, the brightness column vectors of every two adjacent frames of the at least three frames of images are subtracted to obtain at least a first two-frame differential brightness column vector diff1 and a second two-frame differential brightness column vector diff2.

[0096] In step S32, the first two-frame differential brightness column vector diff1 and the second two-frame differential brightness column vector diff2 are standardized to obtain a first standardized two-frame differential brightness column vector and a second standardized two-frame differential brightness column vector.

[0097] Furthermore, the step of normalizing the first two-frame differential brightness column vector diff1 and the second two-frame differential brightness column vector diff2 may include: determining the average value diff_mean, the maximum value diff_max and the minimum value diff_min of the differential brightness column vector of each frame; and normalizing the first two-frame differential brightness column vector diff1 and the second two-frame differential brightness column vector diff2 using the following formula:

[0098] max_sq_range

[0099] =max(diff_max2-diff_min2,diff_max1-diff_min1)

[0100]

[0101]

[0102] Among them, max_sq_range is used to indicate the maximum sequence range, Diff_nm1 is used to indicate the first standardized 2-frame differential brightness column vector after standardization, Diff_nm2 is used to indicate the second standardized 2-frame differential brightness column vector after standardization, diff_mean1 is used to indicate the average value of the first differential brightness column vector, diff_mean2 is used to indicate the average value of the second differential brightness column vector, diff_max1 is used to indicate the maximum value of the first differential brightness column vector, diff_max2 is used to indicate the maximum value of the second differential brightness column vector, diff_min 1 is used to indicate the minimum value of the first differential brightness column vector, and diff_min2 is used to indicate the minimum value of the second differential brightness column vector.

[0103] In step S33, the flag information is determined.

[0104] In an embodiment of the present invention, flag information can be determined based on a first standardized two-frame differential brightness column vector and a second standardized two-frame differential brightness column vector; the standardized two-frame differential statistical information can also be filtered, and then the flag information can be determined based on the first filtered differential brightness column vector and the second filtered differential brightness column vector.

[0105] In a first specific implementation of the embodiment of the present invention, the following formula may be used to determine the flag information:

[0106]

[0107] Among them, n is used to represent the number of brightness vectors in the brightness column vector, diff_sign(i) is used to represent the flag information of the i-th brightness vector, Diff_nm1(i) is used to represent the i-th brightness vector of the first standardized processing of 2-frame differential brightness column vectors, and Diff_nm2(i) is used to represent the i-th brightness vector of the second standardized processing of 2-frame differential brightness column vectors.

[0108] In the embodiment of the present invention, the flag information diff_sign may be set to represent the relative relationship between the differential statistical information of two frames after the standardization process.

[0109] In an embodiment of the present invention, the first 2-frame differential brightness column vector diff1 and the second 2-frame differential brightness column vector diff2 can be standardized to obtain a first standardized 2-frame differential brightness column vector and a second standardized 2-frame differential brightness column vector, thereby improving the accuracy of determining the flag information.

[0110] In a second specific implementation of the embodiment of the present invention, the first two-frame differential brightness column vector diff1 adopts the first filtered differential brightness column vector Diff_ft1; and the second two-frame differential brightness column vector diff2 adopts the second filtered differential brightness column vector Diff_ft2.

[0111] Specifically, a sliding window can be used to filter the difference statistical information of the two frames of normalization processing at the same time, thereby effectively eliminating waveform noise and improving the accuracy of subsequent judgment.

[0112] Furthermore, each filtered differential brightness column vector Diff_ft can be determined using the following formula:

[0113]

[0114] Among them, Diff_ft(j) is used to represent the j-th filtered differential brightness column vector, filt_len is used to represent the filtering window length, and Diff_nm(k+j) is used to represent the k+j-th brightness vector of each standardized processing of the 2-frame differential brightness column vector.

[0115] Furthermore, the following formula can be used to determine the flag information:

[0116]

[0117] Among them, diff_ft_sign(i) is used to represent the filter flag information of the i-th brightness vector, n is used to represent the number of brightness vectors in the brightness column vector, Diff_ft1 is used to represent the first filtered differential brightness column vector, and Diff_ft2 is used to represent the second filtered differential brightness column vector.

[0118] In an embodiment of the present invention, by setting the first two-frame differential brightness column vector diff1 to adopt the first filtered differential brightness column vector Diff_ft1, and the second two-frame differential brightness column vector diff2 to adopt the second filtered differential brightness column vector Diff_ft2, waveform noise can be eliminated through filtering, thereby further improving the accuracy of determining the filter flag information.

[0119] In step S34, a stroboscopic statistical value is determined at least according to the flag information.

[0120] Among them, diff_2nd can be defined as the transition information of diff_sign(i) to characterize the intersection of Diff_nm1(i) and Diff_nm2(i); diff_ft_2nd is defined as the transition information of diff_ft_sign(i) to characterize the intersection of Diff_ft1(f) and Diff_ft2(f).

[0121] Further, at least based on the flag information, the step of determining the flicker statistics may include: determining one or more points where Diff_ft1(i) and Diff_ft2(i) intersect, recorded as filter transition points; taking each filter transition point as the origin, using each preset search window in the preset search window group, searching for the number of times the value is 0 and the number of times the value is 1 in diff_ft_sign(i) in the first direction and the second direction, respectively, recorded as the number of times the value is 0 in the first direction, the number of times the value is 0 in the second direction, the number of times the value is 1 in the first direction, and the number of times the value is 0 in the second direction, respectively; determining the flicker statistics based on the number of times the value is 0 in the first direction, the number of times the value is 0 in the second direction, the number of times the value is 1 in the first direction, and the number of times the value is 0 in the second direction obtained from each filter transition point and each preset search window; wherein the first direction and the second direction are different, and the first direction and the second direction are selected from the direction in which the sequence number of the brightness vector in the brightness column vector increases or decreases.

[0122] Specifically, by setting each filtering transition point as the origin, the number of calculations can be effectively reduced, and the calculation complexity can be effectively reduced.

[0123] Furthermore, according to the number of times the first direction takes 0, the number of times the second direction takes 0, the number of times the first direction takes 1, and the number of times the second direction takes 0 obtained from each filter transition point and each preset search window, the step of determining the flicker statistical value may include: for each preset search window of each filter transition point, if |pos_l-peak_dist| and |neg_r-peak_dist| are both ≤peak_dist / x or |neg_l-peak_dist| and |pos_r-peak_dist| are both ≤peak_dist / x, determining that the count of the flicker statistical value flicker_val is increased by a preset flicker weight value; traversing each preset search window of each filter transition point until the traversal is completed or the statistical flicker statistical value flicker_val reaches a preset upper limit value; wherein pos_l is used to indicate the number of times the first direction takes 1, pos_r is used to indicate the number of times the second direction takes 1, neg_l is used to indicate the number of times the first direction takes 0, pos_r is used to indicate the number of times the second direction takes 0, peak_dist is used to indicate the window length of each preset search window, and x is used to indicate a preset integer value.

[0124] In a specific implementation, whenever |pos_l-peak_dist| and |neg_r-peak_dist| are both ≤peak_dist / x, or |neg_l-peak_dist| and |pos_r-peak_dist| are both ≤peak_dist / x, a preset strobe weight value is added to the count of the strobe statistic value flicker_val.

[0125] As a non-limiting example, x may be set to 4-6, for example, x is 5. 1 / x may also be set to 15%-35%, for example, 1 / x is 20%.

[0126] Furthermore, the initial value of the stroboscopic statistical value may be 0, and the preset stroboscopic weight value may be 1.

[0127] In the embodiment of the present invention, 1 may be added only when the above condition is met, thereby effectively reducing the amount of calculation.

[0128] Furthermore, the image stroboscopic detection method may also include a scheme for detecting stroboscopic images based on images with more frames and longer brightness column vectors.

[0129] Reference Figure 4 , Figure 4 is a partial flow chart of another image stroboscopic detection method in an embodiment of the present invention. The another image stroboscopic detection method may include Figure 1The steps S11 to S14 shown may include steps S41 to S45, and each step is described below.

[0130] Among them, the number of images can be at least five frames, the first filtered differential brightness column vector Diff_ft1 and the second filtered differential brightness column vector Diff_ft2 are obtained based on the first three frames of images, and steps S41 to S45 can be before step S14, and can also be before or after other appropriate steps.

[0131] In step S41 , a third filtered differential brightness column vector Diff_ft1 ′ and a fourth filtered differential brightness column vector Diff_ft2 ′ are determined according to the third frame image to the fifth frame image.

[0132] The third filtered differential brightness column vector Diff_ft1' may be determined by using the difference between the fourth frame image and the third frame image, and the fourth filtered differential brightness column vector Diff_ft2' may be determined by using the difference between the fifth frame image and the fourth frame image.

[0133] In step S42, Diff_ft1 and Diff_ft2 are cascaded after performing a modulo operation to generate a first cascaded brightness column vector with a length of 2n, and Diff_ft1' and Diff_ft2' are cascaded after performing a modulo operation to generate a second cascaded brightness column vector with a length of 2n.

[0134] Wherein, n is used to represent the number of brightness vectors in the brightness column vector.

[0135] It should be noted that, taking n as 480 as an example, the length of the first cascade brightness column vector with a length of 2n may be 960, and the vector may be temporarily stored in the memory space for cascading with the subsequent second cascade brightness column vector with a length of 960.

[0136] In step S43, the first cascaded brightness column vector and the second cascaded brightness column vector are cascaded to obtain a cascaded column vector diff_long with a length of 4n.

[0137] It should be pointed out that, taking n as 480 as an example, the length of the concatenated column vector diff_long with a length of 4n is 1920.

[0138] In step S44, a part or multiple parts of the brightness vector in the cascade column vector of length 4n are selected, m values ​​are uniformly sampled for each part in turn, and a subtraction operation is performed between every two adjacent values ​​to obtain m-1 gradient values.

[0139] Specifically, in the cascade column vector of length 4n, screening is required to select a portion of points that meet the conditions to form a point set valley, thereby effectively reducing the amount of subsequent operations and further reducing the computational complexity.

[0140] Further, the step of selecting a part of the brightness vectors in the concatenated column vectors with a length of 4n may include: in the concatenated column vectors diff_long with a length of 4n, using the following conditions to filter the brightness vectors, recorded as a brightness vector set:

[0141] diff_still_ft(s+t)≥diff_still_ft(s)&&

[0142] diff_still_ft(st)≥diff_still_ft(s)&&

[0143] diff_still_ft≤300(t=1,2,...,T)

[0144] Determine the serial number difference between adjacent brightness vectors in the brightness vector set, and if the serial number difference is between 40 and 150, select diff_long between the serial numbers to which the serial number difference belongs as a part of the brightness vector in the cascade column vector with a length of 4n; traverse the brightness vector set to obtain a part or multiple parts of the brightness vector in the cascade column vector with a length of 4n; wherein s is used to represent the sth brightness vector traversed, T is a preset positive integer value, and t is a positive integer selected from the range from 1 to T.

[0145] In the embodiment of the present invention, in the cascade column vector of length 4n, the above formula is used to screen and select a part of points that meet the conditions to form a point set valley, which can effectively reduce the amount of subsequent operations and further reduce the computational complexity.

[0146] In step S45, the weight value of the stroboscopic statistical value is determined according to the m-1 gradient values ​​of each part.

[0147] Further, determining the weight value of the stroboscopic statistical value according to the m-1 gradient values ​​of each part includes: among the m-1 gradient values ​​of each part, counting the number of negative gradient values; and determining the weight value of the stroboscopic statistical value according to the number of negative gradient values; wherein, the greater the number of negative gradient values, the greater the weight value of the stroboscopic statistical value.

[0148] In a specific implementation, m=13 can be taken as an example. For example, all points in the valley are traversed, and the points are assigned to peak_l and peak_r in turn during the traversal process. When peak_r-peak_l is between 40 and 150, 13 values ​​are uniformly sampled between diff_long(peak_l)ˉdiff_long(peak_r) and the difference between two adjacent values ​​is taken to obtain the gradient. Then, the number of 12 gradient values ​​whose latter are smaller than the former is counted, and this is used as the weight item for the accumulation of the cascaded strobe statistical value flicker_long.

[0149] Furthermore, at least according to the stroboscopic statistical value, the step of determining whether the image has stroboscopic may include: determining a stroboscopic characteristic quantization value Flk_val according to the stroboscopic statistical value and a weight value of the stroboscopic statistical value by using the following formula;

[0150] Flk_val=flicker_val×WEIGHT(flicker_long, thrd_n) / 4

[0151]

[0152] At least according to the quantified value of the flicker feature, determine whether the image has flicker; wherein Flk_val is used to represent the quantified value of the flicker feature, thrd_n is used to represent a preset flicker threshold, the WEIGHT(x,y) function is used to indicate the weight value of the flicker statistical value, and x and y are used to represent input variables of the WEIGHT(x,y) function; the larger the quantified value of the flicker feature, the greater the probability that the image has flicker.

[0153] In an embodiment of the present invention, by cascading the brightness column vectors and then determining the weight value of the flicker statistics, flicker can be detected based on images with more frames and longer brightness column vectors, and the weight value can be used to effectively adjust the flicker detection, thereby effectively improving the accuracy of detection.

[0154] Further, the step of determining whether the image has stroboscopic characteristics at least according to the stroboscopic characteristic quantization value Flk_val may include: determining a long sequence stroboscopic characteristic value Flk_v_val according to the stroboscopic characteristic quantization value Flk_val by using the following formula:

[0155]

[0156] According to the long sequence stroboscopic characteristic value Flk_v_val being greater than a preset long sequence stroboscopic characteristic threshold, it is determined that stroboscopic exists in the image; wherein Flk_v_val is used to indicate the long sequence stroboscopic characteristic value, and thrd_v is used to indicate the preset stroboscopic threshold.

[0157] In an embodiment of the present invention, by determining the long-sequence stroboscopic characteristic value Flk_v_val, and then determining that stroboscopic exists based on the long-sequence stroboscopic characteristic value Flk_v_val being greater than a preset long-sequence stroboscopic characteristic threshold, a brightness column vector with a longer length can be further used to detect stroboscopic, thereby further effectively improving the accuracy of detection.

[0158] In an embodiment of the present invention, by setting the brightness column vector of each frame of image to be extracted, determining the flicker statistics based on the brightness column vectors of the at least three frames of image, and setting the flicker statistics to indicate the number of flickers of each of the brightness vectors, it is possible to perform calculations only within the pixel domain to determine whether the image has flicker. Compared with the prior art, which requires a conversion operation from the pixel domain to the frequency domain during the processing, resulting in higher computational complexity, the solution of the embodiment of the present invention can effectively reduce the computational complexity.

[0159] Reference Figure 5 , Figure 5 : is a schematic diagram of the structure of an image stroboscopic detection device in an embodiment of the present invention. The image stroboscopic detection device may include:

[0160] Providing a module 51, for providing at least three consecutive frames of images;

[0161] An extraction module 52, used to extract a brightness column vector of each frame of the image, wherein each brightness vector in the brightness column vector is used to indicate a brightness value of a minimum unit of each row in the image;

[0162] A stroboscopic statistical value determination module 53, configured to determine a stroboscopic statistical value based on the brightness column vectors of the at least three frames of images, wherein the stroboscopic statistical value is used to indicate the number of stroboscopic times of each of the brightness vectors;

[0163] The stroboscopic determination module 54 is configured to determine whether stroboscopic exists in the at least three frames of images at least according to the stroboscopic statistical value.

[0164] For the principle, specific implementation and beneficial effects of the image stroboscopic detection device, please refer to the previous article and Figures 1 to 4 The related description about the image stroboscopic detection method shown is not repeated here.

[0165] The embodiment of the present invention further provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are executed. The storage medium may be a computer-readable storage medium, for example, may include a non-volatile memory or a non-transitory memory, and may also include an optical disk, a mechanical hard disk, a solid-state hard disk, etc.

[0166] The embodiment of the present invention further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of the above method when running the computer program. The terminal includes but is not limited to terminal devices such as mobile phones, computers, and tablet computers.

[0167] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A method for detecting image stroboscopic frequency, characterized in that: The following steps are involved: Providing at least three consecutive frames of images; Extracting a brightness column vector of each frame of the image, wherein each brightness vector in the brightness column vector is used to indicate a brightness value of a minimum unit of each row in the image; Determine a stroboscopic statistic value based on the brightness column vectors of the at least three frames of images, wherein the stroboscopic statistic value is used to indicate the number of stroboscopic times of each of the brightness vectors; The method of determining the stroboscopic statistical value based on the brightness column vectors of the at least three frames of images includes: performing subtraction processing on the brightness column vectors of each two adjacent frames of images in the at least three frames of images to obtain at least a first 2-frame differential brightness column vector diff1 and a second 2-frame differential brightness column vector diff2; performing filtering processing on the first 2-frame differential brightness column vector diff1 and the second 2-frame differential brightness column vector diff2 to obtain a first filtered differential brightness column vector and a second filtered differential brightness column vector; determining flag information based on the first filtered differential brightness column vector and the second filtered differential brightness column vector; and determining the stroboscopic statistical value at least according to the flag information; Determining whether there is stroboscopic frequency in the at least three frames of images at least according to the stroboscopic frequency statistics; Wherein, the at least three frames of images are all Bayer array images; Extracting the brightness column vector of each frame of image includes: Extract the R value, Gr value, Gb value and B value of each minimum unit in each frame image; Determine a brightness vector matrix of each frame of image according to the R value, Gr value, Gb value and B value; Based on the brightness vector matrix, a brightness column vector of each frame of image is determined.

2. The image stroboscopic detection method according to claim 1, characterized in that: The following formula is used to determine the brightness vector matrix of each frame of the image according to the R value, Gr value, Gb value and B value: Y=(77×R+75×(Gr+Gb)+29×B)>>10 Among them, Y is used to represent the brightness vector matrix, R is used to represent the R value matrix, Gr is used to represent the Gr value matrix, Gb is used to represent the Gb value matrix, and B is used to represent the B value matrix.

3. The image stroboscopic detection method according to claim 1, characterized in that: Based on the brightness vector matrix, determining the brightness column vector of each frame of image includes: Downsampling each frame of the image to obtain a sampled image of a preset size, wherein the preset size includes a preset number of rows and a preset number of columns; A sum operation is performed on the brightness vector of each row of the sampled image to generate brightness column vectors with the preset number of columns.

4. The image stroboscopic detection method according to claim 3, characterized in that: The following formula is used to perform a summation operation on the brightness vector of each row of the sampled image: Among them, Y(p,q) is used to represent the brightness vector of the pth row and qth column of the sampled image, Y_acc(p) is used to represent the brightness vector of the pth row of the sampled image, and width is used to represent the number of minimum units in each minimum unit of each row in the sampled image.

5. An image stroboscopic detection device, characterized in that: include: Providing a module for providing at least three consecutive frames of images; An extraction module, used to extract a brightness column vector of each frame of the image, wherein each brightness vector in the brightness column vector is used to indicate a brightness value of a minimum unit of each row in the image; A stroboscopic statistical value determination module, used to determine a stroboscopic statistical value based on the brightness column vectors of the at least three frames of images, wherein the stroboscopic statistical value is used to indicate the number of stroboscopic times of each of the brightness vectors; The method of determining the stroboscopic statistical value based on the brightness column vectors of the at least three frames of images includes: performing subtraction processing on the brightness column vectors of each two adjacent frames of images in the at least three frames of images to obtain at least a first 2-frame differential brightness column vector diff1 and a second 2-frame differential brightness column vector diff2; performing filtering processing on the first 2-frame differential brightness column vector diff1 and the second 2-frame differential brightness column vector diff2 to obtain a first filtered differential brightness column vector and a second filtered differential brightness column vector; determining flag information based on the first filtered differential brightness column vector and the second filtered differential brightness column vector; and determining the stroboscopic statistical value at least according to the flag information; A stroboscopic determination module, configured to determine whether stroboscopic exists in the at least three frames of images at least according to the stroboscopic statistical value; Wherein, the at least three frames of images are all Bayer array images; The extraction module comprises: The extraction submodule is used to extract the R value, Gr value, Gb value and B value of each minimum unit in each frame of the image; A brightness vector matrix determination module, used to determine the brightness vector matrix of each frame of image according to the R value, Gr value, Gb value and B value; The brightness column vector determination module is used to determine the brightness column vector of each frame of image based on the brightness vector matrix.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image stroboscopic detection method according to any one of claims 1 to 4 are executed.

7. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the image stroboscopic detection method according to any one of claims 1 to 4 are performed.

Images