Image processing device, method, imaging equipment and computer-readable storage medium

Abstract

The present application discloses an image processing device, method, imaging equipment, and computer-readable storage medium. The image processing device includes: an image processing module for obtaining a brightness difference value array including multiple difference values, determining a detection value based on the brightness difference value array after difference processing, wherein each difference value is the difference in the total brightness of one or more rows of pixels in two consecutive frames of images, and the detection value represents the change period and / or number of periods of each difference value; and a judgment module for determining whether there is stroboscopic movement between two consecutive frames of images based on the detection value and a predetermined range, wherein the predetermined range represents a predetermined change period and / or a predetermined number of periods obtained by theoretically calculating the difference value when stroboscopic movement exists. The image processing device can accurately and efficiently detect stroboscopic movement in an image by detecting the regularity of the brightness change of row pixels.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and more particularly, to an image processing apparatus, method, imaging device, and computer-readable storage medium. Background Art

[0002] Rolling shutter exposure (an exposure method in which different rows of pixels begin exposure at different times) is one of the most basic operating modes of CMOS (Complementary Metal Oxide Semiconductor) image sensors. Adjacent rows of pixels start exposure at intervals, and the voltage of the alternating current (AC) varies over time in the form of a sine wave, with an energy wave frequency twice that of the AC. This can result in different rows of pixels receiving different amounts of light energy during their exposure time, and the same row receiving different amounts of energy at different exposure start times. This results in dynamic, alternating light and dark stripes appearing in photos and videos captured by the camera. This phenomenon is called stroboscopic light.

[0003] In order to eliminate flicker, it is first necessary to detect the flicker phenomenon. However, the efficiency of existing flicker detection technology is low, and the accuracy of flicker detection needs to be improved. Summary of the Invention

[0004] In view of the above problems, an object of the present invention is to provide an image processing apparatus, method, imaging device and computer-readable storage medium to accurately and efficiently detect stroboscopic phenomena in images.

[0005] According to one aspect of the present invention, there is provided an image processing device, comprising: an image processing module for obtaining a brightness difference value array comprising a plurality of difference values, determining a detection value based on the brightness difference value array after differential processing, wherein each of the difference values ​​is the difference in total brightness between corresponding rows or rows of pixels in two consecutive frames of images, and the detection value represents a change period and / or number of periods of each of the difference values; and a judgment module for determining whether there is flicker in the two consecutive frames of images based on the detection value and a predetermined range, wherein the predetermined range represents a predetermined change period and / or a predetermined number of periods obtained by theoretically calculating the difference values ​​when flicker exists.

[0006] Optionally, the image processing module includes a differential processing unit, which is used to perform first-order differential processing, median filtering processing and first-order differential processing on the brightness difference value array in sequence.

[0007] Optionally, the differential processing unit performs median filtering on the brightness difference array, including: setting a filtering window; performing sliding summation on each element in the brightness difference array within the filtering window; and medianizing each element in the brightness difference array within the filtering window based on the result of the sliding summation, and assigning the element to the corresponding median.

[0008] Optionally, the image processing module includes a preprocessing unit for preprocessing the brightness difference array, where the preprocessing includes sliding filtering and / or centering processing.

[0009] Optionally, the image processing module includes a period detection unit for determining the detection value, and the period detection unit searches for each extreme point that meets preset conditions in the brightness difference value array after differential processing, determines the change period of the brightness difference value array according to the number of pixel rows between adjacent extreme points, and determines the number of periods contained in the brightness difference value array.

[0010] Optionally, the preset conditions set by the period detection unit include: the value of the corresponding position of the maximum point in the brightness difference array after differential processing in the brightness difference array after preprocessing is greater than 0, and the value of the corresponding position of the minimum point in the brightness difference array after differential processing in the brightness difference array after preprocessing is less than 0; and / or the absolute values ​​of the maximum point and the minimum point are greater than / equal to a predetermined value; and / or when there is no minimum point before two adjacent maximum points, and / or there is no maximum point between two adjacent minimum points, weighted calculation is used to merge two adjacent maximum points and / or two adjacent minimum points.

[0011] Optionally, the period detection unit determines the change period of the brightness difference array based on the number of pixel rows of the spacing between adjacent extreme points, including: determining half of the change period of the brightness difference array based on the number of pixel rows of the spacing between adjacent maximum points and minimum points; or determining the change period of the brightness difference array based on the number of pixel rows of the spacing between two adjacent maximum points; or determining the change period of the brightness difference array based on the number of pixel rows of the spacing between two adjacent minimum points.

[0012] Optionally, the image processing module includes a difference calculation unit, which divides the image statistical areas of the two consecutive frames of images into horizontal strips of equal height, each of the horizontal strips including one or more consecutive rows of pixels; sums the pixel brightness of each horizontal strip in the two consecutive frames of images to obtain a first brightness array and a second brightness array; and performs a subtraction operation on the first brightness array and the second brightness array to obtain the brightness difference array.

[0013] Optionally, the number of pixel rows contained in the horizontal strips divided by the difference calculation unit is an even number.

[0014] Optionally, when each of the pixels includes multiple sub-pixels, the difference calculation unit uses the brightness of any sub-pixel to represent the brightness of the pixel.

[0015] Optionally, the image processing module determines the predetermined range by a method comprising: calculating the predetermined change period and the predetermined number of periods of change in row pixel brightness difference based on the light source frequency and the parameters of the image acquisition device used to acquire the image; and determining the predetermined range based on the predetermined change period and the predetermined number of periods.

[0016] Optionally, the judgment module is used to score the detection value according to the predetermined range to obtain a score value, and when the score value exceeds a threshold, it is determined that there is stroboscopic light in the two consecutive frames of images.

[0017] According to a second aspect of the present invention, there is provided an image processing method, comprising: obtaining a brightness difference value array comprising a plurality of difference values, determining a detection value based on the brightness difference value array after differential processing, wherein each of the difference values ​​is the difference in total brightness between corresponding rows or rows of pixels in two consecutive frames of images, and the detection value represents a change period and / or number of periods of each of the difference values; and determining whether there is flicker in the two consecutive frames of images based on the detection value and a predetermined range, wherein the predetermined range represents a predetermined change period and / or a predetermined number of periods obtained by theoretically calculating the difference values ​​when flicker exists.

[0018] According to a third aspect of the present invention, there is provided an imaging device comprising:

[0019] An image acquisition device, configured to capture image frames to be detected, wherein the image frames to be detected include at least two consecutive image frames; and

[0020] The image processing device as described above is used to receive and determine whether there is stroboscopic light in the two consecutive frames of images.

[0021] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the image processing method described above is implemented.

[0022] The image processing apparatus, method, imaging device, and computer-readable storage medium provided by the present invention determine a detection value based on a brightness difference array after differential processing, and use the detection value to determine whether there is stroboscopic motion. The detection steps are simple, facilitating efficient stroboscopic motion detection. Furthermore, the differential processing step can effectively remove noise from the brightness difference array, thereby improving the accuracy of stroboscopic motion detection.

[0023] Furthermore, the differential processing includes first-order differential processing, median filtering processing, and first-order differential processing, which is beneficial to further improve the accuracy compared with the traditional second-order differential processing, and uses sliding summation instead of the sorting process in the median filtering process, which reduces the time complexity and space complexity and is conducive to hardware implementation.

[0024] Furthermore, preprocessing the brightness difference array before differential processing can further remove noise, improve algorithm robustness, and thus further improve the accuracy of stroboscopic detection.

[0025] Furthermore, after the differential processing, the extreme points that do not meet the preset conditions are removed, which can further remove noise and improve the accuracy of stroboscopic detection.

[0026] Furthermore, dividing the image statistical area helps save hardware costs and improve computing efficiency; setting the width of the horizontal stripe to an even number can avoid the green balance problem and help improve the accuracy of flicker detection; using sub-pixel brightness instead of pixel brightness helps further reduce the amount of calculation and improve computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0028] Figure 1 A voltage variation curve of a display device and a corresponding light source brightness variation curve are shown;

[0029] Figure 2 A block diagram of an image processing apparatus according to an embodiment of the present invention is shown;

[0030] Figure 3 A pixel map of an image according to an embodiment of the present invention is shown;

[0031] Figure 4 A flowchart of an image processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] The present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, identical elements are represented by similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale. In addition, some well-known parts may not be shown in the drawings.

[0033] Many specific details of the present invention are described below, such as device structures, materials, dimensions, processing techniques, and technologies, to provide a clearer understanding of the present invention. However, as will be appreciated by those skilled in the art, the present invention may be practiced without these specific details.

[0034] It should be understood that the connection / coupling of A and B in the embodiment of the present application means that A and B can be connected in series or in parallel, or A and B can be connected through other devices, and the embodiment of the present application is not limited to this.

[0035] The following describes embodiments of the image processing apparatus, method, imaging device, and computer-readable storage medium provided by the present application in conjunction with the accompanying drawings.

[0036] Figure 1 The figure shows a voltage variation curve of a display device and its corresponding light source brightness variation curve. Figure 1 In the figure, curve S1 is the voltage variation curve of the AC power supply, where the horizontal axis is time and the vertical axis is voltage amplitude; curve S2 is the light source brightness variation curve, where the horizontal axis is time and the vertical axis is light source brightness. It should be understood that Figure 1 It is made to clearly show the periodic relationship between voltage and brightness. It does not represent its actual size, period, frequency and other parameters. This application does not limit the voltage size, period, frequency and other parameters of the display device.

[0037] Display devices usually use AC power as their power source, and their voltage variation curve is as follows: Figure 1 As shown in the curve S1 in the figure, the brightness of the light source is positively correlated with the voltage amplitude. The light source brightness change curve is as follows: Figure 1 As shown in the middle curve S2.

[0038] The relationship between the brightness of the light generated by the AC power supply and time can be expressed as:

[0039]

[0040] Where L is the peak brightness and f is the AC power frequency (100 Hz in China). This periodic change in energy is usually imperceptible to the naked eye, but it has a significant impact on the image sensor. When the sensor exposure mode is rolling shutter exposure, assuming that the exposure start time of the i-th row of the n-th frame is t n,i , the corresponding end time is t′ n,i , then:

[0041] Δt=t n,i+1 -t n,i

[0042] expt=t′ n,i -t n,i

[0043] Δt is the exposure start interval between two adjacent rows, expt is the exposure time of the current frame, and the light energy E obtained in the i-th row of the n-th frame can be calculated. n,i for:

[0044]

[0045] Therefore, to obtain the same energy when exposing the i-th row and the i+1-th row, E is required. n,i =E n,i+1 , we can know from calculation that when

[0046] When the energy obtained by each line in the image is the same, no stripes will appear. The reason for the stroboscopic effect is that the energy obtained by each line in the image is different (E n,i ≠E n,i+1 ).

[0047] In addition, in order to prevent the image from flickering, the energy obtained from the i-th row of two adjacent frames must be equal, that is, E n,i =E n+1,i , that is: E n+1,i -E n,i =0

[0048] Consider Δ i =E n+1,i -E n,i , we can simplify it to get:

[0049]

[0050] It can be seen that Δ i It is t n,i The function of the change, It can be regarded as a constant term, and the above formula can be simplified to Δ i =Asin[πf(2t n,i +Δt)], if you want to prevent the image from flickering, you need to make A=0, and this should also satisfy It can be seen that when Δ i ≠0, Δ i The relationship with the number of rows is equivalent to sampling the form f = Asin[πf(2t+Δt)] according to the number of rows. The sampling interval is the time interval between the start and end of the row exposure. The period of this function is

[0051] Based on the above analysis, we can know that, in theory, if there is flicker in the image, the brightness difference between the corresponding rows of adjacent frame images and the row number have a sinusoidal function relationship. By analyzing its period, the frequency of ambient light energy change can be obtained.

[0052] Figure 2 A block diagram of an image processing apparatus according to an embodiment of the present invention is shown; Figure 3 A pixel map of an image according to an embodiment of the present invention is shown.

[0053] like Figure 2 As shown, the image processing apparatus 100 includes an image processing module 110 and a determination module 120 .

[0054] Image processing module 110 is configured to obtain a brightness difference array comprising a plurality of difference values, and determine a detection value based on the brightness difference array after differential processing, wherein each difference value is the difference in total brightness between corresponding rows of pixels in two consecutive image frames, wherein the detection value represents the variation period and / or number of periods of each difference value, and the predetermined range represents the predetermined variation period and / or number of periods obtained by theoretically calculating the difference value when stroboscopic light is present. Determination module 120 is configured to determine whether stroboscopic light is present in two consecutive image frames based on the detection value and the predetermined range, wherein the predetermined range represents the predetermined variation period and / or number of periods obtained by theoretically calculating the difference value when stroboscopic light is present. The image processing device 100 of this embodiment has a simple detection step, facilitating efficient stroboscopic light detection, and the differential processing step can effectively remove noise from the brightness difference array, thereby improving the accuracy of stroboscopic light detection.

[0055] As an example, the image processing module 110 includes a difference calculation unit 111, a preprocessing unit 112, a difference processing unit 113, and a period detection unit 114. It should be understood that in actual hardware / software implementation, the various units in this example may be independent, integrated in pairs, or integrated into multiple units, and this application does not limit the specific implementation methods.

[0056] The difference calculation unit 111 is used to obtain a brightness difference array between two consecutive frames of images. Specifically, the difference calculation unit 111 divides the image statistical area of ​​the two consecutive frames of images into horizontal strips of equal height, each horizontal strip including one or more rows of consecutive pixels; sums the pixel brightness of each horizontal strip in the two consecutive frames of images to obtain a first brightness array and a second brightness array; and performs a subtraction operation on the first brightness array and the second brightness array to obtain a brightness difference array. Optionally, the number of pixel rows contained in the horizontal strips divided by the difference calculation unit 111 is an even number. Optionally, when each pixel contains multiple sub-pixels, the difference calculation unit 111 uses the brightness of any sub-pixel to represent the brightness of the pixel.

[0057] like Figure 3 As shown in the figure, if the size of a frame image is H×W, the image statistical area is divided into horizontal strips of equal height along the vertical direction, and the number of strips is M, which is beneficial to saving hardware costs and improving calculation efficiency. In addition, setting the strip width to an even number can avoid the green balance problem and help improve the accuracy of stroboscopic detection. The height of each horizontal strip is The floor function rounds down. The total brightness of the sub-pixels (e.g., G pixels) of each horizontal strip is then counted separately, which helps further reduce the amount of computation and improve computational efficiency. This step can obtain two one-dimensional arrays (a first brightness array and a second brightness array). Subsequently, a subtraction operation is performed on the first brightness array and the second brightness array to obtain a brightness difference array D. The brightness difference array D includes multiple difference values, each of which is the difference in the total brightness of the corresponding horizontal strips of the two frames of image.

[0058] The pre-processing unit 112 is connected to the difference calculation unit 111 and is used to pre-process the brightness difference array. The pre-processing includes sliding filtering and / or centering. For example, the brightness difference array D is subjected to m times of sliding mean filtering with a size of n to obtain the array D. S , then for array D S Perform centralization to obtain array D S-C Specifically, the sliding mean filter treats the differences in the brightness difference array D as a queue. Each time a new data point is sampled, it is placed at the end of the queue and the data at the head of the queue is discarded. All the data in the queue are averaged to obtain a new filtering result. Centralization is to average the differences in the brightness difference array D, and then subtract each difference from the mean to obtain a new array.

[0059] Experiments show that the raw image brightness data typically contains a lot of noise, and sliding mean filtering can make the data smoother. Furthermore, theoretical analysis shows that the energy acquired by each row of the image during the exposure time has a sinusoidal relationship with the row number. However, the brightness and row number curves drawn based on actual results exhibit a certain vertical offset. A centering operation can shift the curve to facilitate subsequent data processing.

[0060] The difference processing unit 113 is connected to the pre-processing unit 112 and is configured to perform difference processing on the pre-processed brightness difference array. The difference processing may include, for example, sequentially performing first-order difference processing, median filtering, and first-order difference processing on the pre-processed brightness difference array. In other embodiments, second-order difference processing may be performed directly on the pre-processed brightness difference array.

[0061] For example, the difference processing unit 113 performs the calculation on the array D obtained by the pre-processing unit 112. S-CThe array Dr is obtained by sequentially performing a difference calculation (values ​​greater than 0 are marked as 1, values ​​less than 0 are marked as -1), median filtering (scaling summation within the filter window followed by sign determination), and a difference calculation. Specifically, the median filtering process performed by the difference processing unit 113 includes: setting a filter window; performing a sliding summation on each element in the luminance difference array within the filter window; and, based on the sliding summation result, calculating the median of each element in the luminance difference array within the filter window, and assigning the corresponding median value to each element.

[0062] Specifically, for array D s_c Calculate and obtain the brightness difference change trend array D diff ,in D diff [0] = D diff [1], this step is equivalent to the first-order difference operation; then D diff Perform median filtering of size s to obtain D diff_s , D diff The values ​​in are only + / -1. Therefore, the median value in the filter window is the mode, and the sign of the sum of the values ​​in the window is the same as the mode. In particular, the sorting process in the median filter can be converted into a sliding summation, which is convenient for hardware implementation. Finally, D diff_s Perform first-order difference calculation to obtain D r , where D r [i]=D diff_s [i]-D diff_s [i-1], D r Equivalent to D s_c The second-order difference result of , where -2 corresponds to the maximum point and +2 corresponds to the minimum point.

[0063] The period detection unit 114 is connected to the differential processing unit 113 and is used to find each extreme point (including maximum point and minimum point) that meets the preset conditions in the brightness difference array after differential processing, determine the change period of the brightness difference array based on the number of pixel rows between adjacent extreme points, and determine the number of periods contained in the brightness difference array to obtain a detection value.

[0064] Optionally, after obtaining the detection value, the period detection unit 114 or an additional scoring unit (not shown) scores the detection value according to a predetermined range to obtain a score value, wherein the predetermined range represents a predetermined change period and / or a predetermined number of cycles obtained by theoretically calculating the difference when the frequency flash is present. For example, if the change period obtained by theoretically calculating the difference when the frequency flash is present is T and the number of cycles is N, and the detection value obtained by the period detection unit 114 includes a difference change period of 0.9T and a number of cycles of 0.9N, the period detection unit 114 or an additional scoring unit scores on a percentage basis and can obtain a score value of 90. In addition, the period detection unit 114 or an additional scoring unit can perform a weighted calculation on the difference change period and the number of cycles according to actual conditions to obtain a score value, and this application does not impose any restrictions on this.

[0065] The predetermined range may be pre-stored in the image processing module 110 or calculated by the image processing module 110. As an example, the method by which the image processing module 110 determines the predetermined range includes: calculating a predetermined change cycle and a predetermined number of cycles of a change in the brightness difference value of a row of pixels based on the frequency of the light source and parameters of the image acquisition device used to acquire the image; and determining the predetermined range based on the predetermined change cycle and the predetermined number of cycles.

[0066] Specifically, the preset conditions set by the period detection unit 114 include: the value of the corresponding position of the maximum point in the brightness difference value array after differential processing in the brightness difference value array after preprocessing is greater than 0, and the value of the corresponding position of the minimum point in the brightness difference value array after differential processing in the brightness difference value array after preprocessing is less than 0; and / or the absolute values ​​of the maximum point and the minimum point are greater than / equal to a predetermined value; and / or when there is no minimum point before two adjacent maximum points, and / or there is no maximum point between two adjacent minimum points, weighted calculation is used to merge two adjacent maximum points and / or two adjacent minimum points.

[0067] After determining the extreme points that meet the preset conditions, the period detection unit 114 determines the change period of the brightness difference array according to the number of pixel rows between adjacent extreme points, including: determining half of the change period of the brightness difference array according to the number of pixel rows between adjacent maximum points and minimum points; or determining the change period of the brightness difference array according to the number of pixel rows between two adjacent maximum points; or determining the change period of the brightness difference array according to the number of pixel rows between two adjacent minimum points.

[0068] For example, according to the array D S-C Set the predetermined values ​​for the maximum and minimum values ​​in the array and find the array D through the index position corresponding to + / -2 in the array Dr S-CThe values ​​at the corresponding positions in the array are removed, and the extreme points whose absolute values ​​of the maximum (minimum) are less than the predetermined value, the maximum points whose maximum values ​​are less than 0, and the extreme minimum points whose minimum values ​​are greater than 0 are eliminated. The adjacent maximum / minimum points are merged, and the distances between the adjacent maximum and minimum values ​​are subtracted to obtain half of the variation period of the brightness difference array, thereby obtaining the variation period of the brightness difference array and the number of complete periods contained therein.

[0069] Finally, the judgment module 120 determines whether the image exhibits stroboscopic motion based on the detection value and the predetermined range. In other embodiments, the judgment module 120 directly determines whether the image exhibits stroboscopic motion based on the score value provided by the image processing module 110. When the score value exceeds a threshold, it is determined that stroboscopic motion exists between two consecutive image frames. For example, the image processing module 110 scores the detection values ​​of two consecutive image frames on a percentage basis, with a score of 100 indicating that the brightness difference between the row pixels of the two image frames completely conforms to the theoretical condition for stroboscopic motion. The judgment module 120 may set a threshold of 60. Then, when the score value obtained by the image processing module 110 is greater than 60, it is determined that stroboscopic motion exists between the two image frames.

[0070] In addition, the present application also provides an imaging device, which includes an image acquisition device (not shown) and Figure 2 In the image processing device shown, the image acquisition device is used to capture the image frames to be detected, and the image frames to be detected include at least two consecutive frames of images. The image processing device receives and determines whether there is stroboscopic light in the two consecutive frames of images.

[0071] Figure 4 A flowchart of an image processing method according to an embodiment of the present invention is shown. Figure 4 The flowcharts shown are for illustrative purposes only and should not be overly interpreted as limiting the scope of the claims. A person skilled in the art will recognize that many variations, substitutions, and modifications may exist in the various embodiments of the present disclosure. For example, Figure 4 The steps shown may be added, deleted, substituted, rearranged and repeated.

[0072] like Figure 4 As shown, the image processing method according to the embodiment of the present invention includes steps S1-S3.

[0073] In step S1 , a brightness difference value array including a plurality of difference values ​​is obtained, where each difference value is the difference in total brightness between one or more rows of pixels in two consecutive frames of images.

[0074] Optionally, obtaining a brightness difference value array including multiple difference values ​​includes: dividing the image statistical areas of two consecutive frames of images into horizontal strips of equal height, each horizontal strip including one or more consecutive rows of pixels; summing the pixel brightness of each horizontal strip in the two consecutive frames of images to obtain a first brightness array and a second brightness array; and performing a subtraction operation on the first brightness array and the second brightness array to obtain a brightness difference value array.

[0075] Optionally, the horizontal stripe contains an even number of pixel rows.

[0076] Optionally, when each pixel includes multiple sub-pixels, the brightness of any sub-pixel is used to represent the brightness of the pixel.

[0077] For example, if the size of a frame image is H×W, the image statistical area is divided into horizontal strips of equal height along the vertical direction, and the number of strips is M, which is beneficial to saving hardware costs and improving calculation efficiency; in addition, setting the strip width to an even number can avoid the green balance problem and help improve the accuracy of stroboscopic detection; the height of each horizontal strip is The floor function rounds down. The total brightness of the sub-pixels (e.g., G pixels) of each horizontal strip is then counted separately, which helps further reduce the amount of computation and improve computational efficiency. This step can obtain two one-dimensional arrays (a first brightness array and a second brightness array). Subsequently, a subtraction operation is performed on the first brightness array and the second brightness array to obtain a brightness difference array D. The brightness difference array D includes multiple difference values, each of which is the difference in the total brightness of the corresponding horizontal strips of the two frames of image.

[0078] Optionally, the image processing method further includes: preprocessing the brightness difference array, where the preprocessing includes sliding filtering and / or centering.

[0079] For example, the brightness difference array D is subjected to m times of sliding mean filtering with a size of n to obtain the array D S , then for array D S Perform centralization to obtain array D S-C Specifically, the sliding mean filter treats the differences in the brightness difference array D as a queue. Each time a new data point is sampled, it is placed at the end of the queue and the data at the head of the queue is discarded. All the data in the queue are averaged to obtain a new filtering result. Centralization is to average the differences in the brightness difference array D, and then subtract each difference from the mean to obtain a new array.

[0080] Experiments show that the raw image brightness data typically contains a lot of noise, and sliding mean filtering can make the data smoother. Furthermore, theoretical analysis shows that the energy acquired by each row of the image during the exposure time has a sinusoidal relationship with the row number. However, the brightness and row number curves drawn based on actual results exhibit a certain vertical offset. A centering operation can shift the curve to facilitate subsequent data processing.

[0081] In step S2, a detection value is determined based on the array of brightness difference values ​​after difference processing. The detection value represents the period of change and / or the number of periods of each difference value. Difference processing, for example, includes sequentially performing first-order difference processing, median filtering, and first-order difference processing on the pre-processed array of brightness difference values. In other embodiments, second-order difference processing may be performed directly on the pre-processed array of brightness difference values.

[0082] Optionally, performing median filtering on the brightness difference array includes: setting a filtering window; performing sliding summation on each element in the brightness difference array within the filtering window; and finding the median of each element in the brightness difference array within the filtering window based on the result of the sliding summation, and assigning the element to the corresponding median.

[0083] For example, for the array D S-C The array Dr is obtained by sequentially performing a difference calculation (values ​​greater than 0 are marked as 1, values ​​less than 0 are marked as -1), median filtering (scaling summation within the filter window followed by sign determination), and a difference calculation. Specifically, the median filtering process performed by the difference processing unit 113 includes: setting a filter window; performing a sliding summation on each element in the luminance difference array within the filter window; and, based on the sliding summation result, calculating the median of each element in the luminance difference array within the filter window, and assigning the corresponding median value to each element.

[0084] Specifically, for array D S-C Calculate and obtain the brightness difference change trend array D diff ,in D diff [0] = D diff [1], this step is equivalent to the first-order difference operation; then D diff Perform median filtering of size s to obtain D diff_s , D diff The values ​​in are only + / -1. Therefore, the median value in the filter window is the mode, and the sign of the sum of the values ​​in the window is the same as the mode. In particular, the sorting process in the median filter can be converted into a sliding summation, which is convenient for hardware implementation. Finally, D diff_s Perform first-order difference calculation to obtain D r , where D r [i]=D diff_s [i]-D diff_s[i-1], D r Equivalent to D s_c The second-order difference result of , where -2 corresponds to the maximum point and +2 corresponds to the minimum point.

[0085] Optionally, determining the detection value includes: searching for extreme points that meet preset conditions in the brightness difference array after differential processing, determining a change period of the brightness difference array based on the number of pixel rows between adjacent extreme points, and determining the number of periods contained in the brightness difference array.

[0086] Optionally, the preset conditions include: the value of the maximum point in the brightness difference array after differential processing at the corresponding position in the brightness difference array after preprocessing is greater than 0, and the value of the minimum point in the brightness difference array after differential processing at the corresponding position in the brightness difference array after preprocessing is less than 0; and / or the absolute values ​​of the maximum point and the minimum point are greater than / equal to a predetermined value; and / or when there is no minimum point before two adjacent maximum points, and / or there is no maximum point between two adjacent minimum points, weighted calculation is used to merge two adjacent maximum points and / or two adjacent minimum points.

[0087] Optionally, determining the change period of the brightness difference array based on the number of pixel rows of the spacing between adjacent extreme points includes: determining half of the change period of the brightness difference array based on the number of pixel rows of the spacing between adjacent maximum points and minimum points; or determining the change period of the brightness difference array based on the number of pixel rows of the spacing between two adjacent maximum points; or determining the change period of the brightness difference array based on the number of pixel rows of the spacing between two adjacent minimum points.

[0088] For example, according to the array D S-C Set the predetermined values ​​for the maximum and minimum values ​​in the array and find the array D through the index position corresponding to + / -2 in the array Dr S-C The values ​​at the corresponding positions in the array are removed, and the extreme points whose absolute values ​​of the maximum (minimum) are less than the predetermined value, the maximum points whose maximum values ​​are less than 0, and the extreme minimum points whose minimum values ​​are greater than 0 are eliminated. The adjacent maximum / minimum points are merged, and the distances between the adjacent maximum and minimum values ​​are subtracted to obtain half of the variation period of the brightness difference array, thereby obtaining the variation period of the brightness difference array and the number of complete periods contained therein.

[0089] In step S3, whether there is stroboscopic motion between two consecutive frames of images is determined based on the detection value and a predetermined range, wherein the predetermined range represents a predetermined change period and / or a predetermined number of periods obtained by theoretically calculating the difference when stroboscopic motion exists.

[0090] Optionally, in this step, first, the detection value is scored according to a predetermined range to obtain a score value, and the predetermined range represents a predetermined change period and / or a predetermined number of periods obtained by theoretically calculating the difference when the flash memory is present.

[0091] The predetermined range represents a predetermined change period and / or predetermined number of cycles obtained by theoretically calculating the difference value when the flash memory is present. For example, if the change period obtained by theoretically calculating the difference value when the flash memory is present is T and the number of cycles is N, and the detection value obtained in step S2 includes a change period of 0.9T for the difference value and a number of 0.9N for the cycle value, if the score is given on a percentage basis, the score value would be 90. In addition, the change period and the number of cycles of the difference value can be weighted to obtain a score value based on actual circumstances, and this application does not impose any restrictions on this.

[0092] Optionally, the method for determining the predetermined range includes: calculating a predetermined change period and a predetermined number of periods of change in row pixel brightness difference based on the light source frequency and parameters of the image acquisition method used to acquire the image; and determining the predetermined range based on the predetermined change period and the predetermined number of periods.

[0093] Optionally, when the score exceeds a threshold, it is determined that two consecutive image frames have flicker. For example, the detection values ​​of two consecutive image frames are scored on a percentage basis, with a score of 100 indicating that the difference in row pixel brightness between the two frames completely meets the theoretical condition for flicker. A threshold value of 60 can be set, and then when the score is greater than 60, it can be determined that flicker has occurred in the two frames.

[0094] In addition, the present application also provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the following Figure 4 The image processing method shown.

[0095] In summary, the present application provides an image processing device, an image processing method, an imaging device, and a computer-readable storage medium, which determine a detection value based on a brightness difference array after differential processing, and use the detection value to determine whether flicker exists. The detection steps are simple, which facilitates efficient flicker detection, and the differential processing step can effectively remove noise in the brightness difference array, thereby improving the accuracy of flicker detection.

[0096] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0097] While embodiments of the present invention have been described above, these embodiments do not exhaustively describe all details and do not limit the invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the above description. These embodiments are selected and described in detail in this specification in order to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better utilize the present invention and its modifications. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An image processing device, comprising: an image processing module, configured to obtain a brightness difference value array comprising a plurality of difference values, and determine a detection value based on the brightness difference value array after difference processing, wherein each difference value is a difference in total brightness between corresponding rows or rows of pixels in two consecutive frames of images, and the detection value represents a variation period and / or number of periods of each difference value; and a judgment module, configured to determine whether stroboscopic movement exists in the two consecutive frames of image based on the detection value and a predetermined range, wherein the predetermined range represents a predetermined change cycle and / or a predetermined number of cycles obtained by theoretically calculating the difference value when stroboscopic movement exists; Wherein, the image processing module includes: a difference processing unit, configured to sequentially perform first-order difference processing, median filtering processing, and first-order difference processing on the brightness difference value array; and a period detection unit for determining the detection value, wherein the period detection unit searches for each extreme point that meets a preset condition in the brightness difference value array after differential processing, determines a change period of the brightness difference value array based on the number of pixel rows between adjacent extreme points, and determines the number of periods contained in the brightness difference value array; The preset conditions include: the value of the corresponding position of the maximum point in the brightness difference value array after the difference processing in the brightness difference value array after the preprocessing is greater than 0, and the value of the corresponding position of the minimum point in the brightness difference value array after the difference processing in the brightness difference value array after the preprocessing is less than 0; and / or The absolute values ​​of the maximum point and the minimum point are greater than / equal to a predetermined value; and / or When there is no minimum point before two adjacent maximum points, and / or there is no maximum point between two adjacent minimum points, weighted calculation is used to merge the two adjacent maximum points and / or the two adjacent minimum points.

2. The image processing apparatus according to claim 1, wherein: The difference processing unit performing median filtering on the brightness difference array includes: Set the filter window; Performing a sliding summation on each element in the brightness difference array within the filtering window; and Based on the result of the sliding summation, the median of each element in the brightness difference array is calculated within the filtering window, and the element is assigned a corresponding median value.

3. The image processing apparatus according to claim 1, wherein: The image processing module includes a preprocessing unit for preprocessing the brightness difference array, wherein the preprocessing includes sliding filtering and / or centering processing.

4. The image processing apparatus according to claim 1, wherein: The period detection unit, which determines the change period of the brightness difference array according to the number of pixel rows of the interval between adjacent extreme value points, includes: Determine half of the variation period of the brightness difference array according to the number of pixel rows between adjacent maximum value points and minimum value points; or Determine the variation period of the brightness difference array according to the number of pixel rows between two adjacent maximum value points; or The variation period of the brightness difference array is determined according to the number of pixel rows between two adjacent minimum value points.

5. The image processing apparatus according to claim 1, wherein: The image processing module includes a difference calculation unit, which divides the image statistical areas of the two consecutive frames of images into horizontal strips of equal height, each of which includes one or more consecutive rows of pixels; sums the pixel brightness of each horizontal strip in the two consecutive frames of images to obtain a first brightness array and a second brightness array; and performs a subtraction operation on the first brightness array and the second brightness array to obtain the brightness difference array. The image processing apparatus according to claim 5 , wherein: The number of pixel rows included in the horizontal strips divided by the difference calculation unit is an even number.

7. The image processing apparatus according to claim 5, wherein: When each of the pixels includes multiple sub-pixels, the difference calculation unit uses the brightness of any sub-pixel to represent the brightness of the pixel.

8. The image processing apparatus according to claim 1, wherein: The image processing module, wherein the method for determining the predetermined range includes: Calculating the predetermined change period and the predetermined number of periods of the change in the brightness difference of the row pixels according to the frequency of the light source and the parameters of the image acquisition device used to acquire the image; as well as The predetermined range is determined according to the predetermined change cycle and the predetermined number of cycles.

9. The image processing apparatus according to claim 1, wherein: The judgment module is used to score the detection value according to the predetermined range to obtain a score value, and when the score value exceeds a threshold, it is determined that stroboscopic light exists in the two consecutive frames of images.

10. An image processing method, comprising: Obtaining a brightness difference value array comprising a plurality of difference values, and determining a detection value based on the brightness difference value array after difference processing, wherein each difference value is a difference between the total brightness of one or more rows of pixels in two consecutive frames of images, and the detection value represents a variation period and / or number of periods of each difference value; and Determining whether there is stroboscopic motion between the two consecutive frames of image based on the detection value and a predetermined range, wherein the predetermined range represents a predetermined change cycle and / or a predetermined number of cycles obtained by theoretically calculating the difference when stroboscopic motion exists; Wherein, determining the detection value based on the brightness difference value array after the difference processing includes: performing first-order difference processing, median filtering processing, and first-order difference processing on the brightness difference array in sequence; and Finding each extreme point in the brightness difference array after the difference processing that meets the preset conditions, determining the change period of the brightness difference array according to the number of pixel rows between adjacent extreme points, and determining the number of periods contained in the brightness difference array, The preset conditions include: the value of the corresponding position of the maximum point in the brightness difference value array after the difference processing in the brightness difference value array after the preprocessing is greater than 0, and the value of the corresponding position of the minimum point in the brightness difference value array after the difference processing in the brightness difference value array after the preprocessing is less than 0; and / or The absolute values ​​of the maximum point and the minimum point are greater than / equal to a predetermined value; and / or When there is no minimum point before two adjacent maximum points, and / or there is no maximum point between two adjacent minimum points, weighted calculation is used to merge the two adjacent maximum points and / or the two adjacent minimum points.

11. An imaging device comprising: An image acquisition device is used to capture image frames to be detected, wherein the image frames to be detected include at least two consecutive frames of images; as well as The image processing device according to any one of claims 1 to 9, configured to receive and determine whether stroboscopic light is present in the two consecutive frames of image. 12 . A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the image processing method according to claim 10 when executed by a processor.

Images