Image sensor trigger signal generation method and image correction method for eliminating flicker phenomenon

Abstract

The invention discloses an image sensor trigger signal generation method for eliminating a flicker phenomenon. The method comprises the following steps: acquiring working data information of a target image sensor; calculating and configuring the output frame length of the target image sensor; and generating a trigger signal which enables the initial phase difference of two adjacent frames of the target image sensor to be a set value when the two adjacent frames are exposed so as to complete the generation of the image sensor trigger signal for eliminating the flicker phenomenon. The invention further discloses an image correction method comprising the image sensor trigger signal generation method capable of eliminating the flicker phenomenon. According to the invention, by controlling the generation of the trigger signal of the image sensor, the exposure time of two adjacent frames of images is controlled, and finally the bright and dark fringes of the image are controlled at the required position, so that better effect and more convenient image fusion correction are realized; therefore, the image sensor trigger signal generation method and the image correction can be realized, the reliability is higher, and the accuracy is better.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to an image sensor trigger signal generation method and an image correction method for eliminating flicker phenomenon. Background Art

[0002] The rolling shutter mode is a line-by-line exposure method, which is relatively simple to implement and has low cost. Therefore, the rolling shutter mode has become one of the most commonly used exposure methods of image sensors.

[0003] However, when using the rolling shutter mode for exposure, flicker phenomenon is very likely to occur, such as Figure 1 The light and dark stripes phenomenon shown in the figure. The fundamental reason for the flicker phenomenon is that the power frequency is a periodic sine wave, and the light emitted by the light source driven by this industrial frequency power supply will produce an alternating change with a frequency twice the sine wave frequency. Since the exposure method of the image sensor using the rolling shutter mode is performed by row, when the exposure time is not an integer multiple of the light source's stroboscopic period, it will result in the same frame of data. Although the exposure time is the same for each row, the intensity of the light source during the exposure period is different, which is ultimately reflected in the captured image as alternating light and dark stripes. Specifically, Figure 2 As shown: In rolling shutter mode, when each line is exposed, the brightness of the corresponding light source will change over time. Figure 2 In the frame n, row 0 (R0) starts at the darkest light source, while row 200 (R200) ​​starts at the brightest light source. In this way, row 0 is the darkest row and row 200 is the brightest row. Since the rolling shutter has different exposure timings for each row, spanning one or more light source alternation cycles, the final result reflected in the image is the phenomenon of bright and dark stripes.

[0004] At present, the traditional solution to eliminate the bright and dark stripes phenomenon is to set the exposure parameters at an integer multiple of the stroboscopic period. However, the restrictions on exposure imposed by this solution severely restrict the application scenarios and scope of image sensors. In addition, although there are solutions to correct the bright and dark stripes through image processing algorithms, the effect of this type of correction solution is not ideal because the amount of information at the brightest and darkest parts of the stripes is determined when the image is exposed. Summary of the invention

[0005] One of the purposes of the present invention is to provide a method for generating a trigger signal of an image sensor with high reliability and good accuracy for eliminating the flicker phenomenon.

[0006] A second objective of the present invention is to provide an image correction method including the image sensor trigger signal generation method for eliminating the flicker phenomenon.

[0007] The method for generating a trigger signal of an image sensor for eliminating the flicker phenomenon provided by the present invention comprises the following steps:

[0008] S1. Obtaining the working data information of the target image sensor;

[0009] S2. Calculate and configure the output frame length of the target image sensor according to the data information obtained in step S1;

[0010] S3. According to the data information obtained in step S2, based on the frame rate of the target image sensor, a trigger signal is generated so that the starting phase difference between two adjacent frames of the target image sensor when exposed is equal to a set value, so as to complete the generation of the image sensor trigger signal for eliminating the flicker phenomenon.

[0011] The step S1 of obtaining the working data information of the target image sensor specifically includes the following steps:

[0012] Obtain working data information of the target image sensor;

[0013] The working data information includes the frame rate and synchronization pulse period of the target image sensor.

[0014] The step S2 of calculating and configuring the output frame length of the target image sensor according to the data information obtained in step S1 specifically includes the following steps:

[0015] The output frame length of the target image sensor is calculated using the following formula:

[0016]

[0017] Where T zc is the output frame length of the target image sensor; z l is the acquisition frame rate of the target image sensor; T tm is the synchronization pulse period;

[0018] The target image sensor is configured according to the calculated output frame length of the target image sensor.

[0019] The step S3 of generating a trigger signal that makes the initial phase difference between two adjacent frames of exposure of the target image sensor equal to a set value comprises the following steps:

[0020] The starting phase difference between two adjacent frames of exposure of the target image sensor is generated to be An odd multiple of the trigger signal.

[0021] The step S3 of generating a trigger signal that makes the initial phase difference between two adjacent frames of exposure of the target image sensor equal to a set value specifically includes the following steps:

[0022] The starting phase difference between two adjacent frames of exposure of the target image sensor is generated to be The trigger signal;

[0023] When a trigger signal is generated, the frame rate remains unchanged or changes.

[0024] The method of keeping the frame rate unchanged specifically includes the following steps:

[0025] The trigger signal delay is calculated using the following formula:

[0026]

[0027] Where T trg is the trigger signal delay; n is the trigger signal count, indicating the nth trigger signal; % is the remainder operation;

[0028] At this time, the synchronization signal period is calculated using the following formula:

[0029]

[0030] Where T tb is the synchronization signal period; z l is the acquisition frame rate of the target image sensor.

[0031] The changing of the frame rate specifically comprises the following steps:

[0032] The trigger signal delay is calculated using the following formula:

[0033]

[0034] Where T trg is the trigger signal delay; n is the trigger signal count, indicating the nth trigger signal; % is the remainder operation;

[0035] At this time, the synchronization signal period is calculated using the following formula:

[0036]

[0037] Where T tb is the synchronization signal period; z l is the acquisition frame rate of the target image sensor.

[0038] The present invention also provides an image correction method including the image sensor trigger signal generation method for eliminating the flicker phenomenon, and further includes the following steps:

[0039] S4. Using the trigger signal generated in step S3, triggering the target image sensor, and acquiring the corresponding image;

[0040] S5. Perform image fusion on the images obtained in step S4 to obtain a final fused image, thereby completing image correction.

[0041] Step S5, in which the images obtained in step S4 are fused to obtain a final fused image, specifically comprises the following steps:

[0042] A. For the two images obtained in step S4, select n columns of data on the image, take the average of the corresponding position of each column of data, and obtain the brightness distribution array F1{v0,v1,...,v i ,...,v m} and the brightness distribution array F2{v0,v1,...,v i ,...,v m}; Among them, F1{v i} is the average brightness at height i in the first image, F2{v i} is the average brightness value at the height m in the second image; n is 1 to N, N is the width of the image; m is the height of the image

[0043] B. Select the maximum brightness value F1 of the first image from the brightness distribution array of the first image max , select the maximum brightness value F2 of the second image in the brightness distribution array of the second image max ;

[0044] C. Calculate the first coefficient array C1{c0,c1,...,c i ,...,c m} and the second coefficient array C2{c0,c1,...,c i ,...,c m}; where C1{c i} is the i-th coefficient in the first coefficient array, and C2{c i} is the i-th coefficient in the second coefficient array, and

[0045] D. Compare the element values ​​at corresponding positions of the brightness distribution array of the first image and the brightness distribution array of the second image:

[0046] If F1{v i}>F2{v i}, then set K1{k i}=C1{c i} and K2{k i} = 0;

[0047] If F1{v i}≤F2{v i}, then set K1{k i}=0 and K2{k i}=C2{c i};

[0048] where K1{k0,k1,...,k i ,...,k m} is the first image weighted array, K2{k0,k1,...,k i ,...,k m} is the second image weighted array;

[0049] E. Use the following formula to calculate the fused image:

[0050] P x,y =K1{k y}*Q x,y +K2{k y}*R x,y

[0051] Where P x,y is the pixel value of the fused image at position (x, y); Q x,y is the pixel value of the first image at position (x, y); R x,y is the pixel value of the second image at position (x, y).

[0052] The image sensor trigger signal generation method and image correction method for eliminating the flicker phenomenon provided by the present invention control the exposure time of two adjacent frames of images by controlling the generation of the image sensor trigger signal, and finally control the light and dark stripes of the image at the required position to achieve better and more convenient image fusion correction; therefore, the present invention can not only realize the image sensor trigger signal generation method and image correction, but also has higher reliability and better accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is an existing image with bright and dark stripes.

[0054] Figure 2 A schematic diagram of the cause of the existing flicker phenomenon.

[0055] Figure 3 The figure is a flow chart of the trigger signal generating method of the present invention.

[0056] Figure 4Schematic diagram of a trigger signal for maintaining a constant frame rate in the trigger signal generating method of the present invention.

[0057] Figure 5 Schematic diagram of a trigger signal for frame rate change in the trigger signal generating method of the present invention.

[0058] Figure 6 It is a schematic diagram of an image obtained by the trigger signal generating method of the present invention.

[0059] Figure 7 The figure is a schematic diagram of the method flow of the image correction method of the present invention. DETAILED DESCRIPTION

[0060] like Figure 3 The method flow chart of the trigger signal generating method of the present invention is shown as follows: The image sensor trigger signal generating method for eliminating the flicker phenomenon disclosed in the present invention comprises the following steps:

[0061] S1. Obtaining the working data information of the target image sensor; specifically comprising the following steps:

[0062] Obtain working data information of the target image sensor;

[0063] The working data information includes the frame rate and synchronization pulse period of the target image sensor;

[0064] S2. Calculate and configure the output frame length of the target image sensor according to the data information obtained in step S1; specifically, the steps include:

[0065] The output frame length of the target image sensor is calculated using the following formula:

[0066]

[0067] Where T zc is the output frame length of the target image sensor; z l is the acquisition frame rate of the target image sensor; T tm is the synchronization pulse period;

[0068] According to the calculated output frame length of the target image sensor, the target image sensor is configured;

[0069] Taking 50Hz mains as an example, the synchronous pulse is 100Hz, and the acquisition frame rate is 25FPS, then the output frame length is configured as

[0070] S3. Based on the data information obtained in step S2 and the frame rate of the target image sensor, a trigger signal is generated so that the starting phase difference between two adjacent frames of the target image sensor when exposed is set to a set value, so as to complete the generation of the image sensor trigger signal for eliminating the flicker phenomenon; comprising the following steps:

[0071] The starting phase difference between two adjacent frames of exposure of the target image sensor is generated to be An odd multiple of the trigger signal.

[0072] In specific implementation, the preferred solution is to generate a starting phase difference between two adjacent frames of exposure of the target image sensor of The trigger signal;

[0073] When generating a trigger signal, keep the frame rate unchanged or change the frame rate:

[0074] Among them, keeping the frame rate unchanged specifically includes the following steps:

[0075] like Figure 4 As shown, the trigger signal delay is calculated using the following formula:

[0076]

[0077] Where T trg is the trigger signal delay; n is the trigger signal count, indicating the nth trigger signal; % is the remainder operation;

[0078] At this time, the synchronization signal period is calculated using the following formula:

[0079]

[0080] Where T tb is the synchronization signal period; z l is the acquisition frame rate of the target image sensor;

[0081] In this case, taking 50Hz mains, 100Hz synchronous pulse, and 25FPS acquisition frame rate as an example, the trigger timing is: odd frame, Even-numbered frames: This means: for odd frames, the sync signal is triggered after a delay of 5 milliseconds; for even frames, the sync signal is triggered immediately after it arrives. At this time, the sync signal period The sensor trigger signal period changes alternately between 35ms and 45ms, and the sensor output frame rate remains unchanged at 25fps;

[0082] Among them, changing the frame rate specifically includes the following steps:

[0083] like Figure 5As shown, the trigger signal delay is calculated using the following formula:

[0084]

[0085] Where T trg is the trigger signal delay; n is the trigger signal count, indicating the nth trigger signal; % is the remainder operation;

[0086] At this time, the synchronization signal period is calculated using the following formula:

[0087]

[0088] Where T tb is the synchronization signal period; z l is the acquisition frame rate of the target image sensor.

[0089] In this case, taking 50Hz mains, 100Hz synchronous pulse, and 25FPS acquisition frame rate as an example, the trigger timing is: odd frame, Synchronous signal period Even-numbered frames: Synchronous signal period This means: for odd frames, the sync signal is delayed for 5 milliseconds before triggering; for even frames, the sync signal is triggered immediately after it arrives. At this time, the sensor trigger signal period is fixed at 35ms, and the sensor output frame rate changes from 25fps to 1000 / 35≈28.6fps.

[0090] Regarding the above-mentioned cases of keeping the frame rate unchanged and changing the frame rate, the following are explained:

[0091] Keep the frame rate unchanged, suitable for

[0092] (1) The sensor output frame rate is fixed and cannot be changed;

[0093] (2) The backend processing capacity is limited and cannot process more data;

[0094] (3) Strict requirements on frame rate, such as the need for strict synchronization with other devices such as display;

[0095] Changing the frame rate is suitable for

[0096] (2) There is no strict requirement for frame rate and no need to synchronize with other video devices;

[0097] (2) The device processing power can support more data;

[0098] (3) Strict requirements on the uniformity of exposure timing, such as trajectory measurement;

[0099] The differences and similarities between the two are: the main difference between the two is that the sensor trigger signal cycle is different, the exposure start point uniformity is different, the frame rate is different, and the data processing capability requirements are different; the common point is that both use the method of exposing the half power synchronization pulse phase difference to obtain an image with the flicker phenomenon eliminated through data fusion. Since the bright and dark stripes of the two frames before and after are just in opposite positions after using this method, the performance requirements for the fusion algorithm can be effectively reduced;

[0100] Finally, according to the generated trigger signal, the image sensor is triggered to obtain the following Figure 6 The image shown; Figure 6 In the image, the light and dark stripes of two adjacent frames are just at orthogonal positions. By fusing the previous and next two frames, the stripes can be easily eliminated, thereby optimizing the image quality.

[0101] like Figure 7 The figure shows a method flow chart of the image correction method of the present invention; the image correction method disclosed in the present invention, including the method for generating a trigger signal of an image sensor for eliminating the flicker phenomenon, comprises the following steps:

[0102] S1. Obtaining the working data information of the target image sensor;

[0103] S2. Calculate and configure the output frame length of the target image sensor according to the data information obtained in step S1;

[0104] S3. According to the data information obtained in step S2, based on the frame rate of the target image sensor, a trigger signal is generated so that the starting phase difference between two adjacent frames of the target image sensor when exposed is set to a trigger signal to complete the generation of the image sensor trigger signal for eliminating the flicker phenomenon;

[0105] S4. Using the trigger signal generated in step S3, triggering the target image sensor, and acquiring the corresponding image;

[0106] S5. The image obtained in step S4 is subjected to image fusion to obtain a final fused image, thereby completing image correction. In specific implementation, a pre-calibration method may be used to obtain the light and dark variation rule of two adjacent frames, and then the fusion process is performed according to the calibration result in actual use. Specifically, the process comprises the following steps:

[0107] A. For the two images obtained in step S4 (i.e., images of two adjacent frames), select n columns of data on the image, take the average of the corresponding position of each column of data, and obtain the brightness distribution array F1{v0,v1,...,v i ,...,v m} and the brightness distribution array F2{v0,v1,...,vi ,...,v m}; Among them, F1{v i} is the average brightness at height i in the first image, F2{v i} is the average brightness value at the height m in the second image; n is 1 to N, N is the width of the image; m is the height of the image

[0108] B. Select the maximum brightness value F1 of the first image from the brightness distribution array of the first image max , select the maximum brightness value F2 of the second image in the brightness distribution array of the second image max ;

[0109] C. Calculate the first coefficient array C1{c0,c1,...,c i ,...,c m} and the second coefficient array C2{c0,c1,...,c i ,...,c m}; where C1{c i} is the i-th coefficient in the first coefficient array, and C2{c i} is the i-th coefficient in the second coefficient array, and

[0110] D. Compare the element values ​​at corresponding positions of the brightness distribution array of the first image and the brightness distribution array of the second image:

[0111] If F1{v i}>F2{v i}, then set K1{k i}=C1{c i} and K2{k i} = 0;

[0112] If F1{v i}≤F2{v i}, then set K1{k i}=0 and K2{k i}=C2{c i};

[0113] where K1{k0,k1,...,k i ,...,k m} is the first image weighted array, K2{k0,k1,...,k i ,...,k m} is the second image weighted array;

[0114] E. Use the following formula to calculate the fused image:

[0115] P x,y =K1{k y}*Q x,y +K2{k y}*R x,y

[0116] Where P x,y is the pixel value of the fused image at position (x, y); Q x,y is the pixel value of the first image at position (x, y); R x,y is the pixel value of the second image at position (x, y).

[0117] The image sensor trigger signal generation method and image correction method for eliminating the flicker phenomenon provided by the present invention can be applied to occasions with alternating lighting interference, and are particularly suitable for indoor stable power frequency lighting equipment applications.

Claims

1. A method for generating a trigger signal of an image sensor for eliminating flicker phenomenon, comprising the following steps: S1. Obtaining the working data information of the target image sensor; S2. Calculate and configure the output frame length of the target image sensor according to the data information obtained in step S1; S3. According to the data information obtained in step S2, based on the frame rate of the target image sensor, a trigger signal is generated so that the starting phase difference between two adjacent frames of the target image sensor when exposed is equal to a set value, so as to complete the generation of the image sensor trigger signal for eliminating the flicker phenomenon.

2. The method for generating a trigger signal for an image sensor for eliminating flicker phenomenon according to claim 1, characterized in that The step S1 of obtaining the working data information of the target image sensor specifically includes the following steps: Obtain working data information of the target image sensor; The working data information includes the frame rate and synchronization pulse period of the target image sensor.

3. The method for generating a trigger signal for an image sensor for eliminating flicker phenomenon according to claim 2, characterized in that The step S2 of calculating and configuring the output frame length of the target image sensor according to the data information obtained in step S1 specifically includes the following steps: The output frame length of the target image sensor is calculated using the following formula: Where T zc is the output frame length of the target image sensor; z l is the acquisition frame rate of the target image sensor; T tm is the synchronization pulse period; The target image sensor is configured according to the calculated output frame length of the target image sensor.

4. The method for generating a trigger signal for an image sensor for eliminating flicker phenomenon according to claim 3, characterized in that The step S3 of generating a trigger signal that makes the initial phase difference between two adjacent frames of exposure of the target image sensor equal to a set value comprises the following steps: The starting phase difference between two adjacent frames of exposure of the target image sensor is generated to be An odd multiple of the trigger signal.

5. The method for generating a trigger signal for an image sensor for eliminating flicker phenomenon according to claim 3, characterized in that The step S3 of generating a trigger signal that makes the initial phase difference between two adjacent frames of exposure of the target image sensor equal to a set value specifically includes the following steps: The starting phase difference between two adjacent frames of exposure of the target image sensor is generated to be The trigger signal; When a trigger signal is generated, the frame rate remains unchanged or changes.

6. The method for generating a trigger signal for an image sensor for eliminating flicker phenomenon according to claim 5, characterized in that The method of keeping the frame rate unchanged specifically includes the following steps: The trigger signal delay is calculated using the following formula: Where T trg is the trigger signal delay; n is the trigger signal count, indicating the nth trigger signal; % is the remainder operation; At this time, the synchronization signal period is calculated using the following formula: Where T tb is the synchronization signal period; z l is the acquisition frame rate of the target image sensor.

7. The method for generating a trigger signal for an image sensor for eliminating flicker phenomenon according to claim 5, characterized in that The changing of the frame rate specifically comprises the following steps: The trigger signal delay is calculated using the following formula: Where T trg is the trigger signal delay; n is the trigger signal count, indicating the nth trigger signal; % is the remainder operation; At this time, the synchronization signal period is calculated using the following formula: Where T tb is the synchronization signal period; z l is the acquisition frame rate of the target image sensor.

8. An image correction method including the image sensor trigger signal generation method for eliminating flicker phenomenon, characterized in that The following steps are also included: S4. Using the trigger signal generated in step S3, triggering the target image sensor, and acquiring the corresponding image; S5. Perform image fusion on the images obtained in step S4 to obtain a final fused image, thereby completing image correction.

9. The image correction method according to step S8, characterized in that Step S5, in which the images obtained in step S4 are fused to obtain a final fused image, specifically comprises the following steps: A. For the two images obtained in step S4, select n columns of data on the image, take the average of the corresponding position of each column of data, and obtain the brightness distribution array F1{v0,v1,...,v i ,...,v m } and the brightness distribution array F2{v0,v1,...,v i ,...,v m }; Among them, F1{v i } is the average brightness at height i in the first image, F2{v i } is the average brightness value at the height m in the second image; n is 1 to N, N is the width of the image; m is the height of the image B. Select the maximum brightness value F1 of the first image from the brightness distribution array of the first image max , select the maximum brightness value F2 of the second image in the brightness distribution array of the second image max ; C. Calculate the first coefficient array C1{c0,c1,...,c i ,...,c m } and the second coefficient array C2{c0,c1,...,c i ,...,c m }; where C1{c i } is the i-th coefficient in the first coefficient array, and C2{c i } is the i-th coefficient in the second coefficient array, and D. Compare the element values ​​at corresponding positions of the brightness distribution array of the first image and the brightness distribution array of the second image: If F1{v i }>F2{v i }, then set K1{k i }=C1{c i } and K2{k i } = 0; If F1{v i }≤F2{v i }, then set K1{k i }=0 and K2{k i }=C2{c i }; where K1{k0,k1,...,k i ,...,k m } is the first image weighted array, K2{k0,k1,...,k i ,...,k m } is the second image weighted array; E. Use the following formula to calculate the fused image: P x,y =K1{k y }*Q x,y +K2{k y }*R x,y Where P x,y is the pixel value of the fused image at position (x, y); Q x,y is the pixel value of the first image at position (x, y); R x,y is the pixel value of the second image at position (x, y).

Images