Anti-overexposure anti-trailing imaging method based on dynamic integral time and multi-frame fusion

Abstract

The invention provides a self-adaptive multi-frame accumulation detection imaging method used in a dynamic background irradiation environment, belongs to the technical field of space target imaging, and aims to solve the problems of low imaging quality, image information loss and a dynamic background of trailing in the dynamic background irradiation environment. Setting initial integral time to shoot the to-be-detected area, and setting panoramic integral time according to a shooting result to shoot again; 2, dividing the re-shot area into a plurality of rectangular areas; step 3, setting different sampling time for each divided area for sampling; step 4, performing multi-frame accumulation on the sampled image until a new faint target is identified; and 5, for a new faint target, repeating the steps 2-4 to perform sampling and multi-frame accumulation until the sampling time reaches the specified total integral time, and outputting a detection image.

Description

TECHNICAL FIELD

[0001] The application relates to an anti-overexposure and anti-tail imaging method based on dynamic integration time and multi-frame fusion, and belongs to the technical field of space target imaging. BACKGROUND

[0002] In the field of space target imaging, with the rapid development of space technology and the increasing demand for space environment monitoring, higher requirements are put forward for high-precision imaging technology of space targets. In order to accurately identify and track targets, high-quality image acquisition needs to be realized under complex lighting conditions and changes in target brightness and position. At present, space target imaging technology is widely used in fields such as space reconnaissance, space target monitoring and astronomical observation, and has important significance for ensuring space safety and scientific research.

[0003] In the process of space target imaging, the identification of dark and weak targets has always been a technical problem. Because the brightness and position of space targets may change over time, and the background noise is large, traditional imaging technology faces many challenges in processing such targets. On the one hand, if the integration time is set too large, the brighter part of the image is prone to overexposure, resulting in information loss; on the other hand, if the integration time is too low, the brightness of the dark and weak target is insufficient and it is difficult to be effectively identified. In order to solve this problem, many detectors use multi-frame accumulation technology to improve the signal-to-noise ratio of the image through multiple sampling, and at the same time, multi-frame accumulation can effectively handle the tailing caused by too long integration time. However, although the multi-frame accumulation technology can improve the identification ability of dark and weak targets to some extent, frequent sampling of the image will lead to a decrease in image quality and loss of part of the image information, and will also increase the burden on the detector, so many detectors use adaptive integration time technology. However, for a background with varying radiation intensity over time and space, and a dynamic background that may produce tailing, how to balance multi-frame accumulation and adaptive integration time and develop a reasonable imaging strategy is still a complex problem. SUMMARY

[0004] The application is to solve the problem of low imaging quality in a dynamic background irradiation environment, image information loss and tailing of a dynamic background, and further proposes an anti-overexposure and anti-tail imaging method based on dynamic integration time and multi-frame fusion.

[0005] The technical scheme adopted by the application to solve the above problems is as follows: Step 1: Obtain the full-well charge number of the pixel, set the initial integration time to take a picture of the region to be measured, and set the panoramic integration time according to the shooting result to take a picture again; Step 2: divide the region taken a picture again into p rectangular regions; Step 3: Sampling with different sampling times for each divided region; Step 4: Multi-frame accumulation of the sampled images until a new dim target is identified; Step 5: For the new dim target, repeat steps 2-4 for sampling and multi-frame accumulation until the sampling time reaches the specified total integration time, and output the detection image.

[0006] Further, step 1 specifically includes: Obtaining the full well charge number of the pixel n Setting the initial integration time to take a picture of the region to be measured; If the detector needs to obtain a non-dynamic image, obtaining the charge number of the brightest pixel of the image based on the shooting result n 0, setting a threshold coefficient according to the gray value of the brightest pixel, and setting the panoramic integration time again based on the threshold coefficient and the initial integration time t 0 to take a picture again, and eliminating identifiable target pixels in the picture taken again; If the detector needs to obtain a dynamic image, obtaining the number of center moving pixels of the bright star in two frames of images based on the shooting result n 1, setting a threshold coefficient according to the number of center moving pixels n 1, and setting the panoramic integration time again based on the threshold coefficient and the initial integration time t 1.

[0007] Further, step 3 specifically includes: For each of the divided p rectangular regions, the charge number of the brightest pixel of each rectangular region is n 1, n 2, n 3, n 4…… n i , i = n , wherein the integration time is , the single-frame sampling integration time or the sampling frequency of the pixel is changed under the condition that the brightness of the region does not exceed the threshold, and the sampling time corresponding to the p rectangular regions is set to T 1, T 2, T 3, T 4…… T i .

[0008] Further, step 4 specifically includes: Multi-frame accumulation of the sampled images, according to the number of multi-frame accumulation frames t / T i , and the new dim target is identified; the average sampling times of each pixel only once region division The calculation formula is: (1); In formula (1), , and , N i is the number of pixels in the first i region after the first division, t is the overall integration time of the detector, n i is the maximum charge number of each pixel in the first i region.

[0009] Further, step 5 specifically includes: Step 5.1: Under the condition that the detector needs to obtain a non-dynamic image, if the specified total integration time is reached, the shooting process is ended, and the detection image is output, if not, it is judged whether there is a new identified target pixel, if yes, the new identified target pixel is removed and steps 2-4 are repeated until the total integration time is reached, if not, it is judged whether the pixel brightness exceeds the first threshold or the brightness is lower than the second preset threshold, if yes, the sampling time is reset and steps 3-4 are repeated, if not, steps 3-4 are repeated until the specified total integration time is reached, and the detection image under the non-dynamic environment is output; Step 5.2: Under the condition that the detector needs to obtain a dynamic image, if the offset has been obtained, the identifiable target pixel is ignored, the image region is divided, and step 5.1 is repeated, if the offset has not been obtained, the bright star is used for navigation, the offset of each frame in the multi-frame accumulation process is obtained, it is judged whether the tailing will be caused based on the offset, the panoramic integration time t 0 to prevent tailing is calculated according to the set threshold, and shooting is performed again, the integration time of each region is calculated, and sampling is performed according to the sampling time; It is judged whether the specified total integration time is reached, if not, it is judged whether the tailing occurs, if the tailing occurs, the sampling time of each region is recalculated for sampling, until the specified total integration time is reached, if the tailing does not occur, sampling is performed according to the calculated sampling time of each sampling region, until the specified total integration time is reached, if the specified total integration time is reached, the detection image under the dynamic environment is output.

[0010] Further, in the sampling process, if the pixel brightness exceeds the first threshold, the sampling adjustment coefficient is obtained based on the threshold charge and the brightest pixel chargem , the sampling adjustment coefficient is adjusted according to the pixel brightness m The sampling time is adjusted. If the pixel brightness is lower than the second preset threshold, the sampling adjustment coefficient is adjusted according to the pixel brightness m The sampling time is adjusted.

[0011] Further, in the case of secondary image division, the average sampling number of each pixel in step 5.1 is: (2); In formula (2), , , and , , N i is the pixel number of the first i region after the first division, t is the overall integration time of the detector, n i is the maximum charge number of each pixel of the first i region, t i is the integration time experienced by the first i block region from the first division to the second division of its sub-regions, p i is the number of sub-blocks of the first i block region after the second division, N ij is the pixel number of the first i sub-region of the first j block region after the secondary division, n ij is the maximum charge number of each pixel of the first i sub-region of the first j block region.

[0012] Further, after reaching the specified total integration time in steps 5.1 and 5.2, for any target pixel region, the equivalent integration time after multi-frame superposition is obtained according to the sampling time and the sampling number, and the splicing between block regions is performed according to the equivalent integration time after multi-frame superposition to obtain a detection image.

[0013] The beneficial effects of the present application are: 1. The present application combines adaptive integration time and multi-frame accumulation technology, by intelligently dynamically adjusting the sampling area and integration time, it can flexibly adjust the sampling strategy according to the brightness change of the background radiation. This method effectively avoids the problem of overexposure or insufficient brightness caused by fixed integration time setting in traditional technology, thereby ensuring the stability of image quality, enhancing the recognizability of the target, and especially under complex lighting conditions, it can still accurately capture target information.

[0014] 2. The present application ensures that the system only samples the area that needs attention through accurate area division, thereby reducing unnecessary area sampling. This intelligent allocation of sampling resources not only effectively reduces the computational burden of the detector, but also improves the image processing efficiency, avoids the low efficiency and information loss caused by excessive sampling in traditional technology, and ensures the ability to process large-scale data efficiently.

[0015] 3. For the problem of target position change trailing, the present application adjusts the integration time adaptively, uses appropriate integration time to obtain multiple frames of images, avoids trailing caused by too few frames, and causes difficulty in recognition, which makes the present application have higher response speed and processing efficiency in dynamic target recognition and real-time imaging applications, and ensures the stability of image quality.

[0016] 4. For the problem of overexposure or information loss caused by rapid change of environmental brightness, the present application introduces an environmental brightness adjustment coefficient, which can adjust the sampling frequency in real time during sampling. When the environmental brightness changes suddenly, the system will automatically adjust the sampling strategy according to the environmental change, thereby avoiding overexposure or image quality loss. This ability to flexibly respond to environmental changes enables the present application to provide stable and high-quality imaging results under complex lighting conditions, improves the accuracy and robustness of target recognition in dynamic scenes, and effectively improves the recognition ability of dark and weak targets. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The flowchart of the anti-overexposure and anti-trailing imaging method based on dynamic integration time and multi-frame fusion provided by the present application; Figure 2 The imaging strategy flowchart for the target under dynamic change of the environment provided by the present application; Figure 3 The integration time diagram after dividing each area; Figure 4 The specific flowchart for calculating the sampling time of each area. DETAILED DESCRIPTION

[0018] In combination Figures 1-4 The present embodiment is described as follows: Figure 1As shown, the steps of the anti-overexposure and anti-smear imaging method based on dynamic integration time and multi-frame fusion according to the embodiment include: S1: Obtain the full well capacity of the pixel, set the initial integration time, take a picture of the region to be measured, and set the panoramic integration time according to the picture to take a picture again; Obtain the full well capacity n of the pixel, take the smallest integration time possible , and obtain the charge number n0 of the brightest pixel of the image. Take the threshold gray scale as 80% of the maximum gray scale, i.e., the threshold coefficient k 0=0.8 (in order to prevent overexposure caused by unstable factors such as the environment, the value can be determined according to experience). Take the panoramic integration time t 0= k 0* t 0 ’ * n / n 0, take a picture again. At this time, the maximum gray scale of the image does not exceed the threshold gray scale, and the identifiable target pixels in the obtained image are removed (these targets can be identified without collecting their information again).

[0019] When the detector needs to obtain a dynamic image, it is necessary to prevent smear. Take the smallest integration time possible t 0 ’ , and the number of moving pixels of the center of the bright star in the two obtained images is n 0. Take the threshold pixel number as 0.1, i.e., the threshold coefficient k 0=0.1. Take the panoramic integration time t 0 as k 0* t 0 ’ / n 0, take a picture again. At this time, it is not necessary to remove the identifiable target pixels.

[0020] S2: Divide the region obtained in S1; In this embodiment, rectangular division is adopted when dividing, and the entire image is divided into p rectangular regions. The division should meet the following rules (which can be adjusted according to the actual detector parameters and experience): 1) Each block has at least 100 pixels; 2) The number of divided blocks is not more than 16; 3) The ratio of the brightest and darkest pixels in each region is <2.

[0021] These parameters can be adjusted according to the actual pixel number of the detector and the environment.

[0022] The present application ensures that the system only samples the area that needs attention by precise regional division, thereby reducing the sampling of unnecessary areas. This intelligent allocation of sampling resources not only effectively reduces the computational burden of the detector, but also improves the image processing efficiency, avoids the low efficiency and information loss caused by over-sampling in traditional techniques, and ensures the ability to efficiently process large-scale data.

[0023] S3: sampling with different sampling times for each divided region; Find the brightest pixel point of each divided region, and count the number of charges of the pixel n 1, n 2, n 3, n 4… n i At this time, the integration time is t0, and different regions are sampled, and different sampling times are set for each region T 1, T 2, T 3, T 4… T i ; The calculation formula of the sampling time is: (1).

[0024] S4: After setting the sampling time, sampling is performed and the sampled image is accumulated for multiple frames.

[0025] The sampled image is accumulated for multiple frames until a new dim target can be identified, at which time the identified target pixel is ignored (the target position remains unchanged), and the remaining part is repeated S2-3, according to the number of accumulated frames t / T i , multiple frames of each region during sampling are superimposed. During the sampling process, if the environment becomes brighter, causing the maximum charge to exceed the threshold charge, the sampling time should be multiplied by the sampling adjustment coefficient m, which can be set by experience. If the environment becomes dark and the charge is not sufficient, the value of the adjustment coefficient m is: threshold charge / brightest charge.

[0026] To solve the problem of overexposure or information loss caused by rapid changes in environmental brightness, the present application introduces an environmental brightness adjustment coefficient, which can adjust the sampling frequency in real time during the sampling process. When the environmental brightness changes suddenly, the system will automatically adjust the sampling strategy according to the environmental change, thereby avoiding overexposure or image quality loss. This ability to flexibly respond to environmental changes enables the present application to provide stable and high-quality imaging results under complex lighting conditions, improving the accuracy and robustness of target recognition in dynamic scenes, and effectively improving the recognition ability of dim targets.

[0027] Average sampling times of each pixel only once region division The calculation formula is: (2); In formula (2), , and , N i is the number of pixels in the first i region after the first division, t is the overall integration time of the detector, n i is the maximum charge number of each pixel in the first i region.

[0028] S5: If the specified total integration time is reached, end the shooting process; S501: Under the condition that the detector needs to acquire a non-dynamic image, if the specified total integration time is reached, end the shooting process and output the detection image, if not, determine whether there is a new identified target pixel, if yes, remove the new identified target pixel and repeat S2-4 until the total integration time is reached, if not, determine whether the pixel brightness exceeds the first threshold or the brightness is lower than the second preset threshold, if yes, reset the sampling time and repeat S3-4, if not, repeat S3-4 until the specified total integration time is reached, and output the detection image under the non-dynamic environment; If a new identified target pixel appears and repeats S2-4, the average sampling times of each pixel in the case of secondary image division are: (3); In formula (3), , , and , , N i is the number of pixels in the first i region after the first division, t is the overall integration time of the detector, n i is the maximum charge number of each pixel in the first i region, t i is the first i block region after the first division, and the integration time experienced from the first division to the second division of its sub-region, p i is the number of sub-blocks of the first i block region after the second division, N ijFor the first i After the secondary division of the block region, the first j The number of pixels of the block sub-region, n ij For the first i The first j The maximum charge number of each pixel of the block sub-region.

[0029] S502: Under the condition that the detector needs to acquire a dynamic image, if the offset has been obtained by using a gyroscope or other means, the identifiable target pixels are ignored, the image region is divided, and S501 is repeated, if the offset cannot be obtained by other means, the navigation is performed by using bright stars, and the offset of each frame in the multi-frame accumulation process is obtained. t Based on the offset, it is judged whether tailing will be caused, the panoramic integration time for preventing tailing is calculated according to a set threshold value It is judged whether the specified total integration time is reached, if the specified total integration time is not reached, it is judged whether tailing is caused, if tailing is caused, the sampling time of each region is recalculated for sampling, until the specified total integration time is reached, if tailing is not caused, sampling is performed according to the calculated sampling time of each sampling region, until the specified total integration time is reached, if the specified total integration time is reached, the detection image in a dynamic environment is output.

[0030] For the tailing problem of the target position change, the present application adjusts the integration time adaptively, obtains multiple frames of images by using appropriate integration time, avoids tailing caused by too few frames, and causes difficulty in identification, which makes the present application have higher response speed and processing efficiency in dynamic target identification and real-time imaging application, and ensures stable image quality.

[0031] For the target on the region boundary, in order to obtain a complete image, the following steps are adopted: the target is located on block A and block B, the sampling times of the two block regions are t A and t B Suppose that the sampling times after shooting are n A and n B , the equivalent integration times after multiple frame superposition are t A * n A and t B * n B It is particularly noted that the splicing can be performed only when the integration times of the two images are the same, therefore, the image gray values of block A and block B after multiple frame superposition are multiplied by the coefficients respectively:t B * n B and t A * n A , after which the splicing can be completed.

[0032] For multi-frame accumulation in dynamic environments, since the position offset of each frame needs to be determined, bright stars need to be used for navigation, so the target pixel culling in S2 needs to be skipped. The imaging strategy flow for non-dynamic and dynamic environments is as follows: Figure 1 、 Figure 2 As shown, it is particularly noted that Figure 1 The strategy in can be applied to dynamic environments to prevent overexposure and improve the signal-to-noise ratio by using multi-frame accumulation. Figure 3 As shown in Figure 2, this step is relatively cumbersome. Therefore, for detectors whose hardware facilities cannot meet the requirements, the steps of area division and regional sampling in step 2 can also be skipped. The calculation of adaptive integration time for specific situations in static environment is as follows: Figure 4 shown.

[0033] In summary, this invention combines adaptive integration time and multi-frame accumulation technology. By intelligently and dynamically adjusting the sampling area and integration time, it can flexibly adjust the sampling strategy based on changes in background radiation brightness. This method effectively avoids the overexposure or insufficient brightness problems caused by fixed integration time settings in traditional technologies, thereby ensuring stable image quality and enhancing target recognizability, especially in complex lighting conditions.

[0034] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical content disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement of the above embodiments made according to the technical essence of the present invention, within the spirit and principles of the present invention, without departing from the content of the technical solution of the present invention, shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. Anti-overexposure and anti-tailing imaging method based on dynamic integration time and multi-frame fusion, characterized by: include: Step 1: Get the full well charge of the pixel, set the initial integration time to shoot the area to be measured, and set the panoramic integration time according to the shooting results to shoot again; Step 2: Divide the area to be photographed again into p A rectangular area; Step 3: Set different sampling times for each divided area; Step 4: Accumulate multiple frames of the sampled image until a new dim target is identified; Step 5: For new dim targets, repeat steps 2-4 for sampling and multi-frame accumulation until the sampling time reaches the specified total integration time, and output the detection image.

2. The anti-overexposure and anti-tailing imaging method based on dynamic integration time and multi-frame fusion according to claim 1, characterized in that: Step 1 specifically includes: Get the full well charge of the pixel n , set the initial integration time to Take photos of the area to be tested; If the detector needs to obtain a non-dynamic image, the charge number of the brightest pixel in the image is obtained based on the shooting results. n 0, set the threshold coefficient according to the gray value of the brightest pixel, and set the threshold coefficient according to the threshold coefficient and initial integration time. Set the panoramic integration time t 0 to shoot again and remove the identifiable target pixels in the retaken image; If the detector needs to obtain dynamic images, the number of pixels of the center movement of the bright star in the two frames of images is obtained based on the shooting results. n 1. Move the number of pixels according to the center n 1 Set the threshold coefficient based on the threshold coefficient and initial integration time Set the panoramic integration time t 0 to shoot again.

3. The anti-overexposure and anti-smearing imaging method based on dynamic integration time and multi-frame fusion according to claim 1, characterized in that: Step 3 specifically includes: For the division p rectangular areas, and the charge of the brightest pixel in each rectangular area is n 1. n 2. n 3. n 4…… n i , i = n , where the integration time is , while ensuring that the brightness of the area does not exceed the threshold, change the single-frame sampling integration time or sampling frequency of the pixel and set p The sampling time corresponding to the rectangular area T 1. T 2. T 3. T 4…… T i Take samples.

4. The anti-overexposure and anti-smearing imaging method based on dynamic integration time and multi-frame fusion according to claim 1, characterized in that: Step 4 specifically includes: Perform multi-frame accumulation on the sampled image, and calculate the number of frames according to the multi-frame accumulation. t / T i , multiple frames of each area in the sampling process are superimposed until a new dim target is identified; The average number of samples per pixel for only one region division The calculation formula is: (1)； In formula (1), ,and , N i After the first division, i The number of pixels in the area, t is the overall integration time of the detector, n i For the i The maximum charge number of each pixel in the area.

5. The anti-overexposure and anti-smearing imaging method based on dynamic integration time and multi-frame fusion according to claim 1, characterized in that: Step 5 specifically includes: Step 5.1: Under the condition that the detector needs to acquire a non-dynamic image, if the specified total integration time is reached, the shooting process is terminated and the detection image is output. If not, it is determined whether there are new identification target pixels. If so, the new identification target pixels are eliminated and steps 2-4 are repeated until the total integration time is reached. If not, it is determined whether the pixel brightness exceeds a first threshold or is lower than a second preset threshold. If it exceeds the first threshold or is lower than the second preset threshold, the sampling time is reset and steps 3-4 are repeated. If it does not exceed the first threshold or is lower than the second preset threshold, steps 3-4 are repeated until the specified total integration time is reached and the detection image in the non-dynamic environment is output. Step 5.2: If the detector needs to obtain dynamic images, if the offset has been obtained, ignore the identifiable target pixels, divide the image area, and repeat step 5.

1. If the offset has not been obtained, use bright stars for navigation, obtain the offset of each frame in the multi-frame accumulation process, determine whether it will cause tailing based on the offset, and calculate the panoramic integration time to prevent tailing based on the set threshold. t 0, and shoot again, calculate the integration time of each area, and sample according to the sampling time; Determine whether the specified total integration time has been reached. If not, determine whether there is tailing. If there is tailing, recalculate the sampling time of each area and perform sampling until the specified total integration time is reached. If not, perform sampling according to the calculated sampling time of each sampling area until the specified total integration time is reached. If the specified total integration time is reached, output the detection image under the dynamic environment.

6. The anti-overexposure and anti-tailing imaging method based on dynamic integration time and multi-frame fusion according to claim 5, characterized in that: During the sampling process, if the pixel brightness exceeds the first threshold, the sampling adjustment coefficient is obtained based on the threshold charge and the brightest pixel charge. m , according to the sampling adjustment coefficient m Adjust the sampling time; If the pixel brightness is lower than the second preset threshold, the sampling adjustment coefficient is adjusted according to m Adjust the sampling time.

7. The anti-overexposure and anti-smearing imaging method based on dynamic integration time and multi-frame fusion according to claim 5, characterized in that: If a new target pixel appears in step 5.1, and steps 2-4 are repeated to perform secondary image division, the average number of sampling times for each pixel is: (2)； In formula (2), , ,and , , N i After the first division, i The number of pixels in the area, t is the overall integration time of the detector, n i For the i The maximum charge number of each pixel in the region, t i After the first division i The integration time from the first division to the second division of the block area is p i For the i The number of sub-blocks in the block area after the second division, N ij For the i After the block area is divided twice, its j The number of pixels in the block sub-region, n ij For the i The block area j The maximum charge count for each pixel in a block sub-region.

8. The anti-overexposure and anti-tailing imaging method based on dynamic integration time and multi-frame fusion according to claim 5, characterized in that: After reaching the specified total integration time in steps 5.1 and 5.2, for the block where any target pixel is located, the equivalent integration time after multi-frame superposition is obtained based on the sampling time and number of sampling times, and the blocks are spliced ​​based on the equivalent integration time after multi-frame superposition to obtain the detection image.

Images