Sequence image distortion and severe thermal radiation effect fusion correction method

Abstract

The invention discloses a sequence image distortion and severe thermal radiation effect fusion correction method, and the method comprises the steps: obtaining an image sequence containing different degrees of atmospheric turbulence distortion and severe thermal radiation degradation, carrying out the preliminary thermal radiation processing of each image in the image sequence, and obtaining a preprocessed image sequence; performing distortion correction on the preprocessed image sequence to obtain a distortion corrected image sequence; and performing normalization processing on the distortion correction image sequence, generating a weight image sequence by calculating a local contrast feature and a brightness feature, and fusing the distortion correction image sequence and the weight image sequence into a correction image by adopting a front-to-back strategy. The method provided by the invention is simple and efficient to implement, can correct the image with atmospheric turbulence distortion and severe thermal radiation superimposed effect, can effectively remove the distortion and thermal radiation degradation effect of the image, improves the image quality and signal-to-noise ratio, and facilitates the subsequent implementation of operations such as target recognition and detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image distortion and aerodynamic optical thermal radiation effect correction, and particularly relates to a sequence image distortion and severe thermal radiation effect fusion correction method. BACKGROUND

[0002] When the aircraft flies at high speed in the atmosphere, flow phenomena such as boundary layer and wake are formed on the surface of the aircraft, and the air temperature, density and pressure in these flow regions are different from the surrounding atmosphere, thereby causing the refractive index to change. When light passes through these non-uniform atmospheric regions, its propagation path is disturbed, thereby producing aberration or optical wave distortion.

[0003] At the same time, the optical head cover of the detector is affected by the airflow during high-speed movement, and the kinetic energy of the airflow in the boundary layer near the surface of the head cover is dissipated and converted into heat energy, causing the temperature of the optical window to rise rapidly. This convective heat transfer between the high-speed airflow and the surface of the head cover is called aerodynamic heating.

[0004] The above two degradation factors will affect the optical system or sensor on the aircraft, causing the detected image to be distorted and thermally degraded, and further causing subsequent target detection and recognition to fail.

[0005] In order to eliminate the influence of distortion and thermal radiation effect on the imaging of optical equipment, there are currently some effective methods, such as replacing the infrared window material with high transmission and low emissivity, variable integration time infrared imaging technology, adaptive optical technology, and deep learning-based methods. Although the above technologies can effectively prevent the saturation of optical equipment, the method is relatively complex to operate and has high cost.

[0006] In addition, the existing image distortion and severe thermal radiation degradation correction methods mostly correct single degradation effect of a single image, and there are few methods for correcting multiple degradation effect sequence images. SUMMARY

[0007] In view of the defects in the prior art, the present application provides a sequence image distortion and severe thermal radiation effect fusion correction method, which can simply and efficiently remove image distortion and severe thermal radiation effect, improve image signal-to-noise ratio, and help subsequent target recognition and detection operations.

[0008] The technical solution adopted by the present application to solve its technical problems is as follows: The first aspect of the present application provides a sequence image distortion and severe thermal radiation effect fusion correction method, which comprises: S1, obtaining an image sequence containing different degrees of atmospheric turbulence distortion and severe thermal radiation degradation, and performing preliminary thermal radiation processing on each image in the image sequence to obtain a preprocessed image sequence; S2, distortion correction is performed on the pre-processed image sequence to obtain a distortion-corrected image sequence; S3, normalization is performed on the distortion-corrected image sequence, a weight image sequence is generated by calculating local contrast features and brightness features, and a from-front-to-back strategy is adopted to fuse the distortion-corrected image sequence and the weight image sequence into a corrected image, specifically comprising: S301, normalization is performed on the distortion-corrected image sequence, and local contrast features and brightness features of each image are calculated, a contrast weight term of each image is calculated based on local contrast features of all images, a brightness feature of each image is taken as a brightness weight term, and a weight image sequence is generated according to the contrast weight term and the brightness weight term of each image; S302, Gaussian filtering and down-sampling are performed on the first and second images of the distortion-corrected image sequence to obtain Gaussian pyramids of the two images, Gaussian filtering and down-sampling are performed on the weight images corresponding to the two images to obtain Gaussian pyramids of the two weight images, the Gaussian pyramids of the two images and the Gaussian pyramids of the corresponding weight images are multiplied pixel by pixel to obtain two Gaussian pyramids containing weight information, the two Gaussian pyramids containing weight information are added in the same scale to form a new Gaussian pyramid, and the new Gaussian pyramid is restored to the original image to obtain a first fused image; meanwhile, the Gaussian pyramids of the two weight images are added in the same scale to obtain a Gaussian pyramid of a weight image corresponding to the fused image; S303, the current fused image and the next image of the distortion-corrected image sequence are taken as two images to be fused, the same fusion logic is adopted for fusion to obtain the next fused image and a Gaussian pyramid of a weight image corresponding to the next fused image; the same from-front-to-back strategy is continuously adopted until all images of the distortion-corrected image sequence are fused into a corrected image.

[0009] In the above scheme, step S1 specifically comprises: S101, weighted least squares filtering is performed on each image in the image sequence; S102, the filtered image is brought into a cubic spline surface equation to fit a thermal radiation effect bias field of each image; S103, the thermal radiation effect bias field of each image is subtracted from the image to obtain a pre-processed image sequence.

[0010] In the above scheme, step S2 specifically comprises: S201, a heavy thermal radiation saturated area and a non-saturated effective image information area block of each image in the pre-processed image sequence are extracted, a median pixel value of each pixel position in the non-saturated effective information area block of the pre-processed image sequence is calculated, and a median pixel value prior image is obtained; S202, determine the overlapping area of the non-saturated effective image information area block of the first image to the last image in the pre-processed image sequence, obtain the overlapping non-saturated effective image information area block of the first image to the last image in the pre-processed image sequence, and extract the feature point pair of the median pixel value prior image and the overlapping non-saturated effective image information area block of the first image to the last image in the pre-processed image sequence; S203, according to the feature point pair of the median pixel value prior image and the overlapping non-saturated effective image information area block of the first image to the last image in the pre-processed image sequence, determine the inverse transformation matrix of the first image to the last image in the pre-processed image sequence to the median pixel value prior image, and calculate the average inverse transformation matrix, and perform distortion correction on the last image in the pre-processed image sequence based on the average inverse transformation matrix; S204, determine the overlapping area of the non-saturated effective image information area block of the first image to the last image in the pre-processed image sequence, and perform distortion correction on the second last image according to the same distortion correction logic; continue to use this strategy from back to front, and sequentially complete the distortion correction of the third last image to the first image, and finally obtain the distortion corrected image sequence.

[0011] In the above scheme, in step S201, the severe thermal radiation saturated area and the non-saturated effective image information area block of each image in the pre-processed image sequence are extracted, including: normalizing the pre-processed image sequence to obtain a normalized image sequence; regarding the area with pixel gray value greater than a preset gray threshold value in each image in the normalized image sequence as a severe thermal radiation saturated area, regarding other areas as non-saturated effective image information area blocks, and extracting the non-saturated effective image information area block of each image from the pre-processed image sequence.

[0012] In the above scheme, step S2 specifically further includes: S205, updating the median pixel value prior image according to the distortion corrected image sequence; S206, performing distortion correction again on the distortion initial correction image sequence based on the updated median pixel value prior image, to obtain the final distortion corrected image sequence.

[0013] In the above scheme, step S301 specifically includes: performing normalization processing on each image in the distortion corrected image sequence Q to obtain a normalized distortion corrected image sequence R; calculating the local contrast feature of the i-th image in the normalized distortion corrected image sequence R R i , and the formula is as follows: ; wherein, x and y is a pixel coordinate; is a local contrast feature; SIFT is a SIFT operator; is an L1 norm; calculating a contrast weight term of the i-th image based on the local contrast features of all images : ; wherein n is the total number of images in the image sequence; calculating a brightness feature of the i-th image as a brightness weight term : ; wherein, is a brightness parameter; calculating a weight image of the i-th image according to the contrast weight term and the brightness weight term of the i-th image : ; normalizing the weight image : ; wherein, is a very small positive number to prevent the denominator from being zero; is the normalized weight image, thus obtaining a weight image sequence E.

[0014] In the above scheme, step S301 specifically further comprises: performing guided filter optimization on the weight image sequence to obtain an optimized weight image sequence W.

[0015] In the above scheme, step S302 specifically comprises: performing Gaussian filtering and down-sampling on the first image Q 1 and the second image Q 2 of the distortion-corrected image sequence Q to construct Gaussian pyramids of the two images and : ; wherein, m , n is a relative coordinate within the kernel; k is a Gaussian kernel radius; is a Gaussian kernel weight, wherein is a standard deviation of Gaussian distribution; x and y is a pixel coordinate, down-sampling factor; l =0, 1, …, L denotes the number of pyramid layers, L denotes the maximum number of layers, denotes the 0th layer image, l denotes the 0th layer image, denotes the original image Q; Each layer image of the Gaussian pyramid of the two images is adaptively edge-aware guided filtered to obtain filtered pyramid images and : ; In the formula, denotes the filtered pyramid image; r denotes a local window radius, denotes a regularization parameter, denotes an input image; denotes guided filtering; The weight images corresponding to the two images W 1 and W 2 are also Gaussian filtered and down-sampled to obtain Gaussian pyramids of the two weight images and ; The filtered Gaussian pyramids of the two images and the Gaussian pyramids of the corresponding weight images are pixel by pixel multiplied to obtain two Gaussian pyramids containing weight information and The two Gaussian pyramids containing weight information are added in the same scale to form a new Gaussian pyramid : ; The new Gaussian pyramid is up-sampled from the coarsest scale l = L to the finest scale l =0 in sequence and accumulated to obtain a new image, and the image pixel value is normalized to [0, 255] to obtain a first fused image; At the same time, the Gaussian pyramids of the two weight images are added in the same scale to obtain the Gaussian pyramid of the weight image corresponding to the fused image.

[0016] According to a second aspect of the present application, a computer device is provided, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the sequence image distortion and severe thermal radiation effect fusion correction method of any one of the first aspect.

[0017] According to a third aspect of the present application, a computer readable storage medium is provided, which stores a program or instructions, and the program or instructions, when executed by a processor, implement the steps of the sequence image distortion and severe thermal radiation effect fusion correction method according to any one of the first aspect.

[0018] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects: The present application provides a sequence image distortion and severe thermal radiation effect fusion correction method, which realizes the collaborative correction of atmospheric turbulence distortion and severe thermal radiation effect in sequence images. First, weighted least squares filtering and cubic spline surface fitting are used to remove thermal radiation bias field, retaining effective information for subsequent processing. Second, combining the median pixel value prior image and the inverse sequence distortion correction strategy, distortion correction is achieved through feature point matching and non-reflective similarity transformation. Finally, based on the local contrast and brightness features, an adaptive weight map is generated, which is optimized by guided filtering. Then, the Gaussian pyramid multi-scale fusion is used to fuse the distortion correction image sequence and the weight image sequence, and the final corrected image is obtained. The method of the present application is simple and efficient, and can correct images with atmospheric turbulence distortion and severe thermal radiation superposition effect. By executing the method, the distortion and thermal radiation degradation effect of the image can be effectively removed, the image quality and signal-to-noise ratio can be improved, and it is helpful for subsequent target recognition and detection operations. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of a sequence image distortion and severe thermal radiation effect fusion correction method provided by an embodiment of the present application is shown in the figure; Figure 2 A schematic diagram of a raw degraded image sequence Z provided by an embodiment of the present application is shown in the figure; Figure 3 A schematic diagram of a preprocessed image sequence S obtained after initial correction of thermal radiation effect provided by an embodiment of the present application is shown in the figure; Figure 4 A schematic diagram of a distortion initial correction result sequence Y provided by an embodiment of the present application is shown in the figure; Figure 5 A schematic diagram of a distortion final correction result sequence Q provided by an embodiment of the present application is shown in the figure; Figure 6 A clear image provided by an embodiment of the present application is shown in the figure; Figure 7 A final distortion and thermal radiation saturated area correction image provided by an embodiment of the present application is shown in the figure; Figure 8 An image fusion schematic diagram provided by an embodiment of the present application is shown in the figure; Figure 9 A hardware structure schematic diagram of a computer device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of the present application.

[0021] It is obvious that the accompanying drawings in the following description are only some examples or embodiments of the present application, and for those of ordinary skill in the art, the present application can be applied to other similar scenarios without creative efforts based on the accompanying drawings. In addition, it can be understood that although the efforts made in the development process can be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some designs, manufacturing or production changes based on the technical content disclosed in the present application are only routine technical means, and should not be understood as insufficient disclosure of the present application.

[0022] In the present application, "embodiments" means that the specific features, structures or characteristics described in combination with the embodiments can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiments, nor is it necessarily independent or alternative embodiments to other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0023] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terms "a", "an", "the", and "said" used in this specification do not denote the quantity of what is referred to, but rather denote the presence of at least one of the referenced items. The terms "including", "comprising", "having" and their variants are meant to encompass the presence of stated features, steps, or components and / or the occurrence of stated conditions, but do not preclude the presence or addition of one or more other features, steps, components, or conditions, or indeed the presence or addition of yet other steps, components, or conditions. The term "connected" or "coupled" or similar terms used in this specification do not necessarily denote a direct connection or coupling, but also include an indirect connection or coupling. The term "plurality" means two or more. The term "and / or" describes associated objects in association relationship, which means that there are three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the front and rear associated objects. The terms "first", "second", "third", and the like are merely used to distinguish similar objects, and do not represent a specific order of the objects.

[0024] The present application provides a sequence image distortion and severe thermal radiation effect fusion correction method, comprising the following steps: preliminary correction of the thermal radiation effect of the degraded image sequence; extracting an effective information region block to obtain a prior image; according to the feature points matched by feature extraction, adopting a back-to-front strategy to construct a distortion correction image sequence; calculating the local contrast and brightness features of the distortion correction image sequence to generate a weight image sequence; using a guided filter to optimize the weight image sequence, and performing weighted fusion on the distortion correction image sequence to obtain a final correction image. The method of the present application is simple and efficient, and can correct images with atmospheric turbulence distortion and severe thermal radiation superposition effect. By executing the method of the present application, the distortion and thermal radiation degradation effect of the above images can be effectively removed, and the image quality and signal-to-noise ratio can be improved.

[0025] As shown in Figure 1 The sequence image distortion and severe thermal radiation effect fusion correction method of the present application comprises the following steps: S1, obtaining an image sequence containing different degrees of atmospheric turbulence distortion and severe thermal radiation degradation, and preliminarily processing each image in the image sequence for thermal radiation, the specific process comprising: S101, performing weighted least squares filtering on the degraded image sequence; S102, inputting the filtered image into a cubic spline surface equation to fit a thermal radiation effect bias field; S103, preliminary removal of thermal radiation effect, to obtain a preprocessed image sequence.

[0026] S2, based on the unsaturated effective information area block, the preprocessed image sequence is corrected for distortion, and the specific process includes: S201, extract the heavy thermal radiation saturation area and the effective image information area block of each image in the image preprocessing sequence, calculate the median pixel value of each pixel position in the effective information area block, and obtain the median pixel value prior image; S202, the median pixel value prior image and the effective image information area block of each frame of image in the preprocessed image sequence are matched by using SURF algorithm for local irregular area feature extraction and matching, and the matched feature point pairs are saved; S203, according to the matched feature point pairs in step S202, in the unsaturated effective image information area block overlapped from the first image to the last image in the preprocessed image sequence, the median pixel value prior image is referred to, the geometric transformation and average inverse transformation matrix are calculated by non-reflection similarity transformation, and the distortion initial correction result of the last degenerated image of the preprocessed image sequence is constructed; S204, using the strategy from back to front, repeating step S203, based on the unsaturated effective image information area block overlapped from the first image to the second last image in the preprocessed image sequence, the distortion correction result of the second last degenerated image is constructed, and the cycle is repeated in turn until the distortion initial correction result of all degenerated images in the sequence is obtained; S205, according to the distortion initial correction result, the median pixel value prior image is updated, the updated median pixel value prior image is referred to, the feature point pairing of the image sequence after distortion initial correction is carried out again, the geometric transformation and average inverse transformation matrix are calculated by non-reflection similarity transformation, and the distortion final correction result of the last degenerated image of the sequence degenerated image is constructed; S206, using the strategy from back to front, repeating step S205, based on the unsaturated effective image information area block overlapped from the first image to the second last image in the preprocessed image sequence, the distortion correction result of the second last degenerated image is constructed, and the cycle is repeated in turn until the distortion final correction result of all degenerated images in the sequence is obtained.

[0027] S3, using the strategy from front to back, the optimized weight image and the distortion final correction image are fused, so as to correct the heavy thermal radiation effect saturation area, and the specific process includes: S301, normalize the distortion final correction image sequence, calculate the local contrast and brightness features of each image, calculate the weight image sequence according to the local contrast and brightness features, use the guided filter to optimize, and obtain the optimized weight image sequence; S302, set the first two images of the distortion end correction image sequence as the first and second to-be-fused images, use Gaussian filtering and down-sampling processing on the first and second to-be-fused images, obtain a Gaussian pyramid of the to-be-fused images, then perform adaptive edge-aware guided filtering on each scale image of the Gaussian pyramid of the to-be-fused images to obtain a guided-filtered image Gaussian pyramid; use Gaussian filtering and down-sampling processing on the optimized weight image to obtain a Gaussian pyramid of the weight image; multiply the corresponding scale images of the Gaussian pyramid of the to-be-fused images and the Gaussian pyramid of the weight image pixel by pixel to obtain two Gaussian pyramids containing weight information; add the same scale images of the two Gaussian pyramids to form a new Gaussian pyramid; up-sample and accumulate the new Gaussian pyramid from the coarsest scale to the finest scale to obtain a new image, and normalize the pixel value of the fused image to [0, 255] to obtain the first fused image; at the same time, add the same scale images of the two weight image Gaussian pyramids to obtain the Gaussian pyramid of the weight image corresponding to the fused image; S303, according to the strategy from front to back, set the next image of the distortion end correction image sequence as a new second to-be-fused image, repeat step S302 until all images of the distortion end correction image sequence are fused into a correction image.

[0028] In some embodiments, the method in step S1 is specifically: Obtain an image sequence containing different degrees of atmospheric turbulence distortion and severe thermal radiation degradation, preliminarily process the thermal radiation of each image in the image sequence: perform weighted least squares filtering on each image in the degraded image sequence; bring the filtered image into a cubic spline surface equation to fit a thermal radiation effect bias field; then subtract the thermal radiation bias field from the original image to remove the thermal radiation effect, preliminarily remove the thermal radiation effect, and obtain a preprocessed image sequence.

[0029] In some embodiments, the method of step S201 is specifically: Perform normalization processing on the image sequence S, regard the part with a pixel value greater than 0.9 as a severe thermal radiation saturation region, and regard other regions as effective image information region blocks, correspondingly divide the severe thermal radiation saturation region and the effective image information region blocks in the non-normalized image sequence S. Extract all effective image information region blocks in the image preprocessing sequence S, calculate the median pixel value of each pixel position in the effective information region block to obtain a median pixel value prior image I 0 (Here, the median pixel value prior image refers to a new image formed by finding the median pixel of the same position pixels of a group of images, and finding the median pixel of all positions).

[0030] In some embodiments, the method of step S202 is specifically: Feature point detection is performed using the SURF algorithm. In order to match the feature points between two image blocks, Euclidean distance is used for matching. When the Euclidean distance between two descriptors is less than a threshold value of 0.7, the feature points of the two images are regarded as a group of feature point pairs.

[0031] Specifically, the effective image information region blocks are processed, the overlapping region is extracted, the median pixel value prior image I 0 is extracted from each frame of the preprocessed image sequence S, feature point detection is performed using the SURF algorithm, the detection principle is that the Hessian matrix of a pixel point is calculated on different Gaussian scales, and the local maximum point of the determinant is selected as a feature point. After the feature points are calculated, the SURF descriptor d needs to be calculated, d is a 64-dimensional vector formed by splicing the response values of 16 sub-regions. In order to match the feature points between two image blocks, Euclidean distance is used for matching. When the Euclidean distance between two descriptors is less than a threshold value of 0.7, the feature points of the two images are regarded as a group of feature point pairs.

[0032] In some embodiments, the method of step S203 is specifically: A parameterized transformation matrix is constructed based on a non-reflective similarity transformation model, parameters are solved based on the feature point pairs, a transformation matrix is obtained, and an average inverse transformation matrix is calculated based on the transformation matrix obtained for each image, so as to realize preliminary correction of image distortion.

[0033] Specifically, according to the matched feature point pairs in step S202, in the overlapping non-saturated effective image information region blocks of the first image to the last image in the preprocessed image sequence S, the median pixel value prior image I 0 is referenced, a parameterized transformation matrix is constructed based on a non-reflective similarity transformation model; parameters are solved based on the feature point pairs, a transformation matrix is obtained; an average inverse transformation matrix is calculated based on the transformation matrix obtained for each image; and the last image S n The average inverse transformation matrix is applied to obtain a distortion preliminary correction result I n .

[0034] In some embodiments, the method of step S204 is specifically: A back-to-front strategy is adopted until preliminary correction of all images in the image sequence is completed.

[0035] Specifically, step S203 is repeated from back to front, and the first to the second last image in the pre-processed image sequence S, i.e., the first to the 99th image, is used to perform distortion correction on the overlapped non-saturated effective image information region block, so as to construct the distortion correction result of the second last degraded image in the initial distortion correction sequence Y, i.e., the 99th image, and then the process is sequentially repeated until the distortion correction results of all the degraded images in the sequence are obtained, and a complete initial distortion correction image sequence Y is obtained.

[0036] In some embodiments, the method of step S205 and step S206 is specifically as follows: The median pixel value prior image and the feature point pair are updated according to the initial distortion correction result, the final distortion correction of the image is realized, and a complete final distortion correction image sequence is formed.

[0037] Specifically, step S205 updates the median pixel value prior image according to the initial distortion correction result as follows: I 0’ is used to refer to the initial distortion correction image sequence Y, and the feature point pairing is performed again on the initial distortion correction image sequence Y. I 0’ is used to refer to the initial distortion correction image sequence Y, and the feature point pairing is performed again on the initial distortion correction image sequence Y. I n ’.

[0038] Step S206 is repeated from back to front, and step S205 is repeated based on the overlapped non-saturated effective image information region block of the first to the second last image in the initial distortion correction image sequence, so as to construct the second last final distortion correction image in the sequence Q, and then the process is sequentially repeated until the final distortion correction results of all the degraded images in the sequence are obtained, and a complete final distortion correction image sequence Q is obtained.

[0039] In some embodiments, the method of step S301 is specifically as follows: The local contrast and brightness features of the image are calculated, and a weight image sequence is generated.

[0040] Specifically, the distortion final correction image sequence Q is normalized, and the pixel values in all images are normalized from [0, 255] to [0, 1], and a normalized image sequence R is obtained after processing; then the local contrast and brightness features of the image are calculated, so as to generate a weight image sequence E; the generated weight image is guided filtered and optimized to obtain an optimized weight image sequence W.

[0041] In some embodiments, the method of step S302 is specifically as follows: The two images to be fused are subjected to Gaussian filtering and down-sampling to obtain Gaussian pyramids of the two images 、 Then, adaptive edge-aware guided filtering is performed on the images of different scales of the two Gaussian pyramids to obtain two filtered Gaussian pyramids 、 The corresponding weight images of the images to be fused are subjected to Gaussian filtering and down-sampling to obtain Gaussian pyramids of the weight images 、 The corresponding scale images of the Gaussian pyramids of the images to be fused and the Gaussian pyramids of the weight images are multiplied pixel by pixel to obtain two Gaussian pyramids containing weight information 、 The same scale images of the two Gaussian pyramids are added to form a new Gaussian pyramid The new Gaussian pyramid is sequentially up-sampled and accumulated to fuse to obtain a new image from the coarsest scale to the finest scale F 1and the image pixel value is normalized to [0, 255] to obtain a first fused image I 1. Meanwhile, the Gaussian pyramids of the two weight images are added in the same scale to obtain a Gaussian pyramid of a weight image corresponding to the fused image.

[0042] In some embodiments, the method of step S303 is specifically: According to the strategy from front to back, the obtained fused image I 1is set as the first image to be fused in a new round of fusion, and the third image of the sequence Q of the final distortion correction image is set as the second image to be fused in the new round of fusion, and fusion is performed to obtain a new fused image. Repeat this process until all images of the sequence Q are fused into a corrected image I f .

[0043] By implementing the method, image distortion and severe thermal radiation effects can be simply and efficiently removed, the image signal-to-noise ratio is improved, and subsequent target identification and detection operations are facilitated.

[0044] As shown in Figure 1 , the sequence image distortion and severe thermal radiation effect fusion correction method of the embodiment of the application comprises the following steps: S1, an image sequence containing different degrees of atmospheric turbulence distortion and severe thermal radiation degradation is obtained, and each image in the image sequence is preliminarily processed for thermal radiation, and the specific process comprises: S101, weighted least squares filtering is performed on the degraded image sequence; S102, the filtered image is input into a cubic spline surface equation to fit a thermal radiation effect bias field; S103, preliminary removal of thermal radiation effect, to obtain a pre-processing image sequence.

[0045] S2, based on the non-saturated effective information area block, the pre-processing image sequence is corrected for distortion, and the specific process includes: S201, extract the heavy thermal radiation saturation area and the effective image information area block of each image in the image pre-processing sequence, calculate the median pixel value of each pixel position in the effective information area block, and obtain the median pixel value prior image; S202, the median pixel value prior image and the effective image information area block of each frame of image in the pre-processing image sequence are matched by using SURF algorithm for local irregular area feature extraction and matching, and the matched feature point pairs are saved; S203, according to the matched feature point pairs in step S202, in the non-saturated effective image information area block overlapped from the first image to the last image in the pre-processing image sequence, the median pixel value prior image is referred to, the geometric transformation and average inverse transformation matrix are calculated by non-reflection similarity transformation, and the distortion initial correction result of the last degenerated image of the sequence degenerated image sequence is constructed; S204, using the strategy from back to front, repeating step S203, based on the non-saturated effective image information area block overlapped from the first to the second last image in the pre-processing image sequence, the distortion correction result of the second last degenerated image is constructed, and the distortion initial correction result of all degenerated images in the sequence is obtained by circulating in turn; S205, according to the distortion initial correction result, the median pixel value prior image is updated, the updated median pixel value prior image is referred to, the feature point pairing of the image sequence after distortion initial correction is carried out again, the geometric transformation and average inverse transformation matrix are calculated by non-reflection similarity transformation, and the distortion final correction result of the last degenerated image of the sequence degenerated image is constructed; S206, using the strategy from back to front, repeating step S205, based on the non-saturated effective image information area block overlapped from the first to the second last image in the pre-processing image sequence, the distortion correction result of the second last degenerated image is constructed, and the distortion final correction result of all degenerated images in the sequence is obtained by circulating in turn.

[0046] S3, using the strategy from front to back, the optimized weight image and the distortion final correction image are fused, so as to correct the heavy thermal radiation effect saturation area, and the specific process includes: S301, normalize the distortion final correction image sequence, calculate the local contrast and brightness features of each image, calculate the weight image sequence according to the local contrast and brightness features, use the guided filter to optimize, and obtain the optimized weight image sequence; S302. Set the first two images of the final distortion-corrected image sequence as the first and second images to be fused. Apply Gaussian filtering and downsampling to the first and second images to be fused to obtain the Gaussian pyramid of the images to be fused. Then, perform guided filtering with adaptive edge perception on each scale image of the Gaussian pyramid of the images to be fused to obtain the Gaussian pyramid of the images after guided filtering. Apply Gaussian filtering and downsampling to the optimized weight map to obtain the Gaussian pyramid of the weight map. Multiply the corresponding scale images of the Gaussian pyramid of the images to be fused and the Gaussian pyramid of the weight image pixel by pixel to obtain two Gaussian pyramids containing weight information. Then, add the same scale images of the two Gaussian pyramids to form a new Gaussian pyramid. Upsample and accumulate the new Gaussian pyramid from the coarsest scale to the finest scale in sequence to obtain a new image, and normalize the pixel values of the fused image to [0, 255] to obtain the first fused image. At the same time, add the Gaussian pyramids of the two weight images at the same scale to obtain the Gaussian pyramid of the weight image corresponding to the fused image. S303. According to the strategy from front to back, set the next image of the final distortion-corrected image sequence as the new second image to be fused, and repeat step S302 until all images of the final distortion-corrected image sequence are fused into one corrected image.

[0047] Further, the method in step S1 of the embodiment of the present invention is specifically as follows: Obtain an image sequence containing different degrees of atmospheric turbulence distortion and severe thermal radiation degradation, and preliminarily perform thermal radiation processing on each image in the image sequence: perform weighted least squares filtering on the degraded image sequence; substitute the filtered image into the cubic spline surface equation to fit out the thermal radiation effect bias field; preliminarily remove the thermal radiation effect to obtain the preprocessed image sequence.

[0048] The specific process is as follows: For each degraded image Figure 2 shown in the Z i degraded image sequence, where 1 < i < n, and in this example n = 100, after performing weighted least squares filtering, obtain the filtered image G i , and its formula is as follows: ; where w ( x , y ) is the weight function, and its value at the position ([[]] x , y ) determines the importance of this position in the optimization process; is the regularization parameter, which controls the trade-off between the smoothing term and the data fidelity term; is a smoothing term, where G denotes the gradient of the image G .

[0049] into the cubic spline surface equation, the thermal radiation effect bias field is fitted B i , the formula is as follows: ; wherein, is a regularization parameter, S3 is a cubic spline function space, which ensures that the surface is continuous and smooth at the boundary.

[0050] remove the thermal radiation effect to obtain a corrected image S i The formula is as follows: .

[0051] Finally, the preprocessed image sequence S is obtained, as shown in Figure 3 .

[0052] Further, in step S2 of the embodiment of the present application, the method of step S201 is specifically: The image sequence S is normalized, and the part with a pixel value greater than 0.9 is regarded as a severe thermal radiation saturation area, and other areas are regarded as effective image information area blocks. The severe thermal radiation saturation area and the effective image information area block in the normalized image sequence S are divided. All effective image information area blocks in the image preprocessing sequence S are extracted, the median pixel value of each pixel position in the effective information area block is calculated, and a median pixel value prior image I 0 (here, the median pixel value prior image refers to a new image formed by calculating the median pixel value of the same position pixel of a group of images, calculating the median pixel value of all positions, and then forming a new image), the formula is as follows: ; In the formula, denotes the median pixel value of the calculation position x , y ). Thus, the median pixel value prior image is obtained.

[0053] Further, in step S2 of the embodiment of the present application, the method of step S202 is specifically: The effective image information area block is processed, the overlapping area is extracted, and the median pixel value prior image I0 and the overlapped effective image information region block extracted from each frame image in the pre-processed image sequence S is detected by using the SURF algorithm, and the detection principle is that the Hessian matrix of the pixel points is calculated on different Gaussian scales, and the local maximum point of the determinant is selected as the feature point. After the feature points are calculated, the SURF descriptor d needs to be calculated, and d is a 64-dimensional vector formed by splicing the response values of 16 sub-regions. In order to match the feature points between two image blocks, the Euclidean distance is used for matching, and when the Euclidean distance of two descriptors is less than a threshold value 0.7, the feature points of the two images are regarded as a group of feature point pairs.

[0054] The specific process is as follows: Feature point detection is performed: ; In the formula, SURF feature point detection is represented, the median pixel value prior image I 0 is represented, SURF feature point of the pre-processed image sequence S is represented i th image S i SURF feature point of the pre-processed image sequence S is represented

[0055] Generate descriptor: ; In the formula, SURF descriptor generation is represented, k and l is the feature point number.

[0056] Feature point matching is performed: ; In the formula, Euclidean distance is represented, and M is a feature point number pair set.

[0057] Output feature point pair: ; In the formula, is a feature point pair set.

[0058] Thus, the median pixel value prior image I 0 and the feature point pairs of the overlapped effective image information region block extracted from each frame image in the pre-processed image sequence S are obtained.

[0059] Further, in step S2 of the embodiment of the application, the method of step S203 is specifically: Based on the feature point pairs matched in step S202, in the overlapping non-saturated effective image information region block from the first image to the last image in the preprocessed image sequence S, referencing the prior image with the median pixel value. I 0. Construct a parameterized transformation matrix based on the non-reflection similarity transformation model; solve for the parameters based on feature point pairs to obtain the transformation matrix; calculate the average inverse transformation matrix based on the transformation matrix obtained for each image; and transform the last image in sequence S... S n The initial distortion correction result is obtained by applying the average inverse transform matrix. I n .

[0060] The specific process is as follows: Non-reflection similarity transformation is a special type of affine transformation. For any point (x, y) in the effective information region of an image, it can be represented as a combination of rotation, translation, and uniform scaling, and its formula is: ; in,( () are the transformed coordinates. It is the rotation angle, and s is the scaling factor. ) is a translation vector. The transformation matrix is ​​represented as: ; in , This ensures that the scaling factor is non-negative.

[0061] The transformation matrix is ​​obtained by solving for the parameters based on feature point pairs. Assume that from... Si and I Matching N pairs of feature points in 0 The goal is to find the optimal one. This makes the transformed point as close as possible to the target point: .

[0062] For each pair of feature points, write two equations: ; Stack the equations of all point pairs into a matrix form: ; Solving parameters using the least squares method : .

[0063] From the optimized parameter matrix Solve the transformation matrix : ; wherein , encoding rotation and scaling, , .

[0064] Based on the transformation matrix obtained from each image, the average inverse transformation matrix is calculated : ; wherein is the inverse matrix of S i to I 0 transformation. The average operation can suppress single image matching error and improve robustness.

[0065] The last image of the image sequence S S n The average inverse transformation matrix is applied to obtain the initial distortion correction result of the last image I n : ; Thus, the initial distortion correction image of the last image is obtained.

[0066] Further, in step S2 of the embodiment of the present application, the method of step S204 is specifically: Using a back-to-front strategy (since the images closer to the end of the sequence may have more stable atmospheric conditions when collected, and thus have more stable distortion characteristics, using the correction result as a visual reference to remove abnormal images can make the correction result more accurate), repeating step S203, the first to the second-to-last image, i.e., the first to the 99th overlapping non-saturated effective image information region block in the preprocessed image sequence S, is subjected to distortion correction to construct the distortion correction result of the second-to-last degraded image in the initial distortion correction sequence Y, i.e., the 99th, and then sequentially cycle down until the distortion correction results of all degraded images in the sequence are obtained and stacked to form the complete initial distortion correction image sequence Y, as shown in Figure 4 .

[0067] Further, in step S2 of the embodiment of the present application, the method of step S205 is specifically: According to the initial distortion correction result, the median pixel value prior image is updated to I 0', and the reference I0' pairs the feature points of the image sequence Y after the initial distortion correction again, calculates the geometric transformation of the non-saturated area overlapped by the first image to the last image in the image sequence Y after the initial distortion correction through the non-reflection similarity transformation, applies the average inverse transformation matrix to the last image, newly creates an empty sequence Q for storing the images after the final distortion correction, and constructs the last image I after the final distortion correction in the sequence Q n .

[0068] Further, in step S2 of the embodiment of the present application, the method of step S206 is specifically: The step S205 is repeated by adopting the strategy from back to front, the last second image is corrected for distortion based on the non-saturated effective image information area block overlapped by the first image to the last second image in the image sequence after the initial distortion correction, the last second image after the final distortion correction is constructed in the sequence Q, and the cycle is repeated in turn until the final distortion correction result of all the degraded images in the sequence is obtained, and the complete image sequence Q after the final distortion correction is obtained, as shown in Figure 5 .

[0069] Further, in step S3 of the embodiment of the present application, the method of step S301 is specifically: The image sequence Q after the final distortion correction is normalized, the pixel values in all the images are normalized from [0, 255] to [0, 1], and the normalized image sequence R is obtained after processing; the local contrast and brightness features of the images are calculated, so as to generate the weight image sequence E; the generated weight image is guided and filtered to obtain the optimized weight image sequence W.

[0070] As shown in Figure 8 , the specific process is as follows: The images in the image sequence Q Q i are normalized to obtain the images in the normalized image sequence R R i : .

[0071] The local contrast and brightness features are calculated to obtain the weight image. In this embodiment, the SIFT operator is adopted to calculate the local contrast of the image , and the formula is as follows: .

[0072] The contrast weight term is calculated : .

[0073] The brightness feature weight term is calculated Here, we normalize the pixel values of the image in the range [0.1, 0.9] as well-exposed pixels, and the brightness feature weight is set to 1, as follows: .

[0074] When taking a picture with a traditional camera, some areas appear dark in the image, i.e. these areas are underexposed, and some areas appear bright; i.e. overexposed. The basic idea of the brightness feature is to avoid these areas and only take regions from the well-exposed source image. The well-exposed range in this method is 0.1 to 0.9. Any pixel value in the range of 0.1 to 0.9 is considered as a well-exposed pixel, and for these pixels, the brightness feature is set to 1.

[0075] The weight image is calculated according to the computed contrast weight and the brightness feature weight To make the sum of the weight image at each pixel equal to 1, the weight image needs to be normalized to , as follows: ; ; where is a small positive number to prevent the denominator from being zero, for example, 10 -25 .

[0076] The generated weight image is guided filtered to get the optimized weight image W i : ; where is the size of the local window, i.e. the total number of pixels in the window; is the variance of in the local window, which measures the degree of variation of pixel values in the window; is a small positive regularization constant to avoid the denominator being zero and to control the degree of smoothing; is the mean of in the local window; is the mean of in the local window, which is the same as

[0077] Further, in step S3 of the embodiment of the present application, the method of step S302 is specifically: Gaussian filtering and down-sampling are performed on the two images to be fused to obtain the Gaussian pyramids of the two images , ​Then, adaptive edge-aware guided filtering is performed on the images of different scales of the two Gaussian pyramids to obtain two filtered Gaussian pyramids 、 The weight image corresponding to the image to be fused is subjected to Gaussian filtering and down-sampling to obtain a Gaussian pyramid of the weight image 、 The corresponding scale images of the Gaussian pyramid of the image to be fused and the Gaussian pyramid of the weight image are multiplied pixel by pixel to obtain two Gaussian pyramids containing weight information 、 The same scale images of the two Gaussian pyramids are added to form a new Gaussian pyramid The new Gaussian pyramid is successively up-sampled and accumulated to fuse a new image from the coarsest scale to the finest scale F 1and the pixel value of the image is normalized to [0, 255] to obtain a first fused image I 1.

[0078] The specific process is as follows: The two images to be fused are subjected to Q 1、 Q 2 Gaussian filtering and down-sampling to construct a Gaussian pyramid 、 , l =0,1,…, L denote the number of pyramid layers (n=0 is the finest scale, n=0 is the coarsest scale): l l L ; ; wherein, m 、 n are the relative coordinates in the kernel; k is the Gaussian kernel radius; is the Gaussian kernel weight, calculated by ; is the standard deviation of the Gaussian distribution, controlling the smoothing strength;2 x ,2 y control the down-sampling step (the size of each layer is halved).

[0079] Then, adaptive edge-aware guided filtering is performed on the images of each layer of the two Gaussian pyramids to obtain filtered pyramid images 、 : ; wherein, r represents the local window radius, is a regularization parameter, is the input filtered image,​​ representative guided filter .

[0080] weight image of the corresponding to-be-fused image W 1 、W 2 、 : ; wherein, m 、 n is the relative coordinate in the kernel; k is the Gaussian kernel radius; is the Gaussian kernel weight, calculated by ; is the standard deviation of the Gaussian distribution, controlling the smoothing strength;2 x 、2 y control the downsampling step (half size of each layer).

[0081] the filtered pyramid image 、 and the Gaussian pyramid of the weight image 、 corresponding scale image is pixel by pixel multiplied to obtain two Gaussian pyramids containing weight information 、 , and the two Gaussian pyramids are added to the same scale image to form a new Gaussian pyramid : .

[0082] The new Gaussian pyramid is up-sampled and accumulated from the coarsest scale l =L to the finest scale l =0, and a new image is fused. F 1, initialization is performed first (coarsest scale): .

[0083] Then, layer-by-layer up-sampling and accumulation are performed: ; wherein, Up() is an up-sampling operation (such as bilinear interpolation), which expands the image size by 2 times.

[0084] The result of the final fused image is obtained: .

[0085] The pixel value of the fused image F 1 is normalized to [0, 255] to obtainI 1 is the first image to be fused.

[0086] Further, in step S3 of the embodiment of the present application, the method of step S303 is specifically: According to the strategy from front to back (fusion from front to back, following the original order of image acquisition, making the fusion process more in line with the logic of data generation), the obtained fused image I 1 is set as the first image to be fused in a new round of fusion, and the third image of the sequence Q of distortion final correction images is set as the second image to be fused in a new round of fusion, and fusion is performed to obtain a new fused image. Repeat this process until all images of the sequence Q are fused into one corrected image I f As shown in the following figure, the iterative fusion process is as follows: Figure 7 Wherein, I is the fused image, Fuse represents the image fusion process, I I f is the final corrected image.

[0087] Figure 6 A clear image provided by the embodiment of the present application can be seen, and the corrected image obtained by the method of the present application can clearly see the information in Figure 6 .

[0088] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0089] The sequence image distortion and severe thermal radiation effect fusion correction method of the embodiment of the present application described in combination Figure 1 with the above can be realized by a computer device. Figure 9 The figure is a hardware structure schematic diagram of the computer device of the embodiment of the present application. As shown in the figure, the device can include a processor 301 and a memory 302 storing computer program instructions. Figure 9

[0090] Specifically, the above processor 301 can include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or can be configured to implement one or more integrated circuits of the embodiment of the present application.

[0091] ​​​​The memory 302 can include a mass storage for data or instructions. By way of example, and without limitation, the memory 302 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a Digital Versatile Disk (DVD), a Blu-Ray, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 302 can be removable and / or non-removable (or fixed) as appropriate. The memory 302 can be internal or external as appropriate. In certain embodiments, the memory 302 is a Non-Volatile Memory. In certain embodiments, the memory 302 includes a Read-Only Memory (ROM) and a Random-Access Memory (RAM). The ROM can be a Mask-Programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended Data Output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0092] The memory 302 can be used to store or buffer various data files required for processing and / or communication, and possible computer program instructions executed by the processor 301.

[0093] The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any one of the sequence image distortion and severe thermal radiation effect fusion correction methods in the above embodiments.

[0094] In some embodiments, the computer device can further include a communication interface 303 and a bus 300. As shown, the processor 301, the memory 302, and the communication interface 303 are connected through the bus 300 and complete communication with each other. Figure 9

[0095] The communication interface 303 is used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 303 can also realize data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations, etc.

[0096] ​Bus 300 includes hardware, software, or both, to couple components of the computer device to each other and to couple components to other components in the environment. Bus 300 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, a local bus, etc. By way of example and not limitation, bus 300 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, bus 300 can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0097] The computer device can execute the sequence image distortion and severe thermal radiation effect fusion correction method in the embodiments of the present application, thereby realizing the combination of Figure 1 The sequence image distortion and severe thermal radiation effect fusion correction method is described.

[0098] In addition, in combination with the sequence image distortion and severe thermal radiation effect fusion correction method in the above embodiments, the embodiments of the present application can provide a computer readable storage medium for implementation. The computer readable storage medium stores computer program instructions; the computer program instructions are executed by a processor to implement any one of the sequence image distortion and severe thermal radiation effect fusion correction methods in the above embodiments.

[0099] It should be noted that the technical features of the above embodiments can be combined in any manner. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure. In addition, according to the needs of implementation, each step / component described in the present application can be split into more steps / components, or two or more steps / components or parts of the operation of the steps / components can be combined into a new step / component, to achieve the purpose of the present application.

[0100] Those skilled in the art will readily understand that the above embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of protection of the present application patent should be subject to the appended claims.

Claims

1. A method for fusion correction of image distortion and severe thermal radiation effects in a sequence, characterized in that, The method includes: S1. Obtain image sequences containing varying degrees of atmospheric turbulence distortion and severe thermal radiation degradation, and perform preliminary thermal radiation processing on each image in the image sequence to obtain a preprocessed image sequence; S2. Perform distortion correction on the preprocessed image sequence to obtain a distortion-corrected image sequence; S3. Normalize the distortion-corrected image sequence, generate a weighted image sequence by calculating local contrast and brightness features, and fuse the distortion-corrected image sequence and the weighted image sequence into a single corrected image using a front-to-back strategy. Specifically, this includes: S301. Normalize the distortion-corrected image sequence and calculate the local contrast and brightness features of each image. Calculate the contrast weight term of each image based on the local contrast features of all images. Use the brightness features of each image as the brightness weight term. Generate a weighted image sequence based on the contrast weight term and brightness weight term of each image. S302. Perform Gaussian filtering and downsampling on the first and second images of the distortion-corrected image sequence to obtain Gaussian pyramids for the two images. Perform Gaussian filtering and downsampling on the corresponding weight images for the two images to obtain Gaussian pyramids for the two weight images. Multiply the Gaussian pyramids of the two images and the corresponding weight images pixel by pixel to obtain two Gaussian pyramids containing weight information. Add the two Gaussian pyramids containing weight information at the same scale to form a new Gaussian pyramid. Restore the new Gaussian pyramid to the original image to obtain the first fused image. At the same time, add the Gaussian pyramids of the two weight images at the same scale to obtain the Gaussian pyramid of the weight image corresponding to the fused image. S303. Take the current fused image and the next image in the distortion correction image sequence as the two images to be fused, and fuse them according to the same fusion logic to obtain the Gaussian pyramid of the next fused image and its corresponding weight image; continue to use this front-to-back strategy until all images in the distortion correction image sequence are fused into a single correction image.

2. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 1, characterized in that, Step S1 specifically includes: S101. Perform weighted least squares filtering on each image in the image sequence; S102. Substitute the filtered image into the cubic spline surface equation to fit the thermal radiation effect bias field of each image. S103. Subtract the thermal radiation effect bias field of each image from the image to obtain the preprocessed image sequence.

3. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 1, characterized in that, Step S2 specifically includes: S201. Extract the unsaturated effective image information region block of the saturated region of severe thermal radiation in each image of the preprocessed image sequence, calculate the median pixel value of each pixel position in the unsaturated effective information region block of the preprocessed image sequence, and obtain the median pixel value prior image. S202. Determine the overlapping region of the non-saturated effective image information region blocks from the first image to the last image in the preprocessed image sequence, obtain the overlapping non-saturated effective image information region blocks from the first image to the last image in the preprocessed image sequence, and extract feature point pairs between the median pixel value prior image and the overlapping non-saturated effective image information region blocks from the first image to the last image in the preprocessed image sequence. S203. Based on the feature point pairs of the overlapping unsaturated effective image information region blocks of the median pixel value prior image and the first to last images in the preprocessed image sequence, determine the inverse transformation matrix from the first to the last images in the preprocessed image sequence to the median pixel value prior image, calculate the average inverse transformation matrix, and perform distortion correction on the last image in the preprocessed image sequence based on the average inverse transformation matrix. S204. Determine the overlapping area of ​​the non-saturated effective image information region blocks from the first image to the second-to-last image in the preprocessed image sequence, and perform distortion correction on the second-to-last image according to the same distortion correction logic; continue to use this back-to-foreign strategy to complete the distortion correction of the third-to-last image to the first image in sequence, and finally obtain the distortion-corrected image sequence.

4. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 3, characterized in that, In step S201, the heavily saturated thermal radiation region and the unsaturated effective image information region block of each image in the preprocessed image sequence are extracted, including: The preprocessed image sequence is normalized to obtain a normalized image sequence; In the normalized image sequence, the region where the pixel gray value of each image pixel is greater than the preset gray value threshold is regarded as a region of severe thermal radiation saturation, and other regions are regarded as non-saturated effective image information region blocks. The non-saturated effective image information region blocks of each image are extracted from the preprocessed image sequence.

5. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 3 or 4, characterized in that, Step S2 also includes: S205. Update the median pixel value prior image based on the distortion-corrected image sequence; S206. Based on the updated median pixel value prior image, the distortion-corrected image sequence is distorted again to obtain the final distortion-corrected image sequence.

6. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 1, characterized in that, Step S301 specifically includes: Each image in the distortion-corrected image sequence Q is normalized to obtain the normalized distortion-corrected image sequence R; Calculate the i-th image in the normalized distortion-corrected image sequence R. R i The local contrast characteristics are expressed by the following formula: ； In the formula, x and y These are pixel coordinates; It represents local contrast features; SIFT is the SIFT operator; It is an L1 norm; The contrast weight term for the i-th image is calculated based on the local contrast features of all images. : ； In the formula, n is the total number of images in the image sequence; Calculate the brightness features of the i-th image as a brightness weight term. : ； In the formula, For brightness parameters; Calculate the weighted image of the i-th image based on the contrast and brightness weights of the i-th image. : ； For weighted images Normalize: ； In the formula, It should be a very small positive number to prevent the denominator from being 0; The normalized weighted image is used to obtain the weighted image sequence E.

7. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 6, characterized in that, Step S301 further includes: The weighted image sequence is optimized by guided filtering to obtain the optimized weighted image sequence.

8. The method for fusion correction of image distortion and severe thermal radiation effects according to claim 1, characterized in that, Step S302 specifically includes: The first image of the distortion-corrected image sequence Q. Q 1 and the second image Q 2. Perform Gaussian filtering and downsampling to construct a Gaussian pyramid for the two images. and : ； In the formula, m , n Relative coordinates within the kernel; k The radius of the Gaussian kernel; For Gaussian kernel weights, ,in The standard deviation is the Gaussian distribution. x and y For pixel coordinates, To reduce the sampling factor; l =0, 1, ..., L Indicates the number of pyramid levels. L The maximum number of floors. Indicates the first l Layer image, layer 0 image The original image is Q; Adaptive edge-aware guided filtering is applied to each layer of the Gaussian pyramid from both images to obtain the filtered pyramid image. and : ； In the formula, This is the filtered pyramid image; r Represents the local window radius. For regularization parameters, The input image; Indicates guided filtering; Similarly, the weight images corresponding to the two images are... W 1 and W 2. Perform Gaussian filtering and downsampling to obtain the Gaussian pyramids of the two weighted images. and ; The Gaussian pyramids of the two filtered images and the corresponding Gaussian pyramids of the weighted images are multiplied pixel-by-pixel to obtain two Gaussian pyramids containing weight information. and Two Gaussian pyramids containing weight information are added together at the same scale to form a new Gaussian pyramid. : ； The new Gauss Pyramid From the coarsest scale l = L To the finest scale l =0 are upsampled and accumulated sequentially to obtain a new image, and the image pixel values ​​are normalized to [0, 255] to obtain the first fused image; Simultaneously, the Gaussian pyramids of the two weighted images are added together at the same scale to obtain the Gaussian pyramid of the weighted image corresponding to the fused image.

9. A computer device, characterized in that, include: A processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, implement the steps of the sequential image distortion and severe thermal radiation effect fusion correction method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores a program or instructions that, when executed by a processor, implement the steps of the sequential image distortion and severe thermal radiation effect fusion correction method according to any one of claims 1 to 8.

Images