SAR image fusion method, device and equipment in terahertz frequency band beam mode

Abstract

The application relates to a SAR image fusion method, device and equipment in a terahertz frequency band focusing mode. After the terahertz focusing mode video is preprocessed, the reference image and the image to be fused are projected to corresponding projection images on the ground plane by combining radar imaging digital elevation model linear interpolation, an anisotropic diffusion scale space image is constructed for the two types of projection images, a Harris function scale space image is generated according to the extreme value points, a SIFT feature descriptor of the two types of images is constructed based on the extreme value points, an initial feature matching point pair set is obtained through similarity comparison, an optimized point pair set is obtained after multiple screenings, an affine matrix is converted through the optimized point pair set, the image to be fused and the reference image are fused according to the matrix to obtain a fused image, the image is updated as the reference image, and the fusion is repeated with the next frame of image to be fused until all frames are fused to obtain a final fused image. The method can effectively improve the quality of the fused image.

Description

Technical Field

[0001] This application relates to the field of radar signal processing technology, and in particular to a SAR image fusion method, apparatus and equipment in terahertz band spotting mode. Background Technology

[0002] In radar imaging, to suppress speckle noise in SAR images and improve the signal-to-noise ratio, multiple SAR images of the same scene from the same imaging radar are typically synthesized into a single clear SAR image through incoherent overlay or image fusion. To obtain high-quality incoherently overlaid or fused images, the accuracy of the positional pairs of image feature points is crucial; even slight deviations can worsen the quality of the fused image.

[0003] Currently, there are many methods for SAR image stitching, registration, and fusion based on feature points, mainly based on Scale Invariant Feature Transform (SIFT) methods, such as the Synthetic Aperture Radar-Scale Invariant Feature Transform (SAR-SIFT) algorithm, the Position-Scale-Azimuth-Scale Invariant Feature Transform (PSO-SIFT) algorithm, and other improved SIFT algorithms. These methods are effective for stitching optical images or striped SAR images, but for terahertz band spot-mode SAR images with severe geometric deformation, registration, stitching, fusion, and image overlay will be a difficult problem. The main reason is that terahertz band video SAR images have a small imaging range, and the imaging quality of terahertz video SAR is greatly affected by radar platform motion errors and speckle. In particular, the angle between the radar and the center of the imaging scene changes during the imaging process, resulting in severe geometric deformation of the image. Therefore, when stitching, registering, and fusing terahertz band spot-mode SAR images of the same scene from the same radar, in addition to the misalignment of feature point pairs, image scaling and cropping also occur. In feature-point-based registration methods, Random Sample Consensus (RANSAC) and its Fast Sample Consensus (FSC) are used for affine matrix parameter estimation in image transformation. This algorithm is uncertain; the parameters obtained are affected by the number of iterations, and the result may not be optimal. Summary of the Invention

[0004] Therefore, it is necessary to provide a SAR image fusion method, apparatus, and device in terahertz band spotting mode that can effectively improve the quality of fused images, addressing the aforementioned technical problems.

[0005] A SAR image fusion method in terahertz band spotting mode, the method comprising:

[0006] Acquire SAR video in terahertz band spotting mode, preprocess each frame of SAR image in the SAR video, select one frame of SAR image as reference image, and then use the other SAR images as images to be fused.

[0007] A digital elevation model based on radar imaging projects a reference image and an image to be fused onto the ground plane using linear interpolation, resulting in a reference projected image and an image to be fused.

[0008] After Gaussian smoothing is applied to the reference projection image and the projection image to be fused respectively, an anisotropic diffusion scale space image is constructed. Based on the anisotropic diffusion scale space image, a Harris scale space is constructed to generate a Harris function scale space image. Then, extreme points are extracted from the Harris function scale space image.

[0009] SIFT feature descriptors for the reference projection image and the projection image to be fused are constructed based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial set of feature matching point pairs for the reference projection image and the projection image to be fused is obtained. The initial set of feature matching point pairs is then filtered multiple times to obtain an optimized set of feature matching point pairs.

[0010] Based on the optimized feature matching point pair set, the image transformation affine matrix is ​​calculated by least squares matching. The image to be fused and the reference image are fused according to the image transformation affine matrix to obtain the fused image. The fused image is then used as the reference image and fused with the next frame image to be fused until the fusion of all images to be fused is completed, thus obtaining the final SAR fused image.

[0011] In one embodiment, the SAR video includes two or more SAR images.

[0012] In one embodiment, constructing anisotropic diffusion-scale spatial images corresponding to the reference projection image and the projection image to be fused, respectively, includes:

[0013] The projected image filtered by the Gaussian window function is used as the first layer image in the anisotropic diffusion scale space.

[0014] After applying Gaussian filtering to the first layer image, a smooth image is obtained. The first-order gradients in the x and y directions of the smooth image are calculated, and the contrast factor is obtained based on the first-order gradients. The conduction coefficient is then obtained by solving the diffusion equation. The second layer image is obtained by splitting the image using the additive operator AOS algorithm. The next layer image is then obtained based on the second layer image. Through continuous iterative calculation, each layer image of the anisotropic diffusion scale space image is obtained.

[0015] In one embodiment, the diffusion equation is expressed as:

[0016] ;

[0017] In the above formula, Represents the transmission function. Represents a grayscale image. express gradient, The scale parameter is represented by the transfer function:

[0018] ;

[0019] In the above formula, Represents a Gaussian smoothed image. express The gradient of, where, The function is represented as:

[0020] ;

[0021] In the above formula, This represents the contrast factor.

[0022] In one embodiment, the initial set of feature matching point pairs is filtered multiple times to obtain an optimized set of feature matching point pairs, including:

[0023] By comparing the position, scale, number of layers, and angle of the initial feature matching point pairs, duplicate feature matching point pairs in the initial feature matching point pair set are deleted, thus completing the first screening.

[0024] In the initial set of feature matching point pairs after the first screening, three feature point pairs are randomly selected, and the corresponding image affine matrix is ​​constructed by least squares matching. Based on the image affine matrix, the interior points of the matching feature point pairs are screened, and the exterior points are removed to complete the second screening.

[0025] In the initial set of feature matching point pairs after the second screening, a final screening is performed by calculating the Euclidean distance between the matching feature point pairs according to a preset distance threshold, to obtain the optimized set of feature matching point pairs.

[0026] In one embodiment, after obtaining the final SAR fused image, it is also smoothed using a Gaussian window.

[0027] This application also provides a SAR image fusion device in terahertz band spotting mode, the device comprising:

[0028] The SAR video acquisition module is used to acquire SAR video in terahertz band spotting mode. After preprocessing each frame of SAR image in the SAR video, one frame of SAR image is selected as the reference image, and the other SAR images are used as images to be fused.

[0029] The image projection module is used for radar imaging-based digital elevation models. It projects the reference image and the image to be fused onto the ground plane using linear interpolation to obtain the reference projected image and the projected image to be fused.

[0030] The extreme point extraction module is used to perform Gaussian smoothing on the reference projection image and the projection image to be fused respectively, construct the corresponding anisotropic diffusion scale space image, and construct the Harris scale space based on the anisotropic diffusion scale space image to generate the Harris function scale space image, and then extract extreme points in the Harris function scale space image.

[0031] The module for optimizing the feature matching point pair set is used to construct SIFT feature descriptors for the reference projection image and the projection image to be fused based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial feature matching point pair set for the reference projection image and the projection image to be fused is obtained. The initial feature matching point pairs are then filtered multiple times to obtain an optimized feature matching point pair set.

[0032] The final SAR fusion image acquisition module is used to calculate the image transformation affine matrix based on the optimized feature matching point pair set through least squares matching, fuse the image to be fused and the reference image according to the image transformation affine matrix to obtain the fused image, and use the fused image as the reference image to fuse with the next frame of the image to be fused, until the fusion of all the images to be fused is completed, and then the final SAR fusion image is obtained.

[0033] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the SAR image fusion method in the terahertz band spotting mode described above.

[0034] The aforementioned SAR image fusion method, apparatus, and equipment in the terahertz band spotting mode preprocesses the terahertz spotting mode SAR video, selects one frame as the reference image, and the rest as images to be fused. Using a radar imaging digital elevation model and linear interpolation, the reference and images to be fused are projected onto the ground plane to obtain corresponding projected images. Both types of projected images are Gaussian smoothed to construct an anisotropic diffusion scale spatial image, which in turn generates a Harris function scale spatial image. Extreme points are extracted, and SIFT feature descriptors for both types of images are constructed based on these extreme points. An initial set of feature matching point pairs is obtained through similarity comparison. After multiple filtering steps, an optimized set of point pairs is obtained. Using the optimized set, the image transformation affine matrix is ​​calculated using least-squares matching. The images to be fused and the reference image are fused according to the matrix to obtain the fused image. This fused image is then updated as the reference image, and the fusion process is repeated with the next frame of the image to be fused until all frames are fused, resulting in the final SAR fused image. This method can solve the problems of misalignment, image scaling and cropping caused by geometric deformation, and poor image quality caused by speckle in terahertz band video SAR images during image fusion, thereby improving the quality of fused images. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a SAR image fusion method in terahertz band spotting mode in one embodiment.

[0036] Figure 2 This is a schematic diagram illustrating the specific process of a SAR image fusion method in terahertz band spotting mode in one embodiment.

[0037] Figure 3 This is a schematic diagram of a spotlight mode SAR experiment, where... Figure 3 (a) represents the reference image. Figure 3 (b) represents the images to be fused;

[0038] Figure 4 This is a projection of a SAR image onto the ground plane in an experiment. Figure 4 (a) represents the reference projected image. Figure 4 (b) The projected images to be fused;

[0039] Figure 5 This is an optical schematic diagram of a sundial used in an experiment to visualize an image.

[0040] Figure 6 This is a schematic diagram of the overlay used in an experiment with the SAR-SIFT algorithm;

[0041] Figure 7 This is a schematic diagram of a fused image obtained using the method described in this paper in an experiment;

[0042] Figure 8This is a structural block diagram of a SAR image fusion device in terahertz band spotting mode in one embodiment;

[0043] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] When fusing terahertz band spotlight mode video SAR images, significant image quality issues arise when using traditional image stitching, registration, and fusion algorithms such as SAR-SIFT and PSO-SIFT. From an imaging principle perspective, these images to be fused are all generated by the same terahertz video SAR illuminating the same target and using advanced imaging algorithms. Theoretically, only translation and planar rotation geometric transformations should exist between the images. However, in actual fusion, the affine matrix calculated through feature point pair registration additionally includes image scaling and shearing components, directly leading to a significant decrease in the quality of the fused image. Further experimental verification revealed that even after processing with the Random Sample Consensus (RANSAC) algorithm, mismatches still remain in the filtered feature point pairs, indicating that the accuracy of traditional registration algorithms is insufficient to meet the high-quality fusion requirements of terahertz band spotlight mode video SAR images.

[0046] By delving into the underlying reasons, it can be found that, on the one hand, terahertz video SAR images in spotlight mode are affected by speckle noise, which reduces the accuracy of feature extraction; on the other hand, the difference in the angle between the imaging line of sight and the target surface will cause the imaging position of the target to shift in different frames, thereby causing geometric deformation of the image. These two factors together make it difficult for traditional algorithms to achieve accurate registration, ultimately affecting the fusion effect.

[0047] In view of the above-mentioned technical problems, in this application, such as Figure 1 As shown, a SAR image fusion method in terahertz band spotting mode is provided, which specifically includes the following steps:

[0048] Step S100: Obtain SAR video in terahertz band spotting mode. After preprocessing each frame of SAR image in the SAR video, select one frame of SAR image as the reference image, and then use the other SAR images as images to be fused.

[0049] Step S110: Based on the digital elevation model of radar imaging, the reference image and the image to be fused are projected onto the ground plane using a linear interpolation method to obtain the reference projected image and the projected image to be fused.

[0050] Step S120: After Gaussian smoothing of the reference projection image and the projection image to be fused, the corresponding anisotropic diffusion scale space image is constructed. Based on the anisotropic diffusion scale space image, the Harris scale space is constructed to generate the Harris function scale space image. Then, the extreme points are extracted from the Harris function scale space image.

[0051] Step S130: Construct SIFT feature descriptors for the reference projection image and the projection image to be fused based on the extreme points. By comparing the similarity of the SIFT feature descriptors, obtain the initial feature matching point pair set for the reference projection image and the projection image to be fused. Then, perform multiple screenings on the initial feature matching point pairs to obtain the optimized feature matching point pair set.

[0052] Step S140: Based on the optimized feature matching point pair set, the image transformation affine matrix is ​​calculated by least squares matching. The image to be fused and the reference image are fused according to the image transformation affine matrix to obtain the fused image. The fused image is then used as the reference image and fused with the next frame image to be fused until the fusion of all images to be fused is completed, thus obtaining the final SAR fused image.

[0053] In step S100, the SAR video includes two or more SAR images.

[0054] In this embodiment, when performing SAR image fusion, multiple images to be fused are fused sequentially according to time order based on a reference image.

[0055] In step S110, based on the digital elevation model (DEM) of the radar imaging, which contains elevation information of the target area's ground surface, for each pixel in the reference image and the image to be fused, combining its position in the radar coordinate system with the elevation data corresponding to the DEM, the corresponding position of the pixel in the ground plane coordinate system is calculated through linear interpolation. This pixel-by-pixel transformation is performed to map the reference image and the image to be fused onto the ground plane, ultimately obtaining the reference projection image and the projection image to be fused.

[0056] In step S120, constructing anisotropic diffusion scale space images corresponding to the reference projection image and the projection image to be fused includes: taking the projection image filtered by a Gaussian window function as the first layer image of the anisotropic diffusion scale space; then applying Gaussian filtering to the first layer image to obtain a smoothed image; calculating the first-order gradients in the x and y directions of the smoothed image; obtaining the contrast factor based on the first-order gradients; then solving the diffusion equation to obtain the conduction coefficient; obtaining the second layer image through the additive operator splitting AOS algorithm; and then obtaining the next layer image based on the second layer image. Through continuous iterative calculation, each layer image of the anisotropic diffusion scale space image is obtained, thereby obtaining the anisotropic diffusion scale space images corresponding to the reference projection image and the projection image to be fused, respectively.

[0057] In this embodiment, the diffusion equation is expressed as:

[0058] ;

[0059] In the above formula, Represents the transmission function. Represents a grayscale image. express gradient, The scale parameter is represented by the transfer function:

[0060] ;

[0061] In the above formula, Represents a Gaussian smoothed image. express gradient, Represents the contrast factor, where, The function is represented as:

[0062] .

[0063] In this embodiment, based on the obtained anisotropic diffusion scale space image, the Harris response function is calculated at different scale layers to construct a multi-scale Harris scale space. Based on this scale space, a Harris function scale space image containing Harris response values ​​at each scale is generated. Finally, in this function image, the response values ​​of each pixel are compared with those of its neighboring pixels, and the pixels with the largest or smallest response values ​​are selected as the extreme points to be extracted. At the same time, the principal direction of each extreme point is also calculated.

[0064] In step S130, neighborhood pixels are first taken centered on the extreme points of the reference projection image and the projection image to be fused. After aligning them according to the main direction, gradient information is calculated to generate SIFT feature descriptors. Next, the similarity (e.g., Euclidean distance) between each SIFT descriptor of the reference projection image and all SIFT descriptors of the projection image to be fused is calculated. The descriptor pairs with the highest similarity are selected and correspondingly form initial feature matching point pairs. Finally, the initial feature matching point pair sets of the two types of images are obtained. Then, each feature matching point pair in the initial feature matching point pair set is further filtered.

[0065] In this embodiment, the initial set of feature matching point pairs is filtered multiple times to obtain an optimized set of feature matching point pairs. This includes: comparing the position, scale, layer number, and angle of the initial set of feature matching point pairs to remove duplicate feature matching point pairs, completing the first filtering. From the initial set of feature matching point pairs after the first filtering, three feature point pairs are randomly selected, and corresponding image affine matrices are constructed using least-squares matching. Based on the image affine matrices, interior points of the matching feature point pairs are filtered, and outliers are removed, completing the second filtering. From the initial set of feature matching point pairs after the second filtering, a final filtering is performed based on a preset distance threshold, calculating the Euclidean distance between the matching feature point pairs to obtain the optimized set of feature matching point pairs.

[0066] Specifically, in the first screening process, the position, scale, layer, and angle of the initial feature matching point pairs are used as the criteria for judgment, and the point pair set is checked one by one. If there are two or more point pairs, the position coordinates, scale layer, and angle parameters of the corresponding points in the reference projection image and the projection image to be fused are completely consistent (i.e., duplicate records), then one valid point pair is retained, and the remaining duplicates are deleted, completing the initial cleaning of redundant point pairs.

[0067] Specifically, in the second screening process, three sets of non-collinear feature point pairs are randomly selected from the point pair set after the first screening. An image affine matrix is ​​constructed using least squares matching (LSM). The reference image coordinates of all remaining point pairs are substituted into this matrix to calculate the theoretical coordinates of the image to be fused. The deviation between the theoretical coordinates and the actual matching coordinates is then compared. Points with deviations within a set range are considered interior points (i.e., points conforming to the geometric transformation rules), while those with deviations exceeding the range are considered exterior points (i.e., incorrectly matched points). Only interior points are retained for the next round. After repeating the second screening process multiple times, the second round of erroneous point pair removal is completed.

[0068] Specifically, in the third screening process, for the set of interior points after the second screening, a preset distance threshold is first set, and the Euclidean distance between each pair of matching feature points is calculated. If the distance is less than or equal to the preset threshold, it indicates that the two point pairs have high similarity and a reliable match, and they are retained. If the distance exceeds the threshold, they are judged as low-reliability matching point pairs and are directly deleted. The final set of retained point pairs is the optimized feature matching point pair set.

[0069] In step S140, after obtaining the final SAR fused image, it is also smoothed using a Gaussian window. A 3×3 Gaussian window smoothing process is applied to the final fusion result to eliminate edge noise and artifacts, resulting in the final SAR fused image.

[0070] like Figure 2 The diagram shown is a complete flowchart of this method.

[0071] This paper also demonstrates the effectiveness of the proposed method through experimental results. In the experiments, the spotlight SAR images were obtained from terahertz video SAR imaging, and the parameters of the imaging scene are shown in Table 1. The experimental procedure is as follows: Figure 2 As shown.

[0072] Table 1. Parameters of the measured data imaging

[0073]

[0074] Figure 3 These are two selected terahertz spotlight SAR images. Both SAR images are generated by the same terahertz video SAR, and the resulting image is of a circular sundial on the ground. The radar is at a certain angle to the ground, resulting in an elliptical image. The images contain speckle patterns, and the Arabic characters on the inner side of the ellipse are not clear. The angle between the target and the radar's line of sight is different in the different images.

[0075] Figure 4 This is the projected image after the generated image is projected onto the ground plane. The projected image restores the circular shape of the ground and adds frequency labels to the coordinate axes. The projected image still suffers from severe speckle, making it unclear.

[0076] Figure 5 An optical image of a sundial in an imaging scene. Figure 5 There is a sundial, steps around the sundial, and greenery with Roman numeral characters.

[0077] Figure 6 To illustrate the image fusion using the SAR-SIFT algorithm, overlap marks from the image fusion process are visible. Using the first frame as the reference frame and the second frame as the frame to be fused, the resulting affine matrix after fusing the projected image of the second frame with the projected image of the first reference frame is:

[0078] ;

[0079] The parameters of the affine matrix indicate image scaling and cropping, suggesting geometric distortion or mismatched feature point pairs in the target image. After Fast Sample Consistency (FSC) to remove mismatches, 19 feature point pairs remain. The image entropy of the fused image is 12.474536.

[0080] Figure 7 This is the fused projection image obtained using this method. Using the first frame as the reference frame and the second frame as the frame to be fused, the affine matrix obtained after fusing the first frame (the projection image to be fused) with the reference frame is:

[0081] ;

[0082] Similarly, the fused image processed by this method did not show significant image scaling, with a rotation angle of 6.75° and a translation amount of [missing information]. After RANSAC removal of erroneous feature point pairs, there are 16 matching feature point pairs. The image entropy is 12.450894, and the structural similarity value is 0.47543.

[0083] and Figure 6 In contrast, the method proposed in this paper can clearly show the Roman characters around the sundial and the rows of green trees next to the sundial steps. Figure 5 The image entropy is 12.360673. Figure 6 The image entropy is 12.474536, indicating that... Figure 7 Image quality is better than Figure 6 Imaging quality.

[0084] In the aforementioned SAR image fusion method under terahertz band spotting mode, the images to be fused are projected onto the ground plane through interpolation to obtain a projected image. One frame is selected as the reference projected image, and the others are the projected images to be fused. Gaussian smoothing and anisotropic processing are applied to both the reference and projected images to be fused to construct a Harris scale space. Extremum points are extracted, and descriptors of the positions, histograms, and principal directions of the extremum points are constructed. Feature point pairs are selected based on the Euclidean distance between the descriptors of the feature points. The least squares matching LSM algorithm is used to select interior points that conform to the affine matrix and remove exterior points that do not meet the conditions. The positions of the selected feature point pairs are extracted, and erroneous feature point pairs are eliminated by calculating the Euclidean distance between the positions of the feature point pairs. The remaining feature point pairs are then processed using the least squares matching LSM algorithm to obtain the affine matrix of the image transformation. Image fusion is performed using the affine matrix of the image transformation to obtain a SAR image with high fusion quality. The fused image is then smoothed with a 3×3 Gaussian window to obtain the output image. This method projects the SAR image onto the ground imaging plane based on the spotlight SAR imaging scene, correcting the geometric distortion of the SAR image. By using the Euclidean distance between feature point pairs, erroneous feature point pairs are eliminated to obtain the correct affine matrix, thus improving the quality of the fused image. Simultaneously, this method constructs an anisotropic scale space, which, compared to the Gaussian gradient scale space, is better able to extract extreme points from the SAR image. Furthermore, by using Euclidean distance feature point filtering based on the position, histogram, and descriptor established by the principal direction of the feature point pairs, it avoids eliminating correct feature point pairs, retaining sufficient feature point pairs for subsequent matching.

[0085] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0086] In one embodiment, such as Figure 8 As shown, a SAR image fusion device in terahertz band spotting mode is provided, comprising: a SAR video acquisition module 200, an image projection module 210, an extreme point extraction module 220, an optimized feature matching point pair set acquisition module 230, and a final SAR fused image acquisition module 240, wherein:

[0087] The SAR video acquisition module 200 is used to acquire SAR video in terahertz band spotting mode. After preprocessing each frame of SAR image in the SAR video, one frame of SAR image is selected as the reference image, and the other SAR images are used as images to be fused.

[0088] Image projection module 210 is used for a digital elevation model based on radar imaging. It projects a reference image and an image to be fused onto a ground plane using a linear interpolation method to obtain a reference projected image and an image to be fused.

[0089] The extreme point extraction module 220 is used to perform Gaussian smoothing on the reference projection image and the projection image to be fused respectively, construct the corresponding anisotropic diffusion scale space image, and construct the Harris scale space based on the anisotropic diffusion scale space image to generate the Harris function scale space image, and then extract extreme points in the Harris function scale space image.

[0090] The optimized feature matching point pair set is obtained by module 230, which is used to construct SIFT feature descriptors for the reference projection image and the projection image to be fused based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial feature matching point pair set for the reference projection image and the projection image to be fused is obtained. The initial feature matching point pairs are then filtered multiple times to obtain an optimized feature matching point pair set.

[0091] The final SAR fusion image acquisition module 240 is used to calculate the image transformation affine matrix based on the optimized feature matching point pair set by least squares matching, fuse the image to be fused and the reference image according to the image transformation affine matrix to obtain the fused image, and use the fused image as the reference image to fuse with the next frame of the image to be fused, until the fusion of all the images to be fused is completed, and then the final SAR fusion image is obtained.

[0092] Specific limitations regarding the SAR image fusion device in terahertz band spotting mode can be found in the above description of the SAR image fusion method in terahertz band spotting mode, and will not be repeated here. Each module in the aforementioned SAR image fusion device in terahertz band spotting mode can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0093] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a SAR image fusion method in a terahertz band spotting mode. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0094] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0095] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0096] Acquire SAR video in terahertz band spotting mode, preprocess each frame of SAR image in the SAR video, select one frame of SAR image as reference image, and then use the other SAR images as images to be fused.

[0097] A digital elevation model based on radar imaging projects a reference image and an image to be fused onto the ground plane using linear interpolation, resulting in a reference projected image and an image to be fused.

[0098] After Gaussian smoothing is applied to the reference projection image and the projection image to be fused respectively, an anisotropic diffusion scale space image is constructed. Based on the anisotropic diffusion scale space image, a Harris scale space is constructed to generate a Harris function scale space image. Then, extreme points are extracted from the Harris function scale space image.

[0099] SIFT feature descriptors for the reference projection image and the projection image to be fused are constructed based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial set of feature matching point pairs for the reference projection image and the projection image to be fused is obtained. The initial set of feature matching point pairs is then filtered multiple times to obtain an optimized set of feature matching point pairs.

[0100] Based on the optimized feature matching point pair set, the image transformation affine matrix is ​​calculated by least squares matching. The image to be fused and the reference image are fused according to the image transformation affine matrix to obtain the fused image. The fused image is then used as the reference image and fused with the next frame image to be fused until the fusion of all images to be fused is completed, thus obtaining the final SAR fused image.

[0101] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0102] Acquire SAR video in terahertz band spotting mode, preprocess each frame of SAR image in the SAR video, select one frame of SAR image as reference image, and then use the other SAR images as images to be fused.

[0103] A digital elevation model based on radar imaging projects a reference image and an image to be fused onto the ground plane using linear interpolation, resulting in a reference projected image and an image to be fused.

[0104] After Gaussian smoothing is applied to the reference projection image and the projection image to be fused respectively, an anisotropic diffusion scale space image is constructed. Based on the anisotropic diffusion scale space image, a Harris scale space is constructed to generate a Harris function scale space image. Then, extreme points are extracted from the Harris function scale space image.

[0105] SIFT feature descriptors for the reference projection image and the projection image to be fused are constructed based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial set of feature matching point pairs for the reference projection image and the projection image to be fused is obtained. The initial set of feature matching point pairs is then filtered multiple times to obtain an optimized set of feature matching point pairs.

[0106] Based on the optimized feature matching point pair set, the image transformation affine matrix is ​​calculated by least squares matching. The image to be fused and the reference image are fused according to the image transformation affine matrix to obtain the fused image. The fused image is then used as the reference image and fused with the next frame image to be fused until the fusion of all images to be fused is completed, thus obtaining the final SAR fused image.

[0107] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A SAR image fusion method in terahertz band spotting mode, characterized in that, The method includes: Acquire SAR video in terahertz band spotting mode, preprocess each frame of SAR image in the SAR video, select one frame of SAR image as reference image, and then use the other SAR images as images to be fused. A digital elevation model based on radar imaging projects a reference image and an image to be fused onto the ground plane using linear interpolation, resulting in a reference projected image and an image to be fused. After Gaussian smoothing is applied to the reference projection image and the projection image to be fused respectively, an anisotropic diffusion scale space image is constructed. Based on the anisotropic diffusion scale space image, a Harris scale space is constructed to generate a Harris function scale space image. Then, extreme points are extracted from the Harris function scale space image. SIFT feature descriptors for the reference projection image and the projection image to be fused are constructed based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial set of feature matching point pairs for the reference projection image and the projection image to be fused is obtained. The initial set of feature matching point pairs is then filtered multiple times to obtain an optimized set of feature matching point pairs. Based on the optimized feature matching point pair set, the image transformation affine matrix is ​​calculated by least squares matching. The image to be fused and the reference image are fused according to the image transformation affine matrix to obtain the fused image. The fused image is then used as the reference image and fused with the next frame image to be fused until the fusion of all images to be fused is completed, thus obtaining the final SAR fused image.

2. The SAR image fusion method in terahertz band spotting mode according to claim 1, characterized in that, The SAR video includes two or more SAR images.

3. The SAR image fusion method in terahertz band spotting mode according to claim 1, characterized in that, Constructing anisotropic diffusion-scale spatial images corresponding to the reference projection image and the projection image to be fused, respectively, includes: The projected image filtered by the Gaussian window function is used as the first layer image in the anisotropic diffusion scale space. After applying Gaussian filtering to the first layer image, a smooth image is obtained. The first-order gradients in the x and y directions of the smooth image are calculated, and the contrast factor is obtained based on the first-order gradients. The conduction coefficient is then obtained by solving the diffusion equation. The second layer image is obtained by splitting the image using the additive operator AOS algorithm. The next layer image is then obtained based on the second layer image. Through continuous iterative calculation, each layer image of the anisotropic diffusion scale space image is obtained.

4. The SAR image fusion method in terahertz band spotting mode according to claim 3, characterized in that, The diffusion equation is expressed as: ； In the above formula, Represents the transmission function. Represents a grayscale image. express gradient, The scale parameter is represented by the transfer function: ； In the above formula, Represents a Gaussian smoothed image. express The gradient of, where, The function is represented as: ； In the above formula, This represents the contrast factor.

5. The SAR image fusion method in terahertz band spotting mode according to any one of claims 1 to 4, characterized in that, The initial set of feature matching point pairs is filtered multiple times to obtain an optimized set of feature matching point pairs, including: By comparing the position, scale, number of layers, and angle of the initial feature matching point pairs, duplicate feature matching point pairs in the initial feature matching point pair set are deleted, thus completing the first screening. In the initial set of feature matching point pairs after the first screening, three feature point pairs are randomly selected, and the corresponding image affine matrix is ​​constructed by least squares matching. Based on the image affine matrix, the interior points of the matching feature point pairs are screened, and the exterior points are removed to complete the second screening. In the initial set of feature matching point pairs after the second screening, a final screening is performed by calculating the Euclidean distance between the matching feature point pairs according to a preset distance threshold, to obtain the optimized set of feature matching point pairs.

6. The SAR image fusion method in terahertz band spotting mode according to claim 5, characterized in that, After obtaining the final SAR fused image, it is also smoothed using a Gaussian window.

7. A SAR image fusion device in terahertz band spotting mode, characterized in that, The device includes: The SAR video acquisition module is used to acquire SAR video in terahertz band spotting mode. After preprocessing each frame of SAR image in the SAR video, one frame of SAR image is selected as the reference image, and the other SAR images are used as images to be fused. The image projection module is used for radar imaging-based digital elevation models. It projects the reference image and the image to be fused onto the ground plane using linear interpolation to obtain the reference projected image and the projected image to be fused. The extreme point extraction module is used to perform Gaussian smoothing on the reference projection image and the projection image to be fused respectively, construct the corresponding anisotropic diffusion scale space image, and construct the Harris scale space based on the anisotropic diffusion scale space image to generate the Harris function scale space image, and then extract extreme points in the Harris function scale space image. The module for optimizing the feature matching point pair set is used to construct SIFT feature descriptors for the reference projection image and the projection image to be fused based on the extreme points. By comparing the similarity of the SIFT feature descriptors, an initial feature matching point pair set for the reference projection image and the projection image to be fused is obtained. The initial feature matching point pairs are then filtered multiple times to obtain an optimized feature matching point pair set. The final SAR fusion image acquisition module is used to calculate the image transformation affine matrix based on the optimized feature matching point pair set through least squares matching, fuse the image to be fused and the reference image according to the image transformation affine matrix to obtain the fused image, and use the fused image as the reference image to fuse with the next frame of the image to be fused, until the fusion of all the images to be fused is completed, and then the final SAR fusion image is obtained.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images