Multispectral and visible image matching method based on cross-spectral anchoring and distortion logic

Abstract

The application relates to a multispectral and visible image matching method based on cross-spectrum anchoring and distortion logic, and relates to the field of remote sensing image processing. The application is to solve the problems that the existing method has insufficient utilization of single-band information, a global threshold cannot adapt to spatial non-uniform distortion, and a global transformation model has insufficient precision. The application matches the band enhancement and derived image of the multispectral image with the gray reference image of the visible light image respectively; extracts cross-spectrum anchor points from a candidate matching point list; deduces a grid displacement field and a grid tolerance field according to the cross-spectrum anchor point set by using a distortion logic rule description module; performs self-consistency test on each point in each candidate matching point pair in the candidate matching point list by using a horizontal and vertical logic self-consistent matching point screening module; combines the screening result with the cross-spectrum anchor point set, estimates a transformation matrix of the multispectral image to the visible light image, and then transforms the multispectral image, so that the matched multispectral image is obtained.

Description

Technical Field

[0001] This application belongs to the field of agricultural remote sensing image processing technology of UAVs, and in particular relates to a crop multispectral visible light image matching method based on cross-spectral anchoring and distortion logic self-consistent screening. Background Technology

[0002] In precision agriculture and crop phenotypic analysis, drones equipped with both multispectral and visible light cameras capture images. After pixel-level matching and registration of the multispectral and visible light images, joint analysis allows for the overlay of spectral information such as vegetation indices and leaf area indices onto high-resolution visible light images, significantly improving the accuracy of pest and disease identification and the effectiveness of growth assessment. However, multispectral and visible light cameras differ significantly in sensor size, resolution, lens focal length, and distortion characteristics. The pixel coordinates of the two types of images cannot be directly aligned, necessitating registration through feature point matching and geometric transformation estimation. In artificial scenes such as buildings and roads, abundant corner and straight-line features result in high accuracy and few mismatches in matching point extraction. However, in crop-growing areas, dense leaves, complex textures, and a lack of clearly defined corners present challenges in setting a suitable threshold for feature point matching. A high threshold results in too few or even zero matching points, while a low threshold leads to too many matching points mixed with incorrect matches, causing significant deviations in subsequent geometric transformation estimations. Therefore, a multispectral and visible light image matching method suitable for crop vegetation scenarios is urgently needed.

[0003] Currently, matching crop multispectral and visible light images using drones mainly relies on the following technical methods:

[0004] I. Feature point matching methods based on single-band global thresholds select the band in a multispectral image that is closest to visible light (such as the red band), extract feature points using SIFT or ORB algorithms, and then perform matching filtering with a fixed distance ratio threshold. The main limitations of this type of method are: it only uses a single band for matching, wasting the complementary matching information that may be provided by the other bands of the multispectral image and their derived indices (NDVI, LAI, etc.); the globally fixed threshold cannot simultaneously adapt to the different accuracy requirements of the central region (small distortion, accurate matching) and the edge region (large distortion, increased matching deviation), and in crop scenes, it is very easy for matching points in the central region to be incorrectly removed while incorrect matches in the edge region are retained.

[0005] II. Global registration methods based on affine transformation models estimate a global affine transformation matrix using a small number of matching points to perform transformation registration on the entire image. The main limitation of this type of method is that the global affine model assumes that the geometric transformation between images is spatially uniform. However, the lens distortion patterns of multispectral and visible light cameras are different, and the actual deformation between the image center and the edge differs significantly. A single global model cannot accurately describe the spatially non-uniform distortion differences, resulting in low registration accuracy in local areas.

[0006] III. Image registration methods based on mutual information use the mutual information of two images as a similarity metric and solve for the optimal transformation through parameter optimization. The main limitations of this type of method are: the gray-level distributions of multispectral bands and visible light bands differ greatly, the search space for mutual information calculation is large and converges slowly, and in crop scenes, the self-similarity of textures can easily lead to multiple local extrema in mutual information, resulting in incorrect registration; furthermore, this method does not generate explicit matching point pairs and cannot perform point-by-point checks on the matching quality.

[0007] IV. Image matching method based on manually marked control points: Operators manually mark corresponding points on two images to complete the matching. The accuracy of this method depends on the operator's experience, is inefficient, cannot be scaled up for large-scale UAV image processing, and is difficult to accurately identify the corresponding positions manually in areas with dense crop leaves.

[0008] In summary, existing methods suffer from several core shortcomings, including insufficient utilization of single-band information, inability of global thresholds to adapt to spatial non-uniform distortion, insufficient accuracy of global transformation models, and lack of scalability of manual labeling. There is an urgent need to propose an automated method that utilizes all multispectral bands and derived indices for cross-spectral cross-anchoring, derives spatial distortion logic from anchor points, and screens candidate matches point by point using position-related tolerance. Summary of the Invention

[0009] This application aims to address the core shortcomings of existing methods, such as insufficient utilization of single-band information, inability of global thresholds to adapt to spatial non-uniform distortion, insufficient accuracy of global transformation models, and non-scalability of manual labeling. It provides a multispectral and visible light image matching method based on cross-spectral anchoring and distortion logic self-consistent screening.

[0010] This application provides a multispectral and visible light image matching method based on transspectral anchoring and distortion logic self-consistent screening, including:

[0011] The five-band enhanced images and two derived images of the multispectral image were matched with the grayscale reference image of the visible light image to obtain a list of candidate matching points.

[0012] Extract multispectral image location points that satisfy the preset number of repeated matching from the candidate matching point list as cross-spectral anchor points to obtain a set of cross-spectral anchor points;

[0013] The distortion logic law description module derives the grid displacement field and grid tolerance field based on the cross-spectral anchor point set;

[0014] Traverse each point in each candidate matching point pair in the candidate matching point list, and perform self-consistency verification based on the grid displacement field and grid tolerance field using the horizontal and vertical logic self-consistent matching point screening module to obtain the screening results.

[0015] The screening results are merged with the set of transspectral anchor points, and the transformation matrix from multispectral image to visible light image is estimated. The multispectral image is then transformed according to the transformation matrix to obtain the matched multispectral image.

[0016] In one possible design, the process of matching the five-band enhanced images and two derived images of the multispectral image with the grayscale reference image of the visible light image to obtain a list of candidate matching points, including:

[0017] Histogram enhancement was performed on the five band images of the multispectral image, namely the red band image, green band image, blue band image, red edge image, and near-infrared spectral image.

[0018] Calculate the NDVI derived image and LAI derived image of the multispectral image respectively;

[0019] The five enhanced images and two derived images are used to construct a list of seven bands to be matched.

[0020] The grayscale reference image is calculated based on three bands of the visible light image, namely the red light band, the green light band, and the blue light band.

[0021] The SIFT algorithm is applied to the images of each band in the band list to be matched and the grayscale reference images of each band to extract feature points and descriptors. The descriptors are used to filter candidate matching point pairs using the nearest neighbor distance ratio threshold. The obtained candidate matching point pairs are stored in the corresponding band position of the candidate matching point list.

[0022] The candidate matching point pairs include: multispectral image coordinates, visible light image coordinates, and band numbers.

[0023] In one possible design, the calculation of the NDVI-derived image and LAI-derived image of the multispectral image, respectively, includes:

[0024] The NDVI derived image of the multispectral image is calculated according to the following formula:

[0025] ,

[0026] in, The pixel values ​​of the NDVI derived image. These are the near-infrared spectral values ​​from the multispectral image. The red band value of the multispectral image;

[0027] The LAI-derived image of the multispectral image is calculated according to the following formula:

[0028] ,

[0029] in, The pixel value of the LAI-derived image.

[0030] In one possible design, calculating the grayscale reference image based on the three bands of the visible light image includes:

[0031] The grayscale reference image of the visible light image is calculated according to the following formula:

[0032] ,

[0033] in, The pixel values ​​of the grayscale reference image. The red band value of the visible light image. The green band value of the visible light image. This represents the blue light band value of the visible light image.

[0034] In one possible design, the step of extracting multispectral image location points that satisfy the requirement of repeating a preset number of matches from the candidate matching point list as transspectral anchor points, thereby obtaining a set of transspectral anchor points, includes:

[0035] Using the multispectral image coordinates of each band candidate matching point pair in the candidate matching point list as location points, and with a preset location tolerance as the radius, the location points are clustered.

[0036] The number of cluster regions to which each location point belongs is counted. Location points with a number greater than or equal to the preset number of bands are used as anchor points. The multispectral image coordinates and visible light image coordinates corresponding to the anchor points are added to the transspectral anchor point set.

[0037] In one possible design, the distortion logic law description module derives the mesh displacement field and mesh tolerance field based on the transspectral anchor point set, including:

[0038] Calculate the actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates;

[0039] The multispectral image is divided into a grid containing several rows and several columns, and the grid cell to which each point in the set of transspectral anchor points belongs is determined.

[0040] The displacement of each point within its respective grid cell is updated using the actual displacement to obtain the grid displacement field;

[0041] Calculate the distance from the center of each grid to the center of the multispectral image, and calculate the positional tolerance of each grid based on this distance, thus forming a grid tolerance field.

[0042] In one possible design, calculating the actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates includes:

[0043] The actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates is calculated according to the following formula:

[0044] ,

[0045] ,

[0046] in, and These represent the actual X-axis and Y-axis displacements of a transspectral anchor point from the multispectral image coordinates to the visible light image coordinates. This represents the visible light image coordinates of the current transspectral anchor point. This indicates the multispectral image coordinates of the current transspectral anchor point. and These represent the width and height of the multispectral image, respectively. and These are the width and height of the visible light image, respectively;

[0047] Determining the grid cell to which each point in the transspectral anchor point set belongs includes:

[0048] Calculate the row number and column number of the grid cell to which each cross-spectral anchor point belongs. If the row number is less than 1, set the row number to 1. If the row number is greater than the preset number of distorted grid rows, set the row number to the preset number of distorted grid rows. If the column number is less than 1, set the column number to 1. If the column number is greater than the preset number of distorted grid columns, set the column number to the preset number of distorted grid columns.

[0049] The step of updating the displacement of each point within its respective grid cell using the actual displacement to obtain the grid displacement field includes:

[0050] The actual displacement of each transspectral anchor point from the multispectral image coordinates to the visible light image coordinates is superimposed on the expected position of the grid cell to which each transspectral anchor point belongs, and 1 is superimposed on the number of transspectral anchor points in the grid to achieve the update.

[0051] In one possible design, the calculation of the distance from each grid center to the multispectral image center, and the calculation of the position-related tolerance of each grid based on this distance, thereby forming a grid tolerance field, includes:

[0052] The distance from each grid center to the multispectral image center is calculated using the following formula:

[0053] ,

[0054] ,

[0055] ,

[0056] in, The distance from the current grid center to the multispectral image center. and These are the row and column numbers of the current grid, respectively. and These are the width and height of the current grid, respectively. and All are intermediate variables. Indicates the center coordinates of the multispectral image;

[0057] The current grid's position-related tolerance is:

[0058] ,

[0059] in, This is the initial value for the mesh tolerance. This is the margin tolerance growth factor. This represents the maximum center distance of the multispectral image.

[0060] In one possible design, the process involves traversing each point in each candidate matching point pair in the candidate matching point list, performing a self-consistency check based on the grid displacement field and grid tolerance field using a horizontal and vertical logic self-consistent matching point screening module, and obtaining screening results, including:

[0061] Determine the location of each candidate matching point in the multispectral image grid;

[0062] The expected displacement and position-related tolerance corresponding to the location of each candidate matching point are determined based on the grid displacement field and the grid tolerance field.

[0063] Calculate the deviation distance between the actual displacement and the expected displacement of each candidate matching point;

[0064] Candidate matching points whose deviation distance is less than or equal to the corresponding positional tolerance are selected as screening results.

[0065] In one possible design, determining the location of each candidate matching point in the multispectral image grid includes:

[0066] Calculate the row number and column number of the grid cell to which each candidate matching point belongs. If the row number is less than 1, set the row number to 1; if the row number is greater than the preset number of distorted grid rows, set the row number to the preset number of distorted grid rows. If the column number is less than 1, set the column number to 1; if the column number is greater than the preset number of distorted grid columns, set the column number to the preset number of distorted grid columns.

[0067] The calculation of the deviation distance between the actual displacement and the expected displacement of each candidate matching point includes:

[0068] The deviation distance is calculated according to the following formula:

[0069] ,

[0070] in, The deviation distance and These represent the X-axis and Y-axis deviations between the actual displacement and the expected displacement, respectively, expressed as:

[0071] ,

[0072] ,

[0073] in, This indicates taking the absolute value. and These are the expected X-axis and Y-axis displacements obtained from the grid displacement field, respectively. and These are the actual X-axis and Y-axis displacements of the current candidate matching point, respectively, expressed as:

[0074] ,

[0075] ,

[0076] in, This represents the visible light image coordinates of the current candidate matching point. This represents the multispectral image coordinates of the current candidate matching point. and These represent the width and height of the multispectral image, respectively. and These represent the width and height of the visible light image, respectively.

[0077] The beneficial effects of this application are:

[0078] To address the problems of existing technologies, this invention provides a multispectral and visible light image matching method based on cross-spectral anchoring and distortion logic self-consistent screening. On one hand, the distortion logic law description module does not use a global single transformation model to describe the geometric relationship between two images. Instead, it starts from high-confidence anchor points that co-occur across multiple bands and uses a gridded spatial displacement field to characterize the local displacement differences at different locations in the image. At the same time, it automatically generates a tolerance field with a tight center and loose edges based on the distance from the image center, overcoming the limitation that global affine models cannot describe spatial non-uniform distortion. On the other hand, the logic self-consistent matching point screening module does not rely on a fixed global distance ratio threshold to filter matching points indiscriminately. Instead, it queries the expected displacement and position-related tolerance of the distortion logic law point by point for self-consistency verification. The small tolerance in the image center region ensures high accuracy, while the tolerance in the edge region is appropriately widened to avoid incorrect rejection of correct matches. It is unaffected by the complexity of crop textures, significantly reducing the false match mixing rate while improving the matching point retention rate. Attached Figure Description

[0079] Figure 1 This is a flowchart of a multispectral and visible light image matching method based on cross-spectral anchoring and distortion logic self-consistent screening. Detailed Implementation

[0080] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0081] Specific Implementation Method 1: The multispectral and visible light image matching method based on transspectral anchoring and distortion logic self-consistent screening described in this implementation method includes:

[0082] The five-band enhanced images and two derived images of the multispectral image were matched with the grayscale reference image of the visible light image to obtain a list of candidate matching points.

[0083] Extract multispectral image location points that satisfy the preset number of repeated matching from the candidate matching point list as cross-spectral anchor points to obtain a set of cross-spectral anchor points;

[0084] The distortion logic law description module derives the grid displacement field and grid tolerance field based on the cross-spectral anchor point set;

[0085] Traverse each point in each candidate matching point pair in the candidate matching point list, and perform self-consistency verification based on the grid displacement field and grid tolerance field using the horizontal and vertical logic self-consistent matching point screening module to obtain the screening results.

[0086] The screening results are merged with the set of transspectral anchor points, and the transformation matrix from multispectral image to visible light image is estimated. The multispectral image is then transformed according to the transformation matrix to obtain the matched multispectral image.

[0087] In one embodiment, the step of matching the five-band enhanced images and two derived images of the multispectral image with the grayscale reference image of the visible light image to obtain a candidate matching point list includes:

[0088] Histogram enhancement was performed on the five band images of the multispectral image, namely the red band image, green band image, blue band image, red edge image, and near-infrared spectral image.

[0089] Calculate the NDVI derived image and LAI derived image of the multispectral image respectively;

[0090] The five enhanced images and two derived images are used to construct a list of seven bands to be matched.

[0091] The grayscale reference image is calculated based on three bands of the visible light image, namely the red light band, the green light band, and the blue light band.

[0092] The SIFT algorithm is applied to the images of each band in the band list to be matched and the grayscale reference images of each band to extract feature points and descriptors. The descriptors are used to filter candidate matching point pairs using the nearest neighbor distance ratio threshold. The obtained candidate matching point pairs are stored in the corresponding band position of the candidate matching point list.

[0093] The candidate matching point pairs include: multispectral image coordinates, visible light image coordinates, and band numbers.

[0094] In one embodiment, calculating the NDVI-derived image and LAI-derived image of the multispectral image respectively includes:

[0095] The NDVI derived image of the multispectral image is calculated according to the following formula:

[0096] ,

[0097] in, The pixel values ​​of the NDVI derived image. These are the near-infrared spectral values ​​from the multispectral image. The red band value of the multispectral image;

[0098] The LAI-derived image of the multispectral image is calculated according to the following formula:

[0099] ,

[0100] in, The pixel value of the LAI-derived image.

[0101] In one embodiment, calculating the grayscale reference image based on the three bands of the visible light image includes:

[0102] The grayscale reference image of the visible light image is calculated according to the following formula:

[0103] ,

[0104] in, The pixel values ​​of the grayscale reference image. The red band value of the visible light image. The green band value of the visible light image. This represents the blue light band value of the visible light image.

[0105] In one implementation, the step of extracting multispectral image location points that satisfy the requirement of repeating a preset number of matches from the candidate matching point list as transspectral anchor points, thereby obtaining a transspectral anchor point set, includes:

[0106] Using the multispectral image coordinates of each band candidate matching point pair in the candidate matching point list as location points, and with a preset location tolerance as the radius, the location points are clustered.

[0107] The number of cluster regions to which each location point belongs is counted. Location points with a number greater than or equal to the preset number of bands are used as anchor points. The multispectral image coordinates and visible light image coordinates corresponding to the anchor points are added to the transspectral anchor point set.

[0108] In one implementation, the distortion logic law description module derives the grid displacement field and grid tolerance field based on the cross-spectral anchor point set, including:

[0109] Calculate the actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates;

[0110] The multispectral image is divided into a grid containing several rows and several columns, and the grid cell to which each point in the set of transspectral anchor points belongs is determined.

[0111] The displacement of each point within its respective grid cell is updated using the actual displacement to obtain the grid displacement field;

[0112] Calculate the distance from the center of each grid to the center of the multispectral image, and calculate the positional tolerance of each grid based on this distance, thus forming a grid tolerance field.

[0113] In one implementation, calculating the actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates includes:

[0114] The actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates is calculated according to the following formula:

[0115] ,

[0116] ,

[0117] in, and These represent the actual X-axis and Y-axis displacements of a transspectral anchor point from the multispectral image coordinates to the visible light image coordinates. This represents the visible light image coordinates of the current transspectral anchor point. This indicates the multispectral image coordinates of the current transspectral anchor point. and These represent the width and height of the multispectral image, respectively. and These are the width and height of the visible light image, respectively;

[0118] Determining the grid cell to which each point in the transspectral anchor point set belongs includes:

[0119] Calculate the row number and column number of the grid cell to which each cross-spectral anchor point belongs. If the row number is less than 1, set the row number to 1. If the row number is greater than the preset number of distorted grid rows, set the row number to the preset number of distorted grid rows. If the column number is less than 1, set the column number to 1. If the column number is greater than the preset number of distorted grid columns, set the column number to the preset number of distorted grid columns.

[0120] The step of updating the displacement of each point within its respective grid cell using the actual displacement to obtain the grid displacement field includes:

[0121] The actual displacement of each transspectral anchor point from the multispectral image coordinates to the visible light image coordinates is superimposed on the expected position of the grid cell to which each transspectral anchor point belongs, and 1 is superimposed on the number of transspectral anchor points in the grid to achieve the update.

[0122] In one embodiment, the step of calculating the distance from each grid center to the multispectral image center, and calculating the positional tolerance of each grid based on the distance, thereby forming a grid tolerance field, includes:

[0123] The distance from each grid center to the multispectral image center is calculated using the following formula:

[0124] ,

[0125] ,

[0126] ,

[0127] in, The distance from the current grid center to the multispectral image center. and These are the row and column numbers of the current grid, respectively. and These are the width and height of the current grid, respectively. and All are intermediate variables. Indicates the center coordinates of the multispectral image;

[0128] The current grid's position-related tolerance is:

[0129] ,

[0130] in, This is the initial value for the mesh tolerance. This is the margin tolerance growth factor. This represents the maximum center distance of the multispectral image.

[0131] In one implementation, the step of traversing each point in each candidate matching point pair in the candidate matching point list, and performing a self-consistency check based on the grid displacement field and grid tolerance field using a horizontal and vertical logic self-consistent matching point screening module to obtain screening results includes:

[0132] Determine the location of each candidate matching point in the multispectral image grid;

[0133] The expected displacement and position-related tolerance corresponding to the location of each candidate matching point are determined based on the grid displacement field and the grid tolerance field.

[0134] Calculate the deviation distance between the actual displacement and the expected displacement of each candidate matching point;

[0135] Candidate matching points whose deviation distance is less than or equal to the corresponding positional tolerance are selected as screening results.

[0136] In one implementation, determining the position of each candidate matching point in the multispectral image grid includes:

[0137] Calculate the row number and column number of the grid cell to which each candidate matching point belongs. If the row number is less than 1, set the row number to 1; if the row number is greater than the preset number of distorted grid rows, set the row number to the preset number of distorted grid rows. If the column number is less than 1, set the column number to 1; if the column number is greater than the preset number of distorted grid columns, set the column number to the preset number of distorted grid columns.

[0138] The calculation of the deviation distance between the actual displacement and the expected displacement of each candidate matching point includes:

[0139] The deviation distance is calculated according to the following formula:

[0140] ,

[0141] in, The deviation distance and These represent the X-axis and Y-axis deviations between the actual displacement and the expected displacement, respectively, expressed as:

[0142] ,

[0143] ,

[0144] in, This indicates taking the absolute value. and These are the expected X-axis and Y-axis displacements obtained from the grid displacement field, respectively. and These are the actual X-axis and Y-axis displacements of the current candidate matching point, respectively, expressed as:

[0145] ,

[0146] ,

[0147] in, This represents the visible light image coordinates of the current candidate matching point. This represents the multispectral image coordinates of the current candidate matching point. and These represent the width and height of the multispectral image, respectively. and These represent the width and height of the visible light image, respectively.

[0148] To further illustrate the implementation scheme of this application, Figure 1 A multispectral and visible light image matching method based on cross-spectral anchoring and distortion logic self-consistent screening is provided, comprising steps one through four. The numbering of these steps does not necessarily restrict their execution order. Each step is described in detail below:

[0149] S1: Input multispectral image (MSImg), visible light image (VISImg), and global control parameters; perform histogram enhancement on each band of the multispectral image and calculate NDVI (Normalized Difference Vegetation Index) and LAI (Leaf Area Index) derived images to form a band list to be matched; generate a grayscale reference image (VISGray) from the visible light image; establish global vegetation parameters. Details are as follows:

[0150] S101, Input multispectral image and visible light image. The dimensions of the multispectral image are... , Indicates the number of columns in a multispectral image. The number of rows in the multispectral image is indicated by the five bands: red (R), green (G), blue (B), red-edge (RE), and near-infrared (NIR). The dimensions of the visible light image are... , This indicates the column number of the visible light image. This indicates the row number of the visible light image, with three bands: red (R), green (G), and blue (B).

[0151] S102, Input global control parameters, including:

[0152] Distance ratio threshold of SIFT algorithm The default value is 0.80;

[0153] Minimum number of transspectral anchoring bands The default value is 3;

[0154] Anchor point spatial tolerance (pixels), default value is 3.0;

[0155] Distorted mesh row count The default value is 5;

[0156] Number of distorted grid columns The default value is 5;

[0157] Marginal tolerance growth factor The default value is 1.5;

[0158] Basic screening tolerance (pixels), the default value is 5.0.

[0159] S103, Histogram equalization enhancement is performed on each of the five bands of the multispectral image to obtain the enhanced five-band image.

[0160] S104, Calculate NDVI derived image:

[0161] ,

[0162] in, The pixel values ​​of the NDVI derived image. For NIR band values, The R-band value of the multispectral image;

[0163] The NDVI derived image is mapped to the range of integers from 0 to 255 using Min-Max normalization, where Min is the minimum value and Max is the maximum value.

[0164] S105, Calculate LAI derived image:

[0165] ,

[0166] in, For the pixel values ​​of the LAI-derived image, if Then set ;

[0167] The LAI imagery is Min-Max normalized and mapped to an integer range of 0 to 255.

[0168] S106, Create a list of bands to be matched. This list contains 7 elements: enhanced R, G, B, RE, and NIR band images, as well as NDVI and LAI derived images; obtain the total number of bands. .

[0169] S107, Calculate the grayscale reference image based on the three bands of the visible light image:

[0170] ,

[0171] in, The pixel values ​​of the grayscale reference image. The R-band value of the visible light image. The G-band value of the visible light image. This represents the B-band value of the visible light image.

[0172] S108, Calculate the center coordinates of the multispectral image. , ;

[0173] Calculate the maximum center distance of multispectral images .

[0174] S109. Establish a band-by-band candidate matching point master table (Band Match Table). Initially, it is an empty list, containing 7 sub-lists. Each sub-list corresponds to a set of matching points for one band.

[0175] Create a cross-spectral anchor list, initially an empty list.

[0176] S110, Establish a band-by-band SIFT matching counter .

[0177] S111, retrieve the first band from the list of bands to be matched. For each band of imagery, feature points and descriptors are extracted from both the band image and the grayscale reference image using the SIFT (Scale-Invariant Feature Transform) algorithm. Nearest neighbor matching is then performed on the descriptors, retaining those with a distance ratio less than a distance ratio threshold. The matching pairs are stored in the band-by-band candidate matching point master table. Each candidate matching point pair contains the following fields: multispectral image X-axis coordinates. Y-axis coordinate of multispectral image X-axis coordinates of visible light images visible light image Y-axis coordinate and band number .

[0178] S112, making ,if If the condition is met, return to S111; otherwise, execute S113.

[0179] S113, Extract transspectral anchor points: For all matching points in the 7 sub-lists of the band-by-band candidate matching point master table, use the multispectral image coordinates of each sub-list. For each location point, count how many different sublists of different bands each location point appears in (based on the anchor point spatial tolerance). (For location tolerance, determine if they are the same location point); if a location point is greater than or equal to If the location appears in all bands, then the multispectral image coordinates and visible light image coordinates (taking the average coordinates of that location in all bands) of that location are added to the transspectral anchor point list, and the total number of anchor points is obtained. .

[0180] S2, establish the distortion logic law description module (JBLJ Model). This module derives the gridded spatial displacement field from the transspectral anchor point and generates a position-related tolerance field with a tight center and loose edges based on the distance from the image center, thus forming a distortion logic law description. Details are as follows:

[0181] S201, establish the distortion logic law description module, whose input is the distortion description input (JBLJ Input), including: a list of cross-spectral anchor points, , , , , , , , and .

[0182] S202, initialize the mesh displacement field as follows A two-dimensional array, Indicates the row number of the mesh displacement field. This represents the number of columns in the mesh displacement field, with each element containing the expected X-axis displacement. Expected Y-axis displacement and the number of anchor points within the grid All values ​​are initialized to 0; the mesh tolerance field is initialized to... A two-dimensional array, where each element is a tolerance value, initialized to... .

[0183] S203, Calculate the mesh cell size: , .

[0184] S204, Establish anchor point traversal counter .

[0185] S205, take the first item in the transspectral anchor point list. Multispectral image coordinates of the records and visible light image coordinates Calculate the actual displacement along the X and Y axes. and :

[0186] ,

[0187] ,

[0188] Determine the first The row number of the grid cell to which each anchor point belongs and column number :

[0189] ,

[0190] ,

[0191] in, This indicates rounding to the nearest integer.

[0192] if Then set ;if Then set ;if Then set ;if Then set .

[0193] S206, in the grid displacement field Line 1 Expected X-axis displacement of the column Accumulation In the grid displacement field, the first Line 1 Expected Y-axis displacement of the column Accumulation Number of anchor points within the grid Adding 1 above yields the grid displacement field.

[0194] S207, making ,if If the condition is met, return to S205; otherwise, execute S208.

[0195] S208, iterate through each element of the mesh displacement field: if Then make and ;if Then in adjacent and Take from the grid cells and The mean is used as the interpolation value (if adjacent cells are all empty, it remains 0).

[0196] S209, Initialize the mesh tolerance field counter .

[0197] S210, Initialize the grid tolerance field counter .

[0198] S211, Calculate the distance from each grid center to the multispectral image center. :

[0199] ,

[0200] ,

[0201] ;

[0202] Calculate location-related tolerances:

[0203] Grid tolerance field Line 1 Listed as: .

[0204] S212, making ,if If yes, return S211; otherwise, make ,if If the condition is met, return to S210; otherwise, proceed to S213.

[0205] S213, the output of the distortion logic law description module is the distortion description output (JBLJ Output), which includes: the mesh displacement field obtained in S206 and the mesh tolerance field obtained in S211.

[0206] S3, establish a logically consistent matching point screening module (LZSJ Model). This module checks point by point whether the deviation between the actual displacement of the candidate matching point and the expected displacement of the distortion logic law is within the position-related tolerance range. Specifically:

[0207] S301, Establish a logically consistent matching point screening module, whose input is the screening input, including: a candidate matching point pair. , , , Mesh displacement field and mesh tolerance field, and , , , , , , and .

[0208] S302, Initialize screening output .

[0209] S303, Determine the row number of the displacement field element of the grid to which the candidate matching point belongs. and column number :

[0210] ,

[0211] ;

[0212] if Then set ;if Then set ;if Then set ;if Then set .

[0213] S304, extract the first from the mesh displacement field Line 1 Expected displacement of the column and ; Take the first from the grid tolerance field Line 1 Column position related tolerance .

[0214] S305, Calculate the actual displacement of the candidate points:

[0215] ,

[0216] .

[0217] S306, Calculate the deviation between the actual displacement and the expected displacement:

[0218] ,

[0219] ,

[0220] in, Indicates taking the absolute value;

[0221] Calculate the deviation distance:

[0222] .

[0223] S307, if Then set the screening output. Otherwise, the screening output will be checked. Keep it at 0.

[0224] S308, the output of the logically consistent matching point screening module is the screening output. , If the condition is met, then check if the point already exists in the final list of approved points (in terms of...). (For tolerance judgment), if the point does not exist, add it to the final list of filtered points. Keeping the value at 0 indicates rejection.

[0225] S4 uses the distortion logic rule description module established in S2 to derive the distortion logic rule from the anchor point. Then, it uses the logic self-consistent matching point screening module established in S3 to perform logic self-consistency screening on the band-by-band candidate matching points in S1. The anchor points obtained in S1 are merged with the matching points that passed the screening in S3, and the geometric transformation matrix is ​​estimated to complete the registration output. The details are as follows:

[0226] S401 calls the distortion logic law description module established in S2 to obtain the mesh displacement field and mesh tolerance field, and calculates the mesh element size. and .

[0227] S402, Establish a band traversal screening counter .

[0228] S403, take the first [number] from the band-by-band candidate matching point list. A list of sub-lists;

[0229] Establish a point-by-point screening counter .

[0230] S404, take the first The first in the sublist The candidate matching points constitute the screening input for the horizontal and vertical logically consistent matching point screening module described in S3. The logically consistent matching point screening module is called to obtain the screening output results.

[0231] S405, makes ,if If the number of candidate matching pairs is less than or equal to the number of candidate matching pairs in the sublist, return S404; otherwise, make... ,if If the condition is met, return to S403; otherwise, execute S406.

[0232] S406, merge all anchor points in the cross-spectral anchor point list into the final filtered pass point list (with deduplication), and obtain the total number of final matched points in the final filtered pass point list. .

[0233] S407, if Then, using the final filtering, the least squares method is used to estimate the perspective transformation matrix (Trans Matrix, 3×3 matrix) from the multispectral image to the visible light image through all matching point pairs in the point list. The perspective transformation matrix is ​​then used to perform perspective transformation on the multispectral image to obtain the registered multispectral image, which is then used as the registration result image; output. The number of final matching points, of which, One is a transspectral anchor point, and the rest are logically consistent screening points.

[0234] S408, if The output has insufficient matching points, making registration impossible. It is recommended to reduce the output size. Parameters or increase Try again after adjusting the parameters.

[0235] This embodiment provides a crop multispectral visible light image matching method based on cross-spectral anchoring and distortion logic self-consistent screening. The method constructs a distortion logic law description module and a logic self-consistent matching point screening module. After inputting multispectral and visible light images, histogram enhancement is performed on each band and the NDVI and LAI derived images, and SIFT feature matching is performed with the visible light image to obtain band-by-band candidate matching points. Cross-spectral anchoring points appearing in multiple bands are extracted from these. The distortion logic law description module derives the image gridded spatial displacement field and the position correlation tolerance field (tight at the center and loose at the edges) from the anchoring points to form a distortion logic law description. The logic self-consistent matching point screening module checks each candidate matching point in each band to see if the deviation between its actual displacement and the expected displacement of the distortion logic law is within the position correlation tolerance range. The screened matching points are merged with the anchoring points, and the geometric transformation matrix is ​​estimated to complete image registration, achieving automatic matching of multispectral and visible light images for crop vegetation scenes.

[0236] Specific Implementation Method Two: The multispectral and visible light image matching device based on transspectral anchoring and distortion logic self-consistent screening described in this implementation method includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the multispectral and visible light image matching method based on transspectral anchoring and distortion logic self-consistent screening as described in Specific Implementation Method One.

[0237] Specific Implementation Method 3: A computer storage medium as described in this embodiment stores at least one instruction, which is loaded and executed by a processor to implement the multispectral and visible light image matching method based on transspectral anchoring and distortion logic self-consistent screening as described in Specific Implementation Method 1.

[0238] While specific embodiments of this application have been described herein with reference to them, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A multispectral and visible light image matching method based on transspectral anchoring and distortion logic, characterized in that, include: The five-band enhanced images and two derived images of the multispectral image were matched with the grayscale reference image of the visible light image to obtain a list of candidate matching points. Extract multispectral image location points that satisfy the preset number of repeated matching from the candidate matching point list as cross-spectral anchor points to obtain a set of cross-spectral anchor points; The distortion logic law description module derives the grid displacement field and grid tolerance field based on the cross-spectral anchor point set; Traverse each point in each candidate matching point pair in the candidate matching point list, and perform self-consistency verification based on the grid displacement field and grid tolerance field using the horizontal and vertical logic self-consistent matching point screening module to obtain the screening results. The screening results are merged with the set of transspectral anchor points, and the transformation matrix from multispectral image to visible light image is estimated. The multispectral image is then transformed according to the transformation matrix to obtain the matched multispectral image.

2. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 1, characterized in that, The process involves matching the five enhanced band images and two derived images of the multispectral image with the grayscale reference image of the visible light image to obtain a candidate matching point list, including: Histogram enhancement was performed on the five band images of the multispectral image, namely the red band image, green band image, blue band image, red edge image, and near-infrared spectral image. Calculate the NDVI derived image and LAI derived image of the multispectral image respectively; The five enhanced images and two derived images are used to construct a list of seven bands to be matched. The grayscale reference image is calculated based on three bands of the visible light image, namely the red light band, the green light band, and the blue light band. The SIFT algorithm is applied to the images of each band in the band list to be matched and the grayscale reference images of each band to extract feature points and descriptors. The descriptors are used to filter candidate matching point pairs using the nearest neighbor distance ratio threshold. The obtained candidate matching point pairs are stored in the corresponding band position of the candidate matching point list. The candidate matching point pairs include: multispectral image coordinates, visible light image coordinates, and band numbers.

3. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 2, characterized in that, The calculation of the NDVI-derived image and LAI-derived image of the multispectral image includes: The NDVI derived image of the multispectral image is calculated according to the following formula: ， in, The pixel values ​​of the NDVI derived image. These are the near-infrared spectral values ​​from the multispectral image. The red band value of the multispectral image; The LAI-derived image of the multispectral image is calculated according to the following formula: ， in, The pixel value of the LAI-derived image.

4. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 3, characterized in that, The calculation of the grayscale reference image based on the three bands of the visible light image includes: The grayscale reference image of the visible light image is calculated according to the following formula: ， in, The pixel values ​​of the grayscale reference image. The red band value of the visible light image. The green band value of the visible light image. This represents the blue light band value of the visible light image.

5. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 1, characterized in that, The step of extracting multispectral image location points that satisfy the preset number of repetitions from the candidate matching point list as cross-spectral anchor points, thereby obtaining a set of cross-spectral anchor points, includes: Using the multispectral image coordinates of each band candidate matching point pair in the candidate matching point list as location points, and with a preset location tolerance as the radius, the location points are clustered. The number of cluster regions to which each location point belongs is counted. Location points with a number greater than or equal to the preset number of bands are used as anchor points. The multispectral image coordinates and visible light image coordinates corresponding to the anchor points are added to the transspectral anchor point set.

6. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 1, characterized in that, The distortion logic law description module derives the grid displacement field and grid tolerance field based on the cross-spectral anchor point set, including: Calculate the actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates; The multispectral image is divided into a grid containing several rows and several columns, and the grid cell to which each point in the set of transspectral anchor points belongs is determined. The displacement of each point within its respective grid cell is updated using the actual displacement to obtain the grid displacement field; Calculate the distance from the center of each grid to the center of the multispectral image, and calculate the positional tolerance of each grid based on this distance, thus forming a grid tolerance field.

7. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 6, characterized in that, The calculation of the actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates includes: The actual displacement of each point in the transspectral anchor point set from the multispectral image coordinates to the visible light image coordinates is calculated according to the following formula: ， ， in, and These represent the actual X-axis and Y-axis displacements of a transspectral anchor point from the multispectral image coordinates to the visible light image coordinates. This represents the visible light image coordinates of the current transspectral anchor point. This indicates the multispectral image coordinates of the current transspectral anchor point. and These represent the width and height of the multispectral image, respectively. and These are the width and height of the visible light image, respectively; Determining the grid cell to which each point in the transspectral anchor point set belongs includes: Calculate the row number and column number of the grid cell to which each cross-spectral anchor point belongs. If the row number is less than 1, set the row number to 1. If the row number is greater than the preset number of distorted grid rows, set the row number to the preset number of distorted grid rows. If the column number is less than 1, set the column number to 1. If the column number is greater than the preset number of distorted grid columns, set the column number to the preset number of distorted grid columns. The step of updating the displacement of each point within its respective grid cell using the actual displacement to obtain the grid displacement field includes: The actual displacement of each transspectral anchor point from the multispectral image coordinates to the visible light image coordinates is superimposed on the expected position of the grid cell to which each transspectral anchor point belongs, and 1 is superimposed on the number of transspectral anchor points in the grid to achieve the update.

8. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 7, characterized in that, The calculation of the distance from the center of each grid to the center of the multispectral image, and the calculation of the positional tolerance of each grid based on this distance, thereby forming a grid tolerance field, includes: The distance from each grid center to the multispectral image center is calculated using the following formula: ， ， ， in, The distance from the current grid center to the multispectral image center. and These are the row and column numbers of the current grid, respectively. and These are the width and height of the current grid, respectively. and All are intermediate variables. Indicates the center coordinates of the multispectral image; The current grid's position-related tolerance is: ， in, This is the initial value for the mesh tolerance. This is the margin tolerance growth factor. This represents the maximum center distance of the multispectral image.

9. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 1, characterized in that, The process involves traversing each point in each candidate matching point pair in the candidate matching point list, and performing a self-consistency check using the horizontal and vertical logic self-consistent matching point screening module based on the grid displacement field and grid tolerance field to obtain screening results, including: Determine the location of each candidate matching point in the multispectral image grid; The expected displacement and position-related tolerance corresponding to the location of each candidate matching point are determined based on the grid displacement field and the grid tolerance field. Calculate the deviation distance between the actual displacement and the expected displacement of each candidate matching point; Candidate matching points whose deviation distance is less than or equal to the corresponding positional tolerance are selected as screening results.

10. The multispectral and visible light image matching method based on transspectral anchoring and distortion logic according to claim 9, characterized in that, Determining the position of each candidate matching point in the multispectral image grid includes: Calculate the row number and column number of the grid cell to which each candidate matching point belongs. If the row number is less than 1, set the row number to 1; if the row number is greater than the preset number of distorted grid rows, set the row number to the preset number of distorted grid rows. If the column number is less than 1, set the column number to 1; if the column number is greater than the preset number of distorted grid columns, set the column number to the preset number of distorted grid columns. The calculation of the deviation distance between the actual displacement and the expected displacement of each candidate matching point includes: The deviation distance is calculated according to the following formula: ， in, The deviation distance and These represent the X-axis and Y-axis deviations between the actual displacement and the expected displacement, respectively, expressed as: ， ， in, This indicates taking the absolute value. and These are the expected X-axis and Y-axis displacements obtained from the grid displacement field, respectively. and These are the actual X-axis and Y-axis displacements of the current candidate matching point, respectively, expressed as: ， ， in, This represents the visible light image coordinates of the current candidate matching point. This represents the multispectral image coordinates of the current candidate matching point. and These represent the width and height of the multispectral image, respectively. and These represent the width and height of the visible light image, respectively.

Images