Remote sensing image stitching method, device, computer equipment and medium

Abstract

The present application relates to the field of image stitching technology, and specifically to a remote sensing image stitching method, device, computer equipment and medium, the method comprising: continuously collecting a preset number of remote sensing images through a drone; determining the distance difference of any matching point pair based on the distance between feature points in two grayscale images and the distance difference with other feature points; determining the neighborhood distribution characterization value of any matching point pair based on the local density of the two feature points in the grayscale image to which they belong, and the difference in the distribution of the feature points of the two feature points in their neighborhood; combining the distance difference and the neighborhood distribution characterization value, screening the correct matching point pair, and stitching the two remote sensing images. The present application aims to reduce the occurrence of feature point mismatching in two remote sensing images and improve the stitching accuracy of remote sensing images.

Description

Technical Field

[0001] The present application relates to the technical field of image stitching, and in particular to a remote sensing image stitching method, device, computer equipment and medium. Background Art

[0002] The goal of remote sensing image stitching is to merge multiple partially overlapping images into a complete image so as to achieve seamless continuity of geographic information in subsequent analysis and applications. This processing is particularly important for drone or satellite remote sensing images, because drones or satellites usually acquire data in a set orbit, resulting in partial overlap of images of different scenes in terms of geographical location. Through stitching processing, these partially overlapping images can be stitched into a complete image, thus ensuring the integrity and continuity of geographic information.

[0003] The existing image stitching method is to first extract feature points from the image, match the feature points, then use the random sampling consensus algorithm to eliminate mismatched point pairs, and finally perform image registration and stitching. When matching feature points of remote sensing images, since remote sensing images contain large areas of farmland and houses, the color and shape features of farmland or houses at different locations are similar. The existing algorithm is prone to matching similar feature points at different locations, resulting in feature point mismatch, which limits the stitching accuracy of remote sensing images. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a remote sensing image stitching method, device, computer equipment and medium. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a remote sensing image stitching method, the method comprising the following steps:

[0006] Continuously collect a preset number of remote sensing images through drones;

[0007] According to the distribution of feature points in two grayscale images converted from any two continuously collected remote sensing images, the correct matching point pairs are selected; specifically:

[0008] A1, determining a matching point pair based on the distance between feature points in two grayscale images; determining the distance difference between any matching point pair and all other remaining matching point pairs, respectively, in the two grayscale images;

[0009] A2: Determine the density representation value of any matching point pair based on the average level and difference of the local density of the two feature points in any matching point pair in their respective grayscale images; determine the neighborhood distribution representation value of any matching point pair based on the difference in the distribution of the density representation values ​​of the matching point pairs to which the feature points in their neighborhoods belong;

[0010] A3, combining the distance difference and the neighborhood distribution representation value to select the correct matching point pairs;

[0011] The two remote sensing images are stitched based on all the correct matching point pairs in the two grayscale images.

[0012] Preferably, the method for determining the matching point pair is further defined as:

[0013] In one of the grayscale images, the feature points that are closest to a feature point in the other grayscale image are combined to form a feature point pair;

[0014] Obtain the segmentation threshold for the difference between all feature point pairs in two grayscale images;

[0015] The two feature point pairs whose differences are less than the segmentation threshold are regarded as the matching point pairs of the two grayscale images.

[0016] Preferably, the method for determining the distance difference is further defined as: the i-th pair of matching points The distance difference is recorded as , ; Where n represents the number of matching point pairs in two grayscale images; and Represents the two feature points of the jth pair of matching points in the two grayscale images; and Represents the two feature points of the i-th pair of matching points in the two grayscale images; is the absolute value symbol; O is the distance function.

[0017] Preferably, the method for determining the neighborhood distribution characterization value is further defined as:

[0018] Get the segmentation threshold of the density representation value of all matching point pairs in two grayscale images;

[0019] The two feature points in the matching point pair to which the density representation value greater than the segmentation threshold belongs are recorded as potential correct feature points respectively;

[0020] The neighborhood distribution representation value of any matching point pair is determined by using the number and value of potential correct feature points in the neighborhood of the two feature points in any matching point pair, as well as the difference in number.

[0021] Preferably, the screening process of the correct matching point pairs includes:

[0022] The neighborhood distribution representation value and distance difference of any matching point pair are reversely fused to construct the correct matching discrimination value of the matching point pair;

[0023] Get the segmentation threshold of all correctly matched discriminant values;

[0024] The matching point pairs corresponding to the correct matching discriminant values ​​greater than the segmentation threshold are regarded as the correct matching point pairs.

[0025] Preferably, the method for stitching two remote sensing images is:

[0026] All the correctly matched point pairs in the two grayscale images are obtained, and the homography matrix between the two remote sensing images is calculated using the pixel point pairs composed of the pixel points at the corresponding positions in the two corresponding remote sensing images;

[0027] Use homography matrix to transform and align two remote sensing images;

[0028] The two aligned remote sensing images are stitched together to obtain a stitched image.

[0029] Preferably, after the two remote sensing images are stitched together, the method further includes processing all remote sensing images acquired at consecutive times according to the stitching method of the two remote sensing images to obtain a panoramic image after all remote sensing images are stitched together.

[0030] In a second aspect, an embodiment of the present application provides a remote sensing image stitching device, the remote sensing image stitching device comprising:

[0031] A remote sensing image acquisition module is used to continuously acquire a preset number of remote sensing images through a drone;

[0032] The feature point matching module is used to convert any two continuously acquired remote sensing images into the distribution of feature points in two grayscale images and select the correct matching point pairs; specifically:

[0033] A1, determining a matching point pair based on the distance between feature points in two grayscale images; determining the distance difference between any matching point pair and all other remaining matching point pairs, respectively, in the two grayscale images;

[0034] A2: Determine the density representation value of any matching point pair based on the average level and difference of the local density of the two feature points in any matching point pair in their respective grayscale images; determine the neighborhood distribution representation value of any matching point pair based on the difference in the distribution of the density representation values ​​of the matching point pairs to which the feature points in their neighborhoods belong;

[0035] A3, combining the distance difference and the neighborhood distribution representation value to select the correct matching point pairs;

[0036] The remote sensing image stitching module is used to stitch two remote sensing images based on all the correct matching point pairs in the two grayscale images.

[0037] In a third aspect, an embodiment of the present application provides a computer device for remote sensing image stitching, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the remote sensing image stitching method as described in the first aspect is implemented.

[0038] In a fourth aspect, an embodiment of the present application further provides a remote sensing image stitching medium, wherein the remote sensing image stitching medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image stitching method as described in the first aspect is implemented.

[0039] It can be seen from the above embodiments that the remote sensing image stitching method, device, computer equipment and medium provided in the embodiments of the present application have at least the following beneficial effects:

[0040] Based on the distance difference between the matching point pairs in the two grayscale images, the distance difference of each matching point pair is constructed to measure the difference between the two distances of a matching point pair and each other matching point pair in the two grayscale images, and then judge the possibility that each matching point pair belongs to a wrong matching point pair; based on the density distribution characteristics of the two feature points in the matching point pair in the grayscale images to which they belong, the density representation value of each matching point pair is constructed to represent the local density characteristics of the two feature points in the matching point pair in the two grayscale images to which they belong; based on the density representation value and the difference in the dense distribution characteristics of the feature points in the neighborhood where the two feature points in the matching point pair are located, the neighborhood distribution representation value is constructed to judge the matching of the feature points in the neighborhood where the two feature points in the matching point pair are located; based on the distance difference of the matching point pair and the neighborhood distribution representation value, the correct matching discrimination value is constructed to judge the possibility that a pair of matching point pairs belong to the correct matching point pair and exclude the feature points of wrong matching. Through the above methods, the occurrence of wrong matching of feature points in two remote sensing images is greatly reduced, and the stitching accuracy of remote sensing images is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A flowchart of a remote sensing image stitching method provided in one embodiment of the present application;

[0043] Figure 2 A previous remote sensing image with a continuous scene taken at a continuous time provided in one embodiment of the present application;

[0044] Figure 3 A remote sensing image with a continuous scene taken at a continuous time provided in one embodiment of the present application;

[0045] Figure 4 A flow chart of a screening process for correct matching point pairs provided in one embodiment of the present application;

[0046] Figure 5 A schematic diagram of feature point distribution after a remote sensing image with continuous scenes taken at a continuous time is converted into a grayscale image according to an embodiment of the present application;

[0047] Figure 6 A schematic diagram of feature point distribution after a remote sensing image with continuous scenes taken at a continuous time is converted into a grayscale image according to an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the distribution of matching point pairs in overlapping areas and non-overlapping areas provided by one embodiment of the present application;

[0049] Figure 8 An image obtained by stitching two remote sensing images provided in one embodiment of the present application;

[0050] Fig. 9 A schematic diagram of the structure of a remote sensing image stitching device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of a remote sensing image stitching method, device, computer equipment and medium proposed in the present application, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0052] Unless otherwise specified and limited, terms such as "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such articles or devices. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the article or device including the element. In addition, the term "and\or" used herein includes any and all combinations of one or more related listed items. All technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of this application.

[0053] The following is a detailed description of a remote sensing image stitching method, device, computer equipment and medium provided by the present application in conjunction with the accompanying drawings.

[0054] See also Figure 1 , which shows a flowchart of a remote sensing image stitching method provided by an embodiment of the present application, the method comprising the following steps:

[0055] The first step is to continuously collect a preset number of remote sensing images using drones.

[0056] First, the present application utilizes a drone to continuously collect a preset number of remote sensing images along a planned flight path. During the sampling process, the flight speed, flight altitude, and sampling frequency of the drone need to be controlled so that there are overlapping areas in the scenes in the images before and after the collection.

[0057] In this embodiment, the flight speed of the drone is set to 20 meters per second, the flight altitude is 80 meters, and the sampling interval is 3 seconds. The median filter algorithm is used to remove noise in the image. The median filter algorithm is a well-known technology and the specific process is not repeated. Among them, the number of remote sensing images collected by the drone, the flight path of the drone, the flight speed, the flight altitude and the sampling interval can all be set by the implementer.

[0058] In the embodiments of the present application, Figure 2 and Figure 3They are the previous and next remote sensing images with continuous scenes taken at consecutive times.

[0059] At this point, a preset number of remote sensing images can be obtained through the above method.

[0060] The second step is to convert any two continuously acquired remote sensing images into the distribution of feature points in two grayscale images and select the correct matching point pairs.

[0061] In order to improve the accuracy of stitching and make the stitched remote sensing images more reflective of the real appearance of the scene, it is necessary to make the matching results more accurate in the feature point matching stage. When stitching multiple remote sensing images, the repeated areas of one image and another are compared and stitched in pairs.

[0062] Therefore, this embodiment takes the stitching of any two continuously collected remote sensing images as an example, and based on the distribution of matching point pairs in the grayscale images corresponding to the two remote sensing images, selects the correct matching point pairs to complete the design of the remote sensing image stitching method.

[0063] In the embodiment of the present application, the flow chart of the screening process of the correct matching point pair is shown in the attached figure. Figure 4 As shown, specifically:

[0064] A1, determining matching point pairs based on the distance between feature points in two grayscale images; determining the distance difference of any matching point pair based on the distance difference between feature points in the two grayscale images between any matching point pair and all other remaining matching point pairs.

[0065] First, all the collected remote sensing images are converted into grayscale images, and all the grayscale images are used as input of the SIFT (Scale Invariant Feature Transform) algorithm to obtain the feature points and their descriptors in all the grayscale images. The SIFT algorithm is a well-known technology and the specific process will not be repeated here.

[0066] In the embodiment of the present application, the previous and next remote sensing images with continuous scenes taken at consecutive times are converted into grayscale images, and the distribution diagrams of feature points are shown in the attached drawings. Figure 5 and attached Figure 6 shown.

[0067] A101, determines matching point pairs based on the distance between feature points in two grayscale images.

[0068] Preferably, in the embodiment of the present application, the method for determining the matching point pair is further defined as: in one of the grayscale images, the feature points that are closest to a feature point in the other grayscale image are used together to form a feature point pair; the segmentation threshold is obtained for the difference between all feature point pairs in the two grayscale images; and the feature point pairs that are less than the segmentation threshold are used as the matching point pairs of the two grayscale images. The method for obtaining the segmentation threshold is obtained by the embodiment of the present application using a threshold segmentation algorithm, which is a well-known technology and will not be described in detail.

[0069] As an implementation method of an embodiment of the present application, the distance between feature points constituting a feature point pair is represented by the absolute value of the difference between the descriptors of the feature points; the difference between the feature point pairs is represented by the absolute value of the difference in the distance between the feature points.

[0070] As another implementation of the embodiment of the present application, the Euclidean distance between the two feature points after the two feature points are mapped to the same grayscale image is used to represent it.

[0071] In other embodiments of the present application, the feature point pairs may also be directly used as matching point pairs of two grayscale images.

[0072] Furthermore, we arbitrarily select grayscale images corresponding to two remote sensing images taken at consecutive times, and take the feature point A of one of the grayscale images as an example for analysis, specifically:

[0073] Calculate the distance d between the descriptor of feature point A and the descriptor of each feature point in another grayscale image, and take feature point A and the feature point with the smallest distance to feature point A as a pair of feature point pairs.

[0074] In addition, it should be noted that if the distances between feature point A and feature point B and feature point C in another grayscale image are the same and are both minimum, feature point A and feature point B are regarded as a pair of feature point pairs, and feature point A and feature point C are also regarded as a pair of feature point pairs.

[0075] Similarly, all feature point pairs in the two grayscale images are obtained.

[0076] The difference between all feature point pairs in the two grayscale images is used as the input of the Otsu threshold segmentation algorithm, the segmentation threshold c is obtained, and the two feature point pairs corresponding to the difference between the feature point pairs less than the segmentation threshold c are used as the matching point pairs of the two grayscale images. Among them, the Otsu threshold segmentation algorithm is a well-known technology and will not be described in detail.

[0077] So far, the matching point pairs in the two grayscale images have been preliminarily obtained.

[0078] A102, determining the distance difference of any matching point pair based on the distance difference between feature points in two grayscale images between any matching point pair and all other remaining matching point pairs.

[0079] Since there may be some similar features in the two grayscale images that do not belong to the same geographical location, such as similar farmland and similar houses, when matching feature points, similar feature points in the two grayscale images that are not in the same geographical location will be matched, so there will be some mismatched matching point pairs in the obtained matching point pairs.

[0080] Usually, the RANSAC (Random Sampling Consensus) algorithm is used to screen the matching point pairs and remove these mismatched point pairs. Although the RANSAC algorithm can remove some mismatched point pairs, since there are usually fewer repeated areas in the two grayscale images, the number of matching point pairs that truly belong to the same geographical location is small. When the number of mismatched point pairs in the matching point pairs is greater than the number of correct matching point pairs, the fitting effect of the RANSAC algorithm will be poor, resulting in poor removal of mismatched point pairs, and even correct matching point pairs will be removed as mismatched point pairs.

[0081] In order to accurately remove the mismatched point pairs and retain the correct matching point pairs, the embodiment of the present application does not use the RANSAC algorithm to screen the matching point pairs, but obtains the mismatched point pairs and the correct matching point pairs through the RANSAC algorithm, and compares the features of the mismatched point pairs and the correct matching point pairs to distinguish the two, so as to improve the accuracy of the matching point pairs. Among them, the RANSAC algorithm is a well-known technology and will not be described in detail.

[0082] Among all the matching point pairs, the reason for the mismatched point pairs is that there are similar farmlands or houses in the non-overlapping areas of the two grayscale images. These similar farmlands or houses may be distributed at any position in the two grayscale images, so the position distribution of the mismatched point pairs in the two grayscale images is different. The correct matching point pairs are all located in the overlapping area of ​​the two grayscale images, so the distribution of the correct matching point pairs in the two grayscale images is relatively consistent.

[0083] In the embodiment of the present application, the distribution diagram of the matching point pairs in the overlapping area and the non-overlapping area is shown in the attached figure. Figure 7 shown.

[0084] exist Figure 7 In the figure, the gray part is the overlapping area, the white part is the non-overlapping area, and the black dots are feature points. is a mismatched point pair, and For the correct matching point pair. , it and other mismatched point pairs or correct matching point pairs, each feature point in its corresponding grayscale image, the distance between other feature points usually has a large difference, such as feature point With feature points The distance between the feature points With feature points There is a large difference in the distance between the correct matching points. , although there is a large difference between its distances with the mismatched point pairs in the two grayscale images, it has a small difference with the distances of other correct matching point pairs in the two grayscale images, such as the feature point With feature points The distance between With feature points There is a small difference in the distance between them.

[0085] Preferably, in an embodiment of the present application, the distance difference of any matching point pair is determined based on the distance difference between the feature points in the two grayscale images between any matching point pair and all other remaining matching point pairs, which is used to measure the difference in the two distances between a pair of matching point pairs and each other pair of matching point pairs in the two grayscale images.

[0086] As an implementation method of the embodiment of the present application, the i-th pair of matching points is The distance difference As an example, the specific expression is: ; Where n represents the number of matching point pairs in two grayscale images; and Represents the two feature points of the jth pair of matching points in the two grayscale images; and Represents the two feature points of the i-th pair of matching points in the two grayscale images; is the absolute value symbol; O is a distance function used to calculate the distance between two input feature points. This embodiment adopts the Euclidean distance calculation method.

[0087] It should be understood that when When the value of is large, it means that the Euclidean distances of the feature points of the i-th matching point pair and the j-th matching point pair in the two images are greatly different, which further indicates that the i-th matching point pair is more likely to be a mismatched point pair; The larger the value of is, the greater the difference in the Euclidean distances between the feature points of the i-th matching point pair and all other matching point pairs in the two images, which further indicates that the i-th matching point pair is more likely to be a mismatched point pair.

[0088] As another implementation of the embodiment of the present application, the i-th pair of matching points is The distance difference is set to: .

[0089] A2: Based on the local density distribution characteristics of the matching point pairs, a density representation value is constructed; based on the size of the density representation value and the matching of the feature points in the neighborhood, a neighborhood distribution representation value is constructed.

[0090] In addition, all correct matching point pairs are located in the repeated area of ​​the two grayscale images, and the distribution is relatively dense; the positions of incorrect matching point pairs may be distributed at any position of the image, and the distribution is relatively sparse. The local density of the two feature points in the correct matching point pair in the two grayscale images is relatively similar, while the local density of the two feature points in the incorrect matching point pair in the two grayscale images may be quite different.

[0091] A201, determining a density representation value of any matching point pair based on the average level and difference of the local density of two feature points in any matching point pair in their respective grayscale images.

[0092] The coordinates of all feature points belonging to matching point pairs in one of the grayscale images are used as the input of the DCP density clustering algorithm, and 1% of the total number of input feature points is used as the cutoff distance to obtain the local density of each feature point. The DCP density clustering algorithm is a well-known technology, and the specific process is not repeated here. Similarly, the local density of all feature points in the other grayscale image is obtained.

[0093] Preferably, in the embodiment of the present application, the density representation value of any matching point pair is determined based on the average level and difference of the local density of two feature points in any matching point pair in the grayscale image to which they belong.

[0094] As an implementation method of the embodiment of the present application, the density representation value of the i-th pair of matching points is Take as an example, it is used to represent the local density features of the i-th pair of matching points in two grayscale images. The expression is: ; In the formula, and Represents the local density of the two feature points of the i-th pair of matching points in the two grayscale images; It is the preset first constant to prevent the denominator from being 0, and its value is 0.01.

[0095] It should be understood that when The larger the value of , the greater the local density of the two feature points corresponding to the i-th pair of matching points, which means that the possibility that the i-th pair of matching points belongs to the correct matching point pair is greater; when The smaller the value of is, the more similar the local densities of the two feature points in the ith pair of matching points are, which means that the possibility that the ith pair of matching points is the correct matching point pair is greater; The larger the value of , the more likely the i-th pair of matching points is to be the correct matching point pair.

[0096] As another implementation of the embodiment of the present application, the density representation value of the i-th pair of matching points is The expression can also be set as: .

[0097] A202, determining a neighborhood distribution representation value of any matching point pair based on the difference in distribution of density representation values ​​of the matching point pairs to which the two feature points in the neighborhood belong respectively.

[0098] Taking the two feature points in a pair of matching points as the center, take the size of the two feature points in their respective grayscale images as In this embodiment, Q is set to 15. In a correct matching point pair, both feature points in the grayscale image have a large number of correctly matched feature points in the neighborhood, and the number of correctly matched points in the neighborhood of the two feature points in the correct matching point pair in their respective grayscale images is approximately equal; while in an incorrect matching point pair, both feature points in the grayscale image have a small number of correctly matched feature points in the neighborhood, or almost no correctly matched feature points, and the number of correctly matched points in the neighborhood of the two feature points in the incorrect matching point pair in their respective grayscale images may also be different.

[0099] Based on this, the density representation values ​​of all matching point pairs in the two grayscale images are used as the input of the Otsu threshold segmentation algorithm to obtain the segmentation threshold r. According to the size of the segmentation threshold r, it is possible to approximately determine which matching point pairs are likely to be correct matching point pairs and which matching point pairs are likely to be mismatched point pairs. Among them, the Otsu threshold segmentation algorithm is a well-known technology, and the specific process is not repeated here.

[0100] like If is greater than r, the i-th pair of matching points is taken as a potential correct pair of points; If it is less than or equal to r, the i-th pair of matching points is regarded as a potential wrong point pair. The two feature points in a potential correct point pair are called potential correct feature points, and the two feature points in a potential wrong point pair are called potential wrong feature points.

[0101] Preferably, in an embodiment of the present application, the method for determining the neighborhood distribution characterization value is further defined as: obtaining a segmentation threshold of the density characterization values ​​of all matching point pairs in two grayscale images; recording the two feature points in the matching point pairs to which the density characterization values ​​greater than the segmentation threshold belong as potential correct feature points; and determining the neighborhood distribution characterization value of any matching point pair by using the number and value of the potential correct feature points in the neighborhood of the two feature points in any matching point pair, as well as the difference in number.

[0102] As an implementation method of the embodiment of the present application, the neighborhood distribution representation value of the i-th pair of matching points is Take as an example, it is used to judge the matching situation of the feature points in the neighborhood of the i-th pair of matching points, specifically: ; In the formula, and is the number of potential correct feature points contained in the neighborhood of the two feature points in the i-th pair of matching points; It is the preset second constant to prevent the denominator from being 0, and its value is 0.01.

[0103] It should be understood that when and When the value of is large, it means that there are a large number of potential correct feature points in the neighborhood of the two feature points in the i-th pair of matching points, which means that the i-th pair of matching points is more likely to be the correct matching point pair; when When the value of is small, it means that the probability of the potential correct feature points appearing in the neighborhood of the two feature points in the i-th pair of matching points is close, which means that the i-th pair of matching points is more likely to be the correct matching point pair; The larger the value of , the more likely the i-th pair of matching points is to be the correct matching point pair.

[0104] As another implementation of the embodiment of the present application, the neighborhood distribution representation value of the i-th pair of matching points is The expression is set to: .

[0105] A3, combine the distance difference and neighborhood distribution representation value to select the correct matching point pairs.

[0106] Preferably, in an embodiment of the present application, the screening process of the correct matching point pairs includes: reversely fusing the neighborhood distribution representation value and the distance difference of any matching point pair to construct the correct matching discrimination value of the matching point pair; obtaining the segmentation threshold of all correct matching discrimination values; and taking the matching point pairs corresponding to the correct matching discrimination values ​​greater than the segmentation threshold as the correct matching point pairs.

[0107] It can be understood that reverse fusion is a fusion method such as subtraction and division between data. The specific reverse fusion method is determined by the implementer according to the actual situation, and this application does not impose any special restrictions.

[0108] As an implementation method of the embodiment of the present application, the correct matching judgment value of the i-th pair of matching points is Take as an example, according to the distance difference and neighborhood distribution representation value of the ith pair of matching points, the correct matching discrimination value of the ith pair of matching points is constructed to judge the possibility that the ith pair of matching points belongs to the correct matching point pair, specifically: ; In the formula, Represents the neighborhood distribution representation value of the i-th pair of matching points; Represents the i-th pair of matching points The distance difference of It is the preset third constant to prevent the denominator from being 0, and its value is 0.01.

[0109] It should be understood that the smaller the distance difference between the i-th pair of matching points and the larger the neighborhood distribution representation value, the more likely the i-th pair of matching points is to be the correct matching point pair; the correct matching discrimination value is The larger the value of , the more likely the i-th pair of matching points is to be the correct matching point pair.

[0110] Then, the correct matching judgment values ​​of all matching point pairs in the two grayscale images are calculated, and all correct matching judgment values ​​are used as the input of the Otsu threshold segmentation algorithm to obtain the segmentation threshold v. According to the size of the segmentation threshold, it is judged whether each pair of matching point pairs is a correct matching point pair.

[0111] Among them, if , then the i-th pair of matching points is taken as the correct matching point pair; if , then the i-th pair of matching points is regarded as a mismatched pair of points.

[0112] As another implementation of the embodiment of the present application, the correct matching judgment value of the i-th pair of matching points is The expression is set as: ; In the formula, e is a natural constant.

[0113] In other embodiments of the present application, the screening process of the correct matching point pairs includes: clustering the correct matching discrimination values ​​of all matching point pairs, and taking the matching point pairs corresponding to all elements in the cluster where the average value of the correct matching discrimination values ​​of all elements in the cluster is the largest as the correct matching point pairs.

[0114] As some implementation methods in other embodiments of the present application, a K-means clustering algorithm is selected to cluster the correct matching discrimination values ​​of all matching point pairs. The K-means clustering algorithm is a well-known technology and will not be described in detail. Other suitable clustering algorithms can also be selected to cluster the correct matching discrimination values ​​of all matching point pairs.

[0115] At this point, all the correct matching point pairs in the two grayscale images are obtained.

[0116] The third step is to stitch the two remote sensing images based on all the correct matching point pairs in the two grayscale images.

[0117] All the correctly matched point pairs in the two grayscale images are obtained, and the pixel pairs composed of the pixel points at the corresponding positions in the corresponding two remote sensing images are used as the input of the direct linear change (DLT) algorithm to obtain the homography matrix between the two remote sensing images. The direct linear change (DLT) algorithm is a well-known technology and the specific process will not be repeated here.

[0118] Then, a perspective transformation algorithm is used to transform one of the remote sensing images according to the homography matrix so that it is aligned with the other remote sensing image in terms of spatial position. The perspective transformation algorithm is a well-known technology and the specific process will not be described in detail.

[0119] The aligned two remote sensing images are used as inputs of a fade-in and fade-out fusion algorithm to obtain a spliced ​​image. The fade-in and fade-out fusion algorithm is a well-known technology, and the specific process will not be described in detail.

[0120] At this point, the stitched image of the two remote sensing images is obtained.

[0121] In the embodiment of the present application, the image after the two remote sensing images are stitched together is as shown in the attached figure. Figure 8 shown.

[0122] Similarly, all remote sensing images taken at consecutive times are processed according to the above-mentioned stitching method for any two remote sensing images to obtain a panoramic image after all remote sensing images are stitched together.

[0123] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a remote sensing image stitching device provided in an embodiment of the present application. In this embodiment, each unit included in the terminal is used to execute each step in an embodiment corresponding to a remote sensing image stitching method. Fig. 9 , the remote sensing image stitching device 90 comprises:

[0124] A remote sensing image acquisition module 91 is used to continuously acquire a preset number of remote sensing images through a drone;

[0125] The feature point matching module 92 is used to convert any two continuously collected remote sensing images into distribution conditions of feature points in two grayscale images and screen correct matching point pairs; specifically:

[0126] A1, determining a matching point pair based on the distance between feature points in two grayscale images; determining the distance difference between any matching point pair and all other remaining matching point pairs, respectively, in the two grayscale images;

[0127] A2: Determine the density representation value of any matching point pair based on the average level and difference of the local density of the two feature points in any matching point pair in their respective grayscale images; determine the neighborhood distribution representation value of any matching point pair based on the difference in the distribution of the density representation values ​​of the matching point pairs to which the feature points in their neighborhoods belong;

[0128] A3, combining the distance difference and the neighborhood distribution representation value to select the correct matching point pairs;

[0129] The remote sensing image stitching module 93 is used to stitch the two remote sensing images according to all correct matching point pairs in the two grayscale images.

[0130] Based on the same concept as the method embodiment of the present application, a computer device for remote sensing image stitching is proposed, wherein the computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the remote sensing image stitching method as described in the first aspect is implemented.

[0131] Based on the same inventive concept as the above method, an embodiment of the present application also provides a remote sensing image stitching medium, wherein the remote sensing image stitching medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image stitching method as described in the first aspect is implemented. Its specific functions and technical effects can be found in the method embodiment section, which will not be repeated here.

[0132] The various embodiments in the present application are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0133] It should be noted that, unless otherwise specified and limited, terms such as "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of further restrictions, an element defined by the sentence "including one..." does not exclude the existence of other identical elements in the article or device including the element. In addition, the term "and\or" used herein includes any and all combinations of one or more related listed items.

[0134] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not invented by the present application.

[0135] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A remote sensing image stitching method, characterized in that: The method comprises the following steps: Continuously collect a preset number of remote sensing images through drones; According to the distribution of feature points in two grayscale images converted from any two continuously collected remote sensing images, the correct matching point pairs are selected; specifically: A1, determining a matching point pair based on the distance between feature points in two grayscale images; determining the distance difference between any matching point pair and all other remaining matching point pairs, respectively, in the two grayscale images; A2: Determine the density representation value of any matching point pair based on the average level and difference of the local density of the two feature points in any matching point pair in their respective grayscale images; determine the neighborhood distribution representation value of any matching point pair based on the difference in the distribution of the density representation values ​​of the matching point pairs to which the feature points in their neighborhoods belong; A3, combining the distance difference and the neighborhood distribution representation value to select the correct matching point pairs; Based on all the correct matching point pairs in the two grayscale images, the two remote sensing images are stitched together; The method for determining the distance difference is further defined as: The distance difference is recorded as , ; Where n represents the number of matching point pairs in two grayscale images; and Represents the two feature points of the jth pair of matching points in the two grayscale images; and Represents the two feature points of the i-th pair of matching points in the two grayscale images; is the absolute value symbol; O is the distance function; The method for determining the neighborhood distribution characterization value is further defined as: Get the segmentation threshold of the density representation value of all matching point pairs in two grayscale images; The two feature points in the matching point pair to which the density representation value greater than the segmentation threshold belongs are recorded as potential correct feature points respectively; The neighborhood distribution representation value of any matching point pair is determined by using the number and value of potential correct feature points in their neighborhoods and the difference in number of the two feature points in any matching point pair.

2. A remote sensing image stitching method as claimed in claim 1, characterized in that: The method for determining the matching point pair is further defined as: In one of the grayscale images, the feature points that are closest to a feature point in the other grayscale image are combined to form a feature point pair; Obtain the segmentation threshold for the difference between all feature point pairs in two grayscale images; The two feature point pairs whose differences are less than the segmentation threshold are regarded as the matching point pairs of the two grayscale images.

3. A remote sensing image stitching method as claimed in claim 1, characterized in that: The screening process of the correct matching point pairs includes: The neighborhood distribution representation value and distance difference of any matching point pair are reversely fused to construct the correct matching discrimination value of the matching point pair; Get the segmentation threshold of all correctly matched discriminant values; The matching point pairs corresponding to the correct matching discriminant values ​​greater than the segmentation threshold are regarded as the correct matching point pairs.

4. A remote sensing image stitching method as claimed in claim 1, characterized in that: The method for stitching two remote sensing images is: All the correctly matched point pairs in the two grayscale images are obtained, and the homography matrix between the two remote sensing images is calculated using the pixel point pairs composed of the pixel points at the corresponding positions in the two corresponding remote sensing images; Use homography matrix to transform and align two remote sensing images; The two aligned remote sensing images are stitched together to obtain a stitched image.

5. A remote sensing image stitching method as claimed in claim 4, characterized in that: After the two remote sensing images are stitched together, all remote sensing images collected at consecutive times are processed according to the stitching method of the two remote sensing images to obtain a panoramic image after all remote sensing images are stitched together.

6. A remote sensing image stitching device, implementing a remote sensing image stitching method as claimed in claim 1, characterized in that: The remote sensing image stitching device comprises: A remote sensing image acquisition module is used to continuously acquire a preset number of remote sensing images through a drone; The feature point matching module is used to convert any two continuously acquired remote sensing images into the distribution of feature points in two grayscale images and select the correct matching point pairs; specifically: A1, determining a matching point pair based on the distance between feature points in two grayscale images; determining the distance difference between any matching point pair and all other remaining matching point pairs, respectively, in the two grayscale images; A2: Determine the density representation value of any matching point pair based on the average level and difference of the local density of the two feature points in any matching point pair in their respective grayscale images; determine the neighborhood distribution representation value of any matching point pair based on the difference in the distribution of the density representation values ​​of the matching point pairs to which the feature points in their neighborhoods belong; A3, combining the distance difference and the neighborhood distribution representation value to select the correct matching point pairs; The remote sensing image stitching module is used to stitch two remote sensing images based on all the correct matching point pairs in the two grayscale images.

7. A computer device for remote sensing image stitching, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a remote sensing image stitching method as described in any one of claims 1 to 5 are implemented.

8. A remote sensing image stitching medium, wherein the remote sensing image stitching medium stores a computer program, characterized in that: When the computer program is executed by a processor, the steps of a remote sensing image stitching method as described in any one of claims 1 to 5 are implemented.

Images