Matching method, device, computer equipment and medium based on feature descriptor

Abstract

Embodiments of the present invention provide a feature descriptor-based matching method, apparatus, computer equipment, and medium, relating to the field of graphics, image, and point cloud processing technology. The method includes the following steps: obtaining image data and point cloud data of a target surface from input data; performing feature extraction on the image data to generate an image feature descriptor; performing feature extraction on the point cloud data to generate a point cloud feature descriptor; fusing the image feature descriptor with the point cloud feature descriptor to generate a texture and topography fused feature descriptor; normalizing the texture and topography fused feature descriptor to generate a normalized feature descriptor; and using the normalized feature descriptor to calculate the similarity of points on the target surface at different times. When the similarity is greater than or equal to a set threshold, the two points are considered the same point. This solution improves matching accuracy by combining feature descriptors for image texture and point cloud topography.

Description

Technical Field

[0001] The present invention relates to the field of graphic image and point cloud processing technology, and in particular to a matching method, device, computer equipment and medium based on feature descriptors. Background Art

[0002] Images and point clouds are fundamental two-dimensional and three-dimensional data used to describe the real world, respectively. After acquiring images or point cloud data of the three-dimensional world, further analysis of these features can be used to identify target types, locate target positions, estimate pose changes, detect target movement, and monitor target deformation. Data processing technologies based on images or point clouds are widely used in areas such as positioning and navigation, target recognition, change detection, and three-dimensional deformation measurement.

[0003] In order to discriminate target types, detect target positions, and measure deformations, it is first necessary to complete image homonymous point matching or point cloud matching. This means identifying the same point in the corresponding image or point cloud before and after the target moves or deforms. The prerequisite for image matching or point cloud matching is to accurately extract the features of the target points. Existing methods mostly perform feature extraction on a single image or point cloud, and then achieve homonymous point matching in a single image or the same point matching in a single point cloud data. With advances in testing instruments and methods, it is becoming increasingly easier to simultaneously obtain images and point cloud data of the target surface. However, methods for simultaneously extracting target features and determining whether they are the same point using both image and point cloud data are still lacking. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a matching method based on feature descriptors to solve the technical problem in the prior art that it is impossible to simultaneously use image and point cloud data to extract target features and determine whether they are the same point. The method includes:

[0005] Acquire image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor;

[0006] Fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and topography fused feature descriptor, and normalizing the texture and topography fused feature descriptor to generate a normalized feature descriptor;

[0007] The normalized feature descriptor is used to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

[0008] The present invention also provides a matching device based on feature descriptors to solve the technical problem in the prior art that it is impossible to simultaneously use image and point cloud data to extract target features and determine whether they are the same point. The device includes:

[0009] a descriptor generation module, configured to obtain image data and point cloud data of a target surface in input data, perform feature extraction on the image data to generate an image feature descriptor, and perform feature extraction on the point cloud data to generate a point cloud feature descriptor;

[0010] a feature normalization module, configured to fuse the image feature descriptor and the point cloud feature descriptor to generate a texture and topography fused feature descriptor, and perform normalization processing on the texture and topography fused feature descriptor to generate a normalized feature descriptor;

[0011] The homonymous point matching module is used to calculate the similarity of the points on the target surface at different times using the normalized feature descriptor. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

[0012] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor implements any of the above-mentioned feature descriptor-based matching methods to solve the technical problem in the prior art that it is impossible to simultaneously utilize image and point cloud data to extract target features and determine whether they are the same point.

[0013] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned feature descriptor-based matching methods to solve the technical problem in the prior art that it is impossible to simultaneously use image and point cloud data to extract target features and determine whether they are the same point.

[0014] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0015] By combining the texture and morphology fusion feature descriptor and matching method of image texture information and point cloud morphology information, the new feature descriptor simultaneously utilizes the texture information of image data and the morphology information provided by point cloud data, making the data content richer and providing the possibility for more accurate matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is a flow chart of a feature descriptor-based matching method provided by an embodiment of the present invention;

[0018] Figure 2 is a flow chart of an embodiment of the present invention for implementing the above-mentioned feature descriptor-based matching method;

[0019] Figure 3 This is a structural block diagram of a computer device provided by an embodiment of the present invention;

[0020] Figure 4 This is a structural block diagram of a feature descriptor-based matching device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0022] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the features in the following embodiments and embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of this application.

[0023] In an embodiment of the present invention, a matching method based on feature descriptors is provided, such as Figure 1 and Figure 2 As shown, the method includes:

[0024] Step S101: acquiring image data and point cloud data of a target surface in input data, performing feature extraction on the image data to generate an image feature descriptor, and performing feature extraction on the point cloud data to generate a point cloud feature descriptor;

[0025] Step S102: fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and topography fused feature descriptor, and normalizing the texture and topography fused feature descriptor to generate a normalized feature descriptor;

[0026] Step S103: using the normalized feature descriptor, calculate the similarity of the points on the target surface at different times; when the similarity is less than or equal to a set threshold, the two points are the same point; if the similarity is greater than the set threshold, the two points are different points.

[0027] In specific implementation, the following steps are performed to extract features from the image data to generate an image feature descriptor, and to extract features from the point cloud data to generate a point cloud feature descriptor:

[0028] Obtain grayscale texture features of the image data, wherein the grayscale texture features include descriptions of corners, edges, grayscale gradients, and normal vectors; quantitatively describe the grayscale texture features in the form of vectors to generate image feature descriptors; obtain morphological information of the point cloud data, wherein the morphological information includes descriptions of spatial position, curvature, and normal vectors; quantitatively describe the morphological information in the form of vectors to generate point cloud feature descriptors.

[0029] Specifically, the image and point cloud data of the target surface are obtained by including but not limited to using a camera and a laser device.

[0030] Specifically, the image feature descriptor is obtained by processing the image data of the target surface, describing the grayscale texture features in the image, including corners, edges, grayscale gradients and normal vectors, and quantitatively describing them in the form of vectors.

[0031] The point cloud feature descriptor is obtained by processing the acquired target surface point cloud data, describing the morphological information in the point cloud, including spatial position, curvature, normal vector, etc., and quantitatively describing it in the form of a vector.

[0032] In a specific implementation, the image feature descriptor and the point cloud feature descriptor are fused to generate a texture and shape fusion feature descriptor through the following steps:

[0033] The construction mode of the feature descriptor after the fusion of texture and morphology is specified as follows: the image feature descriptor is in front or the point cloud feature descriptor is in front; when the image feature descriptor is in front, the element expressions of the first m elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the m elements in the image feature descriptor, and the element expressions of the m+1 to k elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the n elements in the point cloud feature descriptor, where k=m+n; when the point cloud feature descriptor is in front, the element expressions of the first n elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the n elements in the point cloud feature descriptor, and the element expressions of the n+1 to k elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the m elements in the image feature descriptor.

[0034] Specifically, let A represent the feature descriptor vector of the texture and topography fusion feature descriptor (i.e., a new feature descriptor that combines image texture information and point cloud topography information), with k elements and the element expression a. B represents the image feature descriptor vector, with m elements and the element expression b. C represents the point cloud feature descriptor vector, with n elements and the element expression c. The relationship between the number of elements in the three descriptors is: k = m + n. When the image feature descriptor is used first, the first m elements a in the texture and topography fusion feature descriptor A are equal to the m elements b in B, respectively. The m+1 to kth elements a in the texture and topography fusion feature descriptor A are equal to the nth elements in C, respectively. When the point cloud feature descriptor is used first, the first n elements a in the texture and topography fusion feature descriptor A are equal to the nth elements c in C, respectively. The n+1 to kth elements a in the texture and topography fusion feature descriptor A are equal to the mth elements in B, respectively.

[0035] The specific expression is:

[0036] or .in, a i Represents the elements in the feature descriptor A after the fusion of texture and shape, b i and b i-n An element representing an image feature descriptor. c i and c i-m Represents an element of a point cloud feature descriptor.

[0037] In a specific implementation, the following steps are performed to normalize the feature descriptor after the texture and morphology are fused to generate a normalized feature descriptor:

[0038] Calculate the image feature standard deviation std(B) of the elements of the image feature descriptor; calculate the point cloud feature standard deviation std(C) of the elements of the point cloud feature descriptor; take the sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) as the standard deviation sum; calculate the normalized feature descriptor through the texture and morphology fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the standard deviation sum.

[0039] Specifically, the image feature descriptor and point cloud feature descriptor are adaptively weighted (normalized). The expression is:

[0040] or .

[0041] in, a i Represents the elements of the feature descriptor after the fusion of texture and shape, Represents the elements of the normalized feature descriptor. std(B) represents the standard deviation of the image feature descriptor elements, and std(C) represents the standard deviation of the point cloud feature descriptor elements.

[0042] Using the adaptive weighted normalization method for the texture and topography fusion feature descriptor, the feature information extracted from the texture and topography fusion feature descriptor is effectively highlighted. When texture is missing or sparse in the image and topography features are prominent in the point cloud, std(B) is small and std(C) is large. By assigning higher weights to the point cloud features, the role of the point cloud features in the descriptor is enhanced. When topography features are rare in the point cloud and the normal vectors of each point cloud point tend to be consistent, std(C) is small and std(B) is large. By assigning greater weights to the image feature descriptor, the role of the image features in the descriptor is enhanced.

[0043] Specifically, the feature descriptor after texture and topography fusion is a descriptor that combines image texture information and point cloud topography information. It connects the image feature descriptor and the point cloud feature descriptor and quantitatively describes them in the form of a vector. The first part of the elements of the vector formed by the connection is composed of image feature descriptors or point cloud feature descriptors, and the second part of the elements is composed of point cloud feature descriptors or image feature descriptors. Linear connection is to arrange the elements of the image feature descriptor and the point cloud feature descriptor in sequence, with the last element of the first feature descriptor adjacent to the first element of the second feature descriptor. The total number of elements in the feature descriptor after texture and topography fusion is the sum of the number of elements in the image feature descriptor and the number of elements in the point cloud feature descriptor.

[0044] In a specific implementation, the following steps are performed to calculate the normalized feature descriptor using the texture and topography fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C), and the sum of the standard deviations:

[0045] When the image feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; when the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the element of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, and k=m+n.

[0046] In specific implementation, the following steps are used to calculate the similarity of points on the target surface at different times using the normalized feature descriptor:

[0047] The normalized feature descriptors of the points are obtained respectively, and each first element expression of the first normalized feature descriptor and each second element expression of the second normalized feature descriptor are obtained as the first normalized feature descriptor and the second normalized feature descriptor respectively; the similarity between the first normalized feature descriptor and the second normalized feature descriptor is calculated through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor.

[0048] In a specific implementation, the following steps are performed to calculate the similarity between the first normalized feature descriptor and the second normalized feature descriptor through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor:

[0049] ,in, d is the similarity, is the first normalized feature descriptor, is the kth expression of the first element, is the second normalized feature descriptor, is the kth expression of the second element, and k is the total number of elements of the first normalized feature descriptor and the second normalized feature descriptor.

[0050] Specifically, based on the normalized feature descriptor, an embodiment of the present invention provides a coordinate point matching method, including:

[0051] Image descriptors and point cloud descriptors are extracted from the two acquired image point cloud data sets, respectively. The proposed method is then used to process the descriptors to obtain normalized feature descriptors. The similarity of the feature descriptors in the two sets of data is calculated, such as by using methods like Euclidean distance to determine the distance between the two descriptors. A predetermined algorithm, such as a threshold, is used to determine the similarity between the descriptors, ultimately achieving matching.

[0052] Assume that the normalized feature descriptor obtained for the first time is , the kth element expression is , the normalized feature descriptor obtained for the second time is , the kth element expression is The basis for judging the similarity between two descriptors during the matching process is:

[0053] .

[0054] T represents the set threshold. When d≤T, the match is considered successful and the two points are the same point (i.e., points with the same name). Otherwise, the match fails and the two points are judged not to be the same point.

[0055] Currently, there are many feature descriptors extracted from images, such as SIFT descriptors and SURF descriptors, and there are also many descriptors extracted from point clouds, such as PFH descriptors, FPFH descriptors, and SPFH descriptors. The present invention is further described below in conjunction with the proposed image feature descriptors and point cloud feature descriptors, but the present invention is not limited to the following embodiments. It is particularly important to note that the present invention can use existing image feature descriptors and point cloud feature descriptors to generate feature descriptors after the fusion of texture and morphology, and can also use newly proposed image feature descriptors and point cloud feature descriptors (such as feature descriptors extracted based on deep learning methods) to generate feature descriptors after the fusion of texture and morphology, which is also within the scope of protection of the present invention.

[0056] Example 1:

[0057] First, we extract the SIFT (Scale-Invariant Feature Transform) feature descriptor from the image and the FPFH (Fast Point Feature Histogram) feature descriptor from the point cloud. The SIFT feature descriptor is a 128-dimensional vector, while the FPFH is a 33-dimensional vector.

[0058] The above method is used to construct a 161-dimensional texture and topography fusion feature descriptor. The first 128 elements of the texture and topography fusion feature descriptor correspond to the elements of the SIFT feature descriptor, and the elements from the 129th to the 161st elements correspond to the elements of the FPFH feature descriptor.

[0059] The specific expression is:

[0060] .

[0061] in, a i Represents the elements in the feature descriptor A after the fusion of texture and shape, b i Represents an element of a SIFT feature descriptor. c i-128 Represents an element of the FPFH feature descriptor.

[0062] Then, the proposed adaptive weighted normalization method is used to process the constructed texture and morphology fusion feature descriptor, and the standard deviation of the SIFT feature descriptor and the FPFH feature descriptor vector is calculated. Then, the proposed method is used to process each element of the constructed texture and morphology fusion feature descriptor to obtain a new vector with a modulus of 1.

[0063] The calculation method is:

[0064]

[0065] in, a i Represents the elements of the feature descriptor after the fusion of texture and shape, Represents the elements of the normalized feature descriptor. std(SIFT) represents the standard deviation of the SIFT feature descriptor elements, and std(FPFH) represents the standard deviation of the FPFH feature descriptor elements.

[0066] Finally, the proposed matching method is used to calculate the distance between the two sets of normalized feature descriptors to determine the similarity of the two descriptors.

[0067] The distance calculation formula is:

[0068]

[0069] in, and is the normalized feature descriptor obtained twice, and For its element expression.

[0070] When d≤T, the match is considered successful, where T represents the set threshold; otherwise, the match fails.

[0071] Example 2:

[0072] First, we extract the SURF (Speeded Up Robust Features) feature descriptor from the image and the PFH (Point Feature Histogram) feature descriptor from the point cloud. SURF feature descriptors are 64-dimensional vectors, while PFH feature descriptors are 125-dimensional vectors.

[0073] The method described above preliminarily constructs a 189-dimensional texture and topography fusion feature descriptor. The first 64 elements of the texture and topography fusion feature descriptor correspond to the elements of the SURF feature descriptor, and the elements from the 65th to the 189th correspond to the elements of the PFH feature descriptor.

[0074] The specific expression is:

[0075] .

[0076] in, a i Represents the elements in the feature descriptor after the fusion of texture and shape, b i Represents an element of a SURF feature descriptor. ci-64 Represents an element of the PFH feature descriptor.

[0077] The proposed adaptive weighted normalization method is then used to process the constructed texture and topography fusion feature descriptor, and the standard deviation of the SURF feature descriptor and the PFH feature descriptor vectors is calculated. The proposed method is used to process each element of the constructed texture and topography fusion feature descriptor to obtain a new vector with a modulus of 1.

[0078] The calculation method is:

[0079] .

[0080] in, a i Represents the elements of the feature descriptor after the fusion of texture and shape, Represents the elements of the normalized feature descriptor. std(SURF) represents the element-wise standard deviation of the SURF feature descriptor, and std(PFH) represents the element-wise standard deviation of the PFH feature descriptor.

[0081] Finally, the proposed matching method is used to calculate the distance between the two sets of normalized feature descriptors to determine the similarity of the two descriptors.

[0082] The distance calculation formula is:

[0083] .

[0084] in, and is the normalized feature descriptor obtained twice, and For its element expression.

[0085] When d≤T, the match is considered successful, where T represents the set threshold; otherwise, the match fails.

[0086] In this embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory 301, a processor 302 and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned matching methods based on feature descriptors is implemented.

[0087] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0088] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned feature descriptor-based matching methods.

[0089] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include transitory media such as modulated data signals and carrier waves.

[0090] Based on the same inventive concept, a matching device based on a feature descriptor is also provided in an embodiment of the present invention, as described in the following embodiments. Since the principle of solving the problem by the matching device based on a feature descriptor is similar to that of the matching method based on a feature descriptor, the implementation of the matching device based on a feature descriptor can refer to the implementation of the matching method based on a feature descriptor, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0091] Figure 4 This is a structural block diagram of a matching device based on a feature descriptor according to an embodiment of the present invention. Figure 4 As shown, it includes: a descriptor generation module 401, a feature normalization module 402 and a homonymous point matching module 403. The structure is described below.

[0092] The descriptor generation module 401 is used to obtain image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, and perform feature extraction on the point cloud data to generate a point cloud feature descriptor;

[0093] A feature normalization module 402 is configured to fuse the image feature descriptor and the point cloud feature descriptor to generate a texture and topography fused feature descriptor, and perform normalization processing on the texture and topography fused feature descriptor to generate a normalized feature descriptor;

[0094] The same-name point matching module 403 is used to use the normalized feature descriptor to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

[0095] In one embodiment, the descriptor generation module includes:

[0096] A texture feature acquisition unit, configured to acquire grayscale texture features of the image data, wherein the grayscale texture features include descriptions of corner points, edges, grayscale gradients, and normal vectors;

[0097] An image descriptor generating unit, configured to quantitatively describe the grayscale texture features in the form of vectors to generate image feature descriptors;

[0098] A shape information acquisition unit, configured to acquire shape information of the point cloud data, wherein the shape information includes descriptions of spatial position, curvature, and normal vector;

[0099] The point cloud descriptor unit is used to quantitatively describe the topography information in the form of a vector to generate a point cloud feature descriptor.

[0100] In one embodiment, the feature normalization module includes:

[0101] a composition mode setting unit, used to specify the composition mode of the feature descriptor after the fusion of texture and topography, namely, the image feature descriptor is placed first or the point cloud feature descriptor is placed first;

[0102] a first element correspondence unit, configured to, when the image feature descriptor is in front, correspond one-to-one between the element expressions of the first m elements in the texture and topography fusion feature descriptor and the element expressions of the m elements in the image feature descriptor, and correspond one-to-one between the element expressions of the m+1th to the kth elements in the texture and topography fusion feature descriptor and the element expressions of the n elements in the point cloud feature descriptor, where k=m+n;

[0103] The second element corresponding unit is used to, when the point cloud feature descriptor is in front, correspond one-to-one the element expressions of the first n elements in the feature descriptor after the texture and morphology are fused with the element expressions of the n elements in the point cloud feature descriptor, and correspond one-to-one the element expressions of the n+1 to k-th elements in the feature descriptor after the texture and morphology are fused with the element expressions of the m elements in the image feature descriptor.

[0104] In one embodiment, the feature normalization module further includes:

[0105] An image feature standard deviation calculation unit, configured to calculate an image feature standard deviation std(B) of an element of the image feature descriptor;

[0106] a point cloud feature standard deviation calculation unit, configured to calculate a point cloud feature standard deviation std(C) of an element of the point cloud feature descriptor;

[0107] a standard deviation sum calculation unit, configured to take the sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) as the standard deviation sum;

[0108] A descriptor normalization unit is used to calculate the normalized feature descriptor through the feature descriptor after the texture and morphology fusion, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the sum of the standard deviations.

[0109] In one embodiment, the descriptor normalization unit is configured to, when the image feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; when the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the element of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, and k=m+n.

[0110] In one embodiment, the feature normalization module further includes:

[0111] a normalized feature descriptor generating unit, configured to respectively obtain the normalized feature descriptors of the points as a first normalized feature descriptor and a second normalized feature descriptor;

[0112] an expression acquiring unit, configured to acquire an expression of each first element of the first normalized feature descriptor and an expression of each second element of the second normalized feature descriptor;

[0113] A similarity calculation unit is used to calculate the similarity between the first normalized feature descriptor and the second normalized feature descriptor through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor.

[0114] In one embodiment, the similarity calculation unit is used to ,in, d is the similarity, is the first normalized feature descriptor, is the kth expression of the first element, is the second normalized feature descriptor, is the kth expression of the second element, and k is the total number of elements of the first normalized feature descriptor and the second normalized feature descriptor.

[0115] The embodiments of the present invention achieve the following technical effects:

[0116] The embodiments of the present invention provide a texture and morphology fused feature descriptor and a matching method by combining image texture information and point cloud morphology information. The texture and morphology fused feature descriptor simultaneously utilizes the texture information of the image data and the morphology information provided by the point cloud data, and the data content is richer, which provides the possibility for more accurate matching; an adaptive weighted feature descriptor combination method is provided, which more fully utilizes the rich texture information of the image data and the rich morphology feature information of the point cloud data, and avoids the matching difficulties caused by the lack of image texture and the flatness of the point cloud morphology through adaptive weighting; a matching method is provided to realize the judgment of the similarity of the normalized feature descriptors, which provides a judgment basis for realizing the matching of same-name points based on the proposed normalized feature descriptor.

[0117] Obviously, those skilled in the art should understand that the various modules or steps of the above-mentioned embodiments of the present invention can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0118] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A matching method based on feature descriptors, characterized in that: include: Acquire image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor; Fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and topography fused feature descriptor, and normalizing the texture and topography fused feature descriptor to generate a normalized feature descriptor; Normalizing the texture and topography fusion feature descriptor to generate a normalized feature descriptor, including: Calculating the image feature standard deviation std(B) of the elements of the image feature descriptor; calculating the point cloud feature standard deviation std(C) of the elements of the point cloud feature descriptor; taking the sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) as the standard deviation sum; calculating the normalized feature descriptor by using the texture and topography fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C), and the standard deviation sum; When the image feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; When the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the element of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, k=m+n; The normalized feature descriptor is used to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

2. The matching method based on feature descriptors according to claim 1, wherein: Fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and shape fusion feature descriptor, including: Specifying a construction mode of the feature descriptor after the fusion of texture and topography is that the image feature descriptor is placed first or the point cloud feature descriptor is placed first; When the image feature descriptor is in the front, the element expressions of the first m elements in the texture and shape fusion feature descriptor are matched one-to-one with the element expressions of the m elements in the image feature descriptor, and the element expressions of the m+1th to kth elements in the texture and shape fusion feature descriptor are matched one-to-one with the element expressions of the n elements in the point cloud feature descriptor, where k=m+n; When the point cloud feature descriptor is in front, the element expressions of the first n elements in the feature descriptor after the texture and morphology are fused are matched one-to-one with the element expressions of the n elements in the point cloud feature descriptor, and the element expressions of the n+1 to k elements in the feature descriptor after the texture and morphology are fused are matched one-to-one with the element expressions of the m elements in the image feature descriptor.

3. The matching method based on feature descriptors according to claim 1, wherein: Calculating the similarity of points on the target surface at different times using the normalized feature descriptor includes: Respectively obtaining the normalized feature descriptors of the points on the target surface at different times as a first normalized feature descriptor and a second normalized feature descriptor; Obtaining each first element expression of the first normalized feature descriptor and each second element expression of the second normalized feature descriptor; The similarity between the first normalized feature descriptor and the second normalized feature descriptor is calculated by using all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor.

4. The matching method based on feature descriptors according to claim 3, wherein: Calculating the similarity between the first normalized feature descriptor and the second normalized feature descriptor by using all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor includes: ,in, d is the similarity, is the first normalized feature descriptor, is the kth expression of the first element, is the second normalized feature descriptor, is the kth expression of the second element, and k is the total number of elements of the first normalized feature descriptor and the second normalized feature descriptor.

5. The feature descriptor-based matching method according to any one of claims 1 to 4, wherein: Extracting features from the image data to generate an image feature descriptor, and extracting features from the point cloud data to generate a point cloud feature descriptor, including: Acquire grayscale texture features of the image data, wherein the grayscale texture features include descriptions of corner points, edges, grayscale gradients, and normal vectors; Quantitatively describing the grayscale texture features in the form of vectors to generate image feature descriptors; Acquiring topographic information of the point cloud data, wherein the topographic information includes descriptions of spatial position, curvature, and normal vector; The morphological information is quantitatively described in the form of a vector to generate a point cloud feature descriptor.

6. A matching device based on feature descriptors, characterized in that: include: a descriptor generation module, configured to obtain image data and point cloud data of a target surface in input data, perform feature extraction on the image data to generate an image feature descriptor, and perform feature extraction on the point cloud data to generate a point cloud feature descriptor; a feature normalization module, configured to fuse the image feature descriptor and the point cloud feature descriptor to generate a texture and topography fused feature descriptor, and perform normalization processing on the texture and topography fused feature descriptor to generate a normalized feature descriptor; Feature normalization module, including: An image feature standard deviation calculation unit, configured to calculate an image feature standard deviation std(B) of an element of the image feature descriptor; a point cloud feature standard deviation calculation unit, configured to calculate a point cloud feature standard deviation std(C) of an element of the point cloud feature descriptor; a standard deviation sum calculation unit, configured to take the sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) as the standard deviation sum; a descriptor normalization unit, configured to calculate the normalized feature descriptor using the texture and topography fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C), and the sum of the standard deviations; The descriptor normalization unit is also used when the image feature descriptor is in front. ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; when the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the elements of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, k=m+n; the same-name point matching module is used to use the normalized feature descriptor to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the feature descriptor-based matching method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for executing the feature descriptor-based matching method according to any one of claims 1 to 5.

Images