Matching method and device based on feature descriptors, computer equipment and medium

Abstract

The embodiment of the invention provides a matching method and device based on a feature descriptor, computer equipment and a medium, and relates to the technical field of graphic image and point cloud processing.The method comprises the following steps that image data and point cloud data of a target surface in input data are obtained, feature extraction is conducted on the image data, and a feature descriptor is obtained; image feature descriptors are generated, feature extraction is carried out on the point cloud data, and point cloud feature descriptors are generated; fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and morphology fused feature descriptor, and performing normalization processing on the texture and morphology fused feature descriptor to generate a normalized feature descriptor; and calculating the similarity of points on the target surface at different times by using the normalized feature descriptors, and when the similarity is greater than or equal to a set threshold, determining that the two points are the same point. According to the scheme, the matching accuracy is improved by combining the feature descriptors of the image texture and the point cloud morphology.

Description

Technical Field

[0001] The present invention relates to the technical field of graphic image and point cloud processing, and in particular to a matching method, device, computer equipment and medium based on feature descriptors. Background Art

[0002] Images and point clouds are two-dimensional and three-dimensional basic data used to describe the real world. After obtaining images or point cloud data of the three-dimensional world, the image or point cloud data features can be further analyzed to identify the target type, locate the target position, estimate the posture change, detect the target movement, and monitor the target deformation. Data processing technology based on image or point cloud data is widely used in the fields of positioning navigation, target recognition, change detection and three-dimensional deformation measurement.

[0003] In order to achieve target type discrimination, target position detection and deformation measurement, it is first necessary to complete image homonymous point matching or point cloud matching, that is, to identify the same point in the corresponding image or point cloud before and after the target moves or deforms. The premise of achieving image matching or point cloud matching is to extract accurate features of the target point. Existing methods mostly perform feature extraction on a single image or point cloud, and then achieve homonymous point matching in a single image or the same point matching in a single point cloud data. With the advancement of testing instruments and methods, it has become easier and easier to simultaneously obtain the surface image and point cloud data of the measured target, but there is still a lack of methods to use image and point cloud data to extract target features and determine whether they are the same point. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a matching method based on feature descriptors to solve the technical problem in the prior art that it is impossible to simultaneously use image and point cloud data to extract target features and determine whether they are the same point. The method includes: Acquire image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor; The image feature descriptor and the point cloud feature descriptor are fused to generate a texture and shape fused feature descriptor, and the texture and shape fused feature descriptor is normalized to generate a normalized feature descriptor; The normalized feature descriptor is used to calculate the similarity of points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

[0005] The embodiment of the present invention also provides a matching device based on feature descriptors to solve the technical problem in the prior art that it is impossible to simultaneously use image and point cloud data to extract target features and determine whether they are the same point. The device includes: A descriptor generation module is used to obtain image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor; A feature normalization module, used for fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and shape fused feature descriptor, and performing normalization processing on the texture and shape fused feature descriptor to generate a normalized feature descriptor; The same-name point matching module is used to use the normalized feature descriptor to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

[0006] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned feature descriptor-based matching methods when executing the computer program, so as to solve the technical problem in the prior art that it is impossible to simultaneously utilize image and point cloud data to extract target features and determine whether they are the same point.

[0007] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned feature descriptor-based matching methods, so as to solve the technical problem in the prior art that it is impossible to simultaneously use image and point cloud data to extract target features and determine whether they are the same point.

[0008] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: By combining the texture and morphology fusion feature descriptor and matching method of image texture information and point cloud morphology information, the new feature descriptor simultaneously utilizes the texture information of image data and the morphology information provided by point cloud data. The data content is richer, which provides the possibility for more accurate matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 is a flow chart of a matching method based on feature descriptors provided by an embodiment of the present invention; Figure 2 is a flow chart of implementing the above-mentioned feature descriptor-based matching method provided by an embodiment of the present invention; Figure 3 is a structural block diagram of a computer device provided by an embodiment of the present invention; Figure 4 It is a structural block diagram of a matching device based on feature descriptors provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0012] The following describes the implementation methods of the present application through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0013] In an embodiment of the present invention, a matching method based on feature descriptors is provided, such as Figure 1 and Figure 2 As shown, the method includes: Step S101: acquiring image data and point cloud data of a target surface in input data, performing feature extraction on the image data to generate an image feature descriptor, and performing feature extraction on the point cloud data to generate a point cloud feature descriptor; Step S102: fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and shape fused feature descriptor, and normalizing the texture and shape fused feature descriptor to generate a normalized feature descriptor; Step S103: using the normalized feature descriptor, calculate the similarity of the points on the target surface at different times; when the similarity is less than or equal to a set threshold, the two points are the same point; if the similarity is greater than the set threshold, the two points are different points.

[0014] In specific implementation, the following steps are performed to extract features from the image data to generate an image feature descriptor, and to extract features from the point cloud data to generate a point cloud feature descriptor: Obtain grayscale texture features of the image data, wherein the grayscale texture features include descriptions of corner points, edges, grayscale gradients, and normal vectors; quantitatively describe the grayscale texture features in the form of vectors to generate image feature descriptors; obtain morphological information of the point cloud data, wherein the morphological information includes descriptions of spatial position, curvature, and normal vectors; quantitatively describe the morphological information in the form of vectors to generate point cloud feature descriptors.

[0015] Specifically, the image and point cloud data of the target surface are obtained by including but not limited to using a camera and a laser device.

[0016] Specifically, the image feature descriptor is obtained by processing the image data of the target surface, and describes the grayscale texture features in the image, including corners, edges, grayscale gradients, and normal vectors, and describes them quantitatively in the form of vectors.

[0017] The point cloud feature descriptor is obtained by processing the acquired target surface point cloud data, describing the morphological information in the point cloud, including spatial position, curvature, normal vector, etc., and quantitatively describing it in the form of a vector.

[0018] In a specific implementation, the image feature descriptor and the point cloud feature descriptor are fused to generate a texture and shape fused feature descriptor through the following steps: The construction mode of the feature descriptor after the fusion of texture and morphology is specified as follows: the image feature descriptor is in front or the point cloud feature descriptor is in front; when the image feature descriptor is in front, the element expressions of the first m elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the m elements in the image feature descriptor, and the element expressions of the m+1 to k elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the n elements in the point cloud feature descriptor, where k=m+n; when the point cloud feature descriptor is in front, the element expressions of the first n elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the n elements in the point cloud feature descriptor, and the element expressions of the n+1 to k elements in the feature descriptor after the fusion of texture and morphology are corresponded one-to-one with the element expressions of the m elements in the image feature descriptor.

[0019] Specifically, assume that A represents the feature descriptor vector of the texture and shape fusion feature descriptor (i.e., a new feature descriptor that combines image texture information and point cloud shape information), which has k elements and is expressed as a. B represents the image feature descriptor vector, which has m elements and is expressed as b. C represents the point cloud feature descriptor vector, which has n elements and is expressed as c. The relationship between the number of elements of the three descriptors is: k=m+n. When the image feature descriptor is in the front, the first m elements a in the texture and shape fusion feature descriptor A are equal to the m elements b in B, and the m+1 to k-th elements a in the texture and shape fusion feature descriptor A are equal to the n elements in C. When the point cloud feature descriptor is in the front, the first n elements a in the texture and shape fusion feature descriptor A are equal to the n elements c in C, and the n+1 to k-th elements a in the texture and shape fusion feature descriptor A are equal to the m elements in B.

[0020] The specific expression is: or .in, a i Represents the elements in the feature descriptor A after the fusion of texture and shape, b i and b i-n An element representing an image feature descriptor. c i and c i-m Represents an element of a point cloud feature descriptor.

[0021] In specific implementation, the following steps are performed to normalize the feature descriptor after the texture and shape fusion to generate a normalized feature descriptor: Calculate the image feature standard deviation std(B) of the elements of the image feature descriptor; calculate the point cloud feature standard deviation std(C) of the elements of the point cloud feature descriptor; take the sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) as the sum of standard deviations; calculate the normalized feature descriptor through the feature descriptor after texture and morphology fusion, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the sum of standard deviations.

[0022] Specifically, the image feature descriptor and the point cloud feature descriptor are adaptively weighted (normalized). The expression is: or .

[0023] in, ai Represents the element of the feature descriptor after the fusion of texture and shape, Represents the elements of the normalized feature descriptor. std(B) represents the standard deviation of the image feature descriptor element, and std(C) represents the standard deviation of the point cloud feature descriptor element.

[0024] By using the adaptive weighted normalization method of the feature descriptor after texture and morphology fusion, the feature information extracted from the feature descriptor after texture and morphology fusion is effectively highlighted. When the texture is missing or sparse in the image and the morphological features are obvious in the point cloud, std(B) is small and std(C) is large. By giving a higher weight to the point cloud features, the role of the point cloud features in the descriptor is improved. When there are few morphological features in the point cloud and the normal vectors of each point in the point cloud tend to be consistent, std(C) is small and std(B) is large. By giving a larger weight to the image feature descriptor, the role of the image features in the descriptor is improved.

[0025] Specifically, the feature descriptor after texture and morphology fusion is a descriptor that combines image texture information and point cloud morphology information. It connects the image feature descriptor and the point cloud feature descriptor and describes them quantitatively in the form of a vector. The first part of the elements of the vector formed by connection is composed of image feature descriptors or point cloud feature descriptors, and the second part of the elements is composed of point cloud feature descriptors or image feature descriptors. Linear connection is to arrange the elements of the image feature descriptor and the point cloud feature descriptor in sequence, and the last element of the front feature descriptor is adjacent to the first element of the back feature descriptor. The total number of elements of the feature descriptor after texture and morphology fusion is the sum of the number of elements of the image feature descriptor and the number of elements of the point cloud feature descriptor.

[0026] In a specific implementation, the following steps are performed to calculate the normalized feature descriptor through the texture and shape fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the sum of the standard deviations: When the image feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; when the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the element of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, and k=m+n.

[0027] In specific implementation, the following steps are used to calculate the similarity of points on the target surface at different times using the normalized feature descriptor: The normalized feature descriptors of the points are obtained respectively, and each first element expression of the first normalized feature descriptor and each second element expression of the second normalized feature descriptor are obtained as the first normalized feature descriptor and the second normalized feature descriptor respectively; the similarity between the first normalized feature descriptor and the second normalized feature descriptor is calculated through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor.

[0028] In a specific implementation, the similarity between the first normalized feature descriptor and the second normalized feature descriptor is calculated by using all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor through the following steps: ,in, d is the similarity, is the first normalized feature descriptor, is the kth expression of the first element, is the second normalized feature descriptor, is the kth expression of the second element, and k is the total number of elements of the first normalized feature descriptor and the second normalized feature descriptor.

[0029] Specifically, based on the normalized feature descriptor, an embodiment of the present invention provides a coordinate point matching method, including: The image point cloud data obtained twice are respectively extracted from the image descriptor and the point cloud descriptor, and the normalized feature descriptor is obtained by processing with the proposed method. The similarity of the feature descriptors in the two sets of data is calculated, such as using the Euclidean distance method to determine the distance between the two descriptors, and the similarity between the descriptors is determined by a set algorithm, such as a set threshold, to finally achieve matching.

[0030] Assume that the normalized feature descriptor obtained for the first time is , the kth element expression is , the normalized feature descriptor obtained for the second time is , the kth element expression is The basis for judging the similarity between two descriptors during the matching process is: .

[0031] T represents the set threshold. When d≤T, the match is considered successful and the two points are the same point (i.e., points with the same name). Otherwise, the match fails and the two points are not the same point.

[0032] Currently, there are many feature descriptors extracted from images, such as SIFT descriptors and SURF descriptors, and there are also many descriptors extracted from point clouds, such as PFH descriptors, FPFH descriptors, and SPFH descriptors. The present invention is further described below in conjunction with the proposed image feature descriptors and point cloud feature descriptors, but the present invention is not limited to the following embodiments. It is particularly important to note that the present invention can utilize existing image feature descriptors and point cloud feature descriptors to generate feature descriptors after texture and morphology are fused, and can also utilize newly proposed image feature descriptors and point cloud feature descriptors (such as feature descriptors extracted based on deep learning methods) to generate feature descriptors after texture and morphology are fused, which is also within the scope of protection of the present invention.

[0033] Embodiment 1: First, SIFT (Scale-Invariant Feature Transform) feature descriptors are extracted from the image, and FPFH (Fast Point Feature Histograms) feature descriptors are extracted from the point cloud. SIFT feature descriptors are 128-dimensional vectors, and FPFH are 33-dimensional vectors.

[0034] The above method is used to preliminarily construct a 161-dimensional texture and shape fusion feature descriptor. The first 128 elements of the texture and shape fusion feature descriptor correspond to the elements of the SIFT feature descriptor, and the elements from the 129th element to the 161th element correspond to the elements of the FPFH feature descriptor.

[0035] The specific expression is: .

[0036] in, a i Represents the elements in the feature descriptor A after the fusion of texture and shape, b i Represents an element of a SIFT feature descriptor. c i-128 Represents the element of the FPFH feature descriptor.

[0037] Then, the proposed adaptive weighted normalization method is used to process the constructed feature descriptor after the fusion of texture and morphology, and the standard deviation of the SIFT feature descriptor and the FPFH feature descriptor vector is calculated. Then, the proposed method is used to process each element of the constructed feature descriptor after the fusion of texture and morphology to obtain a new vector with a modulus of 1.

[0038] The calculation method is:

[0039] in, a i Represents the element of the feature descriptor after the fusion of texture and shape, Represents the elements of the normalized feature descriptor. std(SIFT) represents the standard deviation of the elements of the SIFT feature descriptor, and std(FPFH) represents the standard deviation of the elements of the FPFH feature descriptor.

[0040] Finally, the proposed matching method is used to calculate the distance between two sets of normalized feature descriptors to determine the similarity of the two descriptors.

[0041] The distance calculation formula is:

[0042] in, and is the normalized feature descriptor obtained twice, and is its element expression.

[0043] When d≤T, the match is considered successful, where T represents the set threshold; otherwise, the match fails.

[0044] Embodiment 2: First, extract SURF (Speeded Up Robust Features) feature descriptors from the image and PFH (Point Feature Histogram) feature descriptors from the point cloud. SURF feature descriptors are 64-dimensional vectors, and PFH are 125-dimensional vectors.

[0045] The method is used to preliminarily construct a 189-dimensional texture and shape fusion feature descriptor. The first 64 elements of the texture and shape fusion feature descriptor correspond to the elements of the SURF feature descriptor, and the elements from the 65th element to the 189th element correspond to the elements of the PFH feature descriptor.

[0046] The specific expression is: .

[0047] in, a i Represents the elements in the feature descriptor after the fusion of texture and shape, b i Represents an element of a SURF feature descriptor. c i-64 Represents the element of the PFH feature descriptor.

[0048] Then, the proposed adaptive weighted normalization method is used to process the constructed texture and shape fusion feature descriptor, and the standard deviation of the SURF feature descriptor and the PFH feature descriptor vector is calculated. The proposed method is used to process each element of the constructed texture and shape fusion feature descriptor to obtain a new vector with a modulus length of 1.

[0049] The calculation method is: .

[0050] in, a i Represents the element of the feature descriptor after the fusion of texture and shape, Represents the elements of the normalized feature descriptor. std(SURF) represents the element standard deviation of the SURF feature descriptor, and std(PFH) represents the element standard deviation of the PFH feature descriptor.

[0051] Finally, the proposed matching method is used to calculate the distance between two sets of normalized feature descriptors to determine the similarity of the two descriptors.

[0052] The distance calculation formula is: .

[0053] in, and is the normalized feature descriptor obtained twice, and is its element expression.

[0054] When d≤T, the match is considered successful, where T represents the set threshold; otherwise, the match fails.

[0055] In this embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned matching methods based on feature descriptors is implemented.

[0056] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0057] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned feature descriptor-based matching methods.

[0058] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0059] Based on the same inventive concept, a matching device based on a feature descriptor is also provided in an embodiment of the present invention, as described in the following embodiments. Since the principle of solving the problem by the matching device based on a feature descriptor is similar to that of the matching method based on a feature descriptor, the implementation of the matching device based on a feature descriptor can refer to the implementation of the matching method based on a feature descriptor, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0060] Figure 4 is a structural block diagram of a matching device based on a feature descriptor according to an embodiment of the present invention. Figure 4 As shown, it includes: a descriptor generation module 401, a feature normalization module 402 and a homonymous point matching module 403. The structure is described below.

[0061] The descriptor generation module 401 is used to obtain the image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor; A feature normalization module 402 is used to fuse the image feature descriptor and the point cloud feature descriptor to generate a texture and shape fused feature descriptor, and normalize the texture and shape fused feature descriptor to generate a normalized feature descriptor; The same-name point matching module 403 is used to use the normalized feature descriptor to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

[0062] In one embodiment, the descriptor generation module includes: A texture feature acquisition unit, used for acquiring grayscale texture features of the image data, wherein the grayscale texture features include descriptions of corner points, edges, grayscale gradients, and normal vectors; An image descriptor generating unit, used for quantitatively describing the grayscale texture feature in the form of a vector to generate an image feature descriptor; A shape information acquisition unit, used to acquire shape information of the point cloud data, wherein the shape information includes descriptions of spatial position, curvature, and normal vector; The point cloud description subunit is used to quantitatively describe the morphological information in the form of a vector to generate a point cloud feature descriptor.

[0063] In one embodiment, the feature normalization module includes: A composition mode setting unit, used for specifying the composition mode of the feature descriptor after the fusion of texture and shape, that is, the image feature descriptor is in front or the point cloud feature descriptor is in front; A first element correspondence unit is used for, when the image feature descriptor is in front, to correspond the element expressions of the first m elements in the feature descriptor after the fusion of texture and shape to the element expressions of the m elements in the image feature descriptor one by one, and to correspond the element expressions of the m+1th to kth elements in the feature descriptor after the fusion of texture and shape to the element expressions of the n elements in the point cloud feature descriptor one by one, wherein k=m+n; The second element corresponding unit is used to, when the point cloud feature descriptor is in front, make one-to-one correspondence between the element expressions of the first n elements in the feature descriptor after the texture and morphology are fused and the element expressions of the n elements in the point cloud feature descriptor, and make one-to-one correspondence between the element expressions of the n+1 to k elements in the feature descriptor after the texture and morphology are fused and the element expressions of the m elements in the image feature descriptor.

[0064] In one embodiment, the feature normalization module further includes: An image feature standard deviation calculation unit, used to calculate the image feature standard deviation std(B) of the elements of the image feature descriptor; A point cloud feature standard deviation calculation unit, used to calculate the point cloud feature standard deviation std(C) of the elements of the point cloud feature descriptor; a standard deviation sum calculation unit, used for taking the sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) as the standard deviation sum; A descriptor normalization unit is used to calculate the normalized feature descriptor through the texture and shape fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the sum of the standard deviations.

[0065] In one embodiment, the descriptor normalization unit is used when the image feature descriptor is in front. ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; when the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the element of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, and k=m+n.

[0066] In one embodiment, the feature normalization module further includes: A normalized feature descriptor generating unit, used to respectively obtain the normalized feature descriptors of the points as the first normalized feature descriptor and the second normalized feature descriptor; An expression acquisition unit, used to acquire each first element expression of the first normalized feature descriptor and each second element expression of the second normalized feature descriptor; A similarity calculation unit is used to calculate the similarity between the first normalized feature descriptor and the second normalized feature descriptor through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor.

[0067] In one embodiment, the similarity calculation unit is used to ,in, d is the similarity, is the first normalized feature descriptor, is the kth expression of the first element, is the second normalized feature descriptor, is the kth expression of the second element, and k is the total number of elements of the first normalized feature descriptor and the second normalized feature descriptor.

[0068] The embodiments of the present invention achieve the following technical effects: The embodiments of the present invention provide a texture and morphology fused feature descriptor and a matching method by combining image texture information and point cloud morphology information. The texture and morphology fused feature descriptor simultaneously utilizes the texture information of image data and the morphology information provided by point cloud data, and the data content is richer, which provides the possibility for more accurate matching; an adaptive weighted feature descriptor combination method is provided, which more fully utilizes the rich texture information of image data and the rich morphology feature information of point cloud data, and avoids the matching difficulties caused by the lack of image texture and the flat point cloud morphology by adaptive weighting; a matching method is provided to realize the judgment of the similarity of normalized feature descriptors, which provides a judgment basis for realizing the matching of same-name points based on the proposed normalized feature descriptors.

[0069] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A matching method based on feature descriptors, characterized in that: include: Acquire image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor; The image feature descriptor and the point cloud feature descriptor are fused to generate a texture and shape fused feature descriptor, and the texture and shape fused feature descriptor is normalized to generate a normalized feature descriptor; The normalized feature descriptor is used to calculate the similarity of points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

2. The matching method based on feature descriptors according to claim 1, characterized in that: The image feature descriptor and the point cloud feature descriptor are fused to generate a texture and shape fused feature descriptor, including: Specifying a construction mode of a feature descriptor after fusion of texture and shape is that the image feature descriptor is in front or the point cloud feature descriptor is in front; When the image feature descriptor is in front, the element expressions of the first m elements in the feature descriptor after the fusion of texture and shape are matched one by one with the element expressions of the m elements in the image feature descriptor, and the element expressions of the m+1th to kth elements in the feature descriptor after the fusion of texture and shape are matched one by one with the element expressions of the n elements in the point cloud feature descriptor, where k=m+n; When the point cloud feature descriptor is in front, the element expressions of the first n elements in the feature descriptor after the texture and morphology are fused are corresponded one-to-one with the element expressions of the n elements in the point cloud feature descriptor, and the element expressions of the n+1 to k elements in the feature descriptor after the texture and morphology are fused are corresponded one-to-one with the element expressions of the m elements in the image feature descriptor.

3. The matching method based on feature descriptors according to claim 1, characterized in that: The feature descriptor after the texture and shape fusion is normalized to generate a normalized feature descriptor, including: Calculate the image feature standard deviation std(B) of the elements of the image feature descriptor; Calculate the point cloud feature standard deviation std(C) of the elements of the point cloud feature descriptor; The sum of the image feature standard deviation std(B) and the point cloud feature standard deviation std(C) is taken as the sum of standard deviations; The normalized feature descriptor is calculated by using the texture and shape fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the sum of the standard deviations.

4. The matching method based on feature descriptors according to claim 3, characterized in that: The normalized feature descriptor is calculated by using the texture and shape fusion feature descriptor, the image feature standard deviation std(B), the point cloud feature standard deviation std(C) and the sum of the standard deviations, including: When the image feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, m is the total number of elements of the image feature descriptor, i is the number of the element of the normalized feature descriptor, and k is the total number of elements of the normalized feature descriptor; When the point cloud feature descriptor is in front, ,in, a i is the element of the feature descriptor after the fusion of texture and shape, is the element of the normalized feature descriptor, n is the total number of elements of the point cloud feature descriptor, i is the number of the element of the normalized feature descriptor, k is the total number of elements of the normalized feature descriptor, and k=m+n.

5. The matching method based on feature descriptors according to claim 1, characterized in that: Utilizing the normalized feature descriptor, calculating the similarity of points on the target surface at different times, including: Respectively acquiring the normalized feature descriptors of the points on the target surface at different times as a first normalized feature descriptor and a second normalized feature descriptor; Obtaining each first element expression of the first normalized feature descriptor and each second element expression of the second normalized feature descriptor; The similarity between the first normalized feature descriptor and the second normalized feature descriptor is calculated through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor.

6. The matching method based on feature descriptors according to claim 5, characterized in that: Calculating the similarity between the first normalized feature descriptor and the second normalized feature descriptor through all the first element expressions of the first normalized feature descriptor and all the second element expressions of the second normalized feature descriptor, including: ,in, d is the similarity, is the first normalized feature descriptor, is the kth expression of the first element, is the second normalized feature descriptor, is the kth expression of the second element, and k is the total number of elements of the first normalized feature descriptor and the second normalized feature descriptor.

7. The feature descriptor-based matching method according to any one of claims 1 to 6, characterized in that: Extracting features from the image data to generate an image feature descriptor, and extracting features from the point cloud data to generate a point cloud feature descriptor, including: Acquire grayscale texture features of the image data, wherein the grayscale texture features include descriptions of corner points, edges, grayscale gradients, and normal vectors; Describing the grayscale texture features quantitatively in the form of vectors to generate image feature descriptors; Acquire the topographic information of the point cloud data, wherein the topographic information includes descriptions of spatial position, curvature, and normal vector; The morphological information is quantitatively described in the form of a vector to generate a point cloud feature descriptor.

8. A matching device based on feature descriptors, characterized in that: include: A descriptor generation module is used to obtain image data and point cloud data of the target surface in the input data, perform feature extraction on the image data to generate an image feature descriptor, perform feature extraction on the point cloud data to generate a point cloud feature descriptor; A feature normalization module, used for fusing the image feature descriptor and the point cloud feature descriptor to generate a texture and shape fused feature descriptor, and performing normalization processing on the texture and shape fused feature descriptor to generate a normalized feature descriptor; The same-name point matching module is used to use the normalized feature descriptor to calculate the similarity of the points on the target surface at different times. When the similarity is less than or equal to a set threshold, the two points are the same point. If the similarity is greater than the set threshold, the two points are different points.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the feature descriptor-based matching method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the feature descriptor-based matching method according to any one of claims 1 to 7.

Images