Point cloud matching method and apparatus, electronic apparatus, and storage medium

Abstract

A point cloud matching method and apparatus, an electronic apparatus, and a storage medium. The point cloud matching method comprises: dividing a point cloud to be matched into a plurality of point cloud blocks, inputting the plurality of point cloud blocks into a pre-trained point cloud feature encoding module, and obtaining a feature vector of the plurality of point cloud blocks; on the basis of the feature vector of the plurality of point cloud blocks, obtaining a global descriptor vector of said point cloud, matching the global descriptor vector of said point cloud with a global descriptor vector of a point cloud frame in a preset historical database, and determining a point cloud frame in a historical database within a preset matching threshold range to be a point cloud matching result.

Description

Point cloud matching method, device, electronic device and storage medium

[0001] Related applications

[0002] This application claims priority to the Chinese patent application filed on July 28, 2023, with application number 202310940230.5, and invention name “A point cloud matching method, device, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of point cloud matching, and in particular to a point cloud matching method, device, electronic device and storage medium. Background Art

[0004] A point cloud is a collection of three-dimensional geometric data consisting of a set of points. Point clouds can represent information such as an object's shape, surface, and outline, and are an important data format in fields such as computer vision, robotics, and virtual reality. Point clouds can be directly acquired from sensors (such as lidar and RGB-D cameras) or generated through technologies such as 3D reconstruction and 3D scanning.

[0005] Point cloud matching is primarily achieved by comparing the global feature similarities between the current point cloud and historical point clouds. Thanks to the rapid development of deep learning, global features of point clouds are currently extracted primarily through neural networks. However, the influence of neighboring points is not considered in feature extraction. Two main approaches to address this problem are K-nearest neighbor and three-dimensional sparse convolution. However, these algorithms are all supervised deep learning algorithms, and their training data must be annotated with distance relationships in real three-dimensional space. Therefore, additional positioning sensors need to be installed, which not only increases equipment costs but also does not support offline data collection and training in many scenarios, making it impossible to use the algorithm out of the box. For example, in simultaneous localization and mapping (SLAM), point cloud matching technology based on supervised deep learning suffers from low matching accuracy.

[0006] There is currently no effective solution to the problem of low matching accuracy in point cloud matching technology based on supervised deep learning in related technologies.

[0007] Summary of the Invention

[0008] According to various embodiments of the present application, a point cloud matching method, apparatus, electronic device, and storage medium are provided.

[0009] In a first aspect, a point cloud matching method is provided in this embodiment, the method comprising:

[0010] Dividing the point cloud to be matched into a plurality of matching point cloud blocks, inputting the plurality of matching point cloud blocks into a pre-trained point cloud feature encoding module to obtain feature vectors of the plurality of matching point cloud blocks; during pre-training, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset masking ratio, and outputs feature vectors of the visible point cloud blocks and the masked point cloud blocks, respectively, to train the point cloud feature encoding module;

[0011] The global description vector of the point cloud to be matched is obtained based on the feature vectors of multiple matching point cloud blocks, the global description vector of the point cloud to be matched is matched with the global description vector of the point cloud frame in the preset historical database, and the point cloud frame in the historical database within the preset matching threshold range is determined as the point cloud matching result.

[0012] In some embodiments, dividing the point cloud to be matched into a plurality of point cloud blocks includes:

[0013] Filtering the point cloud to be matched according to a preset height and radius;

[0014] The filtered point cloud to be matched is voxelized according to the preset spatial resolution;

[0015] Sampling the voxelized point cloud to be matched, and determining multiple sampled points as key points;

[0016] The point cloud to be matched is divided into a plurality of matching point cloud blocks according to the key point and a set of points within a preset range of the key point.

[0017] In some embodiments, dividing the to-be-matched point cloud into a plurality of matching point cloud blocks, inputting the plurality of matching point cloud blocks into a pre-trained point cloud feature encoding module, and obtaining feature vectors of the plurality of matching point cloud blocks includes:

[0018] Inputting the plurality of matching point cloud blocks into a preset linear dimension-increasing layer to obtain matching point cloud blocks that have been dimensionally upgraded to a preset dimension;

[0019] The matching point cloud blocks upgraded to a preset dimension are input into a preset first self-attention layer to obtain feature vectors of multiple matching point cloud blocks.

[0020] In some embodiments, obtaining the global description vector of the point cloud to be matched based on the feature vectors of the plurality of matching point cloud blocks includes:

[0021] Acquire multiple distance vectors from a preset vector library, where the distance vectors are vectors whose distances from the feature vectors of the multiple matching point cloud blocks are within a preset distance threshold range;

[0022] The feature vectors of a plurality of the matching point cloud blocks and a plurality of the distance vectors are concatenated to obtain a global description vector of the point cloud to be matched.

[0023] In some embodiments, before dividing the to-be-matched point cloud into a plurality of matching point cloud blocks and inputting the plurality of matching point cloud blocks into the pre-trained point cloud feature encoding module, the method includes:

[0024] Filter the point cloud to be trained according to the preset height and radius;

[0025] The filtered point cloud to be trained is voxelized according to the preset spatial resolution;

[0026] Sampling the voxelized point cloud to be trained, and determining multiple sampled points as key points;

[0027] Dividing the point cloud to be trained into a plurality of training point cloud blocks according to the key points and a set of points within a preset range of the key points;

[0028] Covering the plurality of training point cloud blocks according to a preset covering rate to obtain a first point cloud block and a second point cloud block, wherein the first point cloud block is a visible point cloud block and the second point cloud block is a covered point cloud block;

[0029] Obtaining prediction information of the second point cloud block according to the information of the first point cloud block and the information of the second point cloud block;

[0030] A prediction loss is obtained according to the information of the second point cloud block and the prediction information of the second point cloud block, and the point cloud feature encoding module is trained according to the prediction loss.

[0031] In some embodiments, obtaining prediction information of the second point cloud block based on the information of the first point cloud block and the information of the second point cloud block includes:

[0032] Inputting the coordinates of the key points in the first point cloud block into a preset visible position coding layer to obtain the position coding of the first point cloud block;

[0033] Inputting the coordinates of the key points in the second point cloud block into a preset mask position coding layer to obtain a position code of the second point cloud block;

[0034] Inputting the first point cloud block and the second point cloud block into a preset linear dimension-increasing layer to obtain the first point cloud block and the second point cloud block upgraded to a preset dimension;

[0035] Inputting the first point cloud block upgraded to a preset dimension into a preset first self-attention layer to obtain a feature vector of the first point cloud block;

[0036] Adding the feature vector of the first point cloud block and the position code of the first point cloud block, and concatenating the addition result with the position code of the second point cloud block to obtain a tensor of the point cloud to be trained;

[0037] The tensor of the point cloud to be trained is input into a preset second self-attention layer to obtain a predicted feature vector of the second point cloud block, where the predicted feature vector is the prediction information.

[0038] In some embodiments, obtaining the prediction loss based on the information of the second point cloud block and the prediction information of the second point cloud block includes:

[0039] Obtaining a predicted feature vector of the second point cloud block and a plurality of original vectors in the feature vector of the second point cloud block, obtaining replacement vectors corresponding to the plurality of original vectors from a preset vector library, and replacing the plurality of original vectors with the replacement vectors; wherein the feature vector of the second point cloud block is obtained by inputting the second point cloud block upgraded to a preset dimension into a preset first self-attention layer;

[0040] Determine the feature vector and the predicted feature vector of the replaced second point cloud block as the semantics of the second point cloud block and the predicted semantics of the second point cloud block respectively;

[0041] Calculating the square error between the semantics of the second point cloud block and the predicted semantics of the second point cloud block, where the square error is the first prediction loss;

[0042] Inputting the feature vector of the second point cloud block into a preset third self-attention layer and a linear dimensionality reduction layer to obtain the predicted coordinates of each point in the second point cloud block after dimensionality reduction;

[0043] Calculating a chamfer distance between the coordinates of the second point cloud block and the predicted coordinates of the second point cloud block, where the chamfer distance is the second prediction loss;

[0044] The sum of the first predicted loss and the second predicted loss is determined to be the predicted loss.

[0045] In a second aspect, a point cloud matching device is provided in this embodiment, and the device includes: a processing module and a matching module.

[0046] The processing module is used to divide the point cloud to be matched into multiple matching point cloud blocks, input the multiple matching point cloud blocks into the pre-trained point cloud feature encoding module, and obtain the feature vectors of the multiple matching point cloud blocks. During pre-training, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset coverage rate, and outputs the feature vectors of the visible point cloud blocks and the masked point cloud blocks respectively.

[0047] The matching module is used to obtain the global description vector of the point cloud to be matched based on the feature vectors of multiple matching point cloud blocks, match the global description vector of the point cloud to be matched with the global description vector of the point cloud frame in a preset historical database, and determine the point cloud frame in the historical database within a preset matching threshold range as the point cloud matching result.

[0048] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the point cloud matching method described in the first aspect when executing the computer program.

[0049] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the point cloud matching method described in the first aspect is implemented.

[0050] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0052] FIG1 is a block diagram of the hardware structure of a terminal of a point cloud matching method according to some embodiments.

[0053] FIG2 is a flowchart of a point cloud matching method according to some embodiments.

[0054] FIG3 is a flowchart of a pre-training method according to some embodiments.

[0055] FIG4 is an optional flow chart of a pre-training method according to some embodiments.

[0056] FIG5 is a diagram illustrating a training process of a mask semantic modeling and alignment method according to some embodiments.

[0057] FIG6 is an optional flow chart of a point cloud matching method according to some embodiments.

[0058] FIG7 is a structural block diagram of a point cloud matching apparatus according to some embodiments. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0061] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, when running on a terminal, Figure 1 is a hardware structure block diagram of the terminal of the point cloud matching method of this embodiment. As shown in Figure 1, the terminal may include one or more (only one is shown in Figure 1) processors 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU (Microcontroller Unit) or a programmable logic device FPGA (Field Programmable Gate Array). The above-mentioned terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that the structure shown in Figure 1 is only for illustration and does not limit the structure of the above-mentioned terminal. For example, the terminal may also include more or fewer components than those shown in Figure 1, or have a different configuration than that shown in Figure 1.

[0062] Memory 104 can be used to store computer programs, such as software programs and modules for application software, such as the computer program corresponding to the point cloud matching method in this embodiment. Processor 102 executes the computer programs stored in memory 104 to perform various functional applications and data processing, thereby implementing the aforementioned method. Memory 104 may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, memory 104 may further include remote memory located remotely from processor 102, which can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0063] Transmission device 106 is used to receive or transmit data via a network. The aforementioned network includes a wireless network provided by the terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one example, transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0064] In this embodiment, a point cloud matching method is provided. The point cloud matching method involved in this application can be applied to robot navigation, three-dimensional reconstruction, industrial manufacturing, aerospace, intelligent transportation and other aspects. Figure 2 is a flow chart of the point cloud matching method of this embodiment. As shown in Figure 2, the process includes the following steps S210-S220.

[0065] In step S210, the point cloud to be matched is divided into multiple matching point cloud blocks, and the multiple matching point cloud blocks are input into the pre-trained point cloud feature encoding module to obtain feature vectors of the multiple matching point cloud blocks; during pre-training, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset coverage rate, and outputs the feature vectors of the visible point cloud blocks and the masked point cloud blocks respectively to train the point cloud feature encoding module.

[0066] Specifically, the processor receives a frame of original point cloud to be matched, divides the point cloud to be matched into multiple matching point cloud blocks, and inputs the matching point cloud blocks into a pre-trained point cloud feature encoding module to obtain feature vectors of multiple matching point cloud blocks. In particular, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset coverage rate during pre-training, and outputs feature vectors of visible point cloud blocks and feature vectors of masked point cloud blocks respectively, so as to facilitate subsequent training of the point cloud feature encoding module based on the feature vectors of the visible point cloud blocks and the feature vectors of the masked point cloud blocks. By partially masking the training point cloud blocks input into the point cloud feature encoding module during pre-training, and training the point cloud feature encoding module based on the output feature vectors of the visible point cloud blocks and the feature vectors of the masked point cloud blocks, the accuracy of the feature vectors of the multiple matching point cloud blocks output by the point cloud feature encoding module is improved.

[0067] Step S220, obtain the global description vector of the point cloud to be matched based on the feature vectors of multiple matching point cloud blocks, match the global description vector of the point cloud to be matched with the global description vector of the point cloud frame in the preset historical database, and determine that the point cloud frame in the historical database within the preset matching threshold range is the point cloud matching result.

[0068] Specifically, the processor outputs feature vectors of multiple matching point cloud blocks according to the point cloud matching module to obtain a global description vector of the point cloud to be matched; and matches the global description vector of the point cloud to be matched with the global description vector of the point cloud frame in the preset historical database, and determines the point cloud frame in the historical database within the preset matching threshold range as the point cloud matching result.

[0069] Through the above steps S210 to S220, the processor receives a frame of original point cloud to be matched, divides the point cloud to be matched into multiple matching point cloud blocks, and inputs the multiple matching point cloud blocks into the pre-trained point cloud feature encoding module to obtain feature vectors of the multiple matching point cloud blocks. During pre-training, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset coverage rate, and outputs feature vectors of the visible point cloud blocks and feature vectors of the masked point cloud blocks, respectively. This facilitates subsequent training of the point cloud feature encoding module based on the feature vectors of the visible point cloud blocks and the feature vectors of the masked point cloud blocks. Based on the feature vectors of the multiple matching point cloud blocks output by the point cloud matching module, a global description vector of the point cloud to be matched is obtained. The global description vector of the point cloud to be matched is matched with the global description vector of the point cloud frame in a preset historical database, and the point cloud frame in the historical database within a preset matching threshold range is determined as the point cloud matching result, thereby solving the problem of low matching accuracy of point cloud matching technology based on supervised deep learning. During pre-training, the point cloud feature encoding module is trained by partially masking multiple training point cloud blocks input into the module and training the module based on the feature vectors of the output visible point cloud blocks and the feature vectors of the masked point cloud blocks. This helps improve the accuracy of the feature vectors output by the module for multiple matching point cloud blocks. The global description variables of the point cloud to be matched are obtained using the feature vectors of the multiple matching point cloud blocks. These global description variables are then matched with the global description variables of the point cloud frames in a preset historical database. The matched point cloud frames in the historical database that fall within a preset matching threshold are then designated as point cloud matching results, further improving the accuracy of the point cloud matching results.

[0070] In some embodiments, step S210 includes steps S211 to S214.

[0071] Step S211: filtering the point cloud to be matched according to the preset height and radius.

[0072] Specifically, the processor filters the point cloud to be matched according to preset height and radius range thresholds.

[0073] Step S212 : voxelizing the filtered point cloud to be matched according to a preset spatial resolution.

[0074] Specifically, the processor performs voxel processing on the filtered point cloud to be matched according to a preset spatial resolution. The voxel processing is helpful for filtering the data of the point cloud to be matched while maintaining the integrity of the point cloud to be matched.

[0075] In step S213 , the voxelized point cloud to be matched is sampled, and a plurality of sampled points are determined as key points.

[0076] Specifically, the processor obtains the to-be-matched point cloud after voxelization processing, samples the to-be-matched point cloud, and uses the sampled points as key points of the to-be-matched point cloud.

[0077] Step S214 : dividing the point cloud to be matched into a plurality of matching point cloud blocks according to the key points and the set of points within a preset range of the key points.

[0078] Specifically, the processor divides the point cloud to be matched into multiple matching point cloud blocks based on the key points obtained by sampling and the set of points within the preset range of the key points, where each matching point cloud block includes a key point and a set of points within the preset range of the key point.

[0079] Through the above steps S211 to S214, the processor filters the point cloud to be matched based on the preset height and radius range thresholds; performs voxel processing on the filtered point cloud to be matched according to the preset spatial resolution; then samples the processed point cloud to be matched, and uses the sampled points as the key points of the point cloud to be matched; and divides the point cloud to be matched into multiple matching point cloud blocks based on the sampled key points and the set of points within the preset range of the key points. Voxelizing the filtered point cloud to be matched facilitates filtering the data of the point cloud to be matched while maintaining the integrity of the point cloud to be matched. Dividing the point cloud to be matched into multiple matching point cloud blocks facilitates the subsequent processing of the multiple matching point cloud blocks, further improving the accuracy of point cloud matching.

[0080] In some embodiments, step S210 further includes steps S215 to S216.

[0081] In step S215 , the plurality of matching point cloud blocks are input into a preset linear dimension-upgrading layer to obtain matching point cloud blocks that have been upgraded to a preset dimension.

[0082] Specifically, after the processor divides the point cloud to be matched into multiple matching point cloud blocks, the multiple matching point cloud blocks are input into the pre-trained point cloud feature encoding module, that is, the multiple matching point cloud blocks are first input into the preset linear dimensionality increase layer to increase the dimension of the matching point cloud blocks to the preset dimension.

[0083] In step S216, the matching point cloud blocks upgraded to a preset dimension are input into a preset first self-attention layer to obtain feature vectors of multiple matching point cloud blocks.

[0084] Specifically, after the processor upgrades the dimensions of the multiple matching point cloud blocks, it inputs the upgraded point cloud blocks into a preset first self-attention layer, thereby obtaining feature vectors of the multiple matching point cloud blocks.

[0085] After the processor divides the to-be-matched point cloud into multiple matching point cloud blocks through steps S215 and S216, it then inputs the multiple matching point cloud blocks into a pre-trained point cloud feature encoding module. Specifically, the multiple matching point cloud blocks are first input into a preset linear dimensionality increase layer to increase the dimensionality of the matching point cloud blocks to a preset dimensionality. Subsequently, the multiple matched point cloud blocks after dimensionality increase are input into a preset first self-attention layer to obtain feature vectors for the multiple matching point cloud blocks. Inputting the multiple matching point cloud blocks into the pre-trained point cloud feature encoding module helps improve the accuracy of the feature vectors obtained for the multiple matching point cloud blocks.

[0086] In some embodiments, step S220 further includes steps S221 to S222.

[0087] In step S221 , a plurality of distance vectors are obtained from a preset vector library, where the distance vectors are vectors whose distances from the feature vectors of the plurality of matching point cloud blocks are within a preset distance threshold range.

[0088] Specifically, the processor obtains a plurality of vectors from a preset vector library, the distances between which and the feature vectors of the plurality of matching point cloud blocks are within a preset distance threshold range, namely, distance vectors.

[0089] Step S222 : Concatenate the feature vectors and the distance vectors of the multiple matching point cloud blocks to obtain a global description vector of the point cloud to be matched.

[0090] Specifically, the processor concatenates the feature vectors and the distance vectors of the multiple matching point cloud blocks to obtain a global description vector of the point cloud to be matched.

[0091] Through steps S221 to S222, the processor retrieves from a preset vector library multiple vectors whose distances from the feature vectors of multiple matching point cloud blocks are within a preset distance threshold. A vector whose distances from the feature vectors of multiple matching point cloud blocks are within the preset distance threshold is defined as a distance vector. The processor then concatenates the feature vectors of the multiple matching point cloud blocks with the multiple distance vectors to obtain a global description vector for the point cloud to be matched. Obtaining distance vectors and concatenating the feature vectors of the multiple matching point cloud blocks with the multiple distance vectors to obtain a global description vector for the matching point cloud further improves the matching accuracy of the point cloud to be matched.

[0092] This embodiment also provides a method for pre-training the point cloud feature encoding module. FIG3 is a flow chart of the pre-training method of this embodiment. As shown in FIG3 , the process includes the following steps S310 to S370.

[0093] Step S310: filtering the point cloud to be trained according to the preset height and radius.

[0094] Specifically, the processor filters the point cloud to be trained according to preset height and radius range thresholds.

[0095] In step S320 , the filtered point cloud to be trained is voxelized according to a preset spatial resolution.

[0096] Specifically, the processor performs voxel processing on the filtered point cloud to be trained according to a preset spatial resolution. The voxel processing is helpful for filtering the data of the point cloud to be trained and for maintaining the integrity of the point cloud to be trained.

[0097] Step S330 : Sampling the voxelized point cloud to be trained, and determining a plurality of sampled points as key points.

[0098] Specifically, the processor obtains the to-be-trained point cloud after voxelization processing, samples the to-be-trained point cloud, and uses the sampled points as key points of the to-be-trained point cloud.

[0099] Step S340 : dividing the point cloud to be trained into a plurality of training point cloud blocks according to a set of key points and points within a preset range of the key points.

[0100] Specifically, the processor divides the point cloud to be trained into multiple training point cloud blocks based on the sampled key points and the set of points within the preset range of the key points, where each training point cloud block includes a key point and a set of points within the preset range of the key point.

[0101] In step S350 , multiple training point cloud blocks are covered according to a preset covering rate to obtain a first point cloud block and a second point cloud block, wherein the first point cloud block is a visible point cloud block and the second point cloud block is a covered point cloud block.

[0102] Specifically, the processor covers the multiple training point cloud blocks according to a preset covering rate to obtain visible point cloud blocks and covered point cloud blocks, which are the first point cloud blocks and the second point cloud blocks respectively.

[0103] Step S360: Obtain prediction information of the second point cloud block based on the information of the first point cloud block and the information of the second point cloud block.

[0104] Specifically, after the processor divides the multiple point cloud blocks into multiple first point cloud blocks and second point cloud blocks, it obtains information of the first point cloud blocks and partial information of the second point cloud blocks, and predicts other information of the second point cloud blocks based on this information.

[0105] Step S370: Obtain prediction loss based on the information of the second point cloud block and the prediction information of the second point cloud block, and train the point cloud feature encoding module based on the prediction loss.

[0106] Specifically, the processor calculates the prediction loss based on the information of the second point cloud block and the prediction information of the second point cloud block, and trains the point cloud feature encoding module based on the obtained prediction loss.

[0107] Through steps S310 to S370, the processor filters the training point cloud based on preset height and radius thresholds; voxelizes the filtered training point cloud at a preset spatial resolution; then samples the processed training point cloud, using the sampled points as key points of the training point cloud; and divides the training point cloud into multiple training point cloud blocks based on the sampled key points and the set of points within the preset key point range. The processor masks the multiple training point cloud blocks based on a preset coverage ratio, obtaining visible point cloud blocks and masked point cloud blocks, which are respectively referred to as first point cloud blocks and second point cloud blocks. After dividing the multiple training point cloud blocks into the first point cloud blocks and the second point cloud blocks, the processor obtains information about the first point cloud blocks and partial information about the second point cloud blocks, and based on this information, predicts other information about the second point cloud blocks. The processor calculates a prediction loss based on the information about the second point cloud blocks and the predicted information of the second point cloud blocks, and trains the point cloud feature encoding module based on the obtained prediction loss. By voxelizing the filtered points to be trained, it is beneficial to filter the data of the points to be trained while maintaining the integrity of the point cloud to be trained. By dividing the points to be trained into multiple training point cloud blocks, it is beneficial to subsequently process the multiple training point cloud blocks, and further helps to improve the accuracy of point cloud matching. By dividing the multiple training point cloud blocks into a first point cloud block and a second point cloud block according to a preset cover rate, and predicting other information of the second point cloud block based on the information of the first point cloud block and the second point cloud block, and calculating the prediction loss of the second point cloud block, and then training the point cloud feature encoding module through the prediction loss, it is beneficial to better train the point cloud feature encoding module, and thus improve the accuracy of the feature vector output by the point cloud feature encoding module.

[0108] In some embodiments, step S360 includes steps S361 to S366.

[0109] In step S361 , the coordinates of the key points in the first point cloud block are input into a preset visible position coding layer to obtain the position coding of the first point cloud block.

[0110] Specifically, the processor inputs the coordinates of the key points in the first point cloud block into a preset visible position coding layer to obtain the position coding related to the first point cloud block, wherein the coordinates of the key points can be the coordinates of the center point.

[0111] In step S362 , the coordinates of the key points in the second point cloud block are input into a preset mask position coding layer to obtain the position coding of the second point cloud block.

[0112] Specifically, the processor inputs the coordinates of the key points in the second point cloud block into a preset visible position coding layer to obtain the position coding related to the second point cloud block.

[0113] Step S363: Input the first point cloud block and the second point cloud block into a preset linear dimension-upgrading layer to obtain the first point cloud block and the second point cloud block upgraded to a preset dimension.

[0114] Specifically, the processor inputs the first point cloud block and the second point cloud block into a preset linear dimension-increasing layer to obtain the first point cloud block and the second point cloud block after dimension-increasing. Setting the linear dimension-increasing layer is beneficial to improving the accuracy of the feature vector output by the point cloud position encoding module.

[0115] Step S364: input the first point cloud block and the second point cloud block upgraded to the preset dimension into the preset first self-attention layer to obtain the feature vector of the first point cloud block and the feature vector of the second point cloud block.

[0116] Specifically, the processor inputs the dimensionally upgraded first point cloud block and the second point cloud block into the preset first self-attention layer, respectively, and then obtains the feature vector of the first point cloud block and the feature vector of the second point cloud block, respectively.

[0117] In step S365 , the feature vector of the first point cloud block and the position code of the first point cloud block are added, and the addition result is concatenated with the position code of the second point cloud block to obtain a tensor of the point cloud to be trained.

[0118] Specifically, the processor adds the feature vector of the first point cloud block and the position code of the first point cloud block, and concatenates the result of the addition with the position code of the second point cloud block to obtain a tensor of the training point cloud. This helps improve the accuracy of subsequent predictions of other information of the second point cloud block.

[0119] In step S366, the tensor of the point cloud to be trained is input into the preset second self-attention layer to obtain the predicted feature vector of the second point cloud block, where the predicted feature vector is the prediction information.

[0120] Specifically, the processor inputs the tensor of the point cloud to be trained into a preset second self-attention layer to obtain a predicted feature vector of the second point cloud block.

[0121] Through steps S361 to S366, the processor inputs the coordinates of the key points in the first point cloud block into a preset visible position encoding layer to obtain the position code associated with the first point cloud block, where the coordinates of the key points can be the coordinates of the center point. The coordinates of the key points in the second point cloud block are input into a preset visible position encoding layer to obtain the position code associated with the second point cloud block. The first and second point cloud blocks are input into a preset linear dimensionality increase layer to obtain the upgraded first and second point cloud blocks. Thereafter, the processor inputs the upgraded first and second point cloud blocks into a preset first self-attention layer, respectively, to obtain the feature vector of the first and second point cloud blocks, respectively. The processor then adds the feature vector of the first point cloud block to the position code of the first point cloud block and concatenates the result of the addition with the position code of the second point cloud block to obtain a tensor for the point cloud to be trained. The processor then inputs the tensor of the point cloud to be trained into a preset second self-attention layer to obtain a predicted feature vector for the second point cloud block. The provision of a linear dimensionality increase layer helps improve the accuracy of the feature vector output by the point cloud position encoding module. By adding the feature vector and position code of the first point cloud block and then splicing it with the position code of the second point cloud block, it is beneficial to improve the accuracy of subsequent prediction of other information of the second point cloud block and further improve the accuracy of the feature vector output by the point cloud feature coding module.

[0122] In some embodiments, step S370 includes steps S371 to S376.

[0123] Step S371: obtain the predicted feature vector of the second point cloud block and multiple original vectors in the feature vector of the second point cloud block, obtain replacement vectors corresponding to the multiple original vectors in a preset vector library, and replace the multiple original vectors with the replacement vectors.

[0124] Specifically, the processor obtains the predicted feature vector of the second point cloud block and multiple original vectors in the feature vector of the second point cloud block, and obtains replacement vectors corresponding to the multiple original vectors in a preset vector library, and then uses the replacement vectors to replace the multiple original vectors.

[0125] Step S372 , respectively determining the feature vector and the predicted feature vector of the replaced second point cloud block as the semantics of the second point cloud block and the predicted semantics of the second point cloud block.

[0126] Specifically, the processor replaces the feature vector and the predicted feature vector of the second point cloud block and determines them as the semantics of the second point cloud block and the predicted semantics of the second point cloud block, respectively.

[0127] Step S373 , calculating the square error between the semantics of the second point cloud block and the predicted semantics of the second point cloud block, where the square error is the first prediction loss.

[0128] Specifically, the processor calculates the first prediction loss, that is, the square error between the semantics of the second point cloud block and the predicted semantics of the second point cloud block, where the square error is the first prediction loss.

[0129] In step S374, the feature vector of the second point cloud block is input into the preset third self-attention layer and linear dimensionality reduction layer to obtain the predicted coordinates of each point in the second point cloud block after dimensionality reduction.

[0130] Specifically, the processor inputs the feature vector of the second point cloud block into the preset third self-attention layer and linear dimensionality reduction layer in sequence to obtain the predicted coordinates of each point in the second point cloud block after dimensionality reduction.

[0131] Step S375 , calculating the chamfer distance between the coordinates of the second point cloud block and the predicted coordinates of the second point cloud block, where the chamfer distance is the second prediction loss.

[0132] Specifically, the processor calculates a chamfer distance between the coordinates of the second point cloud block and the predicted coordinates of the second point cloud block, and determines the chamfer distance as the second prediction loss.

[0133] Step S376: Determine the sum of the first predicted loss and the second predicted loss as the predicted loss.

[0134] Specifically, the processor uses the sum of the first prediction loss and the second prediction loss as the prediction loss to train the point cloud feature encoding module using the prediction loss.

[0135] Through steps S371 to S376, the processor obtains the predicted feature vector of the second point cloud block and multiple original vectors from the feature vector of the second point cloud block, obtains replacement vectors corresponding to the multiple original vectors from a preset vector library, and then replaces the multiple original vectors with the replacement vectors. The feature vector and predicted feature vector of the second point cloud block after replacement are determined as the semantics of the second point cloud block and the predicted semantics of the second point cloud block, respectively. A first prediction loss is calculated based on the semantics and predicted semantics of the second point cloud block, which is the squared error between the semantics of the second point cloud block and the predicted semantics of the second point cloud block. The processor then sequentially inputs the feature vector of the second point cloud block into a preset third self-attention layer and a linear dimensionality reduction layer to obtain the predicted coordinates of each point in the second point cloud block after dimensionality reduction. The processor calculates the chamfer distance between the coordinates of the second point cloud block and the predicted coordinates of the second point cloud block and determines the chamfer distance as the second prediction loss. The processor uses the sum of the first prediction loss and the second prediction loss as the prediction loss to train the point cloud feature encoding module. By obtaining the semantic alignment error of the second point cloud block and the predicted chamfer distance of the second point cloud block as the prediction loss, and training the point cloud feature encoding module according to the prediction loss, it is beneficial to further improve the accuracy of the training, thereby improving the accuracy of the feature vector output by the point cloud feature encoding, and further improving the accuracy of the point cloud matching.

[0136] The present embodiment is described and illustrated below through optional embodiments.

[0137] FIG4 is an optional flow chart of the pre-training method of this embodiment. As shown in FIG4 , the pre-training method includes the following steps:

[0138] Step S401: data processing.

[0139] Specifically, the input is a single point cloud frame and the coverage ratio, and the output is a visible point cloud block and a covered point cloud block. The single point cloud frame and the coverage ratio R are input into the corresponding data processing module, namely the data processing module, which filters the point cloud according to the preset height and radius range thresholds; the filtered point cloud is voxelized according to the preset spatial resolution; then, G key points are sampled from the voxelized point cloud, and the K points closest to each key point are selected as its neighborhood, resulting in G training point cloud blocks. Among them, the set of a key point and its neighborhood is a training point cloud block (Patch), and the values ​​of G and K are both preset. The G training point cloud blocks are randomly masked according to the preset coverage rate R, and a total of G*R point cloud blocks are covered. In particular, if G*R is not an integer, it is rounded down. Therefore, the output of the data processing module is G*(1-R) ​​visible point cloud blocks XV and G*R covered point cloud blocks XM; the sizes of the visible point cloud blocks and the covered point cloud blocks are G*(1-R)*N and G*R*N respectively, where N represents the coordinate dimension of a point in three-dimensional space.

[0140] For example, a frame of original point cloud is input to the data processing module, and points with a height between -20cm and 200cm and a radius within 30m are selected as filtered points; then, the filtered point cloud is voxelized, and the voxelized training point cloud is sampled. Specifically, the farthest point sampling algorithm is used. Sampling (FPS) samples 128 key points from the voxelized training point cloud. For each key point, the 64 points closest to it are selected as its neighborhood. The number of point clouds per frame is controlled to 4096. The point cloud frame is named F, so 128 training point cloud blocks can be obtained. The preset coverage rate R = 0.75 is determined, so 96 training point cloud blocks need to be randomly selected for covering. The method for selecting the covered point cloud blocks can be: simple random sampling and stratified sampling, etc. Finally, 32 visible point cloud blocks XV and 96 covered point cloud blocks XM are output. The sizes of the two training point cloud blocks are 32*3 and 96*3 respectively, where 3 represents the coordinate dimension of a point in three-dimensional space.

[0141] Step S402: point cloud feature encoding.

[0142] Specifically, the visible point cloud block XV and the covered point cloud block XM are input to the module corresponding to the point cloud feature encoding, namely the point cloud feature encoding module. The output of the point cloud feature encoding module is the visible point cloud block feature and the covered point cloud block feature. The first layer of the point cloud feature encoding module is a linear layer, which is used to increase the dimension of the point cloud to be trained, that is, to upgrade the training point cloud blocks of size G*(1-R)*N and G*R*N to point cloud blocks of size G*(1-R)*C and G*R*C, where the value of N is less than the value of C, and the value of C is a predefined value; the point cloud feature encoding module starts from the second layer for D E In the self-attention layer, the visible point cloud block and the covered point cloud block are input into the point cloud feature encoding module separately, and the output is the feature FV of the visible point cloud block; the covered point cloud block is input into the point cloud feature encoding module, and the output is the feature FM of the covered point cloud block.

[0143] Exemplarily, the first layer of the point cloud feature encoding module is a linear layer used to increase the dimensionality of the training point cloud. Specifically, the value of C is 256, which upscales training point cloud blocks of sizes 32*3 and 96*3 to training point cloud blocks of sizes 32*256 and 96*256, respectively. The second layer of the point cloud feature encoding module consists of 12 self-attention layers. The input of the visible point cloud block XV of size 32*3 is fed into the 12 self-attention layers, and the output is the features FV of the visible point cloud block. The input of the masked point cloud block XM of size 96*256 is fed into the 12 self-attention layers, and the output is the features FM of the masked point cloud block.

[0144] Step S403: predicting mask point cloud features.

[0145] Specifically, the visible point cloud block features, the coordinates of the center of the visible point cloud block, and the coordinates of the center of the covered point cloud block are input to the module corresponding to the mask point cloud feature prediction, that is, the mask point cloud feature prediction module, and the output is the predicted covered point cloud block features. P Layer self-attention layer (Self-attentionLayer) layer group. The coordinates of the center of the covered point cloud block, the size of the covered point cloud block is G*R*N, and are input into the cover position encoding layer to obtain the covered point cloud block position encoding of size G*R*C; the coordinates of the center of the visible point cloud block, the size of the visible point cloud block is G*(1-R)*N, are input into the visible position encoding layer to obtain the visible point cloud block position encoding of size G*(1-R)*C; the visible point cloud block feature FV and the visible point cloud block position encoding are added, and its size is still G*(1-R)*C. After the addition result is spliced ​​with the cover point cloud block position encoding, a tensor EP of size G*C is obtained, and the tensor EP is input into the mask point cloud feature prediction module; the tensor EP is processed by D PAfter the self-attention layer, the predicted feature FP is obtained, and the result of the position corresponding to the covered point cloud block in FP is taken as the predicted covered point cloud block feature FPM.

[0146] Exemplarily, the coordinates of the center of the covered point cloud block are input into the covered position coding layer to obtain a covered point cloud block position code of size 96*256, wherein the size of the input covered point cloud block is 96*3; the coordinates of the center of the visible point cloud block are input into the visible position coding layer to obtain a visible point cloud block position code of size 32*256, wherein the size of the input visible point cloud block is 32*3; the visible point cloud block feature FV and the visible point cloud block position code are added, and its size is still 32*256. The addition result is spliced ​​with the covered point cloud block position code to obtain a 128*256 tensor EP, and EP is input into the mask point cloud feature prediction module; the mask point cloud feature prediction module consists of 4 layers of self-attention layers. After EP passes through 4 layers of self-attention layers, the predicted feature FP is obtained, and the result of the position corresponding to the covered point cloud block in FP is taken as the predicted covered point cloud block feature FPM.

[0147] Step S404: semantic alignment of mask point clouds.

[0148] Specifically, the input covered point cloud block feature FM and the predicted covered point cloud block feature FPM are put into the module corresponding to the mask point cloud semantic alignment, that is, the mask point cloud semantic alignment module, and the outputs are the covered point cloud block semantic S M And the predicted covered point cloud block semantics S PM Pre-build the point cloud word vector library (PCV), which consists of A 1*C vectors, where the value of A is pre-set and the vector value is obtained through training; for the vector f in FM and FPM, find the vector v closest to f in PCV and use vector v to replace f, thereby obtaining the semantics S of the covered point cloud block M And the predicted covered point cloud block semantics S PM , where the distance to the vector closest to f, that is, the metric vector, is found and determined using the square error method; and S is calculated M and S PM The squared error is taken as the semantic alignment error of the covered point cloud.

[0149] For example, a point cloud word vector library consisting of 1024 1*256 vectors is pre-built; the vector v in PCV that is closest to each vector f in FM and FPM is found, and the vector v is used to replace the vector f in FM and FPM, thereby obtaining the semantics of the covered point cloud block S respectively. M And the predicted covered point cloud block semantics S PM , and calculate S M and S PMThe square error of L is used as the semantic alignment error of the covered point cloud. The formula for the semantic alignment error of the covered point cloud is: voc =||S M -S PM || 2

[0150] Among them, L voc That is the semantic alignment error of the covered point cloud, S M is the semantics of the covered point cloud block, S PM is the semantics of the predicted covered point cloud block, 2 represents the square, i.e. ||S M -S PM || 2 Represents the semantics of the covered point cloud block S M And the predicted covered point cloud block semantics S PM The square of the norm of the vector difference .

[0151] Step S405: Reconstructing the mask point cloud.

[0152] Specifically, the predicted covered point cloud block feature FPM is input to the module corresponding to the mask point cloud reconstruction, that is, the mask point cloud reconstruction module, and the output is the predicted coordinate X of each point in the covered point cloud block. PM The mask point cloud reconstruction module consists of a DD layer self-attention layer and a linear dimensionality reduction layer. After the FPM passes through the DD layer self-attention layer, its output size remains unchanged, still G*R*C. The FPM after the DD layer self-attention layer is input into the linear dimensionality reduction layer to obtain the predicted coordinates X of each point in the masked point cloud block. PM , whose size is G*R*3, and calculate X M and X PM Chamfer Distance.

[0153] For example, the mask point cloud reconstruction module consists of 4 self-attention layers and a linear dimensionality reduction layer. After the FPM passes through the 4 self-attention layers, its output size remains unchanged at 96*256. The output is put into the linear dimensionality reduction layer to obtain the predicted coordinates X of each point in the masked point cloud block. PM , whose size is 96*3; calculate X M and X PM The chamfer distance formula is:

[0154] Among them, L rec (X M ,X PM ), which is X M and X PM Chamfer distance, X MIncluding the coordinates of each point in the covered point cloud block, p is the coordinate of a point in the covered point cloud block, X PM Including the predicted coordinates of each point in the covered point cloud block, q is the predicted coordinates of a point in the covered point cloud block; where ||qp||2 represents a matrix norm, namely the spectral norm, where 2 indicates that the matrix norm is the 2-norm, and the 2-norm represents the distance in the usual sense, which represents the square root of the maximum matrix eigenvalue obtained by multiplying the (qp) matrix by the transposed conjugate matrix of (qp), that is, the maximum singular value of the (qp) matrix. Therefore, The square of the maximum singular value of the vector difference between the coordinate p of a point in the covered point cloud block and the predicted coordinate q of a point in the covered point cloud block; The square of the maximum singular value of the vector difference between the predicted coordinate q of a point in the covered point cloud block and the coordinate p of a point in the covered point cloud block; Represents X M Any point p to X PM The sum of the minimum distances; Represents X PM Any point q to X M If the calculated chamfer distance is large, it means that the reconstruction effect of the mask point cloud is poor; if the distance is small, it means that the reconstruction effect of the mask point cloud is good.

[0155] Step S406: pre-training.

[0156] Specifically, the semantic alignment error L of the masked point cloud will be voc and X M and X PM Chamfer distance L rec (X M ,X PM ) is taken as the total loss, and the point cloud feature encoding module is trained according to the total loss to improve the accuracy of the feature vector output by the point cloud feature encoding module.

[0157] FIG5 is a diagram of the training process of the mask semantic modeling and alignment method of the present embodiment. Referring to FIG5 , the input is the point cloud to be trained. The point cloud to be trained is divided into a plurality of visible point cloud blocks and covered point cloud blocks through the aforementioned data processing module. The visible point cloud blocks are represented by circular images with dots in circles in FIG5 , and the covered point cloud blocks are represented by circular images with slashes in circles. The obtained plurality of visible point cloud blocks and covered point cloud blocks are then input into the aforementioned point cloud feature encoding module to obtain the feature vectors of the visible point cloud blocks and the feature vectors of the covered point cloud blocks, respectively. In FIG5 , the feature vectors of the visible point cloud blocks are represented by the actual The square with the center is represented by the square with a slash, and the feature vector of the covered point cloud block is represented by a square with a slash; then the aforementioned mask point cloud semantic alignment module is used, and based on the predicted feature vector of the covered point cloud block, the predicted feature vector of the covered point cloud block is represented by a circle with a vertical line in the circle in Figure 5. The vector v closest to each vector f in FM and FPM is found in the point cloud word vector library PCV. The point cloud word vector library is represented by a square set composed of 16 squares arranged in sequence in Figure 5, and the vector v is used to replace the vector f in FM and FPM, thereby obtaining the semantics S of the covered point cloud block respectively. M And the predicted covered point cloud block semantics S PM , and according to S M and S PM Get the semantic alignment error L of the covered point cloud voc ; Then, the predicted feature vector of the masked point cloud block is input through the aforementioned mask point cloud reconstruction module. The predicted feature vector of the masked point cloud block is represented by a square with a vertical line inside in Figure 5, and the predicted coordinates in the masked point cloud block and the chamfer distance L between the coordinates in the masked point cloud block and the predicted coordinates are obtained. rec ,The above process is the mask semantic modeling and alignment method training process of the point cloud ,feature encoding module.

[0158] FIG6 is an optional flow chart of the point cloud matching method of this embodiment. As shown in FIG6 , the point cloud matching method includes the following steps:

[0159] Step S601: data processing.

[0160] Specifically, the input is a single point cloud frame, and the output is a matching point cloud block. The single point cloud frame is input into the module corresponding to data processing, namely the data processing module. The data processing module filters the point cloud frame according to pre-set height and radius range thresholds; the filtered point cloud frame is voxelized according to a pre-set spatial resolution; then, G key points are sampled from the voxelized point cloud frame, and the K points closest to each key point are selected as its neighborhood, resulting in G matching point cloud blocks; where a key point and its neighborhood are called a block; the values ​​of G and K are both pre-set values. Therefore, the output of the data processing module is G matching point cloud blocks; the size of the matching point cloud block is G*N, where N represents the coordinate dimension of a point in three-dimensional space.

[0161] Step S602: point cloud feature encoding.

[0162] Specifically, the matching point cloud block is input to the module corresponding to the point cloud feature encoding, namely the point cloud feature encoding module, and the output of the point cloud feature encoding module is the feature vector of the matching point cloud block. The first layer of the point cloud feature encoding module is a linear layer, which is used to increase the dimension of the point cloud to be matched, that is, to increase the dimension of the matching point cloud block of size G*N to G*C, where the value of N is less than the value of C, and the value of C is a predefined value; the point cloud feature encoding module starts from the second layer to D E The self-attention layer inputs the matching point cloud blocks into the point cloud feature encoding module, and outputs the feature vector F of all matching point cloud blocks. ALL .

[0163] Step S603: point cloud matching.

[0164] Specifically, obtain the vector feature v of each vector in the point cloud word vector library, i.e. PCV n , and calculate the feature vector F of all matching point cloud blocks ALL The local aggregate descriptor vector (Vector of Locally Aggregated Descriptors, VLAD) is given by:

[0165] Among them, n represents the nth vector in PCV, F ALL Represents the feature vectors of all matching point cloud blocks, f all Represents a feature vector of a matching point cloud block. For example, its value range is 1-1024. n =1 indicates v n is the distance f in PCV all The nearest vector, when a n =0 means v n Not the distance f in PCVall nearest vector; Indicates the distance F in PCV ALL The feature f of each matching point cloud block all The set of nearest vectors.

[0166] For example, G n Represents a vector of length 1*256, G1-G 2014 The 1024 vectors are concatenated into a vector of length 1024*256, which is the global description vector of the current point cloud to be matched. Subsequently, the global description vector of the current point cloud to be matched is compared with the global description vectors extracted from the historical database, and the historical point cloud frame with the closest Euclidean distance to the current global description vector is returned as the point cloud matching result.

[0167] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a group of computer executable instructions.

[0168] This embodiment also provides a point cloud matching device 100 for implementing the above-described embodiments and optional implementations. Details already described will not be repeated. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0169] FIG7 is a structural block diagram of a point cloud matching device 100 according to this embodiment. As shown in FIG7 , the device includes a processing module 10 and a matching module 20 .

[0170] The processing module 10 is used to divide the point cloud to be matched into multiple matching point cloud blocks, input the multiple matching point cloud blocks into the pre-trained point cloud feature encoding module, and obtain the feature vectors of the multiple matching point cloud blocks. During pre-training, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset coverage rate, and outputs the feature vectors of the visible point cloud blocks and the masked point cloud blocks respectively.

[0171] The matching module 20 is used to obtain the global description vector of the point cloud to be matched based on the feature vectors of multiple matching point cloud blocks, match the global description vector of the point cloud to be matched with the global description vector of the point cloud frame in the preset historical database, and determine that the point cloud frame in the historical database within the preset matching threshold range is the point cloud matching result.

[0172] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0173] This embodiment further provides an electronic device 200, comprising a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0174] Optionally, the electronic device 200 may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0175] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0176] Step 1: Divide the point cloud to be matched into multiple matching point cloud blocks, input the multiple matching point cloud blocks into the pre-trained point cloud feature encoding module, and obtain the feature vectors of the multiple matching point cloud blocks; during pre-training, the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset coverage rate, and outputs the feature vectors of the visible point cloud blocks and the masked point cloud blocks respectively to train the point cloud feature encoding module.

[0177] Step 2: Obtain the global description vector of the point cloud to be matched based on the feature vectors of multiple matching point cloud blocks, match the global description vector of the point cloud to be matched with the global description vector of the point cloud frame in the preset historical database, and determine that the point cloud frame in the historical database within the preset matching threshold range is the point cloud matching result.

[0178] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0179] In addition, in conjunction with the point cloud matching method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the point cloud matching method. The storage medium stores a computer program that, when executed by a processor, implements any of the point cloud matching methods in the above embodiments.

[0180] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0181] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0182] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0183] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A point cloud matching method, characterized in that: The method comprises: The point cloud to be matched is divided into a plurality of matching point cloud blocks, and the plurality of matching point cloud blocks are input into a pre-trained point cloud feature encoding module to obtain feature vectors of the plurality of matching point cloud blocks; during pre-training, the point cloud feature encoding module inputs training point cloud blocks to be trained after being masked according to a preset masking rate, and outputs feature vectors of visible point cloud blocks and masked point cloud blocks respectively, so as to train the point cloud feature encoding module; The global description vector of the point cloud to be matched is obtained according to the feature vectors of multiple matching point cloud blocks, the global description vector of the point cloud to be matched is matched with the global description vector of the point cloud frame in a preset historical database, and the point cloud frame in the historical database within a preset matching threshold range is determined as the point cloud matching result.

2. The point cloud matching method according to claim 1, wherein: The step of dividing the point cloud to be matched into a plurality of matching point cloud blocks comprises: Filtering the point cloud to be matched according to a preset height and radius; The filtered point cloud to be matched is voxelized according to the preset spatial resolution; Sampling the voxelized point cloud to be matched, and determining multiple sampled points as key points; According to the key point and a set of points within a preset range of the key point, the point cloud to be matched is divided into a plurality of matching point cloud blocks.

3. The point cloud matching method according to claim 1, wherein: The step of dividing the point cloud to be matched into a plurality of matching point cloud blocks, inputting the plurality of matching point cloud blocks into a pre-trained point cloud feature encoding module, and obtaining feature vectors of the plurality of matching point cloud blocks includes: Inputting the plurality of matching point cloud blocks into a preset linear dimension-increasing layer to obtain matching point cloud blocks that are dimension-increasing to a preset dimension; The matching point cloud blocks upgraded to a preset dimension are input into a preset first self-attention layer to obtain feature vectors of multiple matching point cloud blocks.

4. The point cloud matching method according to claim 1, wherein: The step of obtaining the global description vector of the point cloud to be matched according to the feature vectors of the plurality of matching point cloud blocks comprises: Acquire multiple distance vectors in a preset vector library, where the distance vectors are vectors whose distances from the feature vectors of the multiple matching point cloud blocks are within a preset distance threshold range; The feature vectors of a plurality of the matching point cloud blocks and a plurality of the distance vectors are concatenated to obtain a global description vector of the point cloud to be matched.

5. The point cloud matching method according to claim 1, wherein: The method includes dividing the to-be-matched point cloud into a plurality of matching point cloud blocks and inputting the plurality of matching point cloud blocks into the pre-trained point cloud feature encoding module: Filter the point cloud to be trained according to the preset height and radius; The filtered point cloud to be trained is voxelized according to the preset spatial resolution; Sampling the voxelized point cloud to be trained, and determining multiple sampled points as key points; According to the key point and a set of points within a preset range of the key point, the point cloud to be trained is divided into a plurality of training point cloud blocks; Covering the plurality of training point cloud blocks according to a preset covering rate to obtain a first point cloud block and a second point cloud block, wherein the first point cloud block is a visible point cloud block and the second point cloud block is a covered point cloud block; Obtaining prediction information of the second point cloud block according to the information of the first point cloud block and the information of the second point cloud block; A prediction loss is obtained according to the information of the second point cloud block and the prediction information of the second point cloud block, and the point cloud feature encoding module is trained according to the prediction loss.

6. The point cloud matching method according to claim 5, wherein: The obtaining prediction information of the second point cloud block according to the information of the first point cloud block and the information of the second point cloud block includes: Inputting the coordinates of the key points in the first point cloud block into a preset visible position coding layer to obtain the position coding of the first point cloud block; Inputting the coordinates of the key points in the second point cloud block into a preset cover position coding layer to obtain the position coding of the second point cloud block; Inputting the first point cloud block and the second point cloud block into a preset linear dimension-upgrading layer to obtain the first point cloud block and the second point cloud block upgraded to a preset dimension; Inputting the first point cloud block upgraded to a preset dimension into a preset first self-attention layer to obtain a feature vector of the first point cloud block; Adding the feature vector of the first point cloud block and the position code of the first point cloud block, and concatenating the addition result with the position code of the second point cloud block to obtain the tensor of the point cloud to be trained; The tensor of the point cloud to be trained is input into a preset second self-attention layer to obtain a predicted feature vector of the second point cloud block, where the predicted feature vector is the prediction information.

7. The point cloud matching method according to claim 6, wherein: The obtaining the prediction loss according to the information of the second point cloud block and the prediction information of the second point cloud block includes: Acquire the predicted feature vector of the second point cloud block and a plurality of original vectors in the feature vector of the second point cloud block, and acquire replacement vectors corresponding to the plurality of the original vectors in a preset vector library, and use the replacement vectors to replace the plurality of the original vectors; wherein the feature vector of the second point cloud block is obtained by inputting the second point cloud block upgraded to a preset dimension into a preset first self-attention layer; Determine respectively the feature vector and the predicted feature vector of the replaced second point cloud block as the semantics of the second point cloud block and the predicted semantics of the second point cloud block; Calculating the square error between the semantics of the second point cloud block and the predicted semantics of the second point cloud block, where the square error is the first prediction loss; Inputting the feature vector of the second point cloud block into a preset third self-attention layer and a linear dimensionality reduction layer to obtain the predicted coordinates of each point in the second point cloud block after dimensionality reduction; Calculating a chamfer distance between the coordinates of the second point cloud block and the predicted coordinates of the second point cloud block, where the chamfer distance is the second prediction loss; The sum of the first predicted loss and the second predicted loss is determined as the predicted loss.

8. A point cloud matching device, characterized in that: The device comprises: a processing module and a matching module; The processing module is used to divide the point cloud to be matched into a plurality of matching point cloud blocks, input the plurality of matching point cloud blocks to the pre-trained point cloud feature encoding module, and obtain feature vectors of the plurality of matching point cloud blocks; the point cloud feature encoding module inputs the training point cloud blocks to be trained after being masked according to a preset masking rate during pre-training, and outputs feature vectors of visible point cloud blocks and masked point cloud blocks respectively; The matching module is used to obtain the global description vector of the point cloud to be matched based on the feature vectors of multiple matching point cloud blocks, match the global description vector of the point cloud to be matched with the global description vector of the point cloud frame in a preset historical database, and determine that the point cloud frame in the historical database within a preset matching threshold range is the point cloud matching result.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the point cloud matching method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the point cloud matching method according to any one of claims 1 to 7 are implemented.

Images