Positioning system and method of inspection unmanned vehicle, training method of semantic segmentation model, communication device, storage medium and computer program product

Abstract

The application provides a positioning system and method of an inspection unmanned vehicle, a training method of a semantic segmentation model, a communication device, a storage medium and a computer program product. The positioning system comprises a semantic segmentation module for performing multi-class semantic segmentation on a laser radar point cloud to obtain point-level semantic category information about a target point cloud of an equipment to be inspected, an instance extraction module for extracting a target point cloud instance through a density clustering algorithm, a coarse registration module and an accurate registration module for calculating an initial transformation matrix between the target point cloud instance and a pre-stored standard point cloud model through geometric constraint consistency by using a K4PCS method, and the accurate registration module performs nearest neighbor search, error minimization solving and iterative optimization by using an ICP method to obtain a final pose transformation matrix of the target point cloud instance. According to the technical scheme of the application, the positioning accuracy of the inspection unmanned vehicle to a target in a complex airport environment can be significantly improved.

Description

Technical Field

[0001] This invention relates to the field of aviation intelligence, and more particularly to a positioning system and method for unmanned inspection vehicles, a training method for semantic segmentation models, a communication device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the improvement of airport security supervision and intelligence, unmanned vehicles are being used more and more widely in airport ground inspection tasks. However, the actual airport scene is complex and varied. Objects such as aircraft fuselages, towing vehicles, jet bridges, and ground support equipment can cause serious obstruction, which often leads to problems such as sparseness, missing parts, overlap, and misclassification of aircraft point clouds collected by lidar, thereby reducing the reliability of point cloud registration.

[0003] Therefore, there is a need to provide a positioning system and method for unmanned inspection vehicles, a training method for semantic segmentation models, a communication device, a computer-readable storage medium, and a computer program product to at least partially solve the above problems. Summary of the Invention

[0004] According to a first aspect of the present invention, a positioning system for an unmanned inspection vehicle is provided, comprising: A semantic segmentation module is configured to perform multi-category semantic segmentation on the LiDAR point cloud collected by the inspection unmanned vehicle to obtain point-level semantic category information about the target point cloud of the equipment to be inspected. An instance extraction module is configured to extract target point cloud instances of the equipment to be inspected and remove noise based on the point-level semantic category information provided by the semantic segmentation module using the density clustering (DBSCAN) algorithm. A coarse registration module, configured to establish an initial point cloud correspondence using the K4PCS (Kinetic Four-Point Similarity Set) method through geometric constraint consistency screening, and to calculate the initial transformation matrix between the target point cloud instance extracted by the instance extraction module and the pre-stored standard point cloud model; and The precise registration module is configured to perform nearest neighbor search, error minimization, and iterative optimization based on the initial transformation matrix obtained by the coarse registration module using the ICP (Iterative ClosestPoint) method, in order to obtain the final pose transformation matrix of the target point cloud instance of the equipment to be inspected.

[0005] Preferably, the semantic segmentation module includes: A global attention enhancement module is configured to construct long-range dependencies on the lidar point cloud, enhance the ability to distinguish occluded regions, structural boundaries, and geometric shapes through global context modeling, and output enhanced point cloud features; and A multi-layer feature extraction module is configured to receive the enhanced point cloud features and extract spatial structure information about the equipment to be inspected from coded features at different scales, thereby generating multi-layer semantic features.

[0006] Preferably, the semantic segmentation module further includes: A decoding and restoration module, configured to upsample the multi-layer semantic features to progressively increase the spatial resolution of the multi-layer semantic features, and output high-dimensional semantic features; and An output prediction module is configured to map the high-dimensional semantic features to the point-level semantic category information.

[0007] Preferably, the global attention enhancement module includes: A feature encoding unit is configured to linearly map the features of the lidar point cloud to form a feature representation for processing by an attention network. The feature stitching and difference calculation unit is configured to combine the features of each point in the lidar point cloud based on the feature representation formed by the feature encoding unit, and calculate the feature difference between local and global features to enhance the sensitivity to occluded areas. The self-attention mechanism unit is configured to employ the Transformer's Query-Key-Value structure, calculating attention weights by evaluating the similarity between point cloud features, thereby modeling global contextual information to generate attention-weighted features; and The feature enhancement output unit is configured to fuse, normalize, and adaptively adjust the attention-weighted features, and output the enhanced point cloud features.

[0008] Preferably, the multi-layer feature extraction module includes: A multilayer encoder unit, configured to extract local geometric features and overall structural information of the point cloud features from shallow to deep layers; A cross-layer feature fusion unit is configured to concatenate and enhance features extracted from different layers by the multi-layer encoder unit, and perform difference alignment using global attention weights. An attention optimization unit, configured to perform attention filtering on the features fused by the cross-layer feature fusion unit; and An output unit is configured to output the semantic features filtered by the attention optimization unit to the decoding and restoration module.

[0009] Preferably, the instance extraction module includes: An unvisited point selection module is configured to select unvisited points as candidate points from the target point cloud of the equipment to be inspected and generate a neighborhood point set. The core point determination module is communicatively connected to the unvisited point selection module. The core point determination module is configured to determine whether the neighborhood point set of the candidate point satisfies the core point condition, and to mark the candidate points that do not satisfy the core point condition as noise points. A cluster expansion module is communicatively connected to the core point determination module. The cluster expansion module is configured to add the candidate point to the cluster when the candidate point is determined to be a core point, and to expand the unclassified points in the neighborhood point set in sequence. The expansion includes obtaining the neighborhood point set of the current expanded point through the neighborhood search module, determining whether the neighborhood point set of the candidate point meets the core point condition through the core point determination module, and merging the expanded points that meet the core point condition into the current cluster set. A cluster termination control module is configured to control the iteration process until all points in the target point cloud of the equipment to be inspected have been accessed and the final clustering result has been output, so as to realize density clustering of the target point cloud and extract the target point cloud instance.

[0010] Preferably, the coarse registration module includes: A four-point sampling module is configured to randomly sample four pairs of points from the target point cloud instance and the standard point cloud model to generate a candidate set. A geometric constraint detection module is communicatively connected to the four-point sampling module. The geometric constraint detection module is configured to determine whether the sampled four-point pairs satisfy geometric consistency constraints. An initial transformation solution module is communicatively connected to the geometric constraint detection module. The initial transformation solution module is configured to calculate and output the initial transformation matrix from the target point cloud instance to the standard point cloud model when the four point pairs satisfy the geometric consistency constraint. The transformation application and interior point statistics module is communicatively connected to the initial transformation solution module. The transformation application and interior point statistics module is configured to apply the initial transformation matrix to the target point cloud instance and count the number of point pairs whose distance between the transformed point cloud and the standard point cloud model is less than a preset threshold as the number of interior points. The registration judgment and output module is communicatively connected to the transformation application and interior point statistics module. The registration judgment and output module is configured to determine whether the number of interior points has reached a set threshold. If the number of interior points reaches the set threshold, the initial transformation matrix is ​​output as the initial registration result. If the number of interior points does not reach the set threshold, the four-point sampling module is triggered to resample.

[0011] Preferably, the precise registration module includes: A feature matching initialization module is configured to receive the initial registration result and initialize the point pairs to be matched between the target point cloud instance and the standard point cloud model. The nearest point search module is communicatively connected to the feature matching initialization module. The nearest point search module is configured to find the nearest neighbor in the standard point cloud model for each point in the target point cloud instance in each iteration to form a set of point pairs. A coordinate transformation calculation module is communicatively connected to the nearest point search module. The coordinate transformation calculation module is configured to calculate the optimal rotation matrix and translation vector for the current iteration based on the set of point pairs, and update the position of the target point cloud instance. An error evaluation module, communicatively connected to the coordinate transformation calculation module, is configured to calculate the registration error of the current iteration; and An iteration termination control module is communicatively connected to the error evaluation module and the nearest point search module. The iteration termination control module is configured to determine whether the registration error has converged or whether the number of iterations has reached the upper limit. If the registration error has converged or the number of iterations has reached the upper limit, the final pose transformation matrix is ​​output. If the registration error has not converged or the number of iterations has not reached the upper limit, the next iteration process is triggered.

[0012] According to a second aspect of the present invention, a positioning method for an unmanned inspection vehicle is provided, comprising: S1. The lidar point cloud collected by the inspection unmanned vehicle is subjected to multi-category semantic segmentation through the semantic segmentation module to obtain point-level semantic category information of the target point cloud of the equipment to be inspected. S2. The instance extraction module uses a density clustering algorithm to extract target point cloud instances related to the equipment to be inspected from the target point cloud filtered according to the point-level semantic category information and removes noise points. S3. Using a coarse registration module, the K4PCS method is employed, and geometric constraint consistency filtering is used to establish an initial point cloud correspondence. The initial transformation matrix between the target point cloud instance and the pre-stored standard point cloud model is calculated. S4. Using the precise registration module and the ICP method, the final pose transformation matrix of the target point cloud instance is obtained by performing nearest neighbor search, error minimization solution and iterative optimization on the initial transformation matrix obtained by the coarse registration module.

[0013] Preferably, the semantic segmentation step in step S1 includes: performing point-level feature extraction, neighborhood construction, and semantic classification prediction on the lidar point cloud to obtain the semantic category information of each point in the lidar point cloud.

[0014] Preferably, step S2 further includes: Select any unvisited point from the target point cloud of the equipment to be inspected as a candidate point and generate a neighborhood point set; Determine whether the neighborhood point set of the candidate point satisfies the core point condition, and mark the candidate points that do not satisfy the core point condition as noise points; Candidate points that meet the core point conditions are added to the cluster set, and unclassified points in the neighborhood point set are expanded in turn. The expansion includes obtaining the neighborhood point set of the current expanded point through the neighborhood search module, determining whether the neighborhood point set of the candidate point meets the core point conditions through the core point determination module, and merging the expanded points that meet the core point conditions into the current cluster set. Repeat the above steps until all points in the target point cloud related to the equipment to be inspected have been visited, and output the final clustering result.

[0015] Preferably, step S3 further includes a coarse registration step, which includes: Four point pairs are randomly sampled from the target point cloud instance and the standard point cloud model to generate a candidate set; Determine whether the four-point pair satisfies the geometric consistency constraint; When the four point pairs satisfy the geometric consistency constraint, calculate and output the preliminary transformation matrix from the target point cloud instance to the standard point cloud model; The preliminary transformation matrix is ​​applied to the target point cloud instance, and the number of point pairs whose distance between the transformed point cloud and the standard point cloud model is less than a preset threshold is counted as the number of inliers. If the number of inliers reaches a set threshold, the preliminary transformation matrix is ​​output as the initial registration result; if the number of inliers does not reach the set threshold, resampling is triggered.

[0016] Preferably, step S4 further includes a precise registration step, which includes: Receive the initial registration result and initialize the point pairs to be matched between the target point cloud instance and the standard point cloud model; In each iteration, for each point in the target point cloud instance, find the nearest neighbor in the standard point cloud model to form a set of point pairs; Calculate the optimal rotation matrix and translation vector for this iteration based on the set of point pairs, and update the position of the target point cloud instance; Calculate the registration error for the current iteration; Determine whether the registration error has converged or whether the number of iterations has reached the upper limit. If the registration error has converged or the number of iterations has reached the upper limit, output the final pose transformation matrix. If the registration error has not converged or the number of iterations has not reached the upper limit, trigger the next iteration process.

[0017] According to a third aspect of the present invention, a method for training a semantic segmentation model in the above-described positioning system is provided, comprising: S51. Preprocess the point cloud sample data to obtain the processed data; S52. Input the processed data into the encoder structure to obtain feature information; S53. Input the feature information into the decoder structure to obtain the semantic prediction score; and S54. Construct a loss function based on the feature map and the semantic prediction score, and train a semantic segmentation model based on the loss function.

[0018] Preferably, the preprocessing includes one or more of the following data augmentation operations: point cloud denoising, coordinate normalization, point intensity normalization, random rotation, random scaling, and random sampling.

[0019] Preferably, the encoder structure is configured to construct local and global geometric features from the point cloud to extract semantic representations.

[0020] Preferably, the encoder structure includes: a multilayer MLP, a convolutional structure, an attention structure, or a local graph aggregation structure.

[0021] Preferably, the decoder structure is configured to perform upsampling and fusion operations on features from the encoder structure to output the semantic classification probability of each point.

[0022] Preferably, the loss function includes one or more of cross-entropy loss, Dice loss, and Focal Loss.

[0023] According to a fourth aspect of the present invention, a communication device is provided, comprising a processor, wherein the processor executes computer programs or instructions via logic circuits to implement the method described above.

[0024] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program or instructions are stored therein, which, when executed by a communication device, implement the method described above.

[0025] According to a sixth aspect of the present invention, a computer program product is provided, the computer program product storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described above.

[0026] The above-mentioned solution of this application has the following technical effects: (1) Through the three-level serial architecture of semantic segmentation, instance extraction and point cloud registration, the stability of point cloud matching under sparse, occluded and noisy conditions can be effectively improved, and the positioning accuracy of the inspection unmanned vehicle in the complex airport environment can be significantly improved.

[0027] (2) The point cloud semantic segmentation module, combined with the global attention enhancement module and the multi-layer feature extraction module, can effectively model remote dependencies and improve the semantic recognition ability of occluded areas and complex boundaries, thus providing a reliable semantic foundation for subsequent registration.

[0028] (3) The dual-stage point cloud registration mechanism of “K4PCS coarse registration + ICP precise registration” is adopted. It can still maintain robust initial alignment in sparse point cloud, missing point cloud and strong occlusion scenarios, and obtain high-precision registration results in local iterative optimization, which significantly improves the positioning accuracy and stability of the inspection unmanned vehicle in complex airport environment. Attached Figure Description

[0029] To better understand the above and other objects, features, advantages, and functions of the present invention, reference can be made to the preferred embodiments shown in the accompanying drawings. The same or similar reference numerals in the drawings refer to the same or similar parts. Those skilled in the art should understand that the drawings are intended to schematically illustrate preferred embodiments of the invention and do not limit the scope of the invention in any way; the parts in the drawings are not drawn to scale.

[0030] Figure 1 This is a schematic diagram of the structure of a semantic segmentation module according to a preferred embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the structure of a global attention enhancement module according to a preferred embodiment of the present invention.

[0032] Figure 3 This is a schematic diagram of the structure of a multi-layer feature extraction module according to a preferred embodiment of the present invention.

[0033] Figure 4 This is a flowchart illustrating the workflow of a positioning system according to a preferred embodiment of the present invention.

[0034] Figure 5 This is a flowchart of an extraction method based on a density clustering algorithm (DBSCAN) according to a preferred embodiment of the present invention.

[0035] Figure 6 This is a flowchart of a coarse registration method based on K4PCS according to a preferred embodiment of the present invention.

[0036] Figure 7 This is a flowchart of an ICP precise registration method according to a preferred embodiment of the present invention.

[0037] Figure 8 This is an overall flowchart of a positioning method for an unmanned inspection vehicle according to a preferred embodiment of the present invention.

[0038] Figure 9 This is an overall flowchart of a training method for a semantic segmentation model in a localization system according to a preferred embodiment of the present invention. Detailed Implementation

[0039] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It is to be understood that the following description is merely a preferred embodiment of the present invention, and those skilled in the art can conceive of other ways to implement the present invention based on the preferred embodiments; such other ways also fall within the scope of the present invention.

[0040] The following will refer to Figures 1-9 The preferred technical solutions in the embodiments of the present invention will be described in detail below.

[0041] This invention discloses a positioning system for inspecting unmanned vehicles, which can achieve precise positioning of the equipment to be inspected. The equipment to be inspected can be, for example, an aircraft in an airport, a freight vehicle, or a train or car in a station. The following describes in detail the positioning system and method according to a preferred embodiment of the invention, using an aircraft as an example.

[0042] The positioning system includes a semantic segmentation module, which is mainly used to perform multi-category semantic segmentation on the airport LiDAR point cloud collected by the unmanned aerial vehicle (UAV) for inspection, in order to identify the aircraft target point cloud and separate various types of interfering point clouds. In one embodiment, the semantic segmentation module is configured to input the point cloud to be processed into a trained semantic segmentation model for multi-category semantic segmentation to obtain point-level semantic category information about the aircraft target point cloud. The semantic segmentation model can accurately identify and classify aircraft targets and obstructing interference objects based on the input airport inspection scene point cloud.

[0043] In one implementation, the semantic segmentation model employs an encoder-decoder structure. The encoder extracts multi-level geometric and semantic features through downsampling, while the decoder upsamples layer by layer to restore spatial resolution. "Downsampling" here refers to reducing the density of data points, converting a high-resolution signal into a low-resolution signal. Specifically, in this implementation, "downsampling" refers to progressively shrinking larger feature maps in the network. "Upsampling" refers to converting a signal or image from low resolution to high resolution. Specifically, in this implementation, "upsampling" refers to restoring smaller feature maps in the network to a larger spatial size. The encoder or decoder can employ a Transformer architecture, or other encoder-decoder structures can be used as needed by those skilled in the art.

[0044] In one implementation, the semantic segmentation module includes: a global attention enhancement module, a multi-layer feature extraction module, a decoding and restoration module, and an output prediction module. The global attention enhancement module constructs long-range dependencies on the input point cloud, enhancing the ability to distinguish occluded regions, structural boundaries, and complex geometric shapes through global context modeling. The multi-layer feature extraction module extracts rich spatial structural information from encoded features at different scales. The decoding and restoration module restores spatial resolution layer by layer. The output prediction module maps high-dimensional semantic features to point-level category labels to achieve semantic segmentation of the input point cloud.

[0045] The global attention enhancement module comprises: a feature encoding unit, which linearly maps the coordinates of the input point cloud to the basic features to form a feature representation that can be processed by the attention network; a feature concatenation and difference calculation unit, which combines the features of each point in a set manner and calculates the feature difference between the local and global features to enhance the sensitivity of occluded areas; a self-attention mechanism unit, which adopts the Query-Key-Value structure of Transformer and calculates attention weights based on the similarity between point cloud features to achieve global contextual information modeling (i.e., when the model encodes the local features of the current point, it can aggregate contextual information from the entire point cloud to enhance the semantic understanding of the point); and a feature enhancement output unit, which fuses, normalizes (e.g., layer normalization, pulling the feature distribution back to a stable range to ensure training stability) and adaptively adjusts the attention-weighted features (i.e., recalibrates the normalized features channel by channel using a learnable scaling factor γ and a translation factor β to enhance key semantic features and suppress irrelevant information), and finally outputs the globally enhanced high-dimensional semantic features.

[0046] The multi-layer feature extraction module includes: a multi-layer encoder unit, used to extract local geometric features and overall structural information of point clouds from shallow to deep layers; a cross-layer feature fusion unit, used to concatenate and enhance features from different layers and use global attention weights for difference alignment; an attention optimization unit, used to perform attention filtering on the fused features to strengthen key structural regions and suppress noisy regions; and an output unit, used to output the multi-layer fused semantic features to the decoding and restoration module.

[0047] The following section uses the example of a semantic segmentation model segmenting the point cloud to be processed based on the input point cloud to illustrate this point segmentation.

[0048] like Figure 1 As shown, in one embodiment, the semantic segmentation model cascades multiple global attention enhancement modules and downsampling units along the depth direction during the encoding stage to extract feature representations at different scales layer by layer. During the decoding stage, upsampling units and a multilayer perceptron restore the spatial resolution of the feature map, which is then fused with the multi-scale features from the encoding stage to obtain a high-resolution semantic feature map. To further enhance cross-layer information interaction, a multilayer feature extraction module is set at the end of the encoder. This module aggregates and reorganizes the encoded features from each layer before inputting them to the decoder, thereby improving the ability to represent complex boundaries and occlusion structures.

[0049] The multilayer perceptron used for feature reconstruction in the decoding stage can be implemented using the feedforward network structure in Transformer. The specific implementation method can be implemented in a way known in the art, and will not be elaborated here for the sake of brevity.

[0050] like Figure 2As shown, the global attention enhancement module simultaneously encodes the spatial location and local features of points. This module first calculates the centroid feature difference between the neighboring point cloud and the center point to characterize local geometric changes. Then, a self-attention mechanism is introduced to adaptively model the global features of the point cloud, enabling the model to capture long-range dependencies. After feature enhancement, the enhanced features are concatenated with the original feature encodings to improve the expressive power of global semantic features. It should be noted that all global attention enhancement modules in this invention have similar structures.

[0051] like Figure 3 As shown, the multi-layer feature extraction module is used to aggregate the features output by the multi-layer encoder across scales. This module first stacks local features from each encoding layer to form a combined descriptive vector. Then, it uses a 1×1 convolution to linearly transform the combined features to unify the channel dimensions. Next, it uses the Transformer's Query-Key-Value mechanism to construct a contextual feature association graph. Scaling dot product attention is used to achieve cross-scale alignment and fusion between multi-layer features. Finally, the fusion convolution module performs convolution mapping on the fused feature map to generate the final global feature map, which, after normalization, is input to the decoding and restoration module to recover the semantic segmentation result of the output point cloud.

[0052] like Figure 4 As shown, in one embodiment, the positioning system further includes an instance extraction module (also known as a density clustering module) 11, a K4PCS-based coarse registration module 12, and an ICP-based precise registration module 13.

[0053] Specifically, the instance extraction module 11, based on the point-level semantic category information provided by the semantic segmentation module, uses a density clustering algorithm to extract target point cloud instances related to the aircraft based on point cloud density connectivity, and removes residual misclassified points and nested interference points. The coarse registration module 12 uses the K4PCS method to establish an initial point cloud correspondence through geometric constraint consistency screening, and calculates the initial transformation matrix between the target point cloud instances extracted by the instance extraction module and the pre-stored standard point cloud model. The precise registration module 13 further calculates the precise pose transformation based on the coarse registration. It uses the ICP method to perform nearest neighbor search, error minimization, and iterative optimization based on the initial transformation matrix obtained by the coarse registration module to obtain the final pose transformation matrix of the target point cloud instances related to the equipment to be inspected.

[0054] like Figure 5 As shown, in one embodiment, the instance extraction module 11 includes: an unvisited point selection module, a neighborhood search module, a core point determination module, a cluster expansion module, and a cluster termination control module.

[0055] Specifically, the unvisited point selection module selects any unvisited point from the input point cloud, uses it as the current candidate point, and generates its neighborhood point set. The core point determination module is communicatively connected to the unvisited point selection module and determines whether the neighborhood point set of the candidate point satisfies the core point condition. If not, the point is marked as a noise point. If the candidate point is a core point, the cluster expansion module adds the point to a new cluster and selects unclassified points from its neighborhood point set for sequential expansion. The neighborhood search module calculates the neighborhood point set of the currently expanded point and determines whether the expanded point is a core point. For expanded points that satisfy the core point condition, their neighborhood points are merged into the current cluster set and expansion continues until there are no unclassified points in the neighborhood. The cluster termination control module controls the iteration process until all points in the target point cloud of the relevant equipment to be inspected have been visited and the final clustering result is output. Through the above process, density clustering of the input point cloud can be achieved, thereby extracting instance-level point cloud objects.

[0056] like Figure 6 As shown, in one embodiment, the coarse registration module 12 includes: a four-point sampling module, a geometric constraint detection module, an initial transformation solution module, a transformation application and interior point statistics module, and a registration judgment and output module.

[0057] Specifically, the four-point sampling module randomly samples four point pairs from the source point cloud P (the target point cloud instance) and the target point cloud Q (the standard point cloud model) to generate a candidate four-point set; the geometric constraint detection module determines whether the sampled four-point combination satisfies the geometric consistency constraint; if not, resampling is performed; when the four-point pair satisfies the constraint, the initial transformation solution module calculates the preliminary transformation matrix (including rotation and translation) between the source and target point clouds; the transformation application and interior point statistics module applies the preliminary transformation to the source point cloud to obtain the transformed point cloud P', and counts the number of interior points between P' and Q (i.e., the number of point pairs with a distance less than a preset threshold ε); the registration judgment and output module determines whether the current sampling meets the registration requirements based on the number of interior points; if the number of interior points reaches a set threshold K, the transformation matrix is ​​output as the optimal initial registration result; if the sampling count reaches the upper limit and the condition is still not met, the coarse registration is deemed to have failed. Through the above process, and through multiple sampling and filtering, the source and target point clouds achieve reliable preliminary alignment.

[0058] like Figure 7 As shown, in one embodiment, the precise registration module 13 includes: a feature matching initialization module, a nearest point search module, a coordinate transformation calculation module, an error evaluation module, and an iteration termination control module.

[0059] Specifically, the feature matching initialization module receives the coarse registration results and initializes the point pairs to be matched; the nearest point search module finds the nearest neighbor for each point in the source point cloud within the target point cloud, forming a set of point pairs; the coordinate transformation calculation module calculates the optimal coordinate transformation vector (including rotation and translation vectors) based on the set of point pairs and applies the transformation to the source point cloud; the error evaluation module calculates the matching error of the current iteration, including point-to-point distance error or point-to-surface error; and the iteration termination module determines whether the error has converged or whether the maximum number of iterations has been reached. If the registration error converges or the number of iterations reaches the upper limit, the final pose transformation matrix is ​​output; otherwise, the next iteration is triggered. Through continuous iteration, the source point cloud gradually approaches the target point cloud position, achieving high-precision alignment.

[0060] like Figure 8 As shown, in one embodiment, the present invention also provides a high-precision positioning method for unmanned vehicles based on the above-described positioning algorithm model. Specifically, the high-precision positioning method for unmanned vehicles includes: Step S1: Perform multi-category semantic segmentation on the lidar point cloud collected by the inspection unmanned vehicle through the semantic segmentation module to obtain point-level semantic category information of the target point cloud of the equipment to be inspected.

[0061] The semantic segmentation operation in step S1 can employ any existing mature deep learning point cloud semantic segmentation network, such as PointNet++, RandLA-Net, KPConv, etc. In one embodiment, the segmentation step includes at least, but is not limited to, extracting point-level features from the input point cloud, constructing neighborhoods, and predicting semantic classification to obtain the semantic category information of each point.

[0062] Step S2: Using the instance extraction module, density clustering algorithm is used to extract target point cloud instances related to the equipment to be inspected from the target point cloud filtered according to the point-level semantic category information, and noise points are removed.

[0063] Specifically, the point cloud obtained in step S1 is filtered according to semantic category information, and then instance clustering is performed based on the DBSCAN algorithm to extract target point cloud objects that meet the positioning requirements of unmanned vehicles.

[0064] In one implementation, the extraction process based on the DBSCAN algorithm includes: performing a neighborhood search on unvisited points to determine if they are core points; expanding the clusters of core points; and marking points that do not meet the conditions as noise points or boundary points. This method can form a stable point cloud of target object instances. This extraction method has the advantages of being insensitive to noise and robust to changes in point cloud density.

[0065] Step S3: Using the coarse registration module, the K4PCS method is adopted and the geometric constraint consistency screening is used to establish the initial point cloud correspondence, and the initial transformation matrix between the target point cloud instance and the pre-stored standard point cloud model is calculated.

[0066] Specifically, in this step, the target point cloud object is input into the coarse registration module, and the initial alignment relationship is obtained through the K4PCS coarse registration method. In one embodiment, the coarse registration step specifically includes: randomly sampling four point pairs from the target point cloud instance and the standard point cloud model to generate a candidate set; determining whether the four point pairs satisfy the geometric consistency constraint; when the four point pairs satisfy the geometric consistency constraint, calculating and outputting the preliminary transformation matrix from the target point cloud instance to the standard point cloud model; applying the preliminary transformation matrix to the target point cloud instance, and counting the number of point pairs whose distance between the transformed point cloud and the standard point cloud model is less than a preset threshold as the number of inliers; if the number of inliers reaches the set threshold, outputting the preliminary transformation matrix as the initial registration result; if the number of inliers does not reach the set threshold, triggering resampling.

[0067] Step S4: Using the precise registration module and the ICP method, perform nearest neighbor search, error minimization, and iterative optimization on the initial transformation matrix obtained by the coarse registration module to obtain the final pose transformation matrix of the target point cloud instance.

[0068] Specifically, in this step, the target point cloud object is input into the precise registration module, and the final pose is obtained through the ICP precise registration method to achieve high-precision unmanned vehicle positioning. The precise registration step includes: the precise registration module receives the initial registration result and initializes the point pairs to be matched between the target point cloud instance and the standard point cloud model; in each iteration, it finds the nearest neighbor point in the standard point cloud model for each point in the target point cloud instance, forming a set of point pairs; based on the set of point pairs, it calculates the optimal rotation matrix and translation vector for this iteration and updates the position of the target point cloud instance; it calculates the registration error of the current iteration; it determines whether the registration error has converged or whether the number of iterations has reached the upper limit. If the registration error has converged or the number of iterations has reached the upper limit, the final pose transformation matrix is ​​output; if the registration error has not converged or the number of iterations has not reached the upper limit, the next iteration process is triggered.

[0069] like Figure 9 As shown, in one embodiment, the present invention also provides a training method for a semantic segmentation model in the above-described positioning system, used to obtain a trained semantic segmentation model. The training method specifically includes the following steps: Step S51: Preprocess the normal point cloud sample data to obtain processed data.

[0070] It should be noted that the normal sample data includes multiple standard scene point clouds without missing or occlusion, which are used to learn the semantic rules of the point clouds.

[0071] In one embodiment, the preprocessing includes, but is not limited to, data augmentation operations such as point cloud denoising, coordinate normalization, point intensity normalization, random rotation, scaling, and sampling.

[0072] Step S52: Input the processed data into the encoder structure to obtain feature information.

[0073] In one implementation, the encoder structure constructs local and global geometric features from the point cloud for extracting semantic representations. The encoder structure may include, but is not limited to, multi-layer MLPs, convolutional structures, attention structures, and local graph aggregation structures.

[0074] Step S53: Input the feature information into the decoder structure to obtain the semantic prediction score.

[0075] Specifically, the decoder upsamples and fuses the features from the encoder to output the semantic classification probability of each point.

[0076] Step S54: Construct a loss function based on the feature map and the prediction score, and train a semantic segmentation model based on the loss function.

[0077] In one implementation, the loss function includes, but is not limited to, cross-entropy loss, Dice loss, FocalLoss, or a combination of these losses. The training objective is to minimize the loss function so that the model can stably perform class discrimination and semantic structure prediction, ultimately obtaining a well-trained point cloud semantic segmentation model.

[0078] It should be noted that the methods described in the embodiments of the present invention are not limited to the execution order of the steps listed in the embodiments. Any solutions implemented by adding, deleting, or replacing steps in the prior art based on the principles of the present invention are included within the protection scope of the present invention. The implementation apparatus of the positioning method described in the present invention includes, but is not limited to, the structure of the positioning system listed in the above embodiments of the present invention. Any structural modifications and substitutions made in the prior art based on the principles of the present invention are included within the protection scope of the present invention.

[0079] This invention significantly improves the target recognition accuracy of airport point clouds in complex occlusion scenarios by combining deep semantic segmentation networks with density clustering algorithms. Furthermore, by combining K4PCS coarse registration and ICP iterative nearest-point method ICP fine registration techniques, it achieves robust alignment of sparse, nested, and locally missing point clouds, significantly improving the positioning accuracy and stability of unmanned vehicles in real-world environments.

[0080] This invention also provides a communication device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program stored in the memory to enable the communication device to perform the above-described point cloud semantic segmentation, instance extraction, point cloud registration, and high-precision positioning methods.

[0081] In one embodiment, the development platform provided by the present invention (corresponding to the above-mentioned communication device) is a server system.

[0082] In one implementation, the server system uses Ubuntu 18.04 as its operating system, NVIDIA RTX 3090 as its GPU, and Intel Core i9-14900KF as its CPU. The experimental environment includes Python 3.8, TensorFlow 2.4.0, CUDA 11.0, and CUDNN 8.0. Data processing and visualization are deployed using the integrated PCL, Open3D, and ROS frameworks.

[0083] This invention also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a communication device, implements the above-described high-precision unmanned vehicle positioning method based on point cloud semantic-geometric joint modeling.

[0084] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor, and the program can be stored in a computer-readable storage medium. The storage medium is a non-transitory medium, such as random access memory (RAM), read-only memory (ROM), flash memory, hard disk, solid-state drive (SSD), magnetic tape, floppy disk, optical disc, and any combination thereof. The storage medium can be any available medium accessible to a computer, or a data storage device such as a server, edge computing platform, or data center that integrates one or more available media. This available medium can include, but is not limited to, magnetic media, optical media, or semiconductor storage media.

[0085] It should be understood that the communication device, computer-readable storage medium, and computer program product according to the embodiments of the present invention are all used to implement the methods according to the embodiments of the present invention. The detailed description of the above-described method embodiments has been described above, and for the sake of brevity, it will not be repeated here.

[0086] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules or units, and may be electrical, mechanical, or other forms.

[0087] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules or units can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module or unit may exist physically separately, or two or more modules or units may be integrated into one module or unit.

[0088] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0089] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0090] The above description of various embodiments of the present invention is provided for illustrative purposes to a person skilled in the art. It is not intended to limit the invention to a single disclosed embodiment. As taught above, those skilled in the art will understand various alternatives and variations of the invention. Therefore, although some alternative embodiments have been specifically described, those skilled in the art will understand or relatively easily develop other embodiments. The present invention is intended to include all alternatives, modifications, and variations of the invention described herein, as well as other embodiments falling within the spirit and scope of the invention described above.

Claims

1. A positioning system for an unmanned inspection vehicle, comprising: A semantic segmentation module is configured to perform multi-category semantic segmentation on the input lidar point cloud collected by the inspection unmanned vehicle to obtain point-level semantic category information about the target point cloud of the equipment to be inspected. An instance extraction module is configured to extract target point cloud instances of the equipment to be inspected and remove noise based on the point-level semantic category information provided by the semantic segmentation module using a density clustering algorithm. A coarse registration module is configured to establish an initial point cloud correspondence using the K4PCS method through geometric constraint consistency screening, and to calculate the initial transformation matrix between the target point cloud instance extracted by the instance extraction module and the pre-stored standard point cloud model. as well as The precise registration module is configured to perform nearest neighbor search, error minimization, and iterative optimization based on the initial transformation matrix obtained by the coarse registration module using the ICP method, so as to obtain the final pose transformation matrix of the target point cloud instance of the equipment to be inspected.

2. The positioning system according to claim 1, characterized in that, The semantic segmentation module includes: A global attention enhancement module is configured to construct long-range dependencies on the lidar point cloud, enhance the ability to distinguish occluded regions, structural boundaries, and geometric shapes through global context modeling, and output enhanced point cloud features; and A multi-layer feature extraction module is configured to receive the enhanced point cloud features and extract spatial structure information about the equipment to be inspected from coded features at different scales to generate multi-layer semantic features.

3. The positioning system according to claim 2, characterized in that, The semantic segmentation module further includes: A decoding and restoration module, configured to upsample the multi-layer semantic features to progressively increase the spatial resolution of the multi-layer semantic features, and output high-dimensional semantic features; and An output prediction module is configured to map the high-dimensional semantic features to the point-level semantic category information.

4. The positioning system according to claim 2, characterized in that, The global attention enhancement module includes: A feature encoding unit is configured to linearly map the features of the lidar point cloud to form a feature representation for processing by an attention network. The feature stitching and difference calculation unit is configured to combine the features of each point in the lidar point cloud based on the feature representation formed by the feature encoding unit, and calculate the feature difference between local and global features to enhance the sensitivity to occluded areas. A self-attention mechanism unit, configured to employ the Transformer's Query-Key-Value structure, calculates attention weights by evaluating the similarity between point cloud features, thereby modeling global contextual information to generate attention-weighted features; and The feature enhancement output unit is configured to fuse, normalize, and adaptively adjust the attention-weighted features, and output the enhanced point cloud features.

5. The positioning system according to claim 3, characterized in that, The multi-layer feature extraction module includes: A multilayer encoder unit, configured to extract local geometric features and overall structural information of the point cloud features from shallow to deep layers; A cross-layer feature fusion unit is configured to concatenate and enhance features extracted from different layers by the multi-layer encoder unit, and perform difference alignment using global attention weights. An attention optimization unit, configured to perform attention filtering on the features fused by the cross-layer feature fusion unit; and An output unit is configured to output the semantic features filtered by the attention optimization unit to the decoding and restoration module.

6. The positioning system according to claim 1, characterized in that, The instance extraction module includes: An unvisited point selection module is configured to select unvisited points as candidate points from the target point cloud of the equipment to be inspected and generate a neighborhood point set. The core point determination module is communicatively connected to the unvisited point selection module and is configured to determine whether the neighborhood point set of the candidate point satisfies the core point condition, and mark the candidate points that do not satisfy the core point condition as noise points. A cluster expansion module, communicatively connected to the core point determination module, is configured to add a candidate point to a cluster set when the candidate point is determined to be a core point, and sequentially expand unclassified points within the neighborhood point set. The expansion includes obtaining the neighborhood point set of the current expanded point through a neighborhood search module, determining whether the neighborhood point set of the candidate point satisfies the core point condition through the core point determination module, and merging expanded points that satisfy the core point condition into the current cluster set; and A cluster termination control module is configured to control the iteration process until all points in the target point cloud related to the equipment to be inspected have been accessed and the final clustering result has been output, so as to realize density clustering of the target point cloud and extract the target point cloud instance.

7. The positioning system according to claim 6, characterized in that, The coarse registration module includes: A four-point sampling module is configured to randomly sample four pairs of points from the target point cloud instance and the standard point cloud model to generate a candidate set. A geometric constraint detection module is communicatively connected to the four-point sampling module and is configured to determine whether the sampled four-point pairs satisfy geometric consistency constraints. An initial transformation solution module is communicatively connected to the geometric constraint detection module and is configured to calculate and output an initial transformation matrix from the target point cloud instance to the standard point cloud model when the four point pairs satisfy the geometric consistency constraint. A transformation application and interior point statistics module is communicatively connected to the initial transformation solution module and configured to apply the initial transformation matrix to the target point cloud instance, and count the number of point pairs whose distance between the transformed point cloud and the standard point cloud model is less than a preset threshold as the number of interior points; and The registration judgment and output module is communicatively connected to the transformation application and interior point statistics module, and is configured to determine whether the number of interior points has reached a set threshold. If the number of interior points reaches the set threshold, the initial transformation matrix is ​​output as the initial registration result; if the number of interior points does not reach the set threshold, the four-point sampling module is triggered to resample.

8. The positioning system according to claim 7, characterized in that, The precise registration module includes: A feature matching initialization module is configured to receive the initial registration result and initialize the point pairs to be matched between the target point cloud instance and the standard point cloud model. The nearest neighbor search module is communicatively connected to the feature matching initialization module and is configured to find the nearest neighbor in the standard point cloud model for each point in the target point cloud instance in each iteration to form a set of point pairs. A coordinate transformation calculation module is communicatively connected to the nearest point search module and is configured to calculate the optimal rotation matrix and translation vector for the current iteration based on the set of point pairs, and update the position of the target point cloud instance. An error evaluation module, communicatively connected to the coordinate transformation calculation module, and configured to calculate the registration error of the current iteration; and An iteration termination control module is communicatively connected to the error evaluation module and the nearest point search module, and is configured to determine whether the registration error has converged or whether the number of iterations has reached the upper limit. If the registration error has converged or the number of iterations has reached the upper limit, the final pose transformation matrix is ​​output; if the registration error has not converged or the number of iterations has not reached the upper limit, the next iteration process is triggered.

9. A positioning method for unmanned inspection vehicles, characterized in that, include: S1. The lidar point cloud collected by the inspection unmanned vehicle is subjected to multi-category semantic segmentation through the semantic segmentation module to obtain point-level semantic category information of the target point cloud of the equipment to be inspected. S2. The instance extraction module uses a density clustering algorithm to extract target point cloud instances related to the equipment to be inspected from the target point cloud filtered according to the point-level semantic category information and removes noise points. S3. Using the coarse registration module, the K4PCS method is adopted and geometric constraint consistency screening is used to establish the initial point cloud correspondence, and the initial transformation matrix between the target point cloud instance and the pre-stored standard point cloud model is calculated. as well as S4. Using the precise registration module and the ICP method, the final pose transformation matrix of the target point cloud instance is obtained by performing nearest neighbor search, error minimization solution and iterative optimization on the initial transformation matrix obtained by the coarse registration module.

10. The method according to claim 9, characterized in that, The semantic segmentation step in step S1 includes: performing point-level feature extraction, neighborhood construction, and semantic classification prediction on the lidar point cloud to obtain the semantic category information of each point in the lidar point cloud.

11. The method according to claim 9, characterized in that, Step S2 further includes: Select any unvisited point from the target point cloud of the equipment to be inspected as a candidate point and generate a neighborhood point set; Determine whether the neighborhood point set of the candidate point satisfies the core point condition, and mark the candidate points that do not satisfy the core point condition as noise points; Candidate points that meet the core point conditions are added to the cluster set, and unclassified points in the neighborhood point set are expanded in turn. The expansion includes obtaining the neighborhood point set of the current expanded point through the neighborhood search module, determining whether the neighborhood point set of the candidate point meets the core point conditions through the core point determination module, and merging the expanded points that meet the core point conditions into the current cluster set. Repeat the above steps until all points in the target point cloud related to the equipment to be inspected have been visited, and output the final clustering result.

12. The method according to claim 9, characterized in that, Step S3 further includes a coarse registration step, which includes: Four point pairs are randomly sampled from the target point cloud instance and the standard point cloud model to generate a candidate set; Determine whether the four-point pair satisfies the geometric consistency constraint; When the four point pairs satisfy the geometric consistency constraint, calculate and output the preliminary transformation matrix from the target point cloud instance to the standard point cloud model; The preliminary transformation matrix is ​​applied to the target point cloud instance, and the number of point pairs whose distance between the transformed point cloud and the standard point cloud model is less than a preset threshold is counted as the number of inliers. If the number of inliers reaches a set threshold, the preliminary transformation matrix is ​​output as the initial registration result; if the number of inliers does not reach the set threshold, resampling is triggered.

13. The method according to claim 12, characterized in that, Step S4 further includes a precise registration step, which includes: Receive the initial registration result and initialize the point pairs to be matched between the target point cloud instance and the standard point cloud model; In each iteration, for each point in the target point cloud instance, find the nearest neighbor in the standard point cloud model to form a set of point pairs; Calculate the optimal rotation matrix and translation vector for this iteration based on the set of point pairs, and update the position of the target point cloud instance; Calculate the registration error for the current iteration; Determine whether the registration error has converged or whether the number of iterations has reached the upper limit. If the registration error has converged or the number of iterations has reached the upper limit, output the final pose transformation matrix. If the registration error has not converged or the number of iterations has not reached the upper limit, trigger the next iteration process.

14. A method for training a semantic segmentation model contained in a localization system as described in any one of claims 1-8, characterized in that, include: S51. Preprocess the point cloud sample data to obtain the processed data; S52. Input the processed data into the encoder structure to obtain feature information; S53. Input the feature information into the decoder structure to obtain the semantic prediction score; as well as S54. Construct a loss function based on the feature map and the semantic prediction score, and train a semantic segmentation model based on the loss function.

15. The method according to claim 14, characterized in that, The preprocessing includes one or more of the following data augmentation operations: point cloud denoising, coordinate normalization, point intensity normalization, random rotation, random scaling, and random sampling.

16. The method according to claim 14, characterized in that, The encoder structure is configured to construct local and global geometric features from the point cloud to extract semantic representations.

17. The method according to claim 16, characterized in that, The encoder structure includes: multi-layer MLP, convolutional structure, attention structure, or local graph aggregation structure.

18. The method according to claim 14, characterized in that, The decoder structure is configured to perform upsampling and fusion operations on features from the encoder structure to output the semantic classification probability for each point.

19. The method according to claim 14, characterized in that, The loss function includes one or more of cross-entropy loss, Dice loss, and Focal Loss.

20. A communication device, comprising a processor, wherein, The processor executes computer programs or instructions through logic circuits to implement the method as described in any one of claims 9 to 19.

21. A computer-readable storage medium storing a computer program or instructions, wherein, When the computer program or instructions are executed by the communication device, the method as described in any one of claims 9 to 19 is implemented.

22. A computer program product storing a computer program, the computer program including program instructions, wherein, When the program instructions are executed by a computer, the computer performs the method as described in any one of claims 9 to 19.

Images