Point cloud feature extraction method, device and system based on inspection humanoid robot

Abstract

The present invention discloses a point cloud feature extraction method, device and system based on a patrol humanoid robot, including: acquiring point cloud data of underground space; denoising, filtering and normalization processing; extracting local features and global space enhancement features of point cloud data, local features include geometric structure, curvature, boundary features, and performing curvature feature transformation, and global space features include extracting global space features and convolving to obtain global features for enhanced spatial attention; extracting feature information based on local features and global space enhancement features and classifying. The present invention realizes efficient extraction and enhancement of global features in sparse point cloud semantic segmentation tasks, significantly improving the classification accuracy and robustness of the model in complex scenarios; the present invention has a significant improvement in semantic segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection technology for underground spaces, and more specifically, to a method, device and system for extracting point cloud features based on an inspection humanoid robot. Background Art

[0002] With the gradual development of underground spaces such as subway tunnels and mining spaces, problems such as uneven ground, irregular spatial shapes, and dim light have been exposed in the underground space environment. There may be defects such as water leakage, cracks, and lining misalignment in the pipe segments. Therefore, safety monitoring and hidden danger inspection of the underground space environment are particularly important. With the continuous advancement of technology, SLAM technology allows robots to simultaneously self-position in unknown environments. Through integrated sensors, it collects environmental data in real time, captures features such as uneven ground and irregular spatial shapes, and generates high-precision three-dimensional point cloud data to build a three-dimensional map of the surrounding environment. Combined with AI algorithms and big data analysis, humanoid robots can perform intelligent inspections in digital twin models, automatically identify and mark abnormal areas, detect the size, shape and distribution of cracks through image recognition technology, and use machine learning algorithms to predict the development trend of cracks, evaluate their impact on structural safety, and accurately locate defects such as lining misalignment and leakage points, providing precise guidance for subsequent maintenance and reinforcement. Digital twins are digital mappings of physical entities in virtual space. Through real-time data updates, it synchronizes the physical and digital worlds, accurately reflects the structural characteristics of underground space, and can also predict potential safety hazards such as water leakage in pipe segments and crack expansion trends through data analysis and simulation.

[0003] Research status at home and abroad: The handheld mobile laser radar scanning device SLAM100 can be equipped with multiple platforms such as drones and backpacks. It has a 360° rotating platform that can cover a wider range of scenes, and can obtain high-precision and high-precision point cloud data. The relative accuracy can reach 2cm, the absolute accuracy is 5cm, and it can be used in environments without light and GPS signals. Guanghao Qi et al. (2023) focused on improving the accuracy of tunnel model inspection. By using 3D point cloud data and partial detection technology based on deep learning, they were able to effectively identify complex shapes and damaged areas in tunnels, improving the accuracy and efficiency of detection. In 2014, Ji Zhang and Sanjiv Singh proposed a classic framework of laser radar SLAM LOAM, which mainly consists of high-frequency laser odometer and low-frequency mapping. It solves the distortion problem caused by laser radar during mobile scanning, thereby minimizing the offset. Zlot et al. used laser SLAM and inertial measurement devices to build three-dimensional maps of coal mine devices and generate motion trajectories and point clouds. Xu Wei et al. integrated IMU sensors and proposed Fast-Lio and Fast-Lio2. Fast-Lio is a SLAM algorithm designed mainly for mobile robots. It uses iterative error state Kalman filtering (iESKF) to achieve tight coupling between lidar and IMU, and proposes a back propagation to compensate for the distortion of fast motion, solving the degradation problem caused by fast motion or cluttered environment. The PointNet framework proposed by Charles R. Qi et al. in 2017 realizes the disordered processing of point cloud data. It can directly capture the global features in the point cloud without preprocessing or converting it into other data, realizing a method that can process point clouds globally. Sun Duanzheng et al. proposed a road rod semantic segmentation method based on the improved PointNet++ deep learning network, which significantly improved the segmentation accuracy of the rod class. Rosen et al. used digital twin technology to realize the mapping of real physical data and virtual data, improving the anti-interference and autonomy of the production system. Ji Youwang et al. proposed a visual monitoring system based on digital twins for mobile robots to realize digital, transparent and visual monitoring of mobile robots and improve the interactivity of the monitoring system. Based on digital twin technology, Feng Xiangxi and others have comprehensively used common supporting technologies such as intelligent reconstruction of subway tunnel environment, data interconnection, perception interaction, business collaboration, and combined with various communication networks to build a digital twin system for intelligent management of subway tunnels. In response to the needs of smart pipe networks, Chen Sixun and others have built a digital twin model of the entire life cycle of pipelines, including physical pipelines, virtual pipelines, pipeline service systems, pipeline twin data, and connections. Through data sharing, intelligent analysis and other means, they have optimized resource allocation, enhanced the level of safety warning, and promoted the intelligent transformation of the pipeline industry.Tan et al. combined digital twin technology with formal chronology and used laser scanning, oblique photogrammetry and other tools to conduct archaeological dating research on Xiegong buildings. Funari et al. proposed a parametric scan-to-finite element (Scan-to-FEM) framework to construct digital twins of historical masonry structures, and used the Church of Santo Tokato in Portugal as a pilot case to verify that the model can be used to reproduce past scenes and future destruction scenes of the research object.

[0004] In order to improve detection efficiency and accuracy and reduce the risks of manual inspections, the present invention is committed to developing a point cloud data extraction method and system that can collect point cloud data of underground space and perform feature extraction, and can quickly provide high-precision point cloud features, point cloud classification and segmentation for digital twin technology to enhance the intelligent inspection and fault prevention performance of underground space. Summary of the invention

[0005] An object of the present invention is to solve at least the above problems and to provide at least the advantages which will be described hereinafter.

[0006] In order to achieve these purposes and other advantages according to the present invention, a laser point cloud feature extraction method based on an underground space inspection humanoid robot for digital twin is provided, comprising the following steps:

[0007] S1. Obtaining point cloud data of underground space collected by a patrol humanoid robot equipped with a three-dimensional laser point cloud device traveling along a preset route;

[0008] S2, using statistical analysis methods to perform denoising on the point cloud data, then using a Gaussian filter to filter the point cloud data, and then performing batch normalization to obtain a preprocessed point cloud;

[0009] S3, input point cloud data to a pre-built and trained feature extraction neural network model, and output the spatial features and classification labels of the point cloud;

[0010] The construction and training of the feature extraction neural network model specifically includes:

[0011] T1. Construct a local feature description module, which is used to extract local features. The local feature description module samples the input point cloud data based on the farthest point sampling method, sets different sampling radii to obtain multiple clusters of multiple scales, and each cluster includes the center point and the points within its neighborhood;

[0012] For each cluster, the RANSAC algorithm is used to iterate the fitting equations of different structural geometric shapes to fit the corresponding geometric features. Then, the boundary features are judged based on the curvature, and the curvature of the center point of the cluster is calculated. The points in the neighborhood of the center point perform a sliding window to obtain the average value of the curvature. The center point and the points in its neighborhood are classified based on the geometric features.

[0013] KNN is used to search for similar points in the neighborhood of the curvature feature of each cluster to encode the features of each point, obtain the feature information of each point, assign weights to feature points with similar curvatures and adjust them to close spatial positions, and assign weights to points with dissimilar curvatures and adjust them to far spatial positions, so as to transform the curvature features and enhance the curvature features.

[0014] Complete the local feature extraction of all clusters at the same scale, concatenate the local features of multiple scales, and then use the softmax excitation function to obtain the local features of each point cloud;

[0015] T2. Construct a global spatial feature enhancement module, which is used to extract and enhance the global spatial features of the point cloud. The global spatial feature enhancement module extracts global features based on a multi-layer perceptron, takes the global features as input, performs average pooling and maximum pooling respectively, obtains average pooling global features and maximum pooling global features, convolves the average pooling global features and the maximum pooling global features to obtain spatial attention global features, and then uses a Softmax activation function to process to obtain global spatial features;

[0016] T3, superimposing the global features obtained in step T2 onto the local features of each point cloud obtained in step T1, and then performing the multi-layer perceptron operation again to obtain the spatial feature information and encoding information of each point cloud, wherein the encoding information is mapped to the classification label;

[0017] T4. Use scene classification training data to train the feature extraction neural network model constructed in steps T1 to T3, wherein the scene classification training data includes designed or standard pipeline model data, wall model data, ground model data, lamp model data, reinforcement model data, ceiling model data, and electrical equipment model data.

[0018] Preferably, the specific method for obtaining the scene classification training data and the corresponding model component data is: using the method of steps S1 and S2 to obtain a point cloud of a section of the tunnel, then using a random sampling extraction method to screen out the point cloud through point cloud thinning, and then using a point cloud semantic segmentation and annotation tool to generate the corresponding scene classification training data and the corresponding model component data;

[0019] A laser point cloud feature extraction device based on an underground space inspection humanoid robot is provided, comprising:

[0020] A point cloud data acquisition unit, which is used to acquire point cloud data of underground space collected by a patrol humanoid robot equipped with a three-dimensional laser point cloud device traveling along a preset route;

[0021] A point cloud data preprocessing unit, which is used to perform denoising on the point cloud data using a statistical analysis method, and then perform filtering on the point cloud data using a Gaussian filter, and then perform batch normalization processing to obtain a preprocessed point cloud;

[0022] A point cloud data extraction unit, which is used to input point cloud data into a pre-built and trained feature extraction neural network model, and output spatial features and classification labels of the point cloud;

[0023] A model training unit is used to construct and train the feature extraction neural network model.

[0024] A laser point cloud feature extraction system based on an underground space inspection humanoid robot is provided, comprising:

[0025] An underground space inspection humanoid robot, on which a control system, a laser SLAM device, and a communication control module are installed. The laser SLAM device is used to collect point cloud data of the underground space. The control system controls the underground space inspection humanoid robot to travel along a preset route.

[0026] A remote control system connected to the control system of the underground space inspection humanoid robot and the communication control module;

[0027] The feature extraction device;

[0028] A display is connected to the remote control system and the feature extraction device, and is used to display the input, operation and output of the feature extraction device; and is used to display the control system of the underground space inspection humanoid robot and the travel data of the remote control system.

[0029] An electronic device is provided, comprising:

[0030] one or more processors;

[0031] Memory; and

[0032] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the method.

[0033] A computer-readable storage medium is provided, storing a computer program used in combination with an electronic device, wherein the computer program can be executed by a processor to implement the method described.

[0034] The present invention has at least the following beneficial effects:

[0035] This invention introduces a local neighborhood multi-scale clustering feature representation module that takes into account three-dimensional curvature and an adaptive global feature enhancement module to achieve efficient extraction and enhancement of global features in the sparse point cloud semantic segmentation task, significantly improving the classification accuracy and robustness of the model in complex scenes, and providing new ideas and methods for the semantic segmentation of point cloud data. This innovation not only solves the problem of insufficient contextual information in sparse point clouds, but also improves the generalization ability and robustness of the model to noise by adaptively adjusting feature weights and enhancing key features.

[0036] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a feature extraction method according to one of the technical solutions of the present invention;

[0038] Figure 2 A cross-sectional view of a normal point cloud and a sparse point cloud according to one of the technical solutions of the present invention;

[0039] Figure 3 A schematic diagram of geometric structure fitting and curvature feature transformation of one of the technical solutions of the present invention;

[0040] Figure 4 Schematic diagram of the process of extracting the global spatial feature enhancement module of the present invention;

[0041] Figure 5 An original point cloud of one of the embodiments of the present invention;

[0042] Figure 6 for Figure 5 Classification labels of the original point cloud shown;

[0043] Figure 7 This is a test result of one implementation of the present invention;

[0044] Figure 8 It is a feature extraction result of a normal point cloud and a sparse point cloud in one implementation of the present invention;

[0045] Fig. 9 This is the working process of an underground space inspection humanoid robot according to one of the technical solutions of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0047] It should be noted that the experimental methods described in the following embodiments are conventional methods unless otherwise specified, and the reagents and materials are commercially available unless otherwise specified; in the description of the present invention, the orientation or positional relationship indicated by the terms is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0048] like Figures 1 to 9 As shown, the present invention provides a laser point cloud feature extraction method based on an underground space inspection humanoid robot for digital twin, comprising the following steps:

[0049] S1. Obtaining point cloud data of underground space collected by a patrol humanoid robot equipped with a three-dimensional laser point cloud device traveling along a preset route;

[0050] S2, using statistical analysis methods to perform denoising on the point cloud data, then using a Gaussian filter to filter the point cloud data, and then performing batch normalization to obtain a preprocessed point cloud;

[0051] S3, input point cloud data to a pre-built and trained feature extraction neural network model, and output the spatial features and classification labels of the point cloud;

[0052] The construction and training of the feature extraction neural network model specifically includes:

[0053] T1. Construct a local feature description module, which is used to extract local features. The local feature description module samples the input point cloud data based on the farthest point sampling method, sets different sampling radii to obtain multiple clusters of multiple scales, and each cluster includes the center point and the points within its neighborhood;

[0054] For each cluster, the RANSAC algorithm is used to iterate the fitting equations of different structural geometric shapes to fit the corresponding geometric features. Then, the boundary features are judged based on the curvature, and the curvature of the center point of the cluster is calculated. The points in the neighborhood of the center point perform a sliding window to obtain the average value of the curvature. The center point and the points in its neighborhood are classified based on the geometric features.

[0055] KNN is used to search for similar points in the neighborhood of the curvature feature of each cluster to encode the features of each point, obtain the feature information of each point, assign weights to feature points with similar curvatures and adjust them to close spatial positions, and assign weights to points with dissimilar curvatures and adjust them to far spatial positions, so as to transform the curvature features and enhance the curvature features; the curvature threshold can be used to determine whether the curvatures are similar;

[0056] Complete the local feature extraction of all clusters at the same scale, concatenate the local features of multiple scales, and then use the softmax excitation function to obtain the local features of each point cloud;

[0057] T2. Construct a global spatial feature enhancement module, which is used to extract and enhance the global spatial features of the point cloud. The global spatial feature enhancement module extracts global features based on a multi-layer perceptron, takes the global features as input, performs average pooling and maximum pooling respectively, obtains average pooling global features and maximum pooling global features, convolves the average pooling global features and the maximum pooling global features to obtain spatial attention global features, and then uses a Softmax activation function to process to obtain global spatial features;

[0058] T3, superimposing the global features obtained in step T2 onto the local features of each point cloud obtained in step T1, and then performing the multi-layer perceptron operation again to obtain the spatial feature information and encoding information of each point cloud, wherein the encoding information is mapped to the classification label;

[0059] T4. Use scene classification training data to train the feature extraction neural network model constructed in steps T1 to T3, wherein the scene classification training data includes designed or standard pipeline model data, wall model data, ground model data, lamp model data, reinforcement model data, ceiling model data, and electrical equipment model data.

[0060] In the above technical solution, the local feature description module of step T1 is Figure 1 The LPFA module shown is a local neighborhood multi-scale clustering feature representation module that takes into account three-dimensional curvature. The local feature representation module can effectively capture the geometric topological information around each point in the point cloud data. The influencing factors are mainly the calculation of the relative position relationship between the point and its adjacent points. Multi-scale feature extraction can analyze the point cloud at multiple scales. It extracts local neighborhood features within multiple radius ranges, so as to better capture detailed structural information and extract within the neighborhood range of local feature points. The differential local enhancement module characterizes the different local structural characteristics in the three-dimensional point cloud object by perceiving rich local semantics.

[0061] like Figure 2 As shown, (a), (b), (c), (d), (e), and (f) are cross-sectional views of normal point clouds and sparse point clouds of different shapes, and (g) and (h) are schematic diagrams of the fitted local geometric shapes, which are used for subsequent curvature iterative calculations.

[0062] like Figure 3As shown in the figure, the boundary features of the connection in the scene are very different from the characteristics of another label (such as flange, pipe, cushion, support, ground), so the features are distinguished, that is, the point cloud of the connection is counted according to the label. Specifically, for the point features of the boundary, the RANSAC algorithm is used to select the points to be iterated, and the corresponding geometric features are fitted by iteratively executing the fitting equations of different structural geometric shapes. Then, the boundary features are judged based on the curvature, and the curvature of the center point is calculated. The sliding window is executed on the regional points to obtain its average value, and the local point curvature features can be calculated step by step, the curvature data can be analyzed, and the curvature mutation points can be identified. These correspond to different structures of tunnels such as pipe joints and cushions. That is, an initial classification process is performed on the point clouds with different features. A classification is performed through the local and global modules, and a feature is added and normalized.

[0063] Figure 3 The dotted circle area represents different categories with different curvatures. The center point in the local enlarged image represents the center point of the sampling. The balls of different colors represent different categories. When representing local features, the balls with curvatures close to the sampling center point gather toward the center point. Specifically, KNN is used to search for similar points in the neighborhood of the curvature feature, and feature encoding is performed on each point to obtain the feature information of each point. Feature points with similar curvatures are pulled together, and points with dissimilar curvatures are pulled apart. A curvature feature transformation is performed to bring the semantic feature points of the labels of the same category closer and separate the semantic feature points of the labels of different categories. Whether the curvatures are similar can be determined based on the curvature threshold.

[0064] The global spatial feature enhancement module of step T2, namely Figure 1 In the CGC module shown, global representation is achieved through the maximum pooling layer (Maxpooling MLP), which can extract the most important features from the entire point set and fuse them into a global feature vector. However, in the sparse point cloud semantic segmentation task, due to the low density of the point cloud, it is difficult for the model to extract a complete global feature vector, which makes it difficult to distinguish similar categories. For this purpose, this application constructs and trains a global spatial feature enhancement module.

[0065] like Figure 4As shown in the figure, the input feature fully connected layer obtains its spatial features, and the global features are obtained through maximum pooling. The global features are then input into the spatial attention module (SAM), and the corresponding spatial attention map (M1) is obtained through average pooling and maximum pooling operations and convolution. Through the Softmax activation function, as shown in formula (1), the enhanced global spatial feature information is output, and the expression ability of the global features is further improved through the feature enhancement module, so that the model can still capture key semantic information in a sparse point cloud environment. When extracting global features, the feature expression of easily confused categories is enhanced, the error of the model in distinguishing similar categories is reduced, and the accuracy of the overall semantic segmentation is improved. Among them, Figure 4 In the figure, F indicates the point cloud vector, F i and Schematic diagram of feature extraction. F globel Avgpooling and Maxpooling represent average pooling and maximum pooling respectively, Attention represents spatial attention, and dot product represents convolution.

[0066] (1)

[0067] In formula (1), P ( y ∣ x ) represents the given point cloud feature x Next Category y The conditional probability of It means that after nonlinear and amplified processing, the input is evaluated. x and categories y i compatibility or correlation between them; k represents the index to be summed, n represents the total number of categories, x represents the input features, y represents a set of category labels, y i Indicates a specific category label.

[0068] In another technical solution, the specific method for obtaining the scene classification training data and the corresponding model component data is: using the method of steps S1 and S2 to obtain a point cloud of a section of the tunnel, then using a random sampling extraction method to screen out the point cloud through point cloud thinning, and then using a point cloud semantic segmentation and annotation tool to generate the corresponding scene classification training data and the corresponding model component data;

[0069] In the above technical solution, the specific operations of the data set are illustrated as follows:

[0070] A large-scale dataset based on three scenes of tunnels is designed. The point cloud data obtained by non-overlapping TLS scanning stations are working faces, tunnels and pump rooms, with a total length of 5km and a total of 948,576,350 points. The scenes include cables, pipelines, pipeline interfaces, tracks, conveyors, hydraulic pumps, supports, lights, unstructured machinery and equipment, etc. To prevent overfitting or underfitting, the Adam optimizer is used to divide multiple training test validation areas, and multi-fold cross-validation is used. The average sample size is 5.3MB. The point cloud data includes 10-dimensional information including xyz coordinates, RGB, reflection intensity and normal vector in the TLS built-in coordinate system, which is called Alleyway Large Scale Dataset (ALSD). The processed point cloud is stored in kdTree format for easy indexing. Figure 5 The original point cloud and Figure 6 The category labels are shown.

[0071] In another technical solution, evaluation indicators are selected to evaluate the performance of the test set and validation set, so as to comprehensively and systematically measure the performance of the model on the self-built data set, and the performance of the model on the test set and validation set is carefully evaluated through quantitative evaluation indicators such as accuracy, F1 score, mIOU (mean intersection over union), etc. Error analysis can quantify the deviation between the model's predicted results and the actual results, thereby identifying the error distribution of the model in different scenarios. Indicators such as accuracy and F1 score reflect the comprehensive performance of the model in the classification task, ensuring that the model can not only correctly identify positive samples, but also effectively avoid misjudging negative samples. By comparing the performance on the test set and the validation set, the generalization ability of the model can be further verified. Among them, the F1 score is calculated using formula (2);

[0072] (2)

[0073] In formula (2), TP represents a true positive example, that is, a positive example is predicted to be true; FP represents a false positive example, that is, a negative example is predicted to be true; FN represents a false negative example, that is, a positive example is predicted to be false;

[0074] In order to specifically illustrate the superiority of the feature extraction method of the present invention, a test is conducted based on the self-built data set ALSD data set. After the test, the method of the present invention (Ours) is compared with RandLANet and the ground truth. Figure 7 As shown, in the test, it can be clearly seen that part of the ground recognition is recognized on the label of the support in the direction of the pipeline, while part of the pipeline is recognized at the position of the support, the pad is partially recognized on the support, the pipeline is partially recognized on the pad, and the flange is recognized on the pipeline. The method of the present application has been improved in all aspects, especially along the pipeline direction. The recognition indicators of various semantic labels are shown in Table 1. Method Representation method, Eval accuracy Indicates precision.

[0075] Table 1. Accuracy and mIOU of each method.

[0076]

[0077] Table 2 shows the evaluation data of the validation set and test set of the method of the present invention. It can be seen that the floor is the feature with the best segmentation effect in the network model, with a segmentation accuracy between 93% and 98%. The pipe has the same density as the connecting flange plate and the pad as vertical features, which leads to problems in the segmentation of multiple features at the intersection, and the segmentation results are easily confused to varying degrees.

[0078] Table 2. Validation set and test set segmentation effect data

[0079]

[0080] Note: Flange, Pipe, Cushion, Support, Ground; Validation Validation set, Testing Test set, Class Label.

[0081] The superiority of the present invention in feature extraction of non-uniform density point cloud is described as follows:

[0082] Point clouds with normal density can provide more detailed and precise geometric information, allowing the model to better capture the details of objects. Richer geometric and semantic features can be extracted. Even if there are some noise or missing points, the impact is relatively small, and there are still enough points for accurate semantic segmentation. The scarcity of sparse point clouds leads to the loss of geometric information. It is difficult for the model to capture enough information such as the details and boundaries of objects, resulting in a decrease in feature expression ability, which affects the accuracy of semantic segmentation. Therefore, the information of each point becomes more important, and any noise or missing points may significantly affect the segmentation results.

[0083] The input data in the training process includes the data of the global scene, and lacks the problem of uneven density of point clouds in tunnel scenes, which will cause the model to be unable to adapt well to the data characteristics of sparse point clouds. The method of this application and the RandLANet method are used to analyze the point features in the vertical direction of the tunnel, and the sparse point clouds at the boundaries between scanning stations and the normal point clouds scanned near the scanning stations are analyzed, such as Figure 8 As shown, in the vertical direction of the scene, the left box is a sparse point area, and the right box is a normal density point area. The visual comparison between the method of the present application and RandLANet and the true value is shown. Figure 8It can be seen that whether it is the pipes and support plates, pipes and walls, and support plates and walls in sparse or normal point cloud scenes, there are different degrees of confusion. The same confusion problem also exists at the intersection of the guardrail and the road surface. Although the geometric shape and spatial position features of the support plate are not effectively utilized, compared with RandLANet, as shown in Table 3, in terms of sparse feature expression, this method obtains better segmentation results after using global and local spatial information.

[0084] Table 3 Evaluation index results of various methods on sparse and normal point clouds, (N) represents normal point cloud

[0085]

[0086] The present invention provides a feature extraction device for a laser point cloud feature extraction method, comprising:

[0087] A point cloud data acquisition unit is used to acquire point cloud data of underground space collected by a patrol humanoid robot equipped with a three-dimensional laser point cloud device along a preset route; wherein the patrol humanoid robot itself has an autonomous navigation function, and therefore, when collecting point clouds, it can also be based on the autonomous navigation route of the patrol humanoid robot.

[0088] A point cloud data preprocessing unit, which is used to perform denoising on the point cloud data using a statistical analysis method, and then perform filtering on the point cloud data using a Gaussian filter, and then perform batch normalization processing to obtain a preprocessed point cloud;

[0089] A point cloud data extraction unit, which is used to input point cloud data into a pre-built and trained feature extraction neural network model, and output spatial features and classification labels of the point cloud;

[0090] A model training unit is used to construct and train the feature extraction neural network model.

[0091] In the above technical solution, the point cloud data acquisition unit acquires the point cloud data of the underground space, the point cloud data preprocessing unit performs denoising on the point cloud data, and then uses a Gaussian filter to filter the point cloud data, and then performs batch normalization to obtain a preprocessed point cloud; the point cloud data extraction unit extracts and classifies the preprocessed point cloud to provide a data basis for the digital twin. The model training unit can build and train the model to improve the feature extraction precision and accuracy.

[0092] All relevant contents of each step involved in the aforementioned embodiment of the laser point cloud feature extraction method based on the underground space inspection humanoid robot can be referred to the functional description of the functional module corresponding to the laser point cloud feature extraction device based on the underground space inspection humanoid robot in the embodiment of the present application, and will not be repeated here.

[0093] The division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present invention may be integrated into a processor, or may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module.

[0094] The present invention provides a laser point cloud feature extraction system based on an underground space inspection humanoid robot, comprising:

[0095] An underground space inspection humanoid robot is equipped with a control system, a laser SLAM device, and a communication control module. The laser SLAM device is used to collect point cloud data of the underground space and execute the control system to travel along a preset route;

[0096] A remote control system connected to the control system and the communication control module, wherein the remote control system controls the control system through the communication control module to control the underground space inspection humanoid robot to travel along a preset route;

[0097] The feature extraction device;

[0098] A display is connected to the remote control system and the feature extraction device, and is used to display the input, operation and output of the feature extraction device; and is used to display the travel data of the control system and the remote control system.

[0099] In the above technical solution, the underground space inspection humanoid robot moves along a preset route under the control of a remote control system and uses point cloud data; the feature extraction device processes the point cloud data and displays it on a display for the operator to view.

[0100] The present invention also provides an electronic device, comprising:

[0101] one or more processors;

[0102] Memory; and

[0103] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a program for executing the above method. The electronic device can be any terminal device including a mobile phone, a laptop computer, a desktop computer, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a car computer, etc.

[0104] The present invention provides a computer-readable storage medium storing a computer program used in combination with an electronic device, wherein the computer program can be executed by a processor to implement the above method.

[0105] Through the description of the above implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present invention, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0106] A brief description of the workflow of the underground space inspection humanoid robot:

[0107] like Fig. 9 As shown, it is a workflow diagram, including multi-sensor fusion, front-end data processing and back-end optimization. The underground space inspection humanoid robot is equipped with a lidar and an IMU. The multi-sensor fusion is to synchronize the time and coordinate system of the lidar point cloud data and the IMU data. The front-end processing is to remove the motion-distorted point cloud and correct the point cloud optimization data; the back-end optimization is to use the posture of the scanning device as a variable and four factors as constraints between the variables. These four factors include IMU pre-integration factor, lidar odometer factor, loop detection factor and absolute coordinate factor. When the motion state changes to a certain extent, the posture of the scanning device at the current moment is added as a node to the factor graph, and the adjacent posture changes are optimized through the factor graph. Among them, Fig. 9 In the figure, INPUTf indicates input, N×d input is N point clouds, S and E indicate the extraction process, f i , f j , It indicates feature extraction, Concat indicates concatenation or full connection, Pooling Indicates pooling, Fi Schematic point cloud vector, F globel It indicates global features, and Output indicates output.

[0108] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A laser point cloud feature extraction method based on an underground space inspection humanoid robot for digital twin, characterized in that: The following steps are involved: S1. Obtaining point cloud data of underground space collected by a patrol humanoid robot equipped with a three-dimensional laser point cloud device traveling along a preset route; S2, using statistical analysis methods to perform denoising on the point cloud data, then using a Gaussian filter to filter the point cloud data, and then performing batch normalization to obtain a preprocessed point cloud; S3, input point cloud data to a pre-built and trained feature extraction neural network model, and output the spatial features and classification labels of the point cloud; The construction and training of the feature extraction neural network model specifically includes: T1. Construct a local feature description module, which is used to extract local features. The local feature description module samples the input point cloud data based on the farthest point sampling method, sets different sampling radii to obtain multiple clusters of multiple scales, and each cluster includes the center point and the points within its neighborhood; For each cluster, the RANSAC algorithm is used to iterate the fitting equations of different structural geometric shapes to fit the corresponding geometric features. Then, the boundary features are judged based on the curvature, and the curvature of the center point of the cluster is calculated. The points in the neighborhood of the center point perform a sliding window to obtain the average value of the curvature. The center point and the points in its neighborhood are classified based on the geometric features. KNN is used to search for similar points in the neighborhood of the curvature feature of each cluster to encode the features of each point, obtain the feature information of each point, assign weights to feature points with similar curvatures and adjust them to close spatial positions, and assign weights to points with dissimilar curvatures and adjust them to far spatial positions, so as to transform the curvature features and enhance the curvature features. Complete the local feature extraction of all clusters at the same scale, concatenate the local features of multiple scales, and then use the softmax excitation function to obtain the local features of each point cloud; T2. Construct a global spatial feature enhancement module, which is used to extract and enhance the global spatial features of the point cloud. The global spatial feature enhancement module extracts global features based on a multi-layer perceptron, takes the global features as input, performs average pooling and maximum pooling respectively, obtains average pooling global features and maximum pooling global features, convolves the average pooling global features and the maximum pooling global features to obtain spatial attention global features, and then uses a Softmax activation function to process to obtain global spatial features; T3, superimposing the global features obtained in step T2 onto the local features of each point cloud obtained in step T1, and then performing the multi-layer perceptron operation again to obtain the spatial feature information and encoding information of each point cloud, wherein the encoding information is mapped to the classification label; T4, using scene classification training data to train the feature extraction neural network model constructed in steps T1 to T3, wherein the scene classification training data includes designed or standard pipeline model data, wall model data, ground model data, lamp model data, reinforcement model data, ceiling model data, and electrical equipment model data; Among them, the specific method for obtaining the scene classification training data and the corresponding model component data is: using the method of steps S1 and S2 to obtain a point cloud of a section of tunnel, and then using a random sampling extraction method to filter out the point cloud through point cloud thinning, and then using a point cloud semantic segmentation and annotation tool to generate the corresponding scene classification training data and the corresponding model component data.

2. A feature extraction device for executing the laser point cloud feature extraction method according to claim 1, characterized in that: include: A point cloud data acquisition unit, which is used to acquire point cloud data of underground space collected by a patrol humanoid robot equipped with a three-dimensional laser point cloud device traveling along a preset route; A point cloud data preprocessing unit, which is used to perform denoising on the point cloud data using a statistical analysis method, and then perform filtering on the point cloud data using a Gaussian filter, and then perform batch normalization processing to obtain a preprocessed point cloud; A point cloud data extraction unit, which is used to input point cloud data into a pre-built and trained feature extraction neural network model, and output spatial features and classification labels of the point cloud; A model training unit is used to construct and train the feature extraction neural network model.

3. A laser point cloud feature extraction system based on an underground space inspection humanoid robot, characterized in that: include: An underground space inspection humanoid robot, on which a control system, a laser SLAM device, and a communication control module are installed. The laser SLAM device is used to collect point cloud data of the underground space. The control system controls the underground space inspection humanoid robot to travel along a preset route. A remote control system connected to the control system of the underground space inspection humanoid robot and the communication control module; The feature extraction device as claimed in claim 2; A display connected to the remote control system and the feature extraction device, the display being used to display the input, operation and output of the feature extraction device; And used to display the travel data of the control system of the underground space inspection humanoid robot and the remote control system.

4. An electronic device, characterized in that include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for executing the method of claim 1.

5. A computer-readable storage medium, characterized in that A computer program stored in an electronic device for use in combination, wherein the computer program can be executed by a processor to implement the method of claim 1.

Images