A 3D target component automatic identification and analysis method and system

Abstract

The application discloses a kind of 3D target component automatic identification and analysis method and system, method includes: the point cloud of target component is executed voxel segmentation to the space where;The point cloud of target component is divided into corresponding voxel, and the voxel without point cloud is eliminated;Extract the context feature of remaining voxel, and generate the anchor frame of initial prediction structure as suggestion with this;Fusion target component point cloud and context feature obtain feature map, according to the corresponding relationship of anchor frame and feature map obtain the prediction of boundary box;Using part aggregation network, the point cloud of suggestion region of interest is learned, according to the learning result, boundary box is adjusted, and the identification result of target component is obtained;According to the identification result, search in the database of predetermining, obtain the relevant information of target component.The application adds artificial intelligence identification analysis method in traditional manufacturing industry, intelligently identifies target component, reduces classification workload, analyzes its structure, to be used for the automatic production of application manufacturing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for automatic identification and analysis of 3D target components. Background Technology

[0002] Manufacturing, as a fundamental industrial sector of the national economy, is a key industry determining the level of my country's economic development. Looking at the development process of the machinery manufacturing industry, it has gone through manual manufacturing, Taylorist manufacturing, high automation, flexible automation, and integrated manufacturing. Automation in manufacturing is becoming a future trend, and intelligent design is also a future direction for manufacturing. Traditional design requires a large amount of human and material resources. Assembly line production greatly improves industrial production efficiency and helps to achieve product standardization, and is currently a widely adopted production method. Traditional assembly line production is a type of "passive" production; the assembly line mechanically transports workpieces, and each processing, inspection, and packaging step is performed by workers distributed along both sides of the assembly line. With the discovery of automation technology, more and more assembly lines have introduced various automated operation and judgment technologies, thereby reducing manual input and ensuring product quality.

[0003] Artificial intelligence (AI) technology has been integrated into many industries, enabling many jobs to move beyond simple human resource utilization. As the foundation of assembly line automation, workpiece recognition is the starting point of the automation process, and also a challenging one. Integrating AI technology into traditional mechanical manufacturing allows for the use of smarter, less resource-intensive AI models to achieve intelligent design and simulation. Existing technologies disclose various methods for achieving 3D visual recognition, such as laser contour analysis and stereo imaging. One of the current research hotspots is providing various 3D visual recognition systems; however, these technologies often focus on providing new recognition concepts or mathematical models without deeply considering how to apply the results of 3D visual recognition to specific industrial production. Summary of the Invention

[0004] The technical problem to be solved by this invention is: In view of the technical problems existing in the prior art, this invention provides a method and system for automatic identification and analysis of 3D target components, which adds artificial intelligence methods to traditional manufacturing automation, intelligently identifies and analyzes the required component structure, so as to better apply automated production.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] A method for automatic identification and analysis of 3D target components, comprising:

[0007] S1. Perform voxel segmentation on the space where the point cloud of the target component is located to obtain a 3D network composed of voxels;

[0008] S2. Divide the point cloud of the target component into the corresponding voxels and remove voxels without point cloud.

[0009] S3. Extract the contextual features of the remaining voxels and use them to generate anchor boxes for the initial predicted structure as suggestions;

[0010] S4. Fuse the point cloud and context features of the target component to obtain a feature map, and predict the bounding box based on the correspondence between the anchor box and the feature map.

[0011] S5. Use the component aggregation network to learn the point cloud of the proposed region of interest, adjust the bounding box according to the learning results, and obtain the recognition result of the target component;

[0012] S6. Based on the identification results of the target component, search the preset database to obtain relevant information about the target component.

[0013] Furthermore, step S1 specifically includes: setting up a cuboid 3D space to accommodate all point clouds of the target component, and dividing the cuboid 3D space into stacked voxels according to a custom voxel size.

[0014] Furthermore, step S2, which divides the point cloud of the target component into corresponding voxels, specifically includes: using 3D sparse convolution to convert the point cloud of the target component into downsampled feature quantities at different levels, with each level of downsampled feature quantity serving as a set of voxel feature vectors.

[0015] Furthermore, step S3 specifically includes: obtaining a 2D bird's-eye view feature map by downsampling the set of feature vectors of all voxels, using the 2D bird's-eye view feature map as the context feature of the voxels, and then using the anchor-box-based method to generate anchor boxes for the initial prediction structure on the 2D bird's-eye view feature map.

[0016] Furthermore, the anchor-frame-based method specifically includes:

[0017] The region proposal network uses a 3×3 convolutional kernel that slides across the feature map.

[0018] For each location, k anchor boxes of different sizes and proportions are used to match objects of different proportions on the feature map;

[0019] For each anchor box, feature vectors are extracted from the corresponding object and input into the classification branch to estimate the score of the corresponding class for each object, while input into the regression branch to optimize the position of the bounding box from the original anchor box estimates.

[0020] Standardize the anchor frame.

[0021] Furthermore, step S4 specifically includes:

[0022] The point cloud of the target component is input into the point cloud processing neural network to obtain the global high-dimensional features of the point cloud data;

[0023] The context features are used as convolutional features. The global high-dimensional features and convolutional features are concatenated and then input into a lightweight point cloud processing neural network for feature extraction to obtain the target features. The convolutional features are then input into two fully connected layers to obtain weight coefficients. Each node in the fully connected layer is connected to all nodes in the previous layer.

[0024] The result of multiplying the weight coefficients and the target features is input into a two-layer multilayer perceptron to obtain a prediction of the bounding box.

[0025] Furthermore, the first convolutional block of the lightweight point cloud processing neural network includes six three-dimensional convolutional layers, and the second, third, and fourth convolutional blocks of the lightweight point cloud processing neural network are all two-dimensional convolutional blocks; concatenating the global high-dimensional features and convolutional features and then inputting them into the lightweight point cloud processing neural network for feature extraction to obtain the target features specifically includes:

[0026] The feature maps of the last layer of the second, third, and fourth convolutional blocks are upsampled by deconvolution, and then the upsampled results are concatenated. Positional information is collected in layers 1 and 2, and semantic information is collected in layers 3 and 4 to obtain a feature map that fuses the original point cloud and extracts contextual features.

[0027] Establish the correspondence between anchor boxes and the feature maps, classify each anchor box, perform regression calculation on the position of the target component, vectorize the calculation result into a feature vector, input the feature vector into the classification branch to score the bounding box, and simultaneously input it into the regression branch to optimize the position of the bounding box.

[0028] Furthermore, step S5 specifically includes: performing point cloud aggregation operation within the suggested region of interest, inputting the aggregation result into the part aggregation network to obtain the box parameters of the target component, vectorizing the box parameters into feature vectors, inputting them into the classification branch to evaluate the confidence of the bounding box, and simultaneously inputting them into the regression branch to optimize the position of the bounding box.

[0029] Furthermore, the input regression branch to optimize the bounding box position specifically includes: using a smoothing loss to regress the normalized box parameters, as shown in the following expression:

[0030]

[0031]

[0032]

[0033] Where x, y, z, l, w, h, θ are the parameters of the target component bounding box in the lidar coordinate system, where x, y, z are the center position of the target component, l, w, h are the size of the target component, and θ is the target direction from a bird's-eye view.

[0034] The present invention also proposes an automatic identification and analysis system for 3D target components, including a computer device, said computer device being programmed or configured to perform any of the described automatic identification and analysis methods for 3D target components.

[0035] Compared with the prior art, the advantages of the present invention are as follows:

[0036] This invention first voxelizes the point cloud data of the components, generates initial suggestions (predicted regions) based on an anchor-based method, and then uses a part clustering strategy to directly solve for the parameters of the 3D bounding box by fitting the predicted part positions of all points in the corresponding scheme. The box is then scored to evaluate the quality of the suggestion. Furthermore, position optimization is performed based on part features and information to improve the quality of the suggestion, resulting in an optimized recognition result. After component recognition is complete, the target component is retrieved from a database to obtain its classification information, category relationships, and application fields. This invention uses artificial intelligence algorithms to quickly identify target components, reducing the workload of classification, and can be applied to automated production in the manufacturing industry. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention.

[0038] Figure 2 This is a diagram of the voxel network structure in an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0040] Before introducing specific embodiments, let's first explain the relevant concepts:

[0041] 3D target component recognition technology, also known as machine vision component recognition technology, lays the foundation for the industrial application of this step. The basic principle of 3D target component recognition technology is to acquire images of the workpiece, process them using certain algorithms to obtain physical information such as the workpiece's contour, appearance, texture, and shape, and then digitize and model this physical information to obtain digital signals that can be recognized and processed by a computer. This allows equipment such as robotic arms to identify and process the workpiece based on these digital signals, thereby achieving the goal of automated production.

[0042] Laser profilometry: Laser profilometry is one of the most popular 3D imaging techniques. The object being measured moves through a laser beam, and a camera positioned at a known angle records the changing profile of the laser as the object passes through it. This configuration is particularly popular on factory production floors or packaging lines because it relies on an object moving relative to the laser, meaning it is well-suited for products on conveyor belts.

[0043] Stereo Imaging: Another popular 3D imaging technique is stereo imaging, where two cameras are used to record 2D images of an object, which can then be triangulated and transformed into a 3D image. Similar to laser contour analysis, this technique allows for the movement of the object during measurement and recording. The use of random static lighting patterns also allows for the provision of arbitrary textures for ordinary surfaces and objects without natural edges, which is required by many stereo reconstruction algorithms.

[0044] like Figure 1 This embodiment proposes an automatic identification and analysis method for 3D target components, including the following steps:

[0045] S1. Perform voxel segmentation on the point cloud of the target component. Specifically, perform voxel segmentation on the space where the point cloud of the target component is located to obtain a 3D network composed of voxels, where the smallest unit is a voxel.

[0046] S2. Group the obtained voxels and divide them into specified ranges. Specifically, divide the point cloud of the target component into the corresponding voxels and remove voxels without point clouds.

[0047] S3. Process and generate a small number of initial prediction structures. Specifically, extract the contextual features of the remaining voxels and use them to generate anchor boxes for the initial prediction structures as suggestions.

[0048] S4. The original point cloud and the extracted context features are fused to generate the bounding box features for each prediction. Specifically, the point cloud and context features of the target component are fused to obtain the feature map, and the prediction of the bounding box is obtained according to the correspondence between the anchor box and the feature map.

[0049] S5. Use the component aggregation network to learn the point cloud of the proposed region of interest, adjust the bounding box according to the learning result, normalize the ground truth box and the point cloud, reduce the order uncertainty, and obtain the recognition result of the target component. The ground truth box is selected by manual annotation using the currently publicly available conventional method in the field, and the predicted box is the box calculated and output by the target detection model. The bounding box mentioned above is used to identify the position of the object, and can be either a ground truth box or a predicted box.

[0050] S6. Based on the identification results of the target component, search the preset database to obtain relevant information about the target component. Specifically, establish a database to which the component belongs, search for the component, and further analyze the components.

[0051] The basic principle of the method in this embodiment is as follows: using voxel representation as input, it only includes lightweight convolution operations to generate a small number of high-quality initial predictions. The coordinates and index convolutional features of each point in the initial prediction are effectively fused with an attention mechanism, preserving both accurate localization information and contextual information. Subsequent research focuses on internal points and their fused features to further improve the predictions. Compared with traditional manual recognition methods, this method offers significant improvements in speed and usability.

[0052] In this embodiment, step S1 specifically includes: setting a cuboid 3D space to accommodate all point clouds of the target component, and dividing the cuboid 3D space into stacked voxels according to a custom voxel size.

[0053] Step S1 assumes all point clouds are contained within a single box. A large 3D space is used to accommodate all point cloud data, with depth, height, and width (D, H, W). During voxel segmentation, the cuboid 3D space is divided into fixed unit sizes, with the smallest unit being a voxel. This divides the space into stacked, identically sized voxels. In this embodiment, the voxel dimensions are defined as (vD, vH, vW). Therefore, the number of voxel grids generated at each coordinate in the 3D voxelization result of the entire data is (D / vD, H / vH, W / vW). In Step S1, the dimensions of the box along the XYZ axes are W, H, and D. The voxel segmentation operation ultimately produces a 3D network, and the formula for calculating the dimensions is:

[0054] D′=D / vd, H′=H / vh, W′=W / vw.

[0055] In this embodiment, step S2, which divides the point cloud of the target component into corresponding voxels, specifically includes: using 3D sparse convolution to convert the point cloud of the target component into downsampled feature quantities at different levels, with each level of downsampled feature quantity serving as a set of voxel feature vectors. Specifically, a series of 3×3×3 3D sparse convolutions are used to gradually convert the point cloud of the target component into feature quantities with downsampled sizes of 1×, 2×, 4×, and 8×. Each level of sparse feature quantity can be regarded as a set of voxel feature vectors. Commonly used features are 3D coordinates and reflection intensity. After determining the size of the voxel, all points within a voxel become a single point. This point is located at the centroid of all points within the voxel, i.e., the interior point of the voxel. The coordinates of the interior point of the voxel are the voxel coordinates. The original point cloud is the point cloud before voxel segmentation.

[0056] In this embodiment, in step S2, all point cloud data are divided into defined voxels by using sparse convolution and downsampling final encoding operations. Because point clouds are sparse and have highly variable point density throughout the space, the number of point clouds in each voxel is different after grouping. Voxels without point clouds are removed to reduce the computational resources occupied.

[0057] In this embodiment, step S3 specifically includes: obtaining a 2D bird's-eye view feature map by downsampling the set of feature vectors of all voxels, using the 2D bird's-eye view feature map as the context feature of the voxels, and then using the anchor-based method to generate anchor boxes for the initial prediction structure on the 2D bird's-eye view feature map.

[0058] In this embodiment, step S3 converts the 3D feature volume (which is essentially a set of feature vectors) after 8x downsampling into a 2D bird's-eye view feature map, and generates a small number of suggestions (anchor boxes of the initial predicted structure, also known as proposal boxes) using an anchor-based method. The anchor-based method specifically includes:

[0059] The region proposal network uses a 3×3 convolutional kernel that slides across the feature map.

[0060] For each location, k anchor boxes of different sizes and proportions are used to match objects of different proportions on the feature map;

[0061] For each anchor box, feature vectors are extracted from the corresponding object and input into the classification branch to estimate the score of the corresponding class for each object, while input into the regression branch to optimize the position of the bounding box from the original anchor box estimates.

[0062] Standardize the anchor frame.

[0063] The anchor-box-based method involves the Region Proposal Network (RPN) sliding a 3×3 convolutional kernel across a feature map. For each location, k anchor boxes are considered, each with a different size and scale. These varying sizes and scales allow the network to match objects of different proportions on the feature map. Based on the labeled ground truth boxes in the test set, the most suitable anchor box is used to match the target location, thus obtaining the supervision signal for anchor estimation. For each anchor box, a 256-dimensional feature vector is extracted and fed into two parallel branches—a classification layer and a regression layer. The classification branch estimates the score for each object's class, while the regression branch optimizes the bounding box position from the original anchor box estimates. The bounding box is encoded by four values: (x, y, w, h), representing the center coordinates, width, and height of the proposed bounding box, respectively. Each class has 2*L / 8*W / 8 proposal boxes (average label size, 0 degrees, and 90 degrees). Before inputting the coordinates of each point into the network in S4, they must first be normalized to ensure translation and rotation invariance. This is done by using rotation and translation proposal boxes for clipping and normalization.

[0064] In this embodiment, step S4 specifically includes:

[0065] The point cloud of the target component is input into the point cloud processing neural network to obtain the global high-dimensional features of the point cloud data;

[0066] The context features are used as convolutional features. The global high-dimensional features and convolutional features are concatenated and then input into a lightweight point cloud processing neural network for feature extraction to obtain the target features. The convolutional features are then input into two fully connected layers to obtain weight coefficients. Each node in the fully connected layer is connected to all nodes in the previous layer.

[0067] The result of multiplying the weight coefficients and the target features is input into a two-layer multilayer perceptron to obtain a prediction of the bounding box.

[0068] In this embodiment, step S4, by fusing the original point cloud and contextual features, can predict the fine-grained positions of all bounding box corner points based on the proposed (initial prediction structure) bounding boxes. The contextual features (2D bird's-eye view features) obtained in S3 can be used to input each original point cloud into PointNet (point cloud processing neural network) to obtain global high-dimensional features of the point cloud data. Using a public attention mechanism, the global high-dimensional features of the coordinates and the convolutional features (contextual features) are concatenated and then fed into a lightweight PointNet (point cloud processing neural network) for feature extraction. The convolutional features are fed into two fully connected layers (each node is connected to all nodes in the previous layer to combine the features extracted from the previous layers) to obtain an n*1 matrix as the weight coefficients in the attention mechanism. This weight coefficient is multiplied by the features extracted by PointNet, and the result is fed into two MLP (multilayer perceptron) layers to obtain the prediction of the bounding boxes.

[0069] In this embodiment, step S4 takes the voxelized point cloud (actually a regular grid) as input and outputs a feature map, which is the feature of each predicted bounding box. The network structure of the lightweight point cloud processing neural network is as follows: Figure 2 As shown: For 3D detection, the network needs to clearly filter information from the (x, y, z) dimensions. The backbone network consists of 2D and 3D convolutions. The first convolutional block of the first part includes six 3D convolutional layers, which have only a few filters to reduce computation time. This is followed by three 2D convolutional blocks (2nd, 3rd, and 4th) for further extraction and expansion of the receptive field.

[0070] After concatenating global high-dimensional features and convolutional features, the input is fed into a lightweight point cloud processing neural network for feature extraction to obtain target features. Deconvolution is used to upsample the feature maps of the last layer of convolutional blocks 2, 3, and 4, and then these feature maps are concatenated. Rich positional information is collected in the lower layers 1 and 2, while stronger semantic information is collected in the higher layers 3 and 4. Then, normalized anchors with predefined proportions and angles are obtained. By establishing a one-to-one mapping between the anchor box positions and feature maps, the anchor boxes are used on the fused feature map, i.e., fusing the original point cloud and the feature map from which contextual features are extracted. Finally, features from salient regions may have rich classification information, while features near the boundaries are more conducive to position regression. Therefore, classification and regression heads are run on the salient region feature map and the boundary-near feature map, respectively, to classify each anchor box and perform regression operations on the position of the target component. The result is then vectorized into a feature vector, and the same two branches as in step S3 are added: a classification branch for the final bounding box scoring and a regression branch for location refinement. The feature vector is input into the classification branch to score the bounding box and into the regression branch to optimize the location of the bounding box.

[0071] Step S5 in this embodiment specifically includes: performing a point cloud aggregation operation within the suggested region of interest, inputting the aggregation result into the part aggregation network to obtain the box parameters of the target component, vectorizing the box parameters into feature vectors, inputting them into the classification branch to evaluate the confidence of the bounding box, and simultaneously inputting them into the regression branch to optimize the position of the bounding box.

[0072] In step S5 of this embodiment, the obtained bounding box is adjusted, and the ground truth box and point cloud are normalized. By considering the spatial distribution of the part positions within the predicted object of all 3D points in the proposal (i.e., the point cloud data in the proposed anchor box), the quality of the proposal is evaluated by aggregating the predicted part positions. A RoI (Region of Interest)-aware point cloud aggregation operation is performed on the proposal. The part information in the proposal corresponding to each 3D point is grouped, and the boundary is scored using a part aggregation network. Then, the target bounding box is positionally optimized based on the part features and information (voxel feature information in S2). The target box position is optimized by optimizing the center point coordinates, width, and height values ​​relative to the ground truth bounding box to improve the quality of the proposal. For each 3D proposal, the RoI-aware point cloud aggregation operation is applied to the point-like part positions (average aggregation) and point-like features (maximum aggregation) from the initial prediction structure, respectively. After the aggregation operation, the feature dimensions of the point-like features in the initial prediction structure are used to implement the part aggregation network in a hierarchical manner to learn the box parameters of the target component from the spatial distribution of the part positions within the predicted object. The part aggregation network first stacks several 3D sparse convolutional layers with a kernel size of 3 to aggregate all part features of the proposal as the receptive field increases. A sparse max-pooling layer with a kernel size of 2 and a stride of 2 is inserted between the sparse convolutional layers to reduce the feature volume sampling to 7, thus saving computational cost and parameters. Finally, the box parameters of the target component are vectorized into a feature vector (empty voxels are kept at 0) and fed into two branches: classification and regression, for bounding box scoring and location optimization. The classification branch in this step uses a box scoring branch to evaluate the confidence of the bounding box, using the 3D IoU between the 3D proposal and its corresponding ground truth box as a soft label for proposal quality evaluation, which is also learned using binary cross-entropy loss. Only a portion of the proposals were generated in step S3, serving as the basis for generating and refining the newly generated proposals in this step. The regression branch in this step uses smoothing loss to regress and normalize the box parameters to optimize the bounding box location, as shown in the following expression:

[0073]

[0074]

[0075]

[0076] In the lidar coordinate system, the three-dimensional bounding box is represented as (x,y,z,l,w,h,θ), where (x,y,z) is the target center position, (l,w,h) is the target size, and θ is the target direction from a bird's-eye view.

[0077] In this embodiment, during the calculation of the regression target in step S5, the ground truth bounding box and point cloud are normalized by rotating and translating the proposal bounding box. This operation organizes the corner points of the ground truth bounding box in a specific order, which can reduce the uncertainty of the corner order caused by rotation. Using a part clustering strategy, the parameters of the 3D bounding box are directly solved by fitting the predicted part positions of all points in the corresponding scheme, and the proposed suggestion is evaluated by clustering the predicted part positions. Compared with the simple method of directly vectorizing the merged 3D feature map into feature vectors, the proposed part clustering strategy can effectively learn the spatial distribution of the predicted part positions by clustering features from the local scale to the global scale.

[0078] In this embodiment, step S6 establishes a database of component dependencies, from which components are retrieved and analyzed. This database stores 3D component image samples and tags. The tags include the category, relationship, and application field of all components. After the component identification is completed, the database is searched to obtain the category to which the component belongs. After each component is retrieved, the relationship between them can be checked. Compared with conventional analysis methods, this saves the time of calculating similar components and eliminates the need to call additional resources for each analysis, thus preparing for downstream intelligent assembly.

[0079] This embodiment also proposes an automatic identification and analysis system for 3D target components, including a computer device, which is programmed or configured to execute the automatic identification and analysis method for 3D target components described in this embodiment.

[0080] In summary, the method in this embodiment first voxelizes the point cloud data and feeds it into an anchor-based method—RPN—to generate an initial proposal (predicted region). This process, due to voxelization and convolution, results in the loss of some localization information. The high-dimensional features of the voxel coordinates and the convolutional features are concatenated and then fed into a lightweight PointNet (point cloud processing neural network) for feature extraction. The concatenated features are multiplied by weight coefficients and fed into a two-layer MLP (Multilayer Perceptron) to obtain the predicted bounding box. Then, a part clustering strategy is used to directly solve for the 3D bounding box parameters by fitting the predicted part positions of all points in the corresponding scheme. Prediction confidence is used to evaluate the quality of the proposal and the bounding box optimization. Finally, component analysis is performed to obtain category relationships and application areas. This embodiment's method can intelligently identify basic mechanical parts that meet appearance, physical, and industrial requirements.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for automatic identification and analysis of 3D target components, characterized in that, include: S1. Perform voxel segmentation on the space where the point cloud of the target component is located to obtain a 3D network composed of voxels; S2. Divide the point cloud of the target component into the corresponding voxels and remove voxels without point cloud. S3. Extract the contextual features of the remaining voxels and use them to generate anchor boxes for the initial predicted structure as suggestions; S4. Fuse the point cloud and context features of the target component to obtain a feature map, and predict the bounding box based on the correspondence between the anchor box and the feature map. S5. Use the part aggregation network to learn the point cloud of the proposed region of interest, adjust the bounding box according to the learning result, and obtain the recognition result of the target component. Specifically, this includes: performing point cloud aggregation operation in the proposed region of interest, inputting the aggregation result into the part aggregation network to obtain the box parameters of the target component, vectorizing the box parameters into feature vectors, inputting them into the classification branch to evaluate the confidence of the bounding box, and inputting them into the regression branch to optimize the position of the bounding box. S6. Based on the identification results of the target component, search the preset database to obtain relevant information about the target component.

2. The method for automatic identification and analysis of 3D target components according to claim 1, characterized in that, Step S1 specifically includes: setting up a cuboid 3D space to accommodate all point clouds of the target component, and dividing the cuboid 3D space into stacked voxels according to a custom voxel size.

3. The method for automatic identification and analysis of 3D target components according to claim 1, characterized in that, Step S2, which divides the point cloud of the target component into corresponding voxels, specifically includes: using 3D sparse convolution to convert the point cloud of the target component into downsampled feature quantities at different levels, with each level of downsampled feature quantity serving as a set of voxel feature vectors.

4. The method for automatic identification and analysis of 3D target components according to claim 1, characterized in that, Step S3 specifically includes: obtaining a 2D bird's-eye view feature map by downsampling the set of feature vectors of all voxels, using the 2D bird's-eye view feature map as the context feature of the voxels, and then using the anchor-frame-based method to generate anchor frames for the initial prediction structure on the 2D bird's-eye view feature map.

5. The method for automatic identification and analysis of 3D target components according to claim 4, characterized in that, The anchor-frame-based method specifically includes: The region proposal network uses a 3×3 convolutional kernel that slides across the feature map. For each location, k anchor boxes of different sizes and proportions are used to match objects of different proportions on the feature map; For each anchor box, feature vectors are extracted from the corresponding object and input into the classification branch to estimate the score of the corresponding class for each object, while input into the regression branch to optimize the position of the bounding box from the original anchor box estimates. Standardize the anchor frame.

6. The method for automatic identification and analysis of 3D target components according to claim 1, characterized in that, Step S4 specifically includes: The point cloud of the target component is input into the point cloud processing neural network to obtain the global high-dimensional features of the point cloud data; The context features are used as convolutional features. The global high-dimensional features and convolutional features are concatenated and then input into a lightweight point cloud processing neural network for feature extraction to obtain the target features. The convolutional features are then input into two fully connected layers to obtain weight coefficients. Each node in the fully connected layer is connected to all nodes in the previous layer. The result of multiplying the weight coefficients and the target features is input into a two-layer multilayer perceptron to obtain a prediction of the bounding box.

7. The method for automatic identification and analysis of 3D target components according to claim 6, characterized in that, The first convolutional block of the lightweight point cloud processing neural network includes six three-dimensional convolutional layers, while the second, third, and fourth convolutional blocks are all two-dimensional convolutional blocks. The global high-dimensional features and convolutional features are concatenated and then input into the lightweight point cloud processing neural network for feature extraction to obtain the target features. Specifically, this includes: The feature maps of the last layer of the second, third, and fourth convolutional blocks are upsampled by deconvolution, and then the upsampled results are concatenated. Positional information is collected in layers 1 and 2, and semantic information is collected in layers 3 and 4 to obtain a feature map that fuses the original point cloud and extracts contextual features. Establish the correspondence between anchor boxes and the feature maps, classify each anchor box, perform regression calculation on the position of the target component, vectorize the calculation result into a feature vector, input the feature vector into the classification branch to score the bounding box, and simultaneously input it into the regression branch to optimize the position of the bounding box.

8. The method for automatic identification and analysis of 3D target components according to claim 1, characterized in that, The input regression branch to optimize the bounding box position specifically includes: using a smoothing loss to regress the normalized box parameters, as shown in the following expression: , , , Where x, y, z, l, w, h, θ are the parameters of the bounding box of the target component in the lidar coordinate system, where x, y, z are the center position of the target component, l, w, h are the size of the target component, and θ is the target direction from a bird's-eye view.

9. A 3D target component automatic identification and analysis system, comprising computer equipment, characterized in that, The computer device is programmed or configured to perform the automatic identification and analysis method for 3D target components as described in any one of claims 1 to 8.

Images