A system and method for three-dimensional recognition and pose estimation of similar objects

Abstract

The present invention discloses a system and method for three-dimensional recognition and pose estimation of similar objects. The system includes a standard state prediction model, which includes a point cloud feature extraction module: including several cascaded convolution layers, taking the target object observation point cloud as input, and outputting a high-dimensional feature vector for each point; a category recognition module: including several cascaded convolution layers and fully connected layers, identifying and predicting the category of the target object; a point cloud coordinate prediction module: including multiple cascaded convolution heads and fully connected layers, predicting the standard canonical space coordinates for each point in the input point cloud. The method includes a size and pose voting strategy, which calculates the scale and pose information based on the model's prediction results for similar target objects. The present invention only requires the generation of corresponding training data through a simulation environment, without the need for manual data labeling, and has higher efficiency and lower data costs.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and in particular to a system and method for three-dimensional recognition and position estimation of similar objects. Background Art

[0002] Current 3D recognition and pose estimation methods for rigid objects can be broadly categorized into two main groups: those based on traditional feature descriptors and those based on deep learning. Traditional descriptor-based methods require tailored descriptor design for specific objects and application scenarios, a cumbersome process that also exhibits poor robustness to occlusion and noise. Learning-based methods, on the other hand, have garnered widespread attention in recent years due to their ease of application and significant potential for scene adaptability.

[0003] Learning-based methods often place the target objects to be detected in various positions and postures in real scenes, then collect data, annotate the corresponding true values ​​of the pose, and then train the model until the model converges. The model can then be used to predict the position and pose of the target object in practical applications. However, this learning-based 3D recognition and pose estimation method requires the collection and annotation of extensive datasets for the target objects, which is extremely time-consuming and labor-intensive. Furthermore, the trained model is only effective for targets in the training set and is ineffective for similar objects with the same geometric structure but different structural dimensions as the training targets, limiting its scope of application.

[0004] A prior art method and storage medium for acquiring the 6D pose of class-level objects based on language descriptions (CN114399515A) can achieve pose estimation and tracking of similar objects using a meta-learning model. However, when performing pose recognition on a new object of the same type, this method requires collecting training data for the new object and retraining the network, which reduces its convenience and efficiency. Summary of the Invention

[0005] The purpose of the present invention is to provide a system and method for three-dimensional recognition and pose estimation of similar objects, including a standard state prediction model, a corresponding method for generating simulation training data, a model training method, and a method for use. This method aims to address the time-consuming and labor-intensive data collection and annotation issues of current mainstream learning-based pose estimation methods, as well as their inability to be universally applied to similar target objects. This method eliminates the need for data collection and network training when tracking the pose of new objects, thereby improving the method's ease of use and scope of application. The proposed method is trained using simulation data augmentation and is applicable to similar target objects with similar geometric structures.

[0006] The present invention is achieved through at least one of the following technical solutions.

[0007] A similar object three-dimensional recognition and pose estimation system includes a similar object three-dimensional recognition and pose estimation model, wherein the similar object three-dimensional recognition and pose estimation model includes:

[0008] Point cloud feature extraction module: It includes several cascaded convolutional layers, takes the target object observation point cloud as input, and outputs a high-dimensional feature vector for each point;

[0009] Category recognition module: It contains several cascaded convolutional layers and fully connected layers, which processes the extracted point-by-point high-dimensional features to identify and predict the category of the target object;

[0010] Point cloud coordinate prediction module: This module includes multiple cascaded convolution heads and fully connected layers. It jointly processes the point features output by the point cloud feature extraction module and the intermediate features generated by the category recognition module to predict the standard canonical space state coordinates of each point in the input point cloud, and ultimately obtains the point cloud of the target object in the standard canonical space.

[0011] Furthermore, the standard canonical space state is defined as: the standard canonical space state is a predefined unified state of objects with the same structural composition but different sizes of parts.

[0012] Furthermore, the point cloud coordinate prediction module divides the standard canonical space into an m*m*m spatial grid, and uses the corresponding features of each point in the input point cloud to perform sixty-four classifications on the three dimensions of x, y, and z, thereby realizing the standard canonical space coordinate prediction of each point.

[0013] Furthermore, the point cloud feature extraction module includes:

[0014] Point cloud sampling and grouping unit: The point cloud sampling and grouping unit uses the farthest point sampling method to select a series of key points from the input point cloud. Then, all points within a specified radius from these key points are divided into different groups, thereby achieving sampling and grouping of the point cloud;

[0015] One-dimensional convolution layer: performs convolution operation on each group divided by the point cloud sampling grouping unit to generate corresponding feature points;

[0016] Pooling layer: Use the maximum value selection strategy to reduce the dimensionality of high-dimensional feature points;

[0017] Fully connected layer: fully connect the feature points;

[0018] Upsampling interpolation layer: Perform upsampling interpolation calculations on key feature points to obtain feature points with higher density.

[0019] Furthermore, the category identification module includes:

[0020] Convolutional layer: The point cloud feature extraction module performs further convolution operations on the features generated by each point in the target point cloud;

[0021] Fully connected layer: The high-dimensional point feature vector output by the convolutional layer is fully connected and finally output as a one-dimensional feature vector.

[0022] Furthermore, the point cloud coordinate prediction module structure includes:

[0023] Three-layer fully connected unit: It contains three consecutive fully connected layers. First, the one-dimensional feature vector output by the category recognition module is concatenated with the point features in the model. Then, the concatenated point features are fully connected. Finally, a 64-dimensional vector is output, and the label of the maximum value is used as the predicted value of one dimension in the three-dimensional coordinate.

[0024] A method for implementing the similar object three-dimensional recognition and pose estimation system includes the following steps:

[0025] Collecting model training data: Training data is collected in a virtual simulation environment, including the following steps: first, using the object's mesh model as a standard model, independently scaling different geometric components of the object, and combining the scaled geometric components to form a new object sample mesh model; loading the newly generated sample mesh model into the simulation environment and dropping it onto the simulated tabletop in a random position and posture;

[0026] When the object falls onto the table and stops moving, a virtual camera is used to collect RGBD data of the scene and extract the point cloud of the target object. The mesh state of the target object in the current state is recorded, and the vertex correspondence in the mesh model is used to obtain the true coordinate value of the target object's point cloud on the standard canonical model. Each execution of this process completes the collection of one frame of data, and the process is repeated until the data collection volume reaches the target requirement.

[0027] Model pre-training involves inputting the observed target point cloud data into the model input. The point cloud feature extraction module first extracts features from each point in the point cloud. All point features are then input into the category recognition module for processing to obtain a one-dimensional feature vector at the category level. The category vector is then fully connected and the classification loss is calculated. The category vector is then concatenated with the point feature vector of the point cloud. The concatenated point cloud features are then input into the point cloud coordinate prediction module. The standard spatial coordinates of each point in the point cloud are predicted, and the prediction loss is calculated compared with the collected true value of the standard spatial coordinates. The model's strategy parameters are then updated based on the prediction loss until the model converges stably.

[0028] Pose calculation strategy: After the model predicts the coordinates of each point in the target point cloud in the standard space, it uses a voting-based strategy to calculate the pose of the target object, including the following steps:

[0029] Step 1: Randomly select three points in the target point cloud and record them as p1, p2, and p3, and then obtain the corresponding standard space coordinates n1, n2, and n3 according to the output of the standard state conversion model;

[0030] Step 2: Use the three point coordinates p1, p2, and p3 to calculate the vectors

[0031] Step 3: Use the three points n1, n2, and n3 in the standard space to calculate the vectors

[0032] Step 4: Vector With vector Perform normalization processing, calculate the spatial rotation matrix using the one-to-one correspondence between the three pairs of vectors, and decompose it into the rotation angles θ, λ, and φ around the x, y, and z axes respectively; repeat steps 1 to 4 for the specified number of times N, and record the set of N pairs of rotation angles

[0033] Perform discrete statistics on the rotation angles of the x, y, and z axes respectively, and obtain the optimal estimated angle θ with the largest statistical number. optim ,λ optim 、 First, align the target point cloud with the standard model according to the estimated angle. Then, scale the target point cloud in the x, y, and z axes to make it consistent with the standard model in the x, y, and z axes. The corresponding scaling factor is recorded as s. x ,s y ,s z .

[0034] Furthermore, the prediction loss used in pre-training is:

[0035] L const =-(c x ∑log p x +c y ∑log p y +c z ∑log p z )

[0036] Among them, c x 、c y 、c z is the true value of the coordinates of each point in the standard canonical space, px 、p y 、p z is the predicted probability of the three-dimensional coordinate value of each point by the coordinate prediction unit.

[0037] Furthermore, the classification loss used in pre-training is:

[0038] L category =-c∑log p c

[0039] Among them, c is the true category of the target, p c The predicted probability of each category output by the category recognition module.

[0040] Furthermore, the method of updating the model strategy parameters according to the prediction loss is expressed as:

[0041]

[0042] Where θ′ is the parameter of the model updated based on all losses, λ is the internal learning rate, For all loss joint gradient calculations, ∑(L const (θ; P) + L category (θ; P)) is the sum of the classification loss and coordinate prediction loss of all samples in a batch of training.

[0043] Compared with the existing technology, the present invention has the following beneficial effects and advantages:

[0044] 1. Compared with the current mainstream pose estimation method, this method only requires data collection in a simulation environment and can directly generate the required data annotations, which greatly reduces the time and effort of model training data preparation and greatly improves the efficiency and convenience of model training.

[0045] 2. This method is universally applicable to similar targets with similar structures, so there is no need to collect and label data for all targets to be detected, which greatly improves the applicability of the method in practical application scenarios.

[0046] 3. The proposed method can be used as a prerequisite basic function for robot operation and is widely used in various industries and service fields, such as industrial manufacturing, assembly, and household service operations. It provides an efficient and intelligent three-dimensional recognition and positioning method for robots when handling similar objects with the same structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a schematic structural diagram of a similar object three-dimensional recognition and pose estimation model of the present invention;

[0048] Figure 2This is a flowchart of an application of a similar object three-dimensional recognition and pose estimation method of the present invention;

[0049] Figure 3 This is a schematic diagram of the intermediate process of the intermediate pose voting strategy of a similar object three-dimensional recognition and pose estimation method of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and examples. Note that the following embodiments are merely illustrative in nature and the present invention is not intended to limit its use or purpose, and the present invention is not limited to the following embodiments.

[0051] like Figure 1 As shown, this example provides a similar object 3D recognition and pose estimation system based on simulation data enhancement, including a similar object 3D recognition and pose estimation model, which includes a point cloud feature extraction module, a category recognition module, and a point cloud coordinate prediction module;

[0052] The point cloud feature extraction module consists of several cascaded convolutional layers, which take the target object observation point cloud as input and output a high-dimensional feature vector for each point. Its specific structure includes:

[0053] Point cloud sampling and grouping unit: The point cloud sampling and grouping unit uses the farthest point sampling method to select a series of key points from the input point cloud. Then, all points within a specified radius from these key points are divided into different groups, thereby achieving sampling and grouping of the point cloud;

[0054] One-dimensional convolution layer: The convolution sum size is 1*3, and a convolution operation is performed on each group divided by the point cloud sampling grouping unit to generate corresponding feature points;

[0055] Pooling layer: Use the maximum value selection strategy to reduce the dimensionality of high-dimensional feature points;

[0056] Fully connected layer: fully connect the feature points;

[0057] Upsampling interpolation layer: Perform upsampling interpolation calculations on key feature points to obtain feature points with higher density.

[0058] The category recognition module includes several cascaded convolutional layers and fully connected layers to process the extracted point-by-point high-dimensional features to identify and predict the category of the target object. Its specific structure includes:

[0059] Convolution layer: The convolution sum size is 1*3, and the point cloud feature extraction module performs further convolution operation on the features generated by each point in the target point cloud;

[0060] Fully connected layer: The high-dimensional point feature vector output by the convolutional layer is fully connected and finally output as a one-dimensional feature vector.

[0061] The point cloud coordinate prediction module includes multiple cascaded convolution heads and fully connected layers, which jointly process the point features output by the point cloud feature extraction module and the intermediate features generated by the category recognition module to predict the standard canonical space state coordinates of each point in the input point cloud, and finally obtain the point cloud of the target object in the standard canonical space.

[0062] As an embodiment, the point cloud coordinate prediction module divides the standard canonical space into a 64*64*64 spatial grid, and uses the corresponding features of each point in the input point cloud to perform sixty-four classifications on the three dimensions of x, y, and z, thereby realizing the standard canonical space coordinate prediction of each point.

[0063] The standard canonical space state is defined as follows: the standard canonical space state is a predefined unified state for objects with the same structural composition but different sizes of each part. Take the workpiece screw as an example: for any type of screw, move the center of mass of its mesh model to the center of mass position of the unit space body (a space with a unit length of 1m in length, width and height), and then scale the x, y and z axes of the screw respectively so that the lengths occupied by the three axes are equal to a unified fixed value. At this point, the state of the mesh model of the screw is the standard canonical space state, and the corresponding point cloud is the standard canonical state point cloud facing the screw of this model.

[0064] The point cloud coordinate prediction module structure includes three layers of fully connected units, which contain three consecutive fully connected layers. First, the one-dimensional feature vector output by the category recognition module is spliced ​​with the point features in the model, and then the spliced ​​point features are fully connected. Finally, a sixty-four-dimensional vector is output, and the label where the maximum value is located is used as the predicted value of one dimension in the three-dimensional coordinates.

[0065] As an embodiment, this example is based on Figure 2 Taking the two typical tasks of screws and nuts in the image as an example to identify the target object, based on the above, a method for realizing the similar object three-dimensional recognition and pose estimation system based on simulation data enhancement is introduced. Figure 1 As shown, the method includes the following steps:

[0066] Model training data collection: Training data is collected in a virtual simulation environment. Using a nut as a representative example, the simulation training data generation process is described. The process includes the following steps: First, a mesh model of a nut is used as a standard model. Then, the different geometric components of the nut are independently scaled and resized, and the scaled geometric components are combined to form a new example mesh model of the nut. The newly generated example mesh model is loaded into the simulation environment and dropped onto a simulated tabletop at a random position and posture. When the object lands on the tabletop and stops moving, a virtual camera is used to collect RGBD scene data and extract the target object's point cloud. The current mesh state of the target object is recorded, and the vertex correspondences in the mesh model are used to obtain the true coordinate values ​​of the target object's point cloud on the standard model. Each execution of this process completes the collection of one frame of data. This process is repeated until the target data collection volume is reached.

[0067] Based on the canonical mesh models of screws and nuts, category-level data enhancement is performed on them in a simulation environment. Specifically, the mesh models in the canonical structures of screws and nuts are first decomposed according to the geometry. Then, the different geometric structures of screws and nuts are scaled by 0.5 to 1.5 in the x, y, and z axes, and these structures are reassembled to form an instance target with the same geometric structure but different model sizes. They are then loaded into Pybullet for simulation data collection. In the simulation environment, screw and nut models of different sizes are initialized above the table in random poses and allowed to fall freely onto the table under the action of gravity. After the workpiece is stationary, the virtual camera is used to collect the scene RGBD data, and the workpiece point cloud data P is extracted, and the workpiece category label d is assigned according to the workpiece category. Then, the relationship between the coordinates of the points in the workpiece point cloud and the vertex coordinates of the workpiece mesh model, as well as the vertex correspondence between the workpiece mesh model and the standard canonical mesh model, is used to determine the true value P of the standard canonical space coordinate of each point in the workpiece point cloud. nocs .

[0068] Model training involves inputting the observed workpiece point cloud data into the model input terminal, and then extracting point-by-point features from the point cloud by the feature extraction module, outputting a point-by-point feature vector V. Next, the category recognition module uses the point-by-point feature vector V as input, performs operations such as convolution, and outputs a category vector L. After further full-connection processing, the category vector L is combined with the workpiece category label for category loss calculation. At the same time, the category vector L is concatenated with the point-by-point feature vector V, and the concatenated feature vector is input into the point cloud coordinate prediction module, which, after convolution and other operations, outputs the standard spatial coordinate prediction value P for each point. pre , and the true value P nocsThe prediction loss is calculated jointly. The model's strategy parameters are then updated based on the category loss and the coordinate prediction loss.

[0069] The coordinate prediction loss used in model training is:

[0070] L nocs =-(c x ∑log p x +c y ∑log p y +c z ∑log p z ),

[0071] Among them, (c x ,c y ,c z )∈P nocs is the true value of the coordinates of each point in the standard canonical space, (p x ,p y ,p z )∈P pre is the predicted probability of the three-dimensional coordinate value of each point by the coordinate prediction unit.

[0072] The classification loss used in model pre-training is:

[0073] L category =-c∑log p c ,

[0074] Among them, c is the true category of the target, p c The predicted probability of each category output by the category recognition module.

[0075] The way to update the model strategy parameters based on the prediction loss is expressed as:

[0076]

[0077] where θ ′ are the parameters of the model updated based on all losses, λ is the internal learning rate, For all loss joint gradient calculations, ∑(L const (θ; P) + L category (θ; P)) is the sum of the classification loss and coordinate prediction loss of all samples in a batch of training.

[0078] The model usage method also includes a size and posture voting strategy: the posture parameters are calculated based on the model prediction results: Figure 3 The target point cloud of the workpiece is P, and the model predicts the standard canonical space coordinate value of P as P pre . Choose any three points p1, p2, and p3 in P. preThe coordinates of the three points predicted are counted as p1′, p2′, and p3′.

[0079] like Figure 3 , use p1, p2, p3 to calculate vectors respectively Use the three points p1′, p2′, p3′ in the standard space to calculate the vector Pair Vector With vector Perform normalization, then use the one-to-one correspondence between the three pairs of vectors to calculate the spatial rotation matrix and decompose it into the rotation angles θ, λ, and φ around the x, y, and z axes respectively. Repeat this step to the specified number of times N and record the set of N pairs of rotation angles. Discrete statistics are performed on the rotation angles of the three axes x, y, and z respectively. The angle with the largest statistical number is the optimal estimated angle, θ optim ,λ 1ptim , First, align the target point cloud with the standard model according to the estimated angle, and then scale the target point cloud in the x, y, and z axes to make it consistent with the standard model in the x, y, and z axes. The corresponding scaling factor is recorded as s x ,s y ,s z Then the final θ optim ,λ optim , s x ,s y ,s z are the final pose estimation parameters.

[0080] This method only requires the generation of training data in a simulation environment, eliminating the need for manual data annotation, resulting in higher efficiency and lower data costs. This method can adapt to changes in target size and has good generalization capabilities for similar targets with similar structures. These features enable the invention to have a wider range of applications and rapid deployment in practical scenarios.

[0081] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, numerous modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and utilize the present invention.

Claims

1. A similar object 3D recognition and pose estimation system based on simulation data enhancement, characterized by: The system includes a similar object three-dimensional recognition and pose estimation model, wherein the similar object three-dimensional recognition and pose estimation model includes: Point cloud feature extraction module: It includes several cascaded convolutional layers, takes the target object observation point cloud as input, and outputs a high-dimensional feature vector for each point; Category recognition module: It contains several cascaded convolutional layers and fully connected layers, which processes the extracted point-by-point high-dimensional features to identify and predict the category of the target object; Point cloud coordinate prediction module: This module includes multiple cascaded convolution heads and fully connected layers. It jointly processes the point features output by the point cloud feature extraction module and the intermediate features generated by the category recognition module to predict the standard canonical space state coordinates of each point in the input point cloud, ultimately obtaining the point cloud of the target object in the standard canonical space. The posture calculation strategy includes the following steps: Step 1: Randomly select three points in the target point cloud and record them as , and obtain the corresponding standard space coordinates according to the output results of the standard state conversion model ; Step 2: Use three-point coordinates Calculate vectors separately ; Step 3: Using three points in standard space Calculate vectors separately ; Step 4: Vector With vector Perform normalization, then use the one-to-one correspondence between the three pairs of vectors to calculate the spatial rotation matrix and decompose it into the rotation angles θ, λ, and φ around the x, y, and z axes respectively; repeat steps 1 to 4 for the specified number of times N, and record the N pairs of rotation angles ( , , ); Perform discrete statistics on the rotation angles of the x, y, and z axes respectively, and obtain the optimal estimated angle with the largest statistical number. ; First, align the target point cloud with the standard model according to the estimated angle, and then scale the target point cloud in the x, y, and z axes to make it consistent with the standard model in the x, y, and z axis scales. The corresponding scaling factors are recorded as .

2. The system for three-dimensional recognition and pose estimation of similar objects based on simulation data enhancement according to claim 1, characterized in that: The standard canonical space state is defined as: the standard canonical space state is a predefined unified state of objects with the same structural composition but different sizes of parts.

3. The system for three-dimensional recognition and pose estimation of similar objects based on simulation data enhancement according to claim 2, characterized in that: The point cloud coordinate prediction module divides the standard canonical space into an m*m*m spatial grid, and uses the corresponding features of each point in the input point cloud to perform 64 classifications on the x, y, and z dimensions, thereby realizing the standard canonical space coordinate prediction of each point.

4. The system for three-dimensional recognition and pose estimation of similar objects based on simulation data enhancement according to claim 1, characterized in that: The point cloud feature extraction module includes: Point cloud sampling and grouping unit: The point cloud sampling and grouping unit uses the farthest point sampling method to select a series of key points from the input point cloud. Then, all points within a specified radius from these key points are divided into different groups, thereby achieving sampling and grouping of the point cloud; One-dimensional convolution layer: performs convolution operation on each group divided by the point cloud sampling grouping unit to generate corresponding feature points; Pooling layer: Use the maximum value selection strategy to reduce the dimensionality of high-dimensional feature points; Fully connected layer: fully connect the feature points; Upsampling interpolation layer: Perform upsampling interpolation calculations on key feature points to obtain feature points with higher density.

5. The system for three-dimensional recognition and pose estimation of similar objects based on simulation data enhancement according to claim 1, characterized in that: The category identification module includes: Convolutional layer: The point cloud feature extraction module performs further convolution operations on the features generated by each point in the target point cloud; Fully connected layer: The high-dimensional point feature vector output by the convolutional layer is fully connected and finally output as a one-dimensional feature vector.

6. The system for three-dimensional recognition and pose estimation of similar objects based on simulation data enhancement according to claim 1, characterized in that: The point cloud coordinate prediction module structure includes: Three-layer fully connected unit: It contains three consecutive fully connected layers. First, the one-dimensional feature vector output by the category recognition module is concatenated with the point features in the model. Then, the concatenated point features are fully connected. Finally, a 64-dimensional vector is output, and the label of the maximum value is used as the predicted value of one dimension in the three-dimensional coordinate.

7. A method for implementing the similar object 3D recognition and pose estimation system based on simulation data enhancement as claimed in claim 1, characterized in that: The method comprises the following steps: Collecting model training data: Collecting training data in a virtual simulation environment includes the following steps: first, using the object's mesh model as a standard model, then independently scaling different geometric components of the object, and combining the scaled geometric components to form a new object sample mesh model; loading the newly generated sample mesh model into the simulation environment and dropping it onto the simulated tabletop in a random position and posture; When the object falls onto the table and stops moving, a virtual camera is used to collect RGBD data of the scene and extract the point cloud of the target object. The mesh state of the target object in the current state is recorded, and the vertex correspondence in the mesh model is used to obtain the true coordinate value of the target object's point cloud on the standard canonical model. Each execution of this process completes the collection of one frame of data, and the process is repeated until the data collection volume reaches the target requirement. Model pre-training involves inputting the observed target point cloud data into the model input. The point cloud feature extraction module first extracts features from each point in the point cloud. All point features are then processed in the category recognition module to obtain a one-dimensional feature vector at the category level. The category vector is fully connected and the classification loss is calculated. The category vector is concatenated with the point cloud feature vector, and the concatenated point cloud features are input into the point cloud coordinate prediction module. The standard spatial coordinates of each point in the point cloud are predicted, and the prediction loss is calculated compared with the collected true value of the standard spatial coordinates. The model's strategy parameters are updated based on the prediction loss until the model converges stably. Pose calculation strategy: After the model predicts the coordinates of each point in the target point cloud in the standard space, it uses a voting-based strategy to calculate the pose of the target object.

8. The method according to claim 7, characterized in that The prediction loss used in pre-training is: in, 、 、 is the true value of the coordinates of each point in the standard canonical space, 、 、 is the predicted probability of the three-dimensional coordinate value of each point by the coordinate prediction unit.

9. The method according to claim 7, characterized in that The classification loss used in pre-training is: in, c is the true category of the target, The predicted probability of each category output by the category recognition module.

10. The method according to claim 7, characterized in that The way to update the model strategy parameters based on the prediction loss is expressed as: , in are the parameters of the model updated based on all losses, is the internal learning rate, The gradient is calculated for all losses together, It is the sum of the classification loss and coordinate prediction loss of all samples in a batch of training.

Images