Object segmentation model training method and device, object segmentation method and device and electronic equipment

Abstract

The invention discloses an object segmentation model training method and device, an object segmentation method and device and electronic equipment, and the method comprises the steps: obtaining a plurality of pieces of point cloud data, each piece of point cloud data corresponding to a scene, and the scene comprises at least one object; performing target search on each point cloud data to obtain a point cloud of a reference object in the point cloud data; according to each piece of point cloud data and the point cloud of the reference object in the point cloud data, constructing a plurality of first training samples; a plurality of first training samples are adopted to train a neural network model to obtain an object segmentation model, the object segmentation model is used for segmenting an object in to-be-segmented point cloud data to obtain an object segmentation result, and in the model training process, target search is carried out on the point cloud data to obtain a target segmentation result; the point cloud of the reference object for training can be quickly obtained, manual labeling is not needed, and time and labor cost are effectively saved.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and in particular to a training method for an object segmentation model, an object segmentation method, a device, and an electronic device. Background Art

[0002] The point cloud segmentation task is an important step in the application process of point cloud data. The point cloud segmentation task depends on labeled data. Currently, the labeled data is mainly obtained through manual labeling. The segmentation model is trained using the manually labeled data, and then the segmentation model is used to segment the point cloud.

[0003] In the above method, the training of the segmentation model relies on manually labeled data, which requires a lot of manpower and time costs when obtaining large-scale labeled data. Summary of the Invention

[0004] The embodiments of the present application provide a training method, object segmentation method, device and electronic device for a target segmentation model. During the model training process, by searching for targets in point cloud data, the point cloud of the reference object used for training can be quickly obtained without manual labeling, effectively saving time and labor costs.

[0005] In a first aspect, an embodiment of the present application provides a method for training an object segmentation model, the method comprising:

[0006] Acquire a plurality of point cloud data, each of the point cloud data corresponds to a scene, and the scene includes at least one object;

[0007] Performing target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data;

[0008] Constructing a plurality of first training samples according to each of the point cloud data and the point cloud of the reference object in the point cloud data;

[0009] The neural network model is trained using the plurality of the first training samples to obtain an object segmentation model, and the object segmentation model is used to segment objects in the point cloud data to be segmented to obtain an object segmentation result.

[0010] In one embodiment of the present application, performing target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data includes:

[0011] The following processing is performed on each point cloud data:

[0012] Each time a target search operation is performed on the point cloud data, a feature of each point in the point cloud data is input into a pre-trained cross-attention neural network model to obtain a first search area, wherein the feature of each point in the point cloud data is obtained based on feature extraction of the point cloud data;

[0013] Acquire a first point cloud covered by the first search area in the point cloud data;

[0014] The first point cloud is determined as the point cloud of the reference object in the point cloud data.

[0015] In one embodiment of the present application, the cross-attention neural network model is obtained according to the following steps:

[0016] Acquire a plurality of second training samples, where the second training samples include sample point cloud data and features of each point in the sample point cloud data;

[0017] In each round of training, the feature of each point in the sample point cloud data in the second training sample is input into the cross-attention neural network model to obtain a second search area output by the cross-attention neural network model;

[0018] Acquire a second point cloud covered by the second search area in the sample point cloud data;

[0019] Obtaining a first rotation matrix according to the features of each point in the second point cloud, where the first rotation matrix is obtained by rotating an initial posture of the second point cloud relative to a standard posture;

[0020] Obtaining a third point cloud in a standard posture according to the first rotation matrix and the second point cloud;

[0021] Performing a recovery operation on the third point cloud in the standard posture to obtain a fourth point cloud;

[0022] Adjust the parameters of the cross-attention neural network model according to the chamfer distance between the second point cloud and the fourth point cloud.

[0023] In one embodiment of the present application, the process of outputting the second search area by the cross-attention neural network model includes:

[0024] Selecting one of the change information from a plurality of preconfigured groups of change information, wherein the change information includes a search direction and a search size;

[0025] determining the second search area according to the search direction and search size included in the selected change information;

[0026] The second search area is output.

[0027] In one embodiment of the present application, performing a restoration operation on the third point cloud in the standard posture to obtain a fourth point cloud includes:

[0028] Inputting the third point cloud into an encoder of a variational autoencoder to obtain Gaussian distribution parameters output by the encoder, wherein the Gaussian distribution parameters include a mean and a standard deviation;

[0029] The Gaussian distribution parameters are input into the decoder of the variational autoencoder to obtain the fourth point cloud output by the decoder.

[0030] In one embodiment of the present application, the step of training a neural network model using the plurality of first training samples to obtain an object segmentation model includes:

[0031] In a process of training a neural network model using the plurality of first training samples, inputting point cloud data of the first training samples into the neural network model, and performing calculations based on the point cloud of the segmented object output by the neural network model and the point cloud of the reference object of the first training samples to obtain a loss value of the neural network model;

[0032] When the loss value is less than a preset loss value, the neural network model is used as the object segmentation model.

[0033] In a second aspect, an embodiment of the present application provides an object segmentation method, the method comprising:

[0034] Obtain the point cloud data to be segmented;

[0035] The point cloud data to be segmented is input into an object segmentation model to obtain an object segmentation result, wherein the object segmentation model is obtained based on the object segmentation model training method described in the first aspect.

[0036] In a third aspect, an embodiment of the present application provides a training device for an object segmentation model, the device comprising:

[0037] A first acquisition module is configured to acquire a plurality of point cloud data, each of which corresponds to a scene, and the scene includes at least one object;

[0038] A search module, configured to perform target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data;

[0039] A construction module, configured to construct a plurality of first training samples based on each of the point cloud data and the point cloud of the reference object in the point cloud data;

[0040] The training module is used to train the neural network model using the plurality of the first training samples to obtain an object segmentation model, wherein the object segmentation model is used to segment objects in the point cloud data to be segmented to obtain an object segmentation result.

[0041] In one embodiment of the present application, the search module includes a first determination submodule, a first acquisition submodule, and a second determination submodule;

[0042] The following processing is performed on each point cloud data:

[0043] a first determination submodule, configured to input a feature of each point in the point cloud data into a pre-trained cross-attention neural network model each time a target search operation is performed on the point cloud data, to obtain a first search area, wherein the feature of each point in the point cloud data is obtained by extracting features from the point cloud data;

[0044] A first acquisition submodule, configured to acquire a first point cloud covered by the first search area in the point cloud data;

[0045] The second determining submodule is configured to determine the first point cloud as the point cloud of the reference object in the point cloud data.

[0046] In one embodiment of the present application, the training device for the object segmentation model includes a second acquisition submodule, a third determination submodule, a third acquisition submodule, a processing submodule, and a training submodule;

[0047] A second acquisition submodule is used to acquire a plurality of second training samples, where the second training samples include sample point cloud data and features of each point in the sample point cloud data;

[0048] a third determination submodule, configured to input, in each round of training, the features of each point in the sample point cloud data in the second training sample into the cross-attention neural network model to obtain a second search area output by the cross-attention neural network model;

[0049] A third acquisition submodule, configured to acquire a second point cloud covered by the second search area in the sample point cloud data;

[0050] a processing submodule, configured to obtain a first rotation matrix based on the features of each point in the second point cloud, the first rotation matrix being obtained by rotating the initial pose of the second point cloud relative to the standard pose; obtain a third point cloud in the standard pose based on the first rotation matrix and the second point cloud; and perform a restoration operation on the third point cloud in the standard pose to obtain a fourth point cloud;

[0051] A training submodule is used to adjust the parameters of the cross-attention neural network model according to the chamfer distance between the second point cloud and the fourth point cloud.

[0052] In one embodiment of the present application, the processing submodule includes an encoding subunit and an encoding subunit;

[0053] an encoding subunit, configured to input the third point cloud into an encoder of a variational autoencoder to obtain Gaussian distribution parameters output by the encoder, wherein the Gaussian distribution parameters include a mean and a standard deviation;

[0054] An encoding subunit is used to input the Gaussian distribution parameters into a decoder of the variational autoencoder to obtain the fourth point cloud output by the decoder.

[0055] In one embodiment of the present application, the training module includes a calculation submodule and a fourth determination submodule;

[0056] a calculation submodule, configured to input the point cloud data of the first training samples into the neural network model during training of the neural network model using the plurality of the first training samples, and perform calculations based on the point cloud of the segmented object output by the neural network model and the point cloud of the reference object of the first training samples to obtain a loss value of the neural network model;

[0057] The fourth determination submodule is used to use the neural network model as the object segmentation model when the loss value is less than a preset loss value.

[0058] In a fourth aspect, an embodiment of the present application provides a training device for an object segmentation model, the device comprising:

[0059] The second acquisition module is used to obtain the point cloud data to be segmented;

[0060] The segmentation module is used to input the point cloud data to be segmented into an object segmentation model to obtain an object segmentation result. The object segmentation model is obtained based on the training method of the object segmentation model described in the first aspect.

[0061] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions;

[0062] When the processor executes the computer program instructions, the training method of the object segmentation model described in the first aspect or the object segmentation method described in the second aspect is implemented.

[0063] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the object segmentation model training method as described in the first aspect or the object segmentation method as described in the second aspect.

[0064] In the seventh aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the object segmentation model training method described in the first aspect or the object segmentation method described in the second aspect.

[0065] The object segmentation model training method, object segmentation method, device and electronic device of the embodiments of the present application obtain multiple point cloud data, wherein each point cloud data corresponds to a scene, and the scene includes at least one object. A target search is performed on each point cloud data to obtain a point cloud of a reference object in the point cloud data. A plurality of first training samples are constructed based on each point cloud data and the point cloud of the reference object in the point cloud data. The plurality of first training samples are used to train a neural network model to obtain an object segmentation model. The object segmentation model is used to segment the objects in the point cloud data to be segmented to obtain an object segmentation result. During the model training process, the point cloud of the reference object for training can be quickly obtained by performing a target search on the point cloud data without manual labeling, effectively saving time and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0067] Figure 1 1 is a flow chart of a method for training an object segmentation model provided in an embodiment of the present application;

[0068] Figure 2 This is a flow chart of the object segmentation method provided in an embodiment of the present application;

[0069] Figure 3 This is another flowchart of the object segmentation method provided in an embodiment of the present application;

[0070] Figure 4 This is a flow chart of the model training process provided by the embodiment of the present application;

[0071] Figure 5 Schematic diagram of the structure of the training device of the object segmentation model provided in the embodiment of the present application;

[0072] Figure 6 is a structural diagram of an object segmentation device provided in an embodiment of the present application;

[0073] Figure 7 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0075] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0076] In each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the disclosed embodiment will be obtained.

[0077] In order to solve the problems of the prior art, the embodiments of the present application provide a training method for an object segmentation model, an object segmentation method, an apparatus, and an electronic device. The training method for an object segmentation model provided in the embodiments of the present application is first introduced below.

[0078] Figure 1 FIG. 1 is a flow chart showing a method for training an object segmentation model according to an embodiment of the present application. Figure 1 As shown, the object segmentation model training method provided in the embodiment of the present application is applied to an electronic device and includes the following steps 101 to 104, wherein:

[0079] Step 101: Acquire a plurality of point cloud data, each of which corresponds to a scene, and the scene includes at least one object.

[0080] In this embodiment, the execution subject is an electronic device, which obtains multiple point cloud data, each point cloud data corresponds to a scene, and each scene includes at least one object, such as an object, a person, an animal, etc.

[0081] Step 102 : performing a target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data.

[0082] In this embodiment, each point cloud data includes a point cloud of at least one object, and a target search is performed on each point cloud data to obtain a point cloud of a reference object in the point cloud data, where the reference object is an object in the at least one object.

[0083] Step 103 : constructing a plurality of first training samples according to each of the point cloud data and the point cloud of the reference object in the point cloud data.

[0084] In this embodiment, a plurality of first training samples are obtained based on the features of each point cloud data and the point cloud of the reference object in each point cloud data. Each first training sample includes the point cloud data and the point cloud of the reference object in the point cloud data.

[0085] Step 104 : Use the plurality of the first training samples to train the neural network model to obtain an object segmentation model. The object segmentation model is used to segment objects in the point cloud data to be segmented to obtain an object segmentation result.

[0086] In this embodiment, a plurality of first training samples are used to train the neural network model, wherein the point cloud of the reference object in the point cloud data in the first training sample is annotated. For example, the point cloud data is input into the neural network model to obtain a point cloud of the segmented object output by the neural network model. According to the loss value calculated based on the point cloud of the segmented object and the annotation, that is, the point cloud of the reference object, the neural network model is trained according to the loss value. When the loss value meets the preset requirements, an object segmentation model is obtained. The object segmentation model is used to segment the object in the point cloud data to be segmented to obtain an object segmentation result. The object segmentation result includes the point cloud contained in each segmented object.

[0087] In this embodiment, a plurality of point cloud data are acquired, wherein each point cloud data corresponds to a scene, and the scene includes at least one object. A target search is performed on each point cloud data to obtain a point cloud of a reference object in the point cloud data. A plurality of first training samples are constructed based on each point cloud data and the point cloud of the reference object in the point cloud data. The plurality of first training samples are used to train a neural network model to obtain an object segmentation model. The object segmentation model is used to segment the objects in the point cloud data to be segmented to obtain an object segmentation result. During the model training process, the point cloud of the reference object for training can be quickly obtained by performing a target search on the point cloud data without manual labeling, thereby effectively saving time and labor costs.

[0088] In one embodiment of the present application, step 102, performing a target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data, includes:

[0089] The following processing is performed on each point cloud data:

[0090] Each time a target search operation is performed on the point cloud data, a feature of each point in the point cloud data is input into a pre-trained cross-attention neural network model to obtain a first search area, wherein the feature of each point in the point cloud data is obtained based on feature extraction of the point cloud data;

[0091] Acquire a first point cloud covered by the first search area in the point cloud data;

[0092] The first point cloud is determined as the point cloud of the reference object in the point cloud data.

[0093] In this embodiment, each point cloud data is processed as follows: each time a target search operation is performed on the point cloud data, the features of each point in the point cloud data are input into a pre-trained cross-attention neural network model to obtain a first search area, wherein the features of each point in the point cloud data are obtained based on feature extraction of the point cloud data, such as using a sparse convolutional network SparseConv or a point cloud network PointNet++ for feature extraction, obtaining a first point cloud covered by the first search area in the point cloud data, and determining the first point cloud as the point cloud of the reference object in the point cloud data. The point cloud of the reference object in each point cloud data is obtained in the above manner.

[0094] In this embodiment, a pre-trained neural network model is used to determine the search area, so that the point cloud covered by the search area is used as the point cloud of the reference object, and the target point cloud can be located according to the characteristics of the point cloud, and the target point cloud can be accurately identified and acquired.

[0095] In one embodiment of the present application, the cross-attention neural network model is obtained according to the following steps:

[0096] Acquire a plurality of second training samples, where the second training samples include sample point cloud data and features of each point in the sample point cloud data;

[0097] In each round of training, the feature of each point in the sample point cloud data in the second training sample is input into the cross-attention neural network model to obtain a second search area output by the cross-attention neural network model;

[0098] Acquire a second point cloud covered by the second search area in the sample point cloud data;

[0099] Obtaining a first rotation matrix according to the features of each point in the second point cloud, where the first rotation matrix is obtained by rotating an initial posture of the second point cloud relative to a standard posture;

[0100] Obtaining a third point cloud in a standard posture according to the first rotation matrix and the second point cloud;

[0101] Performing a recovery operation on the third point cloud in the standard posture to obtain a fourth point cloud;

[0102] Adjust the parameters of the cross-attention neural network model according to the chamfer distance between the second point cloud and the fourth point cloud.

[0103] In this embodiment, multiple second training samples are obtained, and the second training samples include sample point cloud data and features of each point in the sample point cloud data. The features of each point are obtained by extracting features from the sample point cloud data, such as using a sparse convolutional network SparseConv or a point cloud network PointNet++ for feature extraction.

[0104] In each round of training, the features of each point in the sample point cloud data in the second sample training are input into the cross-attention neural network model to obtain the second search area output by the cross-attention neural network model, and the second point cloud covered by the second search area in the sample point cloud data is obtained. At this time, the second point cloud is not necessarily in the standard posture, and it needs to be adjusted to the point cloud data of the standard posture. Specifically, the pre-trained PointNet++ and self-attention layer are used. PointNet++ is used to extract features from the point cloud data to obtain the features of each point in the point cloud data, or to extract features from the second point cloud to obtain the features of each point in the second point cloud; the features of each point in the second point cloud are input into the self-attention layer to obtain the first rotation matrix output by the self-attention layer. The first rotation matrix is obtained by rotating the initial posture of the second point cloud relative to the standard posture.

[0105] Furthermore, a third point cloud in a standard pose is obtained based on the first rotation matrix and the second point cloud. Specifically, for the coordinates of each point in the second point cloud, the coordinates of each point in the second point cloud are transformed using the first rotation matrix. That is, the coordinates of each point in the second point cloud are multiplied by the first rotation matrix to obtain a third point cloud in a standard pose. The coordinates of each point in the third point cloud are the transformed coordinates. The third point cloud may have some missing parts, which need to be predicted and completed. The third point cloud in a standard pose is restored to obtain a fourth point cloud.

[0106] Furthermore, based on the chamfer distance between the second point cloud and the fourth point cloud, the reward and punishment value is determined, the parameters of the cross-attention neural network model are adjusted according to the reward and punishment value, and the chamfer distance between the second point cloud and the fourth point cloud is compared with a preset threshold to determine the reward and punishment value, wherein the preset threshold can be set according to actual needs.

[0107] Chamfer distance is a metric used to measure the similarity or distance between two point clouds. For the second point cloud A={a1,a2,....a m} and the fourth point cloud B={b1,b2,...b m}, chamfer distance d from the second point cloud A to the fourth point cloud B AB is each point a in the second point cloud A i The average of the sum of the distances to the nearest point in the fourth point cloud B, the chamfer distance d from the fourth point cloud B to the second point cloud A BA is the average of the sum of the distances of each point bi in the fourth point cloud B to the nearest point in the second point cloud A. The chamfer distance d(A,B) between the second point cloud and the fourth point cloud is d AB with d BA The sum of d(A,B)=d AB +d BA .

[0108] The chamfer distance can be considered as a matching method. When comparing two point clouds, the distance between the points in one point cloud and the nearest point in the other point cloud is measured. In this way, the difference between the two point clouds is measured. If the two point clouds are similar, the chamfer distance between them will be smaller. Conversely, if the difference is large, the chamfer distance will be larger.

[0109] If the chamfer distance between the second point cloud and the fourth point cloud is less than a preset threshold, it means that the second point cloud covered by the second search area is basically consistent with the fourth point cloud obtained after the posture adjustment and recovery operation, and the model is rewarded to obtain a reward value; if the chamfer distance between the second point cloud and the fourth point cloud is greater than or equal to the preset threshold, it means that the second point cloud covered by the second search area is basically inconsistent with the fourth point cloud obtained after the posture adjustment and recovery operation, and may belong to multiple objects, and the model is penalized to obtain a penalty value, and the parameters of the cross-attention neural network model are adjusted according to the reward value or the penalty value to obtain a trained cross-attention neural network model.

[0110] In this embodiment, by pre-training the cross-attention neural network model and using the cross-attention neural network model to determine the search area, the target point cloud can be accurately identified and acquired.

[0111] In one embodiment of the present application, the process of outputting the second search area of the cross-attention neural network model includes:

[0112] Selecting one of the change information from a plurality of preconfigured groups of change information, wherein the change information includes a search direction and a search size;

[0113] determining the second search area according to the search direction and search size included in the selected change information;

[0114] The second search area is output.

[0115] In this embodiment, the process of outputting the second search area of the cross-attention neural network model is as follows: one change information is selected from multiple preconfigured groups of change information, the change information includes a search direction and a search size, the second search area is determined according to the search direction and search size included in the selected change information, and the cross-attention neural network model outputs the second search area.

[0116] In one implementation, the search area can be represented by a search box, a container, or other methods. For example, the container can be a cylinder. The search area of the cylinder is determined based on the search direction and search size included in the selected change information. The search direction is used to determine the direction of movement of the cylinder, and the search size is used to determine the size of the cylinder. For example, search directions include forward, backward, left, right, and empty. The search size includes multiple sizes, each of which is different. For example, the search direction of change information a can be selected from front, backward, left, right, or empty, with empty indicating maintaining the current position. The search size of change information a can be selected from multiple sizes, and at least one of the search directions and search sizes in any two change information differs. By changing the search direction and search size of the cylinder, the number of point clouds covered by the cylinder and the location of the covered point clouds are adjusted. The cross-attention neural network model outputs the search area of the cylinder.

[0117] In this embodiment, the direction and search size of the search area are adjusted according to the change information, so as to accurately locate the point cloud and obtain the required point cloud of the reference object.

[0118] In one embodiment of the present application, performing a restoration operation on the third point cloud in the standard posture to obtain a fourth point cloud includes:

[0119] Inputting the third point cloud into an encoder of a variational autoencoder to obtain Gaussian distribution parameters output by the encoder, wherein the Gaussian distribution parameters include a mean and a standard deviation;

[0120] The Gaussian distribution parameters are input into the decoder of the variational autoencoder to obtain the fourth point cloud output by the decoder.

[0121] In this embodiment, a pre-trained variational autoencoder (VAE) is used for restoration. The VAE consists of an encoder and a decoder. The third point cloud in the standard pose is input into the VAE encoder to obtain the Gaussian distribution parameters (mean and variance) output by the encoder. The Gaussian distribution parameters are then input into the VAE decoder to obtain the fourth point cloud output by the decoder. The above process completes the point cloud restoration process, resulting in the fourth point cloud.

[0122] Optionally, a diffusion model may be used, and the third point cloud may be input into the diffusion model to obtain a point cloud output by the diffusion model, and the restoration operation may be performed through the diffusion model.

[0123] In this embodiment, the variational autoencoder can learn the potential distribution of point cloud data, predict and complete the missing parts, fill in the missing parts in the point cloud, and make the point cloud data more complete.

[0124] In one embodiment of the present application, the step of training a neural network model using the plurality of first training samples to obtain an object segmentation model includes:

[0125] In a process of training a neural network model using the plurality of first training samples, inputting point cloud data of the first training samples into the neural network model, and performing calculations based on the point cloud of the segmented object output by the neural network model and the point cloud of the reference object of the first training samples to obtain a loss value of the neural network model;

[0126] When the loss value is less than a preset loss value, the neural network model is used as the object segmentation model.

[0127] In this embodiment, in the process of training the neural network model using multiple first training samples, the point cloud data of the first training sample is input into the neural network model, and the neural network model outputs the point cloud of the segmented object. Calculation is performed based on the point cloud of the segmented object and the point cloud of the reference object to obtain a loss value of the neural network model, and the loss value is compared with a preset loss value. When the loss value is less than the preset loss value, it means that the difference between the output of the model and the annotation is small, and the parameters of the model have been adjusted to a relatively optimal state, so that the model can better fit the data, and the neural network model is used as an object segmentation model; if the loss value is greater than or equal to the preset loss value, it means that the difference between the output of the model and the annotation is large, and the parameters of the model are updated so that the model is continuously optimized in the direction of minimizing the loss, thereby obtaining a set of parameters that can make the model performance better.

[0128] The loss value is a clear, quantifiable metric that can be used to evaluate and compare the performance of a model at different stages of training. By comparing the magnitude of the loss, you can intuitively understand the training status of the model.

[0129] Figure 2 FIG. 1 is a flow chart showing a method for training an object segmentation model according to an embodiment of the present application. Figure 2 As shown, the object segmentation model training method provided in the embodiment of the present application is applied to an electronic device and includes the following steps 201 to 203, wherein:

[0130] Step 201: Obtain point cloud data to be segmented.

[0131] In this embodiment, point cloud data to be segmented is obtained, and the execution subject of this application is applied to a visual perception scenario. For example, in an autonomous driving scenario, an electronic device is set in the vehicle, and the electronic device obtains the point cloud data to be segmented from the point cloud acquisition device; or, in a robot working scenario, the robot is set with an electronic device, and the electronic device obtains the point cloud data to be segmented.

[0132] In step 202 , the point cloud data to be segmented is input into an object segmentation model to obtain an object segmentation result. The object segmentation model is obtained based on a training method of an object segmentation model.

[0133] In this embodiment, the point cloud data to be segmented is input into the object segmentation model. The object segmentation model is divided into a feature extraction part and a segmentation part. The feature extraction part can be implemented by a sparse convolutional network. The point cloud data to be segmented is input into the sparse convolutional network. The sparse convolutional network is used to extract features. The sparse convolutional network outputs the features of each point in the point cloud data. The segmentation part can be implemented by a cross-attention neural network model. The point cloud data to be segmented is input into the cross-attention neural network model. The cross-attention neural network model is used to perform segmentation. The cross-attention neural network model outputs the object segmentation result. The object segmentation result includes the point cloud contained in each segmented object.

[0134] In this embodiment, an object segmentation model is used to segment the point cloud data to be segmented, which can obtain a relatively accurate object segmentation result and meet the segmentation requirements of different scenes.

[0135] like Figure 3 As shown, the object segmentation method provided in the embodiment of the present application is illustrated below.

[0136] Step 301: Train the object-centric network to obtain a trained object-centric network.

[0137] In this embodiment, the object can be an object, a person, an animal, etc. In this embodiment, an object is taken as an example to illustrate that multiple object point clouds are obtained. The multiple object point clouds are obtained from the scene point cloud, which can be the same scene or different scenes. The scene includes at least one object.

[0138] First, the object-centric network is trained. The object-centric network includes an object direction estimation module and a generative object prior module. The object direction estimation module is used to extract features from the point cloud and adjust the posture. Specifically, the object direction estimation module uses the point cloud network PointNet++ to extract features from the object point cloud, inputs the object point cloud into PointNet++, and obtains the features of each point in the object point cloud output by PointNet++; the object direction estimation module uses the self-attention layer to adjust the posture, inputs the features of each point in the object point cloud into the self-attention layer, and obtains the rotation matrix output by the self-attention layer. The rotation matrix is obtained by rotating the initial posture of the object point cloud relative to the standard posture.

[0139] Furthermore, the object direction estimation module obtains the object point cloud of the standard posture according to the rotation matrix and the object point cloud.

[0140] The generative object prior module is used to perform the restoration operation. The generative object prior module can use a variational autoencoder or a diffusion model for the restoration operation. Taking the variational autoencoder as an example, the variational autoencoder is pre-trained. The variational autoencoder consists of an encoder and a decoder. The object point cloud in a standard pose is input into the encoder of the variational autoencoder to obtain the Gaussian distribution parameters of the encoder output, including the mean and variance; the Gaussian distribution parameters are input into the decoder of the variational autoencoder to obtain the decoded object point cloud output by the decoder. Based on the chamfer distance between the input object point cloud in the standard pose and the output decoded object point cloud, the parameters of the variational autoencoder are adjusted so that the variational autoencoder learns a stable and continuous latent distribution, thereby obtaining a trained variational autoencoder. The above training mainly trains the generative object prior module, and the trained object-centric network is obtained through training.

[0141] Step 302: Train the first cross-attention neural network model to obtain a trained first cross-attention neural network model.

[0142] In this embodiment, scene point cloud data of each scene is obtained based on multiple scenes, and each scene includes at least one object. Figure 4 , P∈R N*3 Representing scene point cloud data, performing object search in the scene point cloud, realizing object search through the trained first cross-attention neural network model, and using the trained object-centric network to assist the first cross-attention neural network model in training.

[0143] Specifically, a plurality of training samples are obtained, wherein the training samples include scene point cloud data and features of each point in the scene point cloud data (i.e., a plurality of second training samples are obtained above, wherein the second training samples include sample point cloud data and features of each point in the sample point cloud data), and the features of each point are extracted using a sparse convolutional neural network, see Figure 4 , F∈R N*128 Represents the features of each point in the scene point cloud data.

[0144] Set a dynamic container in the scene point cloud, such as a cylinder. In each round of training, obtain the first point cloud covered by the initial search area of the cylinder in the scene point cloud data, such as Figure 4 As shown, P 0 c ∈R K*3 represents the first point cloud covered by the cylinder, F 0 c ∈R K*3Representing the features of each point in the first point cloud covered by the cylinder; inputting the features of each point in the first point cloud covered by the initial search area into the first cross-attention neural network model to obtain the first search area of the cylinder output by the first cross-attention neural network model. Obtaining a second point cloud covered by the first search area of the cylinder in the scene point cloud data, using the trained object-centric network to obtain a rotation matrix based on the features of each point in the second point cloud, and obtaining a third point cloud in a standard pose based on the rotation matrix and the second point cloud; and restoring the third point cloud in the standard pose using the trained object-centric network to obtain a fourth point cloud.

[0145] Select one change information from multiple sets of change information, the change information includes the search direction and search size, such as Figure 4 As shown, Indicates the search direction, Indicates the search size. For example, the search direction includes forward, backward, left, right, and empty. The change information also includes the coordinate information corresponding to the search direction. The search size includes multiple sizes, and each size is different. For example, the search direction of change information a can be selected from the five options of forward, backward, left, right, and empty. Empty means maintaining the current position. The search size of change information a can be selected from multiple sizes, such as selecting from the three options of increasing, decreasing, and unchanged. Unchanged means maintaining the current size. At least one of the search direction and search size in any two change information is different. Figure 4 , the selected change information moves to the right and becomes smaller, and the state t0 of the cylinder changes to t1. Figure 4 The red transparent part in the middle right part represents the state of the cylinder before the change, and the yellow part represents the state of the cylinder after the change. It can be seen that the area covered by the front cylinder before the change is different from the area covered by the cylinder after the change, that is, the number and position of point clouds covered by the search area before and after the change are different.

[0146] Furthermore, according to the chamfer distance between the second point cloud and the fourth point cloud, the parameters of the cross-attention neural network model are adjusted to obtain a trained first cross-attention neural network model.

[0147] Alternatively, in each round of training, the features of each point in the scene point cloud data are input into the first cross-attention neural network model to obtain the search area output by the first cross-attention neural network model (that is, the features of each point in the sample point cloud data in the second training sample are input into the cross-attention neural network model to obtain the second search area output by the cross-attention neural network model).

[0148] Obtain a point cloud covered by the search area in the scene point cloud (i.e., obtain the second point cloud covered by the second search area in the sample point cloud data above), use an object-centric network to obtain a rotation matrix based on the features of each point in the point cloud covered by the search area (i.e., obtain the first rotation matrix based on the features of each point in the second point cloud above), and use an object-centric network to obtain a point cloud in a standard posture based on the rotation matrix and the point cloud covered by the search area (i.e., obtain a third point cloud in a standard posture based on the first rotation matrix and the second point cloud above).

[0149] The object-centric network performs a restoration operation on the point cloud in the standard posture to obtain a restored point cloud (i.e., the third point cloud in the standard posture is restored to obtain the fourth point cloud in the above text). The restoration operation is implemented using a generative object prior module with an object-centric network. The generative object prior module is used to input the point cloud in the standard posture into the encoder of the variational autoencoder to obtain the Gaussian distribution parameters output by the encoder, and the Gaussian distribution parameters include the mean and the standard deviation; the generative object prior module is used to input the Gaussian distribution parameters into the decoder of the variational autoencoder to obtain the restored point cloud output by the decoder (i.e., the third point cloud is input into the encoder of the variational autoencoder to obtain the Gaussian distribution parameters output by the encoder, and the Gaussian distribution parameters include the mean and the standard deviation; the Gaussian distribution parameters are input into the decoder of the variational autoencoder to obtain the fourth point cloud output by the decoder).

[0150] Furthermore, the parameters of the first cross-attention neural network model are adjusted according to the chamfer distance between the point cloud covered by the search area and the restored point cloud (i.e., the parameters of the cross-attention neural network model are adjusted according to the chamfer distance between the second point cloud and the fourth point cloud in the above text) to obtain the trained first cross-attention neural network model.

[0151] Step 303: Acquire a plurality of point cloud data, each of which corresponds to a scene, and the scene includes at least one object.

[0152] In this embodiment, multiple point cloud data are acquired, each point cloud data corresponds to a scene, and each scene includes at least one object (that is, multiple point cloud data are acquired above, each point cloud data corresponds to a scene, and the scene includes at least one object).

[0153] In step 304 , a target search is performed on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data. The target search adopts the trained first cross-attention neural network model.

[0154] In this embodiment, each point cloud data includes a point cloud of at least one object, and a target search is performed on each point cloud data to obtain a point cloud of a reference object in the point cloud data (i.e., a target search is performed on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data as described above), and the object object is an object in at least one object.

[0155] Specifically, performing a target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data includes:

[0156] The following processing is performed on each point cloud data:

[0157] Each time a target search operation is performed on the point cloud data, a feature of each point in the point cloud data is input into a first cross-attention neural network model (i.e., the pre-trained cross-attention neural network model mentioned above) to obtain a first search area, wherein the feature of each point in the point cloud data is obtained based on feature extraction of the point cloud data;

[0158] Acquire a first point cloud covered by the first search area in the point cloud data;

[0159] The first point cloud is determined as the point cloud of the reference object in the point cloud data (ie, the first point cloud is determined as the point cloud of the reference object in the point cloud data above).

[0160] Step 305 : construct a plurality of first training samples based on each of the point cloud data and the point cloud of the reference object in the point cloud data.

[0161] In this embodiment, multiple first training samples are constructed based on the features of each point cloud data and the point cloud of a reference object within each point cloud data (i.e., the multiple first training samples are constructed based on each point cloud data and the point cloud of the reference object within the point cloud data). Each first training sample includes the point cloud data and the point cloud of the reference object within the point cloud data. In some application scenarios requiring real-time processing of three-dimensional scenes, such as autonomous driving and augmented reality, it is difficult to quickly obtain large amounts of accurately annotated data. The method of this embodiment enables rapid acquisition of annotated data.

[0162] Step 306: Use the plurality of first training sample pairs to train the neural network model to obtain an object segmentation model.

[0163] like Figure 4As shown, multiple first training samples can be stored in the form of pseudo masks to facilitate data query, retrieval and management. Multiple first training samples are used to train the neural network model, wherein the point cloud of the reference object in the point cloud data of the first training samples is annotated. For example, the point cloud data is input into the neural network model to obtain the point cloud of the object segmented by the neural network model. The neural network model is trained based on the loss value calculated based on the point cloud of the segmented object and the annotation, i.e., the point cloud of the reference object. When the loss value meets the preset requirements, an object segmentation model is obtained.

[0164] Step 307: Obtain the point cloud data to be segmented.

[0165] In this embodiment, the point cloud data to be segmented is obtained, which may be the point cloud data to be segmented in an autonomous driving scenario, or the point cloud data to be segmented in a robot working scenario.

[0166] Step 308: Input the point cloud data to be segmented into an object segmentation model to obtain an object segmentation result.

[0167] In this embodiment, the point cloud data to be segmented is input into the object segmentation model (i.e., the point cloud data to be segmented is input into the object segmentation model to obtain the object segmentation result). The object segmentation model is divided into a feature extraction part and a segmentation part. The feature extraction part can be implemented by a sparse convolutional network, and the point cloud data to be segmented is input into the sparse convolutional network, and the sparse convolutional network is used to extract features, and the sparse convolutional network outputs the features of each point in the point cloud data; the segmentation part can be implemented by a cross-attention neural network model, and the point cloud data to be segmented is input into a second cross-attention neural network model (i.e., the segmentation part can be implemented by a cross-attention neural network model, and the point cloud data to be segmented is input into the cross-attention neural network model), and the second cross-attention neural network model is used for segmentation, and the second cross-attention neural network model outputs the object segmentation result (i.e., the cross-attention neural network model is used for segmentation, and the cross-attention neural network model outputs the object segmentation result). The object segmentation result includes the point cloud contained in each segmented object (i.e., the object segmentation result includes the point cloud contained in each segmented object).

[0168] In the above process, there is no need to rely on a large amount of manually labeled data of three-dimensional scenes, which greatly reduces the cost and time of data collection, allowing three-dimensional object segmentation technology to be deployed and used more quickly in practical applications, significantly improving the versatility and practicality of the method. By learning generative and discriminative object prior knowledge and adopting the object search method described above, it is possible to accurately identify and segment a variety of complex objects in complex three-dimensional point cloud data, with performance far exceeding that of existing unsupervised methods. The variational autoencoder framework, diffusion model, and dual-branch structure of the object search network adopted by the object-centric network give the method of the present invention strong flexibility and scalability. According to different application scenarios and requirements, appropriate model variants and parameter settings can be flexibly selected to achieve the best segmentation effect. In the inference stage, once the object discovery branch training is completed, that is, the above-mentioned segmentation model, only the segmentation model is needed to segment the object on the point cloud, which greatly improves the inference efficiency and significantly reduces the time required for inference. Moreover, the existing three-dimensional object segmentation relies on a large amount of manually labeled data and the unsupervised method is insufficient for complex object segmentation. The embodiment of the present application can accurately identify complex objects in complex three-dimensional point cloud data and does not require human labels for supervision of three-dimensional scenes. It is a general solution.

[0169] Figure 5 A structural diagram of a training device for an object segmentation model provided in an embodiment of the present application is shown.

[0170] like Figure 5 As shown, the training device 500 for the object segmentation model includes:

[0171] A first acquisition module 501 is configured to acquire a plurality of point cloud data, each of which corresponds to a scene including at least one object;

[0172] A search module 502 is configured to perform a target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data;

[0173] A construction module 503 is configured to construct a plurality of first training samples based on each of the point cloud data and the point cloud of the reference object in the point cloud data;

[0174] The training module 504 is used to train the neural network model using the plurality of the first training samples to obtain an object segmentation model. The object segmentation model is used to segment objects in the point cloud data to be segmented to obtain an object segmentation result.

[0175] In one embodiment of the present application, the search module includes a first determination submodule, a first acquisition submodule, and a second determination submodule;

[0176] The following processing is performed on each point cloud data:

[0177] a first determination submodule, configured to input a feature of each point in the point cloud data into a pre-trained cross-attention neural network model each time a target search operation is performed on the point cloud data, to obtain a first search area, wherein the feature of each point in the point cloud data is obtained by extracting features from the point cloud data;

[0178] A first acquisition submodule, configured to acquire a first point cloud covered by the first search area in the point cloud data;

[0179] The second determining submodule is configured to determine the first point cloud as the point cloud of the reference object in the point cloud data.

[0180] In one embodiment of the present application, the training device for the object segmentation model includes a second acquisition submodule, a third determination submodule, a third acquisition submodule, a processing submodule, and a training submodule;

[0181] A second acquisition submodule is used to acquire a plurality of second training samples, where the second training samples include sample point cloud data and features of each point in the sample point cloud data;

[0182] a third determination submodule, configured to input, in each round of training, the features of each point in the sample point cloud data in the second training sample into the cross-attention neural network model to obtain a second search area output by the cross-attention neural network model;

[0183] A third acquisition submodule, configured to acquire a second point cloud covered by the second search area in the sample point cloud data;

[0184] a processing submodule, configured to obtain a first rotation matrix based on the features of each point in the second point cloud, the first rotation matrix being obtained by rotating the initial pose of the second point cloud relative to the standard pose; obtain a third point cloud in the standard pose based on the first rotation matrix and the second point cloud; and perform a restoration operation on the third point cloud in the standard pose to obtain a fourth point cloud;

[0185] A training submodule is used to adjust the parameters of the cross-attention neural network model according to the chamfer distance between the second point cloud and the fourth point cloud.

[0186] In one embodiment of the present application, the processing submodule includes an encoding subunit and an encoding subunit;

[0187] an encoding subunit, configured to input the third point cloud into an encoder of a variational autoencoder to obtain Gaussian distribution parameters output by the encoder, wherein the Gaussian distribution parameters include a mean and a standard deviation;

[0188] An encoding subunit is used to input the Gaussian distribution parameters into a decoder of the variational autoencoder to obtain the fourth point cloud output by the decoder.

[0189] In one embodiment of the present application, the training module includes a calculation submodule and a fourth determination submodule;

[0190] a calculation submodule, configured to input the point cloud data of the first training samples into the neural network model during training of the neural network model using the plurality of the first training samples, and perform calculations based on the point cloud of the segmented object output by the neural network model and the point cloud of the reference object of the first training samples to obtain a loss value of the neural network model;

[0191] The fourth determination submodule is used to use the neural network model as the object segmentation model when the loss value is less than a preset loss value.

[0192] The object segmentation model training device provided in the embodiment of the present application can implement the various processes implemented in the aforementioned object segmentation model training method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0193] Figure 6 FIG. 1 shows a structural diagram of an object segmentation device provided in an embodiment of the present application. Figure 6 As shown, the object segmentation device 600 includes: a second acquisition module 601 and a segmentation module 602;

[0194] The second acquisition module 601 is used to acquire point cloud data to be segmented;

[0195] The segmentation module 602 is used to input the point cloud data to be segmented into an object segmentation model to obtain an object segmentation result. The object segmentation model is obtained based on a training method of the object segmentation model.

[0196] The object segmentation device provided in the embodiment of the present application can implement each process implemented in the aforementioned object segmentation method embodiment and achieve the same technical effect. To avoid repetition, it will not be described here.

[0197] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0198] The electronic device may include a processor 701 and a memory 702 storing computer program instructions.

[0199] Specifically, the processor 701 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0200] The memory 702 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 702 may include removable or non-removable (or fixed) media. Where appropriate, the memory 702 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 702 is a non-volatile solid-state memory.

[0201] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect or the second aspect of the present disclosure.

[0202] The processor 701 reads and executes computer program instructions stored in the memory 702 to implement any one of the object segmentation model training methods or object segmentation methods in the above embodiments.

[0203] In one example, the electronic device may further include a communication interface 703 and a bus 710. Figure 7 As shown, the processor 701, the memory 702, and the communication interface 703 are connected via a bus 710 and communicate with each other.

[0204] The communication interface 703 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0205] Bus 710 includes hardware, software or both, and the training method of object segmentation model or the parts of object segmentation method or electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0206] In addition, embodiments of the present application may be implemented by providing a computer storage medium having computer program instructions stored thereon; when the computer program instructions are executed by a processor, any one of the object segmentation model training methods or object segmentation methods described in the above embodiments may be implemented.

[0207] In addition, an embodiment of the present application may be implemented by providing a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device implements any object segmentation model training method or object segmentation method in the above embodiments.

[0208] It should be understood that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described as examples. However, the method process of the present application is not limited to the specific steps described. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0209] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0210] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0211] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0212] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A method for training an object segmentation model, characterized in that: The method comprises: Acquire a plurality of point cloud data, each of the point cloud data corresponds to a scene, and the scene includes at least one object; Performing target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data; Constructing a plurality of first training samples according to each of the point cloud data and the point cloud of the reference object in the point cloud data; The neural network model is trained using the plurality of the first training samples to obtain an object segmentation model, and the object segmentation model is used to segment objects in the point cloud data to be segmented to obtain an object segmentation result.

2. The object segmentation model training method according to claim 1, characterized in that: The performing target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data includes: The following processing is performed on each point cloud data: Each time a target search operation is performed on the point cloud data, a feature of each point in the point cloud data is input into a pre-trained cross-attention neural network model to obtain a first search area, wherein the feature of each point in the point cloud data is obtained based on feature extraction of the point cloud data; Acquire a first point cloud covered by the first search area in the point cloud data; The first point cloud is determined as the point cloud of the reference object in the point cloud data.

3. The object segmentation model training method according to claim 2, characterized in that: The cross-attention neural network model is obtained according to the following steps: Acquire a plurality of second training samples, where the second training samples include sample point cloud data and features of each point in the sample point cloud data; In each round of training, the feature of each point in the sample point cloud data in the second training sample is input into the cross-attention neural network model to obtain a second search area output by the cross-attention neural network model; Acquire a second point cloud covered by the second search area in the sample point cloud data; Obtaining a first rotation matrix according to the features of each point in the second point cloud, where the first rotation matrix is obtained by rotating an initial posture of the second point cloud relative to a standard posture; Obtaining a third point cloud in a standard posture according to the first rotation matrix and the second point cloud; Performing a recovery operation on the third point cloud in the standard posture to obtain a fourth point cloud; Adjust the parameters of the cross-attention neural network model according to the chamfer distance between the second point cloud and the fourth point cloud.

4. The object segmentation model training method according to claim 3, characterized in that: The process of outputting the second search area of the cross-attention neural network model includes: Selecting one of the change information from a plurality of preconfigured groups of change information, wherein the change information includes a search direction and a search size; determining the second search area according to the search direction and search size included in the selected change information; The second search area is output.

5. The object segmentation model training method according to claim 3, characterized in that: The performing a restoration operation on the third point cloud in the standard posture to obtain a fourth point cloud includes: Inputting the third point cloud into an encoder of a variational autoencoder to obtain Gaussian distribution parameters output by the encoder, wherein the Gaussian distribution parameters include a mean and a standard deviation; The Gaussian distribution parameters are input into the decoder of the variational autoencoder to obtain the fourth point cloud output by the decoder.

6. The object segmentation model training method according to any one of claims 1 to 5, characterized in that: The step of training a neural network model using the plurality of first training samples to obtain an object segmentation model includes: In a process of training a neural network model using the plurality of first training samples, inputting point cloud data of the first training samples into the neural network model, and performing calculations based on the point cloud of the segmented object output by the neural network model and the point cloud of the reference object of the first training samples to obtain a loss value of the neural network model; When the loss value is less than a preset loss value, the neural network model is used as the object segmentation model.

7. An object segmentation method, characterized in that: The method comprises: Obtain the point cloud data to be segmented; The point cloud data to be segmented is input into an object segmentation model to obtain an object segmentation result, wherein the object segmentation model is obtained based on the training method of the object segmentation model described in any one of claims 1 to 6.

8. A training device for an object segmentation model, characterized in that: The device comprises: A first acquisition module is configured to acquire a plurality of point cloud data, each of which corresponds to a scene, and the scene includes at least one object; A search module, configured to perform target search on each of the point cloud data to obtain a point cloud of a reference object in the point cloud data; A construction module, configured to construct a plurality of first training samples based on each of the point cloud data and the point cloud of the reference object in the point cloud data; The training module is used to train the neural network model using the plurality of the first training samples to obtain an object segmentation model, wherein the object segmentation model is used to segment objects in the point cloud data to be segmented to obtain an object segmentation result.

9. An object segmentation device, characterized in that: The device comprises: The second acquisition module is used to obtain the point cloud data to be segmented; A segmentation module is used to input the point cloud data to be segmented into an object segmentation model to obtain an object segmentation result, wherein the object segmentation model is obtained based on the training method of the object segmentation model described in any one of claims 1 to 6.

10. An electronic device, characterized in that: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the object segmentation model training method according to any one of claims 1 to 6 or the object segmentation method according to claim 7.

Images