Point cloud target detection model pre-training method and device, equipment and medium

Abstract

The invention discloses a point cloud target detection model pre-training method and device, equipment and a medium, and relates to the field of point cloud target detection, and the method comprises the steps: employing a clustering algorithm and a bounding box fitting algorithm to carry out the processing of unmarked point cloud data; segmenting the background and the target of the processed point cloud data; multiple frames of point cloud backgrounds and multiple types of point cloud instances obtained after segmentation are used as pre-training data, and an object-level self-supervision pre-training method is adopted to pre-train a point cloud target detection model; the pre-training comprises object-level instance discriminant comparison and bounding box geometric comparison prediction; performing supervised fine tuning on the pre-trained point cloud target detection model by using the marked point cloud data to obtain a trained point cloud target detection model; the trained point cloud target detection model is used for outputting the point cloud target in the to-be-detected scene when the point cloud data of the to-be-detected scene is input, the accuracy and robustness of the point cloud target detection model can be improved, and meanwhile the data labeling cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of point cloud target detection, and in particular to a point cloud target detection model pre-training method, device, equipment and medium. Background Art

[0002] Point cloud data is usually generated by LiDAR and contains the three-dimensional spatial coordinate information of objects. Point cloud target detection methods can accurately detect and identify target objects in point cloud data and are widely used in the fields of autonomous driving, robot navigation, etc. The current problems in the field of point cloud target detection are: first, the cost of manual annotation of point cloud data is high; second, the long-tail problem is significant, resulting in low accuracy and robustness of point cloud target detection models. Summary of the invention

[0003] The purpose of this application is to provide a point cloud target detection model pre-training method, device, equipment and medium, which can improve the accuracy and robustness of the point cloud target detection model while reducing data annotation costs.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a point cloud target detection model pre-training method, comprising:

[0006] Obtain unlabeled point cloud data;

[0007] Processing the unlabeled point cloud data using a clustering algorithm and a bounding box fitting algorithm to obtain processed point cloud data;

[0008] Segmenting the background and target of the processed point cloud data to obtain multi-frame point cloud background and multi-class point cloud instances; the point cloud instance includes a target category and a target bounding box;

[0009] Using multi-frame point cloud backgrounds and multi-class point cloud instances as pre-training data, and adopting an object-level self-supervised pre-training method to pre-train the point cloud target detection model, thereby obtaining a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction;

[0010] The pre-trained point cloud target detection model is fine-tuned in a supervised manner using the labeled point cloud data to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used as the input of the point cloud data of the scene to be detected, the point cloud target in the scene to be detected is output.

[0011] In a second aspect, the present application provides a point cloud target detection model pre-training device, comprising:

[0012] Point cloud data acquisition module, used to acquire unlabeled point cloud data;

[0013] A point cloud data processing module, used to process the unlabeled point cloud data using a clustering algorithm and a bounding box fitting algorithm to obtain processed point cloud data;

[0014] A point cloud data segmentation module is used to segment the background and target of the processed point cloud data to obtain multi-frame point cloud background and multi-class point cloud instances; the point cloud instance includes a target category and a target bounding box;

[0015] A pre-training module is used to use multi-frame point cloud backgrounds and multi-class point cloud instances as pre-training data, and adopt an object-level self-supervised pre-training method to pre-train the point cloud target detection model to obtain a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction;

[0016] The model fine-tuning module is used to perform supervised fine-tuning on the pre-trained point cloud target detection model using the labeled point cloud data to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used as the input of the point cloud data of the scene to be detected, the point cloud target in the scene to be detected is output.

[0017] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described point cloud target detection model pre-training methods.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described point cloud target detection model pre-training methods.

[0019] According to the specific embodiments provided in this application, this application has the following technical effects:

[0020] The present application provides a point cloud target detection model pre-training method, device, equipment and medium, which uses a clustering algorithm and a bounding box fitting algorithm to process unlabeled point cloud data to obtain a point cloud background and a point cloud instance, uses the point cloud background and the point cloud instance as pre-training data, and uses an object-level self-supervised pre-training method to pre-train the point cloud target detection model. The pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction, and uses the labeled point cloud data to perform supervised fine-tuning on the pre-trained point cloud target detection model to obtain a finally trained point cloud target detection model. The point cloud target detection model directly uses massive unlabeled point cloud data for pre-training, which saves costs while improving the generalization of the network model. The present application uses an object-level self-supervised pre-training method to help the model learn effective feature representations without a large amount of labeled data, which can improve the accuracy and robustness of the point cloud target detection model and reduce the data labeling cost. When it is necessary to detect the point cloud data of the scene to be detected, the trained point cloud target detection model can be used to achieve accurate detection of targets in the point cloud data of the scene to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 This is an application environment diagram of a point cloud target detection model pre-training method in one embodiment of the present application;

[0023] Figure 2 A schematic diagram of a process for pre-training a point cloud target detection model provided in one embodiment of the present application;

[0024] Figure 3 A flowchart of a method for determining a trained point cloud target detection model provided in an embodiment of the present application;

[0025] Figure 4 A block diagram of a complete training process of a point cloud object detection model provided in one embodiment of the present application;

[0026] Figure 5 A schematic diagram of functional modules of a point cloud object detection model pre-training device provided in another embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0030] In view of the current defects of high cost and significant long-tail problem of manual labeling of point cloud data, self-supervised pre-training can be used to reduce the amount of data labeling and directly use massive unlabeled data for pre-training, which saves costs and improves the generalization of network models.

[0031] Self-supervised learning uses the structural information of the original data as a supervisory signal by designing specific pretext tasks, and can train network models without manually annotated data. In point cloud object detection, self-supervised learning can help the model learn effective feature representations without a large amount of annotated data, thereby improving the accuracy and robustness of object detection and reducing data annotation costs.

[0032] Self-supervised learning originated from image classification tasks, and there is a gap in modality and tasks with the point cloud target detection task used in this application. In the field of images, self-supervised learning has achieved very good results, such as SimCLR in classification tasks, and MoCo in classification and migration to detection tasks. However, most methods compare overall features, which are suitable for classification tasks, while more sophisticated feature comparison methods are required for detection tasks. Since detection tasks are more challenging than classification tasks, in the field of images, some methods have begun to adopt instance-level feature comparison methods, such as InsLoc. In the field of point clouds, self-supervised pre-training research has gradually become a hot topic. Methods such as PointContrast are more suitable for classification tasks from an overall comparison, and other methods such as BEVContrast are more suitable for semantic segmentation tasks from a feature point level comparison. However, there is no instance-level point cloud comparison learning method. The advantage of the instance-level method is that the features of the point cloud target instance are directly compared, which can prompt the model to directly learn instance-level features and is more compatible with target detection tasks. Therefore, this application proposes an instance-level self-supervised point cloud target detection scheme, designs a corresponding contrast loss function, and provides a point cloud instance acquisition tool to assist in preprocessing a large amount of unlabeled data.

[0033] The point cloud target detection model pre-training method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the unlabeled point cloud data to the server 104. After receiving the unlabeled point cloud data, the server 104 uses a clustering algorithm and a bounding box fitting algorithm to process the unlabeled point cloud data to obtain processed point cloud data; the background and target of the processed point cloud data are segmented to obtain a multi-frame point cloud background and a multi-class point cloud instance; the point cloud instance includes a target category and a target bounding box; the multi-frame point cloud background and the multi-class point cloud instance are used as pre-training data, and the point cloud target detection model is pre-trained using an object-level self-supervised pre-training method to obtain a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction; the pre-trained point cloud target detection model is supervised fine-tuned using the labeled point cloud data to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used for the point cloud data input of the scene to be detected, the point cloud target in the scene to be detected is output.

[0034] The server 104 can feed back the obtained trained point cloud target detection model to the terminal 102. In addition, in some embodiments, the point cloud target detection model pre-training method can also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 can directly process the unlabeled point cloud data, or the server 104 can obtain the unlabeled point cloud data from the data storage system and process the unlabeled point cloud data.

[0035] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0036] In an exemplary embodiment, Figure 2 As shown, a point cloud target detection model pre-training method is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 201 to 205. Among them:

[0037] Step 201, obtaining unlabeled point cloud data.

[0038] Step 202: Process the unlabeled point cloud data using a clustering algorithm and a bounding box fitting algorithm to obtain processed point cloud data.

[0039] Step 203 , segmenting the background and target of the processed point cloud data to obtain multi-frame point cloud background and multi-class point cloud instances; the point cloud instance includes a target category and a target bounding box.

[0040] In step 204, the multi-frame point cloud background and multi-class point cloud instances are used as pre-training data, and the point cloud target detection model is pre-trained using an object-level self-supervised pre-training method to obtain a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction.

[0041] Step 205, using the labeled point cloud data to perform supervised fine-tuning on the pre-trained point cloud target detection model to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used for inputting the point cloud data of the scene to be detected, the point cloud target in the scene to be detected is output.

[0042] The instance discrimination in this embodiment is a method of contrastive learning. The instance used for instance discrimination in this embodiment is at the object level, which is different from the point level, voxel level, etc. in the existing methods. The implementation of the above steps 201 to 205 realizes the pre-training of the object-level self-supervised point cloud target detection model, which can improve the accuracy and robustness of the point cloud target detection model and reduce the data annotation cost.

[0043] In another exemplary embodiment of the present application, Figure 3 As shown, step 204 includes:

[0044] Step 301 , randomly read two frames of point cloud background and t×n point cloud instances.

[0045] Among them, the point cloud backgrounds of the two frames are the first point cloud background and the second point cloud background respectively; t represents the number of target categories; and n represents the number of point cloud instances of each target category.

[0046] Step 302: for any read point cloud instance, perform instance enhancement and contrast enhancement on the point cloud instance to construct a first enhanced instance and a second enhanced instance of the point cloud instance.

[0047] The instance enhancement includes: upsampling, downsampling, random shearing and adding noise. The contrast enhancement includes: rotation and scaling. The first enhanced instance and the second enhanced instance have different sizes and angles.

[0048] Step 303: randomly generate t×n positions within the effective range of the point cloud scene, place the first enhanced instance in sequence at the corresponding t×n positions in the first point cloud background to obtain a first synthetic point cloud, and place the second enhanced instance in sequence at the corresponding t×n positions in the second point cloud background to obtain a second synthetic point cloud.

[0049] Step 304: input the first synthetic point cloud into a first point cloud target detection model, input the second synthetic point cloud into a second point cloud target detection model, train with the goal of minimizing the total loss, and determine the trained first point cloud target detection model or the trained second point cloud target detection model as a pre-trained point cloud target detection model.

[0050] Among them, the first point cloud target detection model and the first point cloud target detection model have the same structure, and the first point cloud target detection model and the first point cloud target detection model can be any commonly used models, including: Ancher_Based, Ancher_free, Query_Based and other types, such as PointPillars, Centerpoint, DETR3D, etc.

[0051] During the training process, the first point cloud target detection model and the second point cloud target detection model share parameters.

[0052] The total loss is determined based on the category features, size features, and angle features output by the first point cloud target detection model and the size features and angle features output by the second point cloud target detection model.

[0053] The total loss includes: object-level instance discrimination contrast loss and bounding box geometry contrast prediction loss. The bounding box geometry contrast prediction loss includes: scale loss and angle loss. The object-level instance discrimination contrast loss is determined according to the category features output by the first point cloud target detection model. The scale loss is determined according to the size features output by the first point cloud target detection model and the size features output by the second point cloud target detection model. The angle loss is determined according to the angle features output by the first point cloud target detection model and the angle features output by the second point cloud target detection model.

[0054] The expression of the object-level instance discriminant contrast loss is:

[0055]

[0056] in, represents the object-level instance discrimination contrast loss; N represents the total number of point cloud instances in a batch (i.e., within 1 batch); L i Represents the category feature q of the i-th point cloud instancei The contrast loss of point cloud instances is q; M represents the same type of point cloud instances, that is, the total number of point cloud instances with the same category as the i-th point cloud instance; i Represents the category feature of the i-th point cloud instance; Represents the category feature of the jth point cloud instance with the same category as the i-th point cloud instance, that is, i Other category characteristics of the same category; k j represents all category features of the jth point cloud instance (i.e., all other category features); τ represents the temperature coefficient, which is used to control the degree of disparity between similarities. i No loss calculation is performed on itself. Since M is not necessarily fixed in each batch and each category, it needs to be divided by M to make L i The value of is normalized.

[0057] The method for determining the scale loss is: splicing the size features output by the first point cloud target detection model and the size features output by the second point cloud target detection model to obtain a size splicing feature; and determining the scale loss according to the size splicing feature and the size scaling ratio between the first enhancement instance and the second enhancement instance.

[0058] The method for determining the angle loss is: splicing the angle features output by the first point cloud target detection model and the angle features output by the second point cloud target detection model to obtain an angle splicing feature; and determining the angle loss based on the angle splicing feature and the angle offset between the first enhancement instance and the second enhancement instance.

[0059] Combine the following Figure 4 , a complete training process of the above-mentioned point cloud target detection model is further described in detail, and the effectiveness of the pre-training method of the above-mentioned point cloud target detection model is experimentally verified.

[0060] Step 1: Data preprocessing.

[0061] Data reading: Randomly read 2 frames of regular point cloud scenes as point cloud background B (Base) and point cloud background C (contrast). Point cloud background B is the first point cloud background, and point cloud background C is the second point cloud background. Randomly read t categories and n point cloud instances in each category. The point cloud instances are pre-collected from a large number of unlabeled point clouds by rule-based point cloud target clustering and bounding box fitting algorithms.

[0062] Instance enhancement of point cloud instances: Enhance the diversity of point cloud instances so that the network model can learn the essential features of the same point cloud instance. Enhancement methods include upsampling, downsampling, random cutting, adding noise, etc.

[0063] Contrast enhancement of point cloud instances: including rotation and scaling, the purpose is to construct different sizes and angle forms of the same point cloud instance, and obtain the size scaling ratio Δs and angle offset Δr between the two enhanced instances as supervision signals for subsequent calculation of size loss and angle loss.

[0064] Point cloud synthesis: Randomly generate t×n [x, y, z] position information within the effective range of the point cloud scene, and place the enhanced instance B (i.e., the first enhanced instance) in the corresponding t×n positions in the point cloud background B in sequence to obtain the synthetic point cloud B (i.e., the first synthetic point cloud), and similarly obtain the synthetic point cloud C (i.e., the second synthetic point cloud). Thus, the label information of the point cloud instance [x, y, z, l, w, h, r] is also obtained, where l is the length, w is the width, and h is the height. l, w, and h come from the point cloud instance label, and r is the point cloud instance orientation angle, which comes from the contrast enhancement generation.

[0065] Step 2: Network model feature extraction and regression.

[0066] Point cloud object detection model: can be any commonly used model.

[0067] Point cloud target detection model setting: Point cloud detection model B (i.e., the first point cloud target detection model) and point cloud detection model C (i.e., the second point cloud target detection model) are initialized by the same point cloud target detection model, where point cloud detection model B performs forward reasoning and back propagation to update parameters, but freezes the network model parameters of point cloud detection model C and does not perform back propagation. Before each round of network model reasoning, the parameters of point cloud detection model B are copied to point cloud detection model C so that the two models have the same parameters, and then the synthesized point cloud B is input into point cloud detection model B, and the synthesized point cloud C is input into point cloud detection model C to obtain their respective features.

[0068] Feature regression: The point cloud detection model B and the point cloud detection model C generate the category feature fc1, size features fs1, fs2, and angle features fr1, fr2 corresponding to each point cloud instance through the detection head, which are used for subsequent loss calculation and model update.

[0069] Step 3: Loss calculation and back propagation to update the model.

[0070] Contrastive loss: The category feature fc1 (shape is [b, f, k], indicating b batches, each batch has k = t × n features, each feature has f dimensions) obtained by process B is used to calculate the self-similarity matrix m = sim (fc1, fc1), and the shape is [b, k, k]. The similarity function is cosine similarity or Euclidean distance, etc. Table 1 is the similarity matrix, a and b represent the category features of different categories of point cloud targets, ↑ and ↓ indicate how the similarity values ​​are expected to change when calculating the loss. Indicates that the similarity values ​​of the same feature do not need to be considered. In actual implementation, the numerator / denominator mask matrix can be used to filter out the numerator / denominator in m corresponding to the contrast loss function. In addition, in order to no longer calculate the loss of heterogeneous features whose similarity is already low enough, this part of the similarity value can be filtered out through the margin_mask matrix.

[0071] Table 1 Similarity matrix

[0072]

[0073]

[0074] According to the similarity matrix, a new expression of contrast loss is designed:

[0075]

[0076] Overall object-level instance discrimination contrastive loss For all L i In addition, a0, b0, etc. are not the category features obtained by the point cloud instance through the network model, but the unique hot encoding of each category, which is used as a fixed feature to guide the contrast loss function to converge to a certain position.

[0077] Size loss: Through contrast enhancement in the data preprocessing part, two point cloud instances with different scaling ratios are obtained, and the relative scaling ratio is Δs. This is used as a supervisory signal for size loss, prompting the network model to learn the size characteristics of the point cloud instance when predicting the scaling size. The specific implementation is: concatenate the size features fs1 and fs2, regress the predicted value Δs* of Δs through the fully connected layer, and use the regression loss function to calculate the size loss.

[0078]

[0079] Among them, MLP is the fully connected layer and MSE is the mean square error.

[0080] Angle loss: Similarly, using Δr, calculate the angle loss This forces the network model to learn the angle features of point cloud instances when predicting the deflection angle.

[0081]

[0082] Final total loss for:

[0083]

[0084] The experimental verification results are described below.

[0085] Using the above point cloud target detection model pre-training method, the CenterPoint algorithm was pre-trained on the KITTI dataset for 20 epochs. Each epoch trained 3714 frames of point cloud, and each frame sampled about 60 point cloud instances to obtain a pre-trained model. This model was used to perform supervised fine-tuning on the KITTI dataset. Compared with the supervised fine-tuning of the randomly initialized model, each was trained for 36 epochs. The detection accuracy was compared on the validation set as shown in Table 2.

[0086] Table 2 Comparison between self-supervised methods and randomly initialized models

[0087]

[0088] From Table 2, we can see that under the two fine-tuning data conditions of 20% and 100%, the models pre-trained using the self-supervised method have higher detection accuracy than the randomly initialized model. In addition, it can be seen that the self-supervised model uses 20% data for supervised fine-tuning, achieving the same detection effect as the randomly initialized model using 100% data.

[0089] The present application also provides an autonomous driving application scenario, which uses a trained point cloud target detection model obtained by the above-mentioned point cloud target detection model pre-training method. Specifically: obtain the point cloud data of the autonomous driving scene to be detected to obtain the point cloud to be detected; input the point cloud to be detected into the trained point cloud target detection model to obtain the target in the autonomous driving scene to be detected; wherein the targets in the autonomous driving scene to be detected include: vehicles and pedestrians. When training the point cloud target detection model, the point cloud background in the training data is the point cloud in the autonomous driving scene except for vehicles and pedestrians; the point cloud instance is the vehicle point cloud and pedestrian point cloud in the autonomous driving scene.

[0090] Based on the same inventive concept, the embodiment of the present application also provides a point cloud target detection model pre-training device for implementing the point cloud target detection model pre-training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more point cloud target detection model pre-training device embodiments provided below can refer to the limitations of the point cloud target detection model pre-training method above, and will not be repeated here.

[0091] In an exemplary embodiment, Figure 5 As shown, a point cloud target detection model pre-training device is provided, comprising:

[0092] The point cloud data acquisition module 501 is used to acquire unlabeled point cloud data.

[0093] The point cloud data processing module 502 is used to process the unlabeled point cloud data using a clustering algorithm and a bounding box fitting algorithm to obtain processed point cloud data.

[0094] The point cloud data segmentation module 503 is used to segment the background and target of the processed point cloud data to obtain multi-frame point cloud background and multi-class point cloud instances; the point cloud instance includes the target category and the target bounding box.

[0095] The pre-training module 504 is used to use multi-frame point cloud backgrounds and multi-category point cloud instances as pre-training data, and adopt an object-level self-supervised pre-training method to pre-train the point cloud target detection model to obtain a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction.

[0096] The model fine-tuning module 505 is used to perform supervised fine-tuning on the pre-trained point cloud target detection model using the labeled point cloud data to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used as the input of the point cloud data of the scene to be detected, the point cloud target in the scene to be detected is output.

[0097] As an optional implementation, the pre-training module 504 includes:

[0098] The data reading unit is used to randomly read two frames of point cloud background and t×n point cloud instances; the two frames of point cloud background are respectively the first point cloud background and the second point cloud background; t represents the number of target categories; and n represents the number of point cloud instances of each target category.

[0099] The data enhancement unit is used to perform instance enhancement and contrast enhancement on any point cloud instance read, and construct a first enhanced instance and a second enhanced instance of the point cloud instance; the first enhanced instance and the second enhanced instance have different sizes and angle shapes.

[0100] A data synthesis unit is used to randomly generate t×n positions within the effective range of the point cloud scene, place the first enhanced instance in the corresponding t×n positions in the first point cloud background in sequence to obtain a first synthetic point cloud, and place the second enhanced instance in the corresponding t×n positions in the second point cloud background in sequence to obtain a second synthetic point cloud.

[0101] A training unit is used to input the first synthetic point cloud into a first point cloud target detection model, input the second synthetic point cloud into a second point cloud target detection model, perform training with the goal of minimizing the total loss, and determine the trained first point cloud target detection model or the trained second point cloud target detection model as a pre-trained point cloud target detection model.

[0102] In which, during the training process, the first point cloud target detection model and the second point cloud target detection model share parameters; the total loss is determined based on the category features, size features, angle features output by the first point cloud target detection model and the size features and angle features output by the second point cloud target detection model.

[0103] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store point clouds to be detected. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an instance-level self-supervised point cloud target detection is implemented.

[0104] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0106] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0107] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0108] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0109] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, etc., but is not limited thereto.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A point cloud target detection model pre-training method, characterized in that: The point cloud target detection model pre-training method comprises: Obtain unlabeled point cloud data; Processing the unlabeled point cloud data using a clustering algorithm and a bounding box fitting algorithm to obtain processed point cloud data; Segmenting the background and target of the processed point cloud data to obtain multi-frame point cloud background and multi-class point cloud instances; the point cloud instance includes a target category and a target bounding box; Using multi-frame point cloud backgrounds and multi-class point cloud instances as pre-training data, and adopting an object-level self-supervised pre-training method to pre-train the point cloud target detection model, thereby obtaining a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction; The pre-trained point cloud target detection model is fine-tuned in a supervised manner using the labeled point cloud data to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used as the input of the point cloud data of the scene to be detected, the point cloud target in the scene to be detected is output.

2. The point cloud target detection model pre-training method according to claim 1, characterized in that: The multi-frame point cloud background and multi-class point cloud instances are used as pre-training data, and the point cloud target detection model is pre-trained using the object-level self-supervised pre-training method to obtain a pre-trained point cloud target detection model, which specifically includes: Randomly read two frames of point cloud background and t×n point cloud instances; the two frames of point cloud background are the first point cloud background and the second point cloud background respectively; t represents the number of target categories; n represents the number of point cloud instances of each target category; For any read point cloud instance, perform instance enhancement and contrast enhancement on the point cloud instance to construct a first enhanced instance and a second enhanced instance of the point cloud instance; the first enhanced instance and the second enhanced instance have different sizes and angle shapes; Randomly generate t×n positions within the effective range of the point cloud scene, sequentially place the first augmented instance at the corresponding t×n positions in the first point cloud background to obtain a first synthetic point cloud, and sequentially place the second augmented instance at the corresponding t×n positions in the second point cloud background to obtain a second synthetic point cloud; Inputting the first synthetic point cloud into a first point cloud target detection model, inputting the second synthetic point cloud into a second point cloud target detection model, training with the goal of minimizing the total loss, and determining the trained first point cloud target detection model or the trained second point cloud target detection model as a pre-trained point cloud target detection model; In which, during the training process, the first point cloud target detection model and the second point cloud target detection model share parameters; the total loss is determined based on the category features, size features, angle features output by the first point cloud target detection model and the size features and angle features output by the second point cloud target detection model.

3. The point cloud target detection model pre-training method according to claim 2, characterized in that: The instance enhancement includes: upsampling, downsampling, random shearing and adding noise; the contrast enhancement includes: rotation and scaling.

4. The point cloud target detection model pre-training method according to claim 2, characterized in that: The total loss includes: object-level instance discrimination contrast loss and bounding box geometric contrast prediction loss; the bounding box geometric contrast prediction loss includes: scale loss and angle loss; The object-level instance discrimination contrast loss is determined according to the category features output by the first point cloud object detection model; The scale loss is determined according to the size feature output by the first point cloud object detection model and the size feature output by the second point cloud object detection model; The angle loss is determined based on the angle features output by the first point cloud target detection model and the angle features output by the second point cloud target detection model.

5. The point cloud target detection model pre-training method according to claim 4, characterized in that: The expression of the object-level instance discriminant contrast loss is: in, represents the object-level instance discrimination contrast loss; N represents the total number of point cloud instances processed in a batch; L i represents the contrast loss of the category feature of the i-th point cloud instance; M represents the total number of point cloud instances with the same category as the i-th point cloud instance; q i Represents the category feature of the i-th point cloud instance; k represents the category feature of the jth point cloud instance of the same category as the i-th point cloud instance; j represents all category features of the jth point cloud instance; τ represents the temperature coefficient.

6. The point cloud target detection model pre-training method according to claim 4, characterized in that: The method for determining the scale loss is: Splicing the size features output by the first point cloud object detection model and the size features output by the second point cloud object detection model to obtain a size splicing feature; Determining a scale loss based on the size stitching feature and a size scaling ratio between the first augmented instance and the second augmented instance; The method for determining the angle loss is: splicing the angle feature output by the first point cloud target detection model and the angle feature output by the second point cloud target detection model to obtain an angle splicing feature; An angular loss is determined based on the angular stitching feature and an angular offset between the first enhancement instance and the second enhancement instance.

7. A point cloud target detection model pre-training device, characterized in that: The point cloud target detection model pre-training device comprises: Point cloud data acquisition module, used to acquire unlabeled point cloud data; A point cloud data processing module, used to process the unlabeled point cloud data using a clustering algorithm and a bounding box fitting algorithm to obtain processed point cloud data; A point cloud data segmentation module is used to segment the background and target of the processed point cloud data to obtain multi-frame point cloud background and multi-class point cloud instances; the point cloud instance includes a target category and a target bounding box; A pre-training module is used to use multi-frame point cloud backgrounds and multi-class point cloud instances as pre-training data, and adopt an object-level self-supervised pre-training method to pre-train the point cloud target detection model to obtain a pre-trained point cloud target detection model; the pre-training includes: object-level instance discrimination comparison and bounding box geometry comparison prediction; The model fine-tuning module is used to perform supervised fine-tuning on the pre-trained point cloud target detection model using the labeled point cloud data to obtain a trained point cloud target detection model; when the trained point cloud target detection model is used as the input of the point cloud data of the scene to be detected, the point cloud target in the scene to be detected is output.

8. The point cloud target detection model pre-training device according to claim 7, characterized in that: The pre-training module comprises: A data reading unit is used to randomly read two frames of point cloud backgrounds and t×n point cloud instances; the two frames of point cloud backgrounds are respectively the first point cloud background and the second point cloud background; t represents the number of target categories; n represents the number of point cloud instances of each target category; A data enhancement unit is used to perform instance enhancement and contrast enhancement on any read point cloud instance, and construct a first enhanced instance and a second enhanced instance of the point cloud instance; the first enhanced instance and the second enhanced instance have different sizes and angle shapes; A data synthesis unit is used to randomly generate t×n positions within a valid range of the point cloud scene, sequentially place the first augmented instance at the corresponding t×n positions in the first point cloud background to obtain a first synthesized point cloud, and sequentially place the second augmented instance at the corresponding t×n positions in the second point cloud background to obtain a second synthesized point cloud; A training unit, configured to input the first synthetic point cloud into a first point cloud target detection model, input the second synthetic point cloud into a second point cloud target detection model, perform training with the goal of minimizing the total loss, and determine the trained first point cloud target detection model or the trained second point cloud target detection model as a pre-trained point cloud target detection model; In which, during the training process, the first point cloud target detection model and the second point cloud target detection model share parameters; the total loss is determined based on the category features, size features, angle features output by the first point cloud target detection model and the size features and angle features output by the second point cloud target detection model.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the point cloud target detection model pre-training method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud target detection model pre-training method described in any one of claims 1 to 6 is implemented.

Images