Object detection methods, systems, network training methods, and computer-readable media

Abstract

The target detection method, system, and network training method provided in this specification acquire J frames of unlabeled point cloud data, each frame of the point cloud data including at least multiple target objects, and iteratively update the target detection network until the number of iterations reaches a preset value. The updating of the target detection network includes: inputting the J frames of unlabeled point cloud data into the target detection network for processing, generating pseudo-labels for the target objects detected in each frame of unlabeled point cloud data, iteratively training a student network using I frames of labeled point cloud data, the J frames of unlabeled point cloud data, and the corresponding pseudo-labels together until a preset number of rounds, and updating the weights of the target detection network using the student network. This minimizes the labeling cost, maximizes the utilization of unlabeled data, and improves the confidence of the predicted labels, which is beneficial to improving the accuracy of target detection.

Description

Technical Field

[0001] This specification relates to the field of deep learning technology, and in particular to an object detection method, system, network training method, and computer-readable medium. Background Technology

[0002] Vehicle-mounted LiDAR-based perception of the external environment has become a mainstream technical solution for realizing autonomous driving, and is also a key technology and research hotspot in the field of autonomous driving. Perceiving external environmental information based on LiDAR refers to acquiring 3D point cloud data of different targets outside the vehicle during driving using vehicle-mounted LiDAR, and then processing the point cloud data through perception algorithms to accurately detect the category and location of different 3D targets.

[0003] Deep learning-based point cloud object detection models require training with a large amount of labeled point cloud data. Unlike the rich image data used in training sets, point cloud data is typically collected by driving a car equipped with an expensive LiDAR sensor suite on roads that comply with local regulations. The unlabeled point cloud data is then manually labeled to obtain labeled point cloud data, which is then used to train the object detection model. Therefore, the acquisition of point cloud data is limited by the driving-based collection scenario, and the labeling cost is high, making it difficult to obtain large amounts of labeled point cloud data. Summary of the Invention

[0004] The target detection methods, systems, and network training methods provided in this manual do not rely excessively on data annotation and can effectively utilize a large amount of unlabeled point cloud data. This solves the problems of scarce labeled point cloud data and lack of diversity in driving scenarios for autonomous driving, reduces the dependence of autonomous driving target detection networks on data labels, and lowers the cost of data annotation.

[0005] In a first aspect, this specification provides a training method for an object detection network, comprising: acquiring J frames of unlabeled point cloud data, each frame of the unlabeled point cloud data including at least a plurality of target objects, wherein J is a positive integer; and iteratively updating the object detection network until the number of iterations reaches a preset value, wherein updating the object detection network comprises: inputting the J frames of unlabeled point cloud data into the object detection network for processing, generating pseudo-labels for the target objects detected in each frame of unlabeled point cloud data, iteratively training a student network based on I frames of labeled point cloud data and the J frames of unlabeled point cloud data and the corresponding pseudo-labels until a preset number of rounds is reached, wherein I is a positive integer, and updating the weights of the object detection network using the student network.

[0006] In some embodiments, the object detection network has been trained based on the labeled point cloud data of the I-frame before the first iteration update.

[0007] In some embodiments, inputting the J frames of unlabeled point cloud data into the target detection network for processing, and generating pseudo-labels for target objects detected in each frame of unlabeled point cloud data, includes: for each frame of unlabeled point cloud data: acquiring the original data and N frames of perturbation data for each frame of unlabeled point cloud data, wherein the N frames of perturbation data are point cloud data obtained after perturbing the original data N times, and N is a positive integer; detecting the original data through the target detection network and generating a first bounding box for the detected target objects; detecting the N frames of perturbation data through the target detection network and generating a second bounding box for the perturbed objects detected in each frame of perturbation data; filtering the second bounding boxes based on the first bounding boxes; and determining pseudo-labels for the target objects in the original data based on the first bounding boxes and the filtered second bounding boxes.

[0008] In some embodiments, the perturbation process includes at least one of the following: flipping the original data, rotating the original data, scaling the original data, and performing a masking operation on the original data.

[0009] In some embodiments, detecting the N frames of perturbation data through the target detection network and generating a second bounding box for each detected perturbation object in the perturbation data frame includes: inputting the N frames of perturbation data into the target detection network to obtain the category of the detected perturbation object in each frame of perturbation data; generating a perturbation bounding box for each perturbation object; and performing inverse perturbation processing on the perturbation bounding box of each perturbation object to obtain the second bounding box of each perturbation object, wherein the coordinate system of the second bounding box is the same as the coordinate system of the first bounding box.

[0010] In some embodiments, when multiple disturbance objects are detected in each frame of disturbance data, the filtering includes: obtaining a second location box for each disturbance object in each frame of disturbance data; using the first location box as a reference, sequentially performing distance matching between the multiple second location boxes in each frame of disturbance data and the first location box; and filtering out the second location box in each frame of disturbance data that is closest to the first location box, determining the closest second location box as the nearest location box, thereby obtaining N nearest location boxes that correspond one-to-one with the N frames of disturbance data.

[0011] In some embodiments, when there is one disturbing object detected in each frame of disturbance data, the filtering includes: determining the second location box of the disturbing object in each frame of disturbance data as the nearest location box, and obtaining N nearest location boxes that correspond one-to-one with the N frames of disturbance data.

[0012] In some embodiments, the filtering further includes: performing error matching between the N nearest position boxes and the first position box, and filtering out the remaining nearest position boxes whose error with the first position box is less than a first preset threshold, wherein the remaining nearest position boxes are the filtered second position boxes.

[0013] In some embodiments, the error matching based on the N nearest location boxes and the first location box includes: sequentially performing error matching between the N nearest location boxes and the first location box.

[0014] In some embodiments, the error matching based on the N nearest position boxes and the first position box includes: removing the nearest position boxes from the N nearest position boxes whose distance from the first position box is greater than a second preset threshold, to obtain M nearest position boxes, where M is a positive integer less than or equal to N; and performing error matching between the M nearest position boxes and the first position box in sequence.

[0015] In some embodiments, determining the pseudo-label of the target object in the original data based on the first location box and the filtered second location box includes: determining a target location box based on the first location box and the filtered second location box; and determining the location information of the target location box and the category of the target object as the pseudo-label of the target object in the original data.

[0016] In some embodiments, determining the target location box based on the first location box and the filtered second location box includes: averaging the first location box and the filtered second location box to obtain the target location box; or determining one of the location boxes in the first location box and the filtered second location box as the target location box.

[0017] In some embodiments, determining the target location box based on the first location box and the filtered second location box includes: averaging the first location box and the filtered second location box to obtain the target location box; or determining one of the location boxes in the first location box and the filtered second location box as the target location box.

[0018] In some embodiments, updating the weights of the object detection network using the student network includes: obtaining the weights of the student network after training for the preset number of rounds; obtaining the old weights of the object detection network after the last update; and determining the target weights for the current update of the object detection network based on the weights of the student network and the old weights.

[0019] Secondly, this specification also provides a target detection method, comprising: acquiring n frames of point cloud data to be detected; inputting the n frames of point cloud data to be detected into a target detection network to generate a target detection result, wherein the target detection network is trained according to the target detection network training method described in the first aspect above.

[0020] Thirdly, this specification also provides a target detection system, including at least one storage medium storing a target detection network for target detection; and at least one processor communicatively connected to the at least one storage medium, wherein when the target detection system is running, the at least one processor reads the target detection network and performs the following: inputting point cloud data to be detected into the target detection network; detecting the point cloud data to be detected through the target detection network to obtain the category of the target object; generating a prediction box for the target object through the target detection network; and determining the category of the target object and the position information of the prediction box as the prediction label of the target object, wherein the target detection network is trained by the method described in the first aspect above.

[0021] Fourthly, this specification also provides a computer-readable medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the methods of the first and second aspects described above.

[0022] As can be seen from the above technical solutions, the target detection method, system, and network training method provided in this specification use a target detection network to generate pseudo-labels for target objects detected in J frames of unlabeled point cloud data. Then, I frames of labeled point cloud data, the J frames of unlabeled point cloud data, and the corresponding pseudo-labels are used together to iteratively train a student network until a preset number of rounds are reached. The weights of the target detection network are updated using the student network, and then the updated target detection network is used to generate pseudo-labels. This process is repeated iteratively until the number of iterations reaches a preset value. In other words, the target detection network is used to predict J frames of unlabeled point cloud data to obtain pseudo-labels for the target objects. Then, based on I frames of labeled point cloud data and the J frames of unlabeled point cloud data with the pseudo-labels, the target detection network is continuously retrained. The trained target detection network can be used to detect targets in n frames of point cloud data, minimizing labeling costs, maximizing the utilization of unlabeled data, and improving the confidence of the predicted labels, thus improving the accuracy of target detection.

[0023] The target detection method, system, and network training method provided in this manual can effectively utilize a large amount of unlabeled point cloud data when applied to the field of autonomous driving scene perception based on point cloud data. This solves the problems of scarce labeled point cloud data and lack of diversity in driving scenarios for autonomous driving, reduces the dependence of autonomous driving target detection networks on data labels, lowers the cost of data labeling, and can quickly adapt to different datasets.

[0024] The target detection method, system, and network training method provided in this manual, when inputting unlabeled point cloud data with disturbances, filter the second location boxes generated in the disturbed data, thereby improving the confidence of predicting pseudo-labels for unlabeled point cloud data and facilitating the training of a target detection network with strong generalization ability.

[0025] Other features of the object detection methods, systems, and network training methods provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the object detection methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A diagram of a purely semi-supervised learning framework provided according to some embodiments of this specification is shown;

[0028] Figure 2 A system architecture diagram 100 is shown for a target detection method, system, and network training method provided according to some embodiments of this specification;

[0029] Figure 3 A flowchart of a training method 300 for an object detection network according to some embodiments of this specification is shown;

[0030] Figure 4 A framework diagram of a teacher network provided according to some embodiments of this specification is shown;

[0031] Figure 5 A schematic diagram illustrating feature extraction using a column feature extractor according to some embodiments of this specification is shown;

[0032] Figure 6 A structural diagram of an FPN provided according to some embodiments of this specification is shown;

[0033] Figure 7 A structural diagram of a U-net provided according to some embodiments of this specification is shown;

[0034] Figure 8 This illustration shows a schematic diagram of generating pseudo-labels for target objects in each frame of unlabeled point cloud data, according to some embodiments of this specification.

[0035] Figure 9 A flowchart illustrating a method for determining pseudo-labels of target objects by filtering a second location box, according to some embodiments of this specification, is shown.

[0036] Figure 10 A schematic diagram illustrating the obtaining of a target location box according to some embodiments of this specification is shown;

[0037] Figure 11 A framework diagram of a training method for an object detection network according to some embodiments of this specification is shown;

[0038] Figure 12 A schematic diagram of point cloud data to be detected is shown, according to some embodiments of this specification; and

[0039] Figure 13 A schematic diagram of a target detection result provided according to some embodiments of this specification is shown. Detailed Implementation

[0040] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0041] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0042] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0043] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0044] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0045] The target detection methods, systems, and network training methods provided in this manual are applicable to the field of autonomous driving scene perception based on 3D point cloud data. Currently, target detection based on point cloud data encounters two serious problems in the field of autonomous driving scene perception: data acquisition and data annotation. Regarding data acquisition, autonomous driving scenarios must be collected by driving a car equipped with an expensive LiDAR sensor suite on roads that comply with local regulations, thus limiting the availability of autonomous driving scenarios. Therefore, existing autonomous driving datasets can only provide a limited amount of scene data. Regarding data annotation, a data acquisition vehicle can typically collect over 200,000 frames of point cloud data in 8 working hours, but a skilled worker can only annotate 100-200 frames per day. The high cost of manual data annotation makes it difficult to acquire large amounts of labeled point cloud data. In summary, in the field of autonomous driving scene perception based on point cloud data, there is only a small amount of labeled data, and the cost of annotating point cloud data is sometimes very high, while a large amount of unlabeled point cloud data is readily available. The target detection method, system, and network training method provided in this specification can effectively utilize a large amount of unlabeled point cloud data, solving the problems of scarce labeled point cloud data and lack of diversity in driving scenarios for autonomous driving. This can reduce the dependence of autonomous driving target detection networks on data labels and lower the cost of data labeling.

[0046] For ease of description, this manual explains the terms that will appear in the following descriptions:

[0047] Semi-supervised object detection (SSOD) is a method that uses a dataset containing a small amount of labeled data and a large amount of unlabeled data to train an object detection network, thereby achieving object detection. The core of semi-supervised object detection methods lies in how to fully utilize a large amount of unlabeled and diverse data to improve the performance of the object detection network. Figure 1 A diagram of a purely semi-supervised learning framework provided according to some embodiments of this specification is shown, such as Figure 1 As shown, purely semi-supervised learning uses a small amount of labeled data and a large amount of unlabeled data to train the model, and the trained model can then be used to predict the results on the test data.

[0048] Point cloud data: Point cloud data is collected by LiDAR. The information of each point includes its position coordinates and reflectivity (x, y, z, r) in the LiDAR coordinate system. For ease of description, "point cloud data" can be referred to as "point cloud" or "point".

[0049] Pseudo-labels: When using an object detection network to predict the label of unlabeled point cloud data, a bounding box with high confidence and a category are selected as the label for the current unlabeled point cloud data. Since the label may not be real, it is called a "pseudo-label".

[0050] Non-Maximum Suppression (NMS): Suppresses elements that are not local maxima, searching for local maxima. In object detection, if we want to locate objects in an image, the object detection network finds a bunch of bounding boxes. NMS is used to determine which bounding boxes are useless, discarding the useless ones and retaining the useful ones.

[0051] Figure 2 A system architecture diagram 100 is shown for a target detection method, system, and network training method provided according to some embodiments of this specification. The target detection method, system, and network training method provided in this specification can be applied to, for example... Figure 2 The system architecture 100 shown is as follows. (For example...) Figure 2 As shown, the system architecture 100 may include a data acquisition device 101 and a target detection system 102. The data acquisition device 101 and the target detection system 102 may be installed as a whole or separately.

[0052] The data acquisition device 101 can acquire unlabeled point cloud data and send it to the target detection system 102. The data acquisition device 101 can be a mechanical lidar, solid-state lidar, etc., and the acquired data is 3D unlabeled point cloud data. Operators can manually or automatically label a small amount of unlabeled point cloud data to obtain a small amount of labeled point cloud data, which is then uploaded to the target detection system 102.

[0053] The target detection system 102 can execute the training method for the target detection network and the target detection method described in this specification. The training method for the target detection network and the target detection method will be described in detail below. The target detection system 102 is an exemplary computing device capable of executing the training method for the target detection network and the target detection method. In some embodiments, the computing device may be a distributed computing device, including multiple distributed computing nodes.

[0054] The target detection system 102 may include at least one processor 120 and at least one storage medium 130. In some embodiments, the target detection system 102 may further include a communication module 150 and an internal communication bus 110. The at least one storage medium 130 stores unlabeled point cloud data, labeled point cloud data, and n frames of point cloud data to be detected. The at least one processor 120 may be communicatively connected to the at least one storage medium 130. When the target detection system 102 is running, the at least one processor 120 may read the unlabeled point cloud data and the labeled point cloud data, and execute the target detection network training method provided in this specification, storing the trained target detection network in the at least one storage medium 130. Thus, the at least one storage medium 130 also stores a target detection network for target detection. When the target detection system 102 is running, the at least one processor 120 may read the target detection network and execute the target detection method provided in this specification, i.e., acquire the n frames of point cloud data to be detected, input the n frames of point cloud data to be detected into the target detection network, and generate a target detection result. Specifically, at least one processor 120 can input the point cloud data to be detected into the target detection network, detect the point cloud data to be detected through the target detection network, obtain the category of the target object, generate a prediction box for the target object through the target detection network, and determine the category of the target object and the position information of the prediction box as the prediction label of the target object.

[0055] At least one storage medium 130 may store unlabeled point cloud data, labeled point cloud data, n frames of point cloud data to be detected, and an object detection network in the form of an instruction set, which may be a collection of one or more instructions. For example, the instruction set may be multiple lines of computer program code, which, when retrieved and executed by processor 120, may instruct processor 120 to execute the program, routine, object, component, data structure, process, module, etc., of the object detection network training method and object detection method provided in this specification. At least one storage medium 130 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 132, a read-only storage medium (ROM) 134, or a random access storage medium (RAM) 136.

[0056] At least one processor 120 can execute the training method of the object detection network and all the steps included in the object detection method. The processor 120 can be in the form of one or more processors. In some embodiments, the processor 120 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof. For illustrative purposes only, only one processor 120 is described in the object detection system 102 in this specification. However, it should be noted that the object detection system 102 in this specification may also include multiple processors 120; therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 120 of the target detection system 102 in this specification executes steps A and B, it should be understood that steps A and B can also be executed jointly or separately by two different processors 120 (e.g., the first processor executes step A and the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0057] Of course, the modules for implementing the training method and target detection method of the target detection network provided in this specification can also be designed as dedicated circuits, hardware modules and / or devices, which can be independently communicated with the target detection system 102 or exist as internal components in the target detection system 102. Figure 2 (Not marked).

[0058] Communication module 150 can be connected to processor 120 for data communication between target detection system 102 and external sources, such as servers and databases. Figure 2 (Not shown). Communication module 150 may include at least one of wired communication module and wireless communication module. Internal communication bus 110 may connect different system components, including storage medium 130, processor 120 and communication module 150.

[0059] Figure 3A flowchart of a training method 300 for an object detection network according to some embodiments of this specification is shown. As previously described, the object detection system 102 can be used to execute the training method 500 for the object detection network described in this specification. Specifically, the processor 120 can read an instruction set stored in its local storage medium and then execute the training method 300 for the object detection network described in this specification according to the instructions in the instruction set. Figure 3 As shown, the method 300 may include:

[0060] S310: Acquire unlabeled point cloud data for frame J.

[0061] Since the data acquisition device 101 is communicatively connected to the target detection system 102, after acquiring J frames of unlabeled point cloud data, the data acquisition device 101 can send the J frames of unlabeled point cloud data to the target detection system 102, and then the processor 120 in the target detection system 102 obtains the J frames of unlabeled point cloud data. Of course, the data acquisition device 101 may also not be communicatively connected to the target detection system 102. This J frames of unlabeled point cloud data can be pre-acquired and stored by the data acquisition device 101, and then input into the target detection system 102 as training data. Then, a small amount of unlabeled point cloud data is manually or automatically labeled to obtain a small amount of labeled point cloud data, and this small amount of labeled point cloud data is uploaded to the target detection system 102 as training data.

[0062] After acquiring the unlabeled point cloud data of frame J, processor 120 continues to execute the following steps of method 300:

[0063] S320: Iteratively update the target detection network until the number of iterations reaches a preset value.

[0064] The object detection network is a pre-trained network. Before the first iteration update of the object detection network, the processor 120 can acquire a small amount of labeled point cloud data and perform preprocessing such as data filtering on the labeled point cloud data. The data filtering refers to filtering out some noisy data. Then, the processor 120 can use the pre-processed labeled point cloud data to pre-train the object detection network, and then begin iteratively updating the pre-trained object detection network. That is, the object detection network has already been trained based on the labeled point cloud data before the first iteration update.

[0065] In some embodiments, the labeled point cloud data used for pre-training the object detection network can be the same as the I-frame labeled point cloud data used for iteratively training the student network, that is, the processor 120 can use the I-frame labeled point cloud data to pre-train the object detection network. Of course, the processor 120 can also use labeled point cloud data that is completely different or partially different from the I-frame labeled point cloud data to pre-train the object detection network, and this specification does not limit this.

[0066] The target detection network can be a teacher network. Figure 4 A framework diagram of a teacher network provided according to some embodiments of this specification is shown. Figure 4 As shown, the teacher network includes a feature extractor (e.g., a pillar feature extractor), a backbone network, and a detection head. The processor 120 can input labeled point cloud data into the teacher network. The point cloud data passes through the pillar feature extractor, the backbone network, and the detection head, and then outputs detection results. The pillar feature extractor converts the 3D point cloud data into a 2D pseudo-image. The backbone network processes the pseudo-image to obtain deep features of the point cloud data. The detection head processes the deep features of the point cloud data to detect the category of target objects in the point cloud data and generates bounding boxes for the target objects, thereby outputting detection results containing the category and bounding boxes of the target objects.

[0067] Figure 5 Schematic diagrams illustrating feature extraction using a column feature extractor according to some embodiments of this specification are shown. For example... Figure 5 As shown, processor 120 can partition point cloud data into regions, discretizing the point cloud data into a uniformly spaced grid in the xy plane, thereby creating a set of pillars containing multiple pillars, each pillar containing multiple points, and the pillars having no height subdivision in the z-direction. Additionally, the index of each pillar, i.e., the coordinates (x, y) corresponding to each pillar, is recorded. Processor 120 can encode each point in a pillar into a 9-dimensional vector D(x, y, z, r, x). C ,y C ,z C ,x P ,y P ), where x, y, z, r are the three-dimensional coordinates and reflectivity of each point in the lidar coordinate system, x C ,y C ,z C The x-coordinate is the three-dimensional coordinate of the center point of the pillar containing this point. P =xx CThis represents the distance between this point and the center point in the x-direction, and y-direction... P =yy C This represents the distance in the y-direction between this point and the center point. Due to the uneven spatial distribution of the point cloud data, it is necessary to truncate and pad the pillars and the points within them. If the number of non-empty pillars exceeds P, truncation is performed; if the number of non-empty pillars is less than P, padding with 0 is used. If the number of points in a single pillar exceeds N, truncation is performed; if the number of points in a single pillar is less than N, padding with 0 is used. Finally, a pillar set of size (D, P, N) is selected. Next, the processor 120 can use a convolutional network to extract features from the tensorized point cloud data (D, P, N), generating a new tensor of size (C, P, N). This convolutional network includes linear layers, BN (Batch Normalization) layers, and ReLU (Rectified Linear Units) layers, where C is the number of channels. Then, for all points in the pillars, the processor 120 can perform max pooling along the channel dimension to obtain a tensor of size (C, P), which is used to describe the learned features. Since the coordinates (x, y) of each pillar were recorded during the creation of the pillar set, a pseudo-image of size (C, H, W) can be constructed based on the tensor (C, P) by referring to the coordinates (x, y) of each pillar, where H is the height of the pseudo-image and W is the width of the pseudo-image. Thus, the processor 120 completes the conversion from 3D point cloud data to a 2D pseudo-image using the pillar feature extractor, i.e., (D, P, N) → (C, P, N) → (C, P) → (C, H, W).

[0068] The backbone network can adopt an FPN (Feature Pyramid Network) structure. Figure 6 A schematic diagram of an FPN structure according to some embodiments of this specification is shown. Figure 6As shown, the FPN includes a bottom-up path, a top-down path, and lateral connections. The bottom-up process is the forward pass of the FPN network. During this process, the size of the feature map of the point cloud data changes after passing through some network layers, but remains unchanged after passing through others. Layers whose feature map size does not change can be grouped into a single stage. The extracted features are the output of the last network layer in each stage, thus forming a feature pyramid. The top-down process uses upsampling. The lateral connections merge the upsampled results with the feature maps of the same size generated from the bottom-up process, thus fusing a high-resolution shallow layer with a deep layer rich in semantic information. This enables the rapid construction of a feature pyramid with strong semantic information at all scales from a single input image at a single scale, without incurring significant costs. The backbone network can also use a U-net structure. Figure 7 A U-net structure diagram is shown according to some embodiments provided in this specification. For example... Figure 7 As shown, the network structure mainly consists of three parts: downsampling, upsampling, and skip connections. The left side represents the compression process, i.e., the encoder. It reduces the image size and extracts some superficial features through convolution and downsampling. The right side represents the decoding process, i.e., the decoder. It obtains deeper features through convolution and upsampling. The image size decreases after each convolution. The middle section uses connections to fuse the feature maps obtained in the encoding and decoding stages, combining deep and shallow features to predict segmentation based on the resulting feature map.

[0069] The detection head is configured to detect the category and bounding box of target objects in the point cloud data, meaning the processor 120 can perform classification and bounding box regression based on the detection head. The detection head can employ an SSD (Single Shot Multibox Detection) network, a YOLO (You Only Look Once) network, or similar methods. The loss function of the target detection network includes a classification loss function and a regression loss function. The classification loss function can be a focus loss function, as shown in Formula 1, and the regression loss function can be a smooth loss function. L1 The loss function is shown in Formula 2:

[0070] Formula 1: FL(p) t )=-α t (1-p t ) γ log(p t )

[0071] Among them, FL(p t ) is the focus loss function, p t It is the classification probability, α t Here, γ is the weight parameter between categories, and γ is the focusing parameter, (1-p t ) γ For the sample adjustment factor, -log(p) t ) is the initial cross-entropy loss function.

[0072] Formula 2:

[0073] Where x is the difference between the true label and the predicted label.

[0074] After pre-training the object detection network, the processor 120 can begin iteratively updating the object detection network until the number of iterations reaches a preset value. Specifically, each time the processor 120 updates the object detection network, it can execute steps S322, S324, and S326.

[0075] S322: Input the J-frame unlabeled point cloud data into the target detection network for processing, and generate pseudo-labels for the target objects detected in each frame of unlabeled point cloud data.

[0076] Figure 8 This diagram illustrates the generation of pseudo-labels for target objects in each frame of unlabeled point cloud data, according to some embodiments of this specification. For example... Figure 8 As shown, the processor 120 inputs the unlabeled point cloud data into the target detection network for processing, and generates pseudo-labels for the target objects detected in each frame of unlabeled point cloud data. The generation of pseudo-labels for unlabeled data using the target detection network only involves a forward propagation process.

[0077] Figure 9 A flowchart illustrating a method for determining pseudo-labels of target objects by filtering a second location box, according to some embodiments of this specification, is shown. Figure 9 As shown, method S322 may include the following steps S322-1, S322-3, S322-5, S322-7 and S322-9.

[0078] S322-1: Obtain the original data and N frames of perturbation data for each frame of unlabeled point cloud data, wherein the N frames of perturbation data are point cloud data obtained after perturbing the original data N times.

[0079] To enhance the robustness of the object detection network, the processor 120 can use each frame of unlabeled point cloud data as the original data and perform N perturbation processes on the original data to obtain N frames of perturbed data, where N is a positive integer. For example, the perturbation process can include one or more combinations of flipping the original data, rotating the original data, scaling the original data, and masking the original data. Obviously, each frame of perturbed data still retains all the objects in the original data, referred to as perturbed objects.

[0080] In some embodiments, flipping the original data may refer to flipping the original data around the X-axis of its own coordinate system and / or flipping the original data around the Y-axis of its own coordinate system. If the processor 120 flips the original data around the X-axis, the perturbed objects in the perturbed data obtained by flipping around the X-axis will be mirror images of the target objects in the original data with respect to the X-axis; if the processor 120 flips the original data around the Y-axis, the perturbed objects in the perturbed data obtained by flipping around the Y-axis will be mirror images of the target objects in the original data with respect to the Y-axis. Flipping the original data around the X-axis can be represented by the following formula three, and flipping the original data around the Y-axis can be represented by the following formula four:

[0081] Formula 3: point[x,y,z]=point[x,-y,z]

[0082] Where point[x,y,z] represents any point in the perturbed data after the original data is flipped around the X-axis.

[0083] Formula 4: point[x,y,z] = point[-x,y,z]

[0084] Where point[x,y,z] represents any point in the perturbed data after the original data is flipped around the Y-axis.

[0085] The aforementioned rotation of the original data refers to rotating the original data around the Z-axis of its own coordinate system. The processor 120's rotation of the original data can be expressed by the following formula:

[0086] Formula 5:

[0087] Where point[x,y,z] represents any point in the original data, and θ z It is the rotation angle of the original data around the Z-axis, and T represents transpose.

[0088] The aforementioned scaling of the original data refers to changing the size of the original data so that the size of the perturbed object in the perturbed data is proportional to the size of the target object in the original data. Scaling the original data can be expressed by Formula Six:

[0089] Formula 6: Points[x,y,z] = S * points[x,y,z]

[0090] Where Points[x,y,z] represents the scaled perturbation data, points[x,y,z] represents the original data, and S represents the scaling factor.

[0091] The processor 120 performs a masking operation on the original data, which means changing a region of the original data so that the area of ​​the background region in the perturbed data is smaller than the area of ​​the background region in the original data. The background region in the perturbed data is the region excluding the area where the perturbed object is located, and the background region in the original data is the region excluding the area where the target object is located. The masking operation on the original data can be expressed by Formula 7:

[0092] Formula 7: Points[x[mask],y[mask],z[mask],r[mask]]=[0,0,0,0]

[0093] Where Points[x[mask],y[mask],z[mask],r[mask]] means using a mask to occlude regions of the original data, such that the value of the occluded region is [0,0,0,0], that is, the region occluded by the mask is blocked.

[0094] The processor 120 may also employ other perturbation processing methods, but this specification does not limit the perturbation processing methods.

[0095] S322-3: The original data is detected by the target detection network, and a first location box is generated for the detected target object.

[0096] The processor 120 can input the raw data into the target detection network for detection, obtain the category of the detected target object, and generate a first set of bounding boxes for each target object. The first set of bounding boxes for each target object refers to the set of multiple first bounding boxes located near the target object. For example, "nearby" can mean that the target object can be located within a corresponding first bounding box. The multiple first bounding boxes in the first set of bounding boxes do not completely overlap, and the areas of the multiple first bounding boxes may be different or the same. The processor 120 can perform NMS processing on the raw data so that each target object in the raw data has only one first bounding box nearby. The first bounding box near each target object can, for example, completely enclose the target object, or it can enclose most of the area of ​​the target object, etc.

[0097] S322-5: The target detection network is used to detect the N frames of perturbation data, and a second location box is generated for the perturbation objects detected in each frame of perturbation data.

[0098] The processor 120 can input the N frames of perturbation data into the target detection network for detection, obtain the category of the detected perturbation object, and generate a second bounding box set for each perturbation object. Similarly, the second bounding box set for each perturbation object refers to the set of multiple second bounding boxes located near the perturbation object. For example, "nearby" can mean that the perturbation object can be located within the corresponding second bounding box. The multiple second bounding boxes in the second bounding box set do not completely overlap, and the areas of the multiple second bounding boxes may be different or the same. The processor 120 can perform NMS processing on each frame of perturbation data, so that each detected perturbation object in each frame of perturbation data has only one second bounding box nearby. The second bounding box near each perturbation object can, for example, completely enclose the perturbation object, or enclose most of the area of ​​the perturbation object, etc.

[0099] To facilitate subsequent filtering of the second location box based on the first location box, the processor 120 can perform inverse perturbation processing on the perturbation location boxes generated in the perturbation data, thereby obtaining the second location box through inverse perturbation processing. The inverse perturbation processing is the reverse operation of the perturbation processing. Specifically, the processor 120 can generate a set of perturbation location boxes for each perturbed object detected in each frame of perturbation data, and perform NMS processing on each frame of perturbation data, ensuring that each perturbed object in each frame of perturbation data has only one perturbation location box nearby. Then, the processor 120 can perform inverse perturbation processing on the perturbation location boxes of each perturbed object to obtain the second location box for each perturbed object. The characteristics of the perturbation location box set are similar to those of the second location box set, and the characteristics of the perturbation location boxes are similar to those of the second location boxes, and will not be repeated here.

[0100] It should be noted that performing certain perturbation processes (such as flipping or rotating) on ​​the original data may change the coordinate system of the original data, making the coordinate system of the perturbed data different from that of the original data. Consequently, the coordinate system of the second location box in the perturbed data will be different from that of the first location box in the original data, making it inconvenient to use the first location box as a reference for subsequent filtering of the second location box in the perturbed data. Therefore, for the second location box in the perturbed data with a coordinate system different from that of the original data, after inverse perturbation processing, the coordinate system of the second location box will be the same as that of the first location box.

[0101] S322-7: Based on the first location box, filter the second location box.

[0102] The processor 120 can filter the second location box based on the first location box, which may include distance filtering and error filtering.

[0103] For distance filtering, when multiple disturbing objects are detected in each frame of perturbation data, the processor 120 can obtain multiple second location boxes corresponding to the multiple disturbing objects detected in each frame of perturbation data. Using the first location box as a reference, the distance between the first location box and the multiple second location boxes in each frame of perturbation data is calculated sequentially. The second location box closest to the first location box is selected from each frame of perturbation data and called the nearest location box. Thus, N frames of perturbation data correspond to N nearest location boxes; that is, the first location box corresponds to N nearest location boxes, and the objects marked by the first location box and its corresponding N nearest location boxes are the same. Calculating the distance between two location boxes refers to calculating the distance between the pixels of the center points of the two location boxes. Taking the nearest neighbor matching method to calculate the distance between two location boxes as an example, the distance between the first location box and the multiple second location boxes in each frame of perturbation data can be calculated sequentially using the following formula:

[0104] Formula 8:

[0105] Among them, box1 i [x,y,z] represents the i-th first position box in the original data, box2 j [x,y,z] represents the j-th second location box in any frame of perturbation data, dist ij This represents the distance between the i-th first location box and the j-th second location box. Using Formula 8, the nearest location box (argmin(dist)) closest to the i-th first location box can be selected from multiple second location boxes in any frame of perturbation data. i ).

[0106] For distance filtering, when only one disturbing object is detected in each frame of disturbance data, the unique second bounding box in each frame of disturbance data is determined as the nearest bounding box by default. Similarly, N nearest bounding boxes corresponding one-to-one with N frames of disturbance data can be obtained. That is, the first bounding box corresponds to N nearest bounding boxes, and the objects marked in the first bounding box and the corresponding N nearest bounding boxes are the same.

[0107] To improve the accuracy of the training object detection network, in some embodiments, the processor 120 may further set a second preset threshold and obtain the distances between the N nearest bounding boxes and the first bounding box, respectively. The processor then removes the nearest bounding boxes whose distance to the first bounding box is greater than the second threshold, resulting in M ​​nearest bounding boxes, where M is a positive integer. Thus, the first bounding box corresponds to M nearest bounding boxes, and the objects labeled in the first bounding box and its corresponding M nearest bounding boxes are the same. If the distances between all N nearest bounding boxes and the first bounding box are less than or equal to the second threshold, then M equals N. If at least one of the N nearest bounding boxes has a distance greater than the second threshold, then M is less than N. In this case, for the NM nearest bounding boxes, the processor 120 can treat the region of that box as a region where no target was detected and generate pseudo-labels for the corresponding unlabeled data. It should be noted that in some embodiments, the pseudo-labels not only represent the category and bounding box of the target object, but can also represent the absence of a target and its corresponding bounding box.

[0108] Even if the first bounding box and its corresponding N or M nearest bounding boxes label the same object, if the differences (position, size, shape, etc.) between different bounding boxes of the same object are too large, it may cause the generated pseudo-labels to be inaccurate, leading to a decrease in the accuracy of the trained object detection network. Therefore, the processor 120 can also perform error filtering.

[0109] For error filtering, the processor 120 can directly perform error matching between the N nearest position boxes and the first position box sequentially, that is, calculate the error between the N nearest position boxes and the first position box respectively, and filter out the remaining nearest position boxes whose error with the first position box is less than a first preset threshold from the N nearest position boxes. In some embodiments, the processor 120 can perform error matching between the M nearest position boxes and the first position box sequentially, that is, calculate the error between the M nearest position boxes and the first position box respectively, and filter out the remaining nearest position boxes whose error with the first position box is less than a first preset threshold from the M nearest position boxes. The error can be the average error between the nearest position box and the first position box in multiple dimensions, which can include three-dimensional coordinates representing the position and length, width, and height representing the size of the position box. The remaining nearest position boxes are the position boxes used to jointly determine the pseudo-label of the target object with the first position box, that is, the filtered second position boxes. The number of remaining nearest position boxes can be one or more. Error matching can be performed using methods such as mean square error, mean absolute error, and root mean square error. Taking mean squared error for error matching as an example, the error between the nearest bounding boxes (N or M) and the first bounding box can be calculated using the following formula nine:

[0110] Formula Nine:

[0111] Where k represents the k-th dimension, there are a total of 6 dimensions, including (x, y, z, w, l, h), where (x, y, z) are the 3D coordinates of the center point of the bounding box, and (w, l, h) are the width, length, and height of the bounding box. σ represents the error between the i-th nearest bounding box and the first bounding box in the k-th dimension. i This represents the final error between the i-th nearest frame and the first frame.

[0112] S322-9: Based on the first location box and the filtered second location box, determine the pseudo-label of the target object in the original data.

[0113] In some embodiments, the processor 120 can average the first location box and the filtered second location box to obtain the target location box. In some embodiments, the processor 120 can determine one of the first location box and the filtered second location box as the target location box. Furthermore, the processor 120 can determine the position information of the target location box and the category of the target object as the pseudo-label of the target object in the original data. Figure 10 A schematic diagram illustrating the obtaining of a target location box according to some embodiments of this specification is shown. For example... Figure 10 As shown, for the first location box of the car, there are two corresponding filtered second location boxes. The target location box of the car is obtained by combining the first location box of the car and these two filtered second location boxes. Similarly, for the first location box of the bicycle, there are also two corresponding filtered second location boxes. The target location box of the bicycle is obtained by combining the first location box of the bicycle and these two filtered second location boxes.

[0114] After the processor 120 generates pseudo-labels for each target object in the unlabeled point cloud data in step S322, the method S320 may further include:

[0115] S324: Using the labeled point cloud data of I-frame, the unlabeled point cloud data of J-frame, and the corresponding pseudo-labels, iteratively train the student network until the preset number of rounds.

[0116] As previously mentioned, the processor 120 can pre-train the object detection network. In some embodiments, the student network can be the same network as the pre-trained object detection network before the first iteration of training. The preset number of epochs can be, for example, 10 epochs, 20 epochs, etc. Each time the training samples (the labeled point cloud data in frame I and the unlabeled point cloud data in frame J) are traversed, the student network can be said to have gone through 1 epoch.

[0117] The loss function of the student network may include a classification loss function and a regression loss function. The regression loss function may be the same as the regression loss function of the object detection network, such as smooth loss. L1 Loss function. For the labeled point cloud data in frame I, the student network can use the same classification loss function as the object detection network, such as the focus loss function, where I is a positive integer. For the unlabeled point cloud data in frame J, the student network can use a classification loss function based on KL divergence.

[0118] After the processor 120 iteratively updates the student network until a preset number of rounds, the method S320 may further include:

[0119] S326: Update the weights of the target detection network using the student network.

[0120] In some embodiments, the processor 120 may update the weights of the object detection network using a moving average method. "Moving" means that the update of the object detection network weights is based on the previous update, and "averaging" is reflected in the summation in Formula 11 below. Specifically, the processor 120 may obtain the weights of the student network after training for the preset number of epochs, and obtain the old weights of the object detection network after the previous update, and then determine the target weights for the current update of the object detection network based on the weights of the student network and the old weights. Specifically, the target weights for the current update of the object detection network can be determined using Formula 11 below:

[0121] Formula 11: p t '=λp t +(1-λ)p s

[0122] Where, p t ' represents the target weight of the target detection network in this current update, p s p represents the weights of the student network after the most recent update. t The old weight is represented by λ, which is a coefficient, typically between 0.99 and 0.999.

[0123] This concludes the introduction to the training method of the object detection network. Figure 11 A framework diagram of a training method for an object detection network according to some embodiments of this specification is shown. Figure 11 As shown, the processor 120 can pre-train an object detection network using labeled point cloud data. Then, the object detection network is used to predict pseudo-labels for each target object detected in unlabeled point cloud data. After that, the student network is iteratively updated (retrained) using labeled and unlabeled point cloud data and the corresponding pseudo-labels. Finally, the weights of the object detection network are updated using the student network (not shown in the figure).

[0124] This specification also provides an object detection method. The processor 120 can acquire n frames of point cloud data to be detected, preprocess the n frames of point cloud data, and then input the n frames of point cloud data into an object detection network trained by method S300 to generate object detection results. Specifically, the processor 120 can input the point cloud data to be detected into the object detection network, and the object detection network can detect the point cloud data to be detected to obtain the category of the target object. Then, the object detection network generates a prediction box for the target object, and the category of the target object and the position information of the prediction box are determined as the prediction label of the target object. Figure 12 A schematic diagram of point cloud data to be detected is shown according to some embodiments of this specification. Figure 13 A schematic diagram of target detection results including predicted labels provided according to some embodiments of this specification is shown.

[0125] Table 1 below shows the detection results of point cloud data to be detected using the target detection network described in this specification and the detection results of point cloud data to be detected using other methods:

[0126] Table 1

[0127] category Other methods This instruction manual provides the method Car bev AP:86.4761 bev AP:86.7771 Car 3d AP: 79.1935 3d AP: 79.3337 pedestrian bev AP:57.1357 bev AP:57.5972 pedestrian 3d AP: 48.4946 3d AP: 49.3396 Cyclists bev AP:69.9346 bev AP:70.4100 Cyclists 3d AP: 63.7930 3d AP: 64.2517 truck bev AP:82.2276 bev AP:82.9697 truck 3d AP:75.2059 3d AP:75.8503

[0128] In Table 1, AP represents Average Precision, 3d represents three-dimensional image data, and bev (bird's eye view) represents a bird's-eye view. The data in Table 1 shows that the object detection network trained using the method described in this specification achieves higher detection accuracy than those achieved using other methods.

[0129] In summary, the target detection method, system, and network training method provided in this specification use a target detection network to generate pseudo-labels for target objects detected in J frames of unlabeled point cloud data. Then, I frames of labeled point cloud data, the J frames of unlabeled point cloud data, and the corresponding pseudo-labels are used together to iteratively train a student network until a preset number of rounds. The weights of the target detection network are updated using the student network, and then the updated target detection network is used to generate pseudo-labels again. This process is repeated iteratively until a preset number of iterations is reached. In other words, the target detection network is used to predict J frames of unlabeled point cloud data to obtain pseudo-labels for the target objects. The target detection network is then continuously retrained based on I frames of labeled point cloud data and the J frames of unlabeled point cloud data with the pseudo-labels. The trained target detection network can be used to detect targets in n frames of point cloud data, minimizing labeling costs and maximizing the utilization of unlabeled data, thus improving the accuracy of target detection.

[0130] The target detection method, system, and network training method provided in this manual can effectively utilize a large amount of unlabeled point cloud data when applied to the field of autonomous driving scene perception based on point cloud data. This solves the problems of scarce labeled point cloud data and lack of diversity in driving scenarios in autonomous driving. It can reduce the dependence of autonomous driving target detection network training on data labels, reduce the cost of data labeling, and can quickly adapt to different datasets.

[0131] The target detection method, system, and network training method provided in this manual, when inputting unlabeled point cloud data with disturbances, filter the second location boxes generated in the disturbed data, thereby improving the confidence of predicting pseudo-labels for unlabeled point cloud data and facilitating the training of a target detection network with strong generalization ability.

[0132] This specification, in another aspect, provides a computer-readable medium that is a non-transitory storage medium. A computer program is stored on the computer-readable medium, which, when executed by a processor, implements the steps of the target detection network training method 300 described herein. In some possible embodiments, various aspects of this specification may also be implemented as a program product comprising program code. When the program product is run on a target detection system 102, the program code causes the target detection system 102 to perform the steps of the target detection network training method 300 described herein. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the target detection system 102. However, the program product of this specification is not limited thereto. In this specification, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. The computer-readable medium may be a readable signal medium or a readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages.The program code can be executed entirely on the target detection system 102, partially on the target detection system 102, as a standalone software package, partially on the target detection system 102 and partially on a remote computing device, or entirely on a remote computing device.

[0133] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0134] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0135] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0136] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0137] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. The entire contents used for all purposes, except for any history of prosecution documents associated with it, that may be inconsistent with or conflict with this document, or that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0138] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A training method for an object detection network, characterized in that, include: Acquire J frames of unlabeled point cloud data, where each frame of point cloud data includes at least multiple target objects, and J is a positive integer; as well as Iteratively update the target detection network until the number of iterations reaches a preset value. Updating the target detection network includes: The J frames of unlabeled point cloud data are input into the target detection network for processing, generating pseudo-labels for the target objects detected in each frame of unlabeled point cloud data, including: Obtain the original data and N frames of perturbed data for each frame of unlabeled point cloud data. The N frames of perturbed data are point cloud data obtained after perturbing the original data N times, where N is a positive integer. The target detection network detects the raw data and generates a first bounding box for each detected target object. The target detection network detects the N frames of perturbation data and generates a second bounding box for each detected perturbation object in each frame. Using the first location box as a reference, the second location box is then filtered. The target location box is determined based on the first location box and the filtered second location box, and The location information of the target location box and the category of the target object are determined as pseudo-labels of the target object in the original data; Based on the labeled point cloud data of frame I and the unlabeled point cloud data of frame J, along with the corresponding pseudo-labels, the student network is iteratively trained until a preset number of rounds is reached, where I is a positive integer; and The weights of the target detection network are updated using the student network.

2. The method as described in claim 1, characterized in that, The target detection network had already been trained based on the labeled point cloud data of the I-frame before the first iteration update.

3. The method as described in claim 1, characterized in that, The disturbance processing includes at least one of the following: The original data is flipped, rotated, scaled, and masked.

4. The method as described in claim 1, characterized in that, The step of detecting the N frames of perturbation data through the target detection network and generating a second location box for the detected perturbation objects in each frame of perturbation data includes: The N frames of perturbation data are input into the target detection network to obtain the category of the perturbation object detected in each frame of perturbation data; Generate a perturbation location bounding box for each perturbing object; and The perturbation location frame of each perturbed object is subjected to inverse perturbation processing to obtain the second location frame of each perturbed object. The coordinate system of the second location frame is the same as that of the first location frame.

5. The method as described in claim 1, characterized in that, When multiple disturbance objects are detected in each frame of disturbance data, the filtering includes: Obtain the second bounding box of each perturbation object in each frame of perturbation data; Using the first location box as a reference, the multiple second location boxes in each frame of perturbation data are sequentially matched with the first location box in terms of distance; and The second location box that is closest to the first location box in each frame of perturbation data is selected, and the closest second location box is determined as the nearest location box, thereby obtaining N nearest location boxes that correspond one-to-one with the N frames of perturbation data.

6. The method as described in claim 1, characterized in that, When only one disturbance object is detected in each frame of disturbance data, the filtering includes: The second location box of the disturbed object in each frame of disturbance data is determined as the nearest location box, thus obtaining N nearest location boxes that correspond one-to-one with the N frames of disturbance data.

7. The method as described in claim 5 or 6, characterized in that, The screening also includes: Based on the error matching between the N nearest position boxes and the first position box, the remaining nearest position boxes with an error less than a first preset threshold are filtered out, and the remaining nearest position boxes are the filtered second position boxes.

8. The method as described in claim 7, characterized in that, The error matching based on the N nearest bounding boxes and the first bounding box includes: The N nearest location boxes are sequentially matched with the first location box for error matching.

9. The method as described in claim 7, characterized in that, The error matching based on the N nearest bounding boxes and the first bounding box includes: From the N nearest position boxes, remove the nearest position boxes whose distance from the first position box is greater than a second preset threshold to obtain M nearest position boxes, where M is a positive integer less than or equal to N; and The M nearest location boxes are sequentially matched with the first location box for error matching.

10. The method as described in claim 1, characterized in that, Determining the target location box based on the first location box and the filtered second location box includes: The target location box is obtained by averaging the first location box and the filtered second location box; or One of the first location box and the filtered second location box is determined as the target location box.

11. The method as described in claim 1, characterized in that, The step of updating the weights of the object detection network using the student network includes: Obtain the weights of the student network after training for the preset number of rounds; Obtain the old weights of the target detection network after the last update; and The target weights of the target detection network are determined based on the weights of the student network and the old weights.

12. A target detection method, comprising: Obtain n frames of point cloud data to be detected; The n frames of point cloud data to be detected are input into the target detection network to generate target detection results, wherein the target detection network is trained according to the target detection network training method as described in any one of claims 1 to 11.

13. A target detection system, characterized in that, include: At least one storage medium storing a target detection network for target detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the target detection system is running, the at least one processor reads the target detection network and executes: The point cloud data to be detected is input into the target detection network; The target detection network is used to detect the point cloud data to be detected, and the category of the target object is obtained. The target detection network generates a prediction bounding box for the target object. as well as The category of the target object and the position information of the prediction box are used to determine the predicted label of the target object. The target detection network is trained by the method described in any one of claims 1 to 11.

14. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 12.

Images