Related method, electronic device and storage medium for constructing point cloud labeling dataset

Abstract

The application discloses a related method for constructing a point cloud labeling data set, an electronic device and a storage medium. The method for constructing the point cloud labeling data set comprises the following steps: acquiring multiple frames of point cloud data in a preset time period; performing a first detection operation on any frame of point cloud data in the multiple frames of point cloud data to obtain at least one detection target of the any frame of point cloud data; selecting corresponding point cloud data from the multiple frames of point cloud data based on inconsistency between the at least one detection target of the any frame of point cloud data and at least one reference target of the any frame of point cloud data, thereby obtaining the point cloud labeling data set; and the at least one reference target is obtained by performing a second detection operation on the any frame of point cloud data. The method has no limitation on the use condition, and can effectively improve the data mining effect by mining data based on the inconsistency between the detection target and the reference target.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method, electronic device, and storage medium for constructing a point cloud annotation dataset. Background Technology

[0002] Training a high-precision point cloud 3D object detection model requires a large amount of labeled point cloud data, but blindly labeling all collected data would consume a lot of manpower and resources. Some existing solutions use active learning algorithms to mine valuable data to obtain the required labeled point cloud data, but the conditions for use are often stringent, resulting in poor data mining results. Summary of the Invention

[0003] This application provides at least one method, electronic device, and storage medium for constructing a point cloud annotation dataset to solve the above-mentioned problems.

[0004] The first aspect of this application provides a method for constructing a point cloud annotation dataset. The method includes: acquiring multiple frames of point cloud data within a preset time period; performing a first detection operation on any frame of point cloud data in the multiple frames to obtain at least one detection target of the any frame; selecting corresponding point cloud data from the multiple frames based on the inconsistency between the at least one detection target of the any frame and at least one reference target of the any frame, thereby obtaining a point cloud annotation dataset; wherein the at least one reference target is obtained by performing a second detection operation on the any frame.

[0005] In one embodiment, the inconsistency between at least one detected target and at least one reference target of any frame of point cloud data includes: a first inconsistency between the successfully paired detected target and the reference target, a second inconsistency between the unpaired detected target and the reference target, and a third inconsistency between the unpaired reference target, wherein the successfully paired detected target and the reference target belong to the same category.

[0006] In one embodiment, successful pairing of any of the detected targets with any of the reference targets includes: determining the cross-union ratio (CUP) between any of the detected targets and each of the at least one reference target to obtain at least one CUP corresponding to any of the detected targets; and obtaining at least one CUP corresponding to any of the detected targets that is greater than a preset threshold, wherein the reference target and the detected target corresponding to the same category and the maximum CUP are characterized as successfully paired.

[0007] In one embodiment, each of the at least one detection target corresponds to a first confidence level, and each of the at least one reference target corresponds to a second confidence level;

[0008] The formula for calculating the first inconsistency is:

[0009] inconRB=(1–IOURB)+|scoreR–scoreB|

[0010] Wherein, inconRB represents the first inconsistency, IOURB represents the intersection-union ratio between the detected target and the reference target of the same category, scoreR represents the first confidence level corresponding to the detected target, and scoreB represents the second confidence level corresponding to the reference target; and / or, the second inconsistency is characterized by the first confidence level corresponding to the detected target; and / or, the third inconsistency is characterized by the second confidence level corresponding to the reference target.

[0011] In one embodiment, the inconsistency corresponding to any frame of point cloud data is the sum of the first inconsistency, the second inconsistency, and the third inconsistency.

[0012] In one embodiment, one of the first detection operation processing and the second detection operation processing includes: performing a folding process on any frame of point cloud data to obtain folded point cloud data; inputting the folded point cloud data into a preset target detection model to obtain at least one initial target; performing a folding process on the at least one initial target to obtain at least one target; the other of the first detection operation processing and the second detection operation processing includes: inputting any frame of point cloud data into the preset target detection model to obtain at least one target.

[0013] In one embodiment, performing a flipping process on any frame of point cloud data includes: obtaining the coordinates of each point in the point cloud data of any frame, and flipping the coordinates of each point based on the horizontal and vertical axes of the coordinate system in which the coordinates of each point are located, thereby obtaining the corresponding flipped point cloud data; and / or, performing a flipping process on the at least one initial target includes: obtaining the coordinates of each initial target in the at least one initial target, and flipping the coordinates of each initial target based on the horizontal and vertical axes of the coordinate system in which the coordinates of each initial target are located, thereby obtaining the corresponding at least one target.

[0014] The second aspect of this application provides a model training method, comprising: acquiring a point cloud annotation dataset, performing annotation operations on the point cloud annotation dataset to obtain annotation results; wherein the point cloud annotation dataset is obtained using the point cloud annotation dataset construction method in the first aspect described above;

[0015] The model is trained using the labeled results to obtain the trained target model.

[0016] A third aspect of this application provides an electronic device including a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the point cloud annotation dataset construction method of the first aspect or the model training method of the second aspect.

[0017] The fourth aspect of this application provides a non-volatile computer-readable storage medium for storing program instructions, which, when executed by a processor, are used to implement the point cloud annotation dataset construction method in the first aspect or the model training method in the second aspect.

[0018] The above scheme obtains at least one detection target by performing a first detection operation on any frame of point cloud data from multiple frames, and obtains at least one reference target by performing a second detection operation on any frame of point cloud data. It then determines the inconsistency between the at least one detection target and the at least one reference target in any frame of point cloud data, thereby obtaining the inconsistency corresponding to each frame of point cloud data. Based on the inconsistency corresponding to each frame of point cloud data, it selects corresponding point cloud data from the multiple frames of point cloud data to obtain a point cloud annotation dataset, thus achieving data mining. The scheme of this application has no restrictions on usage conditions and effectively improves the data mining effect by performing data mining through the inconsistency between the detection target and the reference target.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0021] Figure 1 This is a flowchart illustrating the method for constructing a point cloud annotation dataset in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram showing the distribution of the detection target box and the reference target box in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of a point cloud data flipping scenario in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the initial target folding scenario in an embodiment of this application;

[0025] Figure 5 This is a flowchart illustrating the model training method in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of the electronic device in the embodiments of this application;

[0027] Figure 7 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium in an embodiment of this application. Detailed Implementation

[0028] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the application. Similarly, the following embodiments are only some, not all, embodiments of the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0029] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this document means two or more. Additionally, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. Furthermore, the terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0031] As mentioned above, some existing solutions use active learning algorithms to mine valuable data in order to obtain the required point cloud annotation data. However, the conditions for use are often stringent, resulting in poor data mining results.

[0032] To address these issues, this application provides a method, electronic device, and storage medium for constructing a point cloud annotation dataset.

[0033] Please see Figure 1 , Figure 1 This is a flowchart illustrating the method for constructing the point cloud annotation dataset in this application's embodiments. It should be noted that if substantially the same result is obtained, the method of this application does not necessarily require further elaboration. Figure 1 The illustrated process sequence is limited. This method can be applied to electronic devices with computational functions. These devices can execute this method by receiving information collected by sensor devices, such as millimeter-wave radar, lidar, or cameras equipped on the model training vehicle. The sensor devices perceive the dynamic real-world environment surrounding the vehicle during its operation. Figure 1 As shown, the method for constructing a point cloud annotation dataset includes the following steps:

[0034] S11. Obtain multi-frame point cloud data within a preset time period.

[0035] Point cloud data can be collected using radar sensors, for example, by mounting the radar sensor on a mobile device. This mobile device can be an automated mobile device, such as a robot or a model training vehicle.

[0036] In some embodiments, the radar sensor may be a lidar sensor, such as a mechanical lidar, semi-solid-state lidar, or solid-state lidar. In one embodiment, the radar sensor may be any radar device that provides point cloud data and is used for model training to meet perception accuracy requirements.

[0037] In one application scenario, while an autonomous vehicle is driving on the road, radar sensors equipped on the vehicle collect point cloud data in real time, thereby obtaining multiple frames of point cloud data within a preset time period. The preset time period includes the current time and a preset number of times before the current time, such as three times before the current time, or other achievable numbers, which can be selected according to actual usage requirements and are not specifically limited. For example, during the autonomous vehicle's operation, the radar sensors collect point cloud data in real time. The current time is t5, meaning that the radar sensors collect one frame of point cloud data at time t1, one frame at time t2, one frame at time t3, one frame at time t4, and one frame at time t5, obtaining multiple frames of point cloud data within the preset time period, such as obtaining three frames of point cloud data collected by the radar sensors at times t3, t4, and t5.

[0038] S12. Perform a first detection operation on any frame of point cloud data in the multi-frame point cloud data to obtain at least one detection target in any frame of point cloud data.

[0039] It is understandable that performing a first detection operation on a frame of point cloud data can yield at least one detection target for that frame of point cloud data; performing a first detection operation on each frame of point cloud data in the acquired multi-frame point cloud data can yield at least one detection target for each frame of point cloud data in the multi-frame point cloud data.

[0040] S13. Based on the inconsistency between at least one detected target and at least one reference target in any frame of point cloud data, select corresponding point cloud data from multiple frames of point cloud data to obtain a point cloud annotation dataset. The at least one reference target is obtained by performing a second detection operation on any frame of point cloud data.

[0041] Understandably, performing a second detection operation on any frame of point cloud data in a multi-frame point cloud dataset yields at least one reference target for that frame, thereby obtaining at least one reference target for each frame of point cloud data in the multi-frame point cloud dataset.

[0042] Based on the inconsistency between at least one detected target and at least one reference target in any frame of point cloud data, corresponding point cloud data are selected from multiple frames of point cloud data to obtain a point cloud annotation dataset. It can be understood that determining the inconsistency between at least one detected target and at least one reference target in a frame of point cloud data means that the frame of point cloud data corresponding to at least one detected target and the frame of point cloud data corresponding to at least one reference target are the same frame of point cloud data. Therefore, the inconsistency corresponding to that frame of point cloud data can be obtained, and thus the inconsistency corresponding to each frame of point cloud data in multiple frames of point cloud data can be obtained.

[0043] After obtaining the inconsistencies corresponding to each frame of point cloud data, the multi-frame point cloud data can be sorted according to the specific values ​​of the inconsistencies. Then, using a preset selection method, corresponding point cloud data can be selected from the sorted multi-frame point cloud data to obtain a point cloud annotation dataset. The preset selection method can be set according to actual usage requirements and is not specifically limited. In other embodiments, corresponding point cloud data can also be selected from multi-frame point cloud data as a point cloud annotation dataset using other methods, without specific limitations.

[0044] For example, given one million frames of point cloud data, a first detection operation is performed on any frame to obtain at least one detected target. A second detection operation is then performed on the same frame to obtain at least one reference target. Inconsistencies between the at least one detected target and the at least one reference target in each frame are determined, thus identifying the inconsistencies corresponding to each of the one million frames. Based on these inconsistencies, the one million frames are sorted, either in descending order of inconsistency. The first 300,000 frames from the sorted set are selected as the point cloud annotation dataset; alternatively, a selection threshold can be preset, and point cloud data with inconsistencies exceeding this threshold are selected as the point cloud annotation dataset, thereby achieving data mining.

[0045] The above scheme obtains at least one detection target by performing a first detection operation on any frame of point cloud data from multiple frames, and obtains at least one reference target by performing a second detection operation on any frame of point cloud data. It then determines the inconsistency between the at least one detection target and the at least one reference target in any frame of point cloud data, thereby obtaining the inconsistency corresponding to each frame of point cloud data. Based on the inconsistency corresponding to each frame of point cloud data, it selects corresponding point cloud data from the multiple frames of point cloud data to obtain a point cloud annotation dataset, thus achieving data mining. The scheme of this application has no restrictions on usage conditions and effectively improves the data mining effect by performing data mining through the inconsistency between the detection target and the reference target.

[0046] As described above, based on the inconsistencies corresponding to each frame in multi-frame point cloud data, corresponding point cloud data is selected from the multi-frame point cloud data to obtain a point cloud annotation dataset. In one embodiment of this application, the inconsistency between at least one detected target in any frame of point cloud data and at least one reference target in any frame of point cloud data includes: a first inconsistency between a successfully paired detected target and a reference target, a second inconsistency between unpaired detected targets, and a third inconsistency between unpaired reference targets, wherein the successfully paired detected targets and the reference targets belong to the same category.

[0047] Understandably, performing a first detection operation on any frame of point cloud data yields at least one detected target, where each detected target corresponds to at least one class probability, and the sum of the probabilities of at least one class corresponding to any detected target is 1. Performing a second detection operation on any frame of point cloud data yields at least one reference target, where each reference target corresponds to at least one class probability, and the sum of the probabilities of at least one class corresponding to any reference target is 1. The class corresponding to the highest class probability is used to represent the class of the corresponding detected target, and the class corresponding to the highest class probability is used to represent the class of the corresponding reference target. The detected target and the reference target belong to the same class; that is, the class corresponding to the highest class probability of the detected target is the same as the class corresponding to the highest class probability of the reference target.

[0048] For example, if the probability of the detected target being a vehicle is 0.8 and the probability of the detected target being a pedestrian is 0.2, then the detected target is classified as a vehicle. If the probability of the reference target x1 being a vehicle is 0.1 and the probability of the reference target being a pedestrian is 0.9, then the reference target is classified as a pedestrian. If the probability of the reference target x2 being a vehicle is 0.85 and the probability of the reference target being a pedestrian is 0.15, then the reference target x2 is classified as a vehicle. The detected target and the reference target x2 belong to the same category.

[0049] Understandably, pairing at least one detection target with at least one reference target includes both successful pairing and unsuccessful pairing. The first inconsistency is characterized by the inconsistency between successfully paired detection targets and reference targets; the second inconsistency is characterized by the inconsistency between unsuccessfully paired detection targets; and the third inconsistency is characterized by the inconsistency between unsuccessfully paired reference targets.

[0050] As described above, at least one detection target is paired with at least one reference target. In one embodiment of this application, successful pairing of any detection target with any reference target includes: determining the cross-union ratio (CUP) between any detection target and each of the at least one reference target to obtain at least one CUP corresponding to any detection target; obtaining at least one CUP corresponding to any detection target that is greater than a preset threshold, wherein the reference target and the detection target corresponding to the same category and the maximum CUP are characterized as successfully paired.

[0051] The Intersection-over-Union (IoU) ratio between the detected target and the reference target is used to characterize the overlap rate between them; that is, the ratio of the intersection to the union of the detected and reference targets. Ideally, the detected target and the reference target completely overlap, in which case the ratio is 1, meaning the IoU ratio between the detected target and the reference target is 1. Calculating the IoU ratio between any detected target and each of at least one reference target yields at least one IoU ratio for the detected target.

[0052] Obtain at least one cross-union ratio (CUP) greater than a preset threshold for any detected target, wherein the reference target and the detected target corresponding to the five same categories and the maximum CUP are characterized as successfully paired.

[0053] Understandably, the preset threshold can be set according to actual usage requirements and is not specifically limited. For any detected target, there is at least one cross-union ratio (CUI). The CUI corresponding to at least one detected target that is greater than the preset threshold is obtained, and the magnitudes of the obtained at least one CUI are compared with each other.

[0054] The maximum value of the cross-union ratio can be obtained by comparison, where the 0 reference target of the same category and the maximum cross-union ratio is characterized as a successful pairing with the detected target.

[0055] For example, the detection targets include three targets (n1, n2, n3), where target n1 is classified as a vehicle, target n2 as a pedestrian, and target n3 as a vehicle; the reference targets include three reference targets (m1, m2, m3), where reference target m1 is classified as a vehicle.

[0056] For vehicles, reference target m2 is classified as a vehicle, and reference target m3 is classified as a pedestrian. The cross-union ratio (CUI) between detected target n1 and reference target m1 is calculated to be 0.8. The cross-union ratio between detected target n1 and reference target m1 is calculated.

[0057] The cross-union ratio (CUNR) between reference targets m2 is 0.6, and the CUNR between detected target n1 and reference target m3 is 0.1. Similarly, the CUNR between detected target n2 and the three reference targets (m1, m2, m3) are 0.1, 0.3, and 0.9, respectively; and the CUNR between detected target n3 and the three reference targets (m1, m2, m3) are 0.2, 0.8, and 0.1, respectively.

[0058] Assuming a preset threshold of 0.5, for a detection target n1, there are two targets whose intersection-union ratio (IU) is greater than the preset threshold.

[0059] Let the thresholds be 0.8 for the cross-union ratio (CUB) between detected target n1 and reference target m1, and 0.6 for the CUB between detected target n1 and reference target m2. The maximum CUB between detected target n1 and reference target m1 is 0.8. If detected target n1 and reference target m1 belong to the same category, then the corresponding reference target m1 and detected target n1 are successfully paired.

[0060] 5. For a detected target n2, if one of its intersection-union ratios (IU) is greater than a preset threshold (i.e., the IU between detected target n2 and reference target m3 is 0.9, and detected target n2 and reference target m3 belong to the same category), then the corresponding reference target m3 and detected target n2 are successfully paired. For a detected target n3, if one of its IU is greater than a preset threshold (i.e., the IU between detected target n3 and reference target m2 is 0.8, and detected target n3 and reference target m2 belong to the same category), then the corresponding reference target m2 and detected target n3 are successfully paired.

[0061] In one embodiment of this application, each of the at least one detection target corresponds to a first confidence level, and each of the at least one reference target corresponds to a second confidence level. The formula for calculating the first inconsistency is:

[0062] incon RB = (1 – IOU) RB )+|score R –score B |

[0063] Among them, incon RB Indicates the first inconsistency, IOU RB The score represents the crossover ratio (CoU) between detected targets and reference targets of the same category. R The score represents the first confidence level corresponding to the detected target. B The second confidence level corresponds to the reference target; and / or, the second inconsistency is characterized by the first confidence level corresponding to the detection target; and / or, the third inconsistency is characterized by the second confidence level corresponding to the reference target.

[0064] Understandably, performing a first detection operation on the point cloud data yields at least one detection target, and performing a second detection operation on the point cloud data yields at least one reference target. Each of the at least one detection target corresponds to a first confidence level, and each of the at least one reference target corresponds to a second confidence level.

[0065] The first inconsistency is characterized by the inconsistency between the successfully paired detection target and the reference target, and can be calculated using the formula for the first inconsistency. The second inconsistency is characterized by the inconsistency corresponding to the detection target, and is specifically characterized by the first confidence level corresponding to the detection target. The third inconsistency is characterized by the inconsistency corresponding to the reference target, and is specifically characterized by the second confidence level corresponding to the reference target.

[0066] To facilitate understanding of the scheme of this application, each detection target can be represented as a box to obtain at least one detection target box; each reference target can be represented as a box to obtain at least one reference target box.

[0067] For example, Figure 2 This is a schematic diagram illustrating the distribution of the detected target box and the reference target box in an embodiment of this application, as shown below. Figure 2 As shown, the first confidence level corresponding to the detection target box 101 is 0.8, the first confidence level corresponding to the detection target box 102 is 0.6, the first confidence level corresponding to the detection target box 103 is 0.3, and the first confidence level corresponding to the detection target box 104 is 0.3; the second confidence level corresponding to the reference target box 201 is 0.6, the second confidence level corresponding to the reference target box 202 is 0.5, the second confidence level corresponding to the reference target box 203 is 0.5, and the second confidence level corresponding to the reference target box 204 is 0.4.

[0068] Among them, the detected target box 101 and the reference target box 201 are successfully matched, and the corresponding intersection-union ratio is 0.7; then the first inconsistency between the detected target box 101 and the reference target box 201 is (1

[0069] 0.7) + |0.8 – 0.6| = 0.5. The detection target box 102 successfully matches the reference target box 202, with a corresponding intersection-union ratio (IU / R) of 0.6; therefore, the first inconsistency between the detection target box 102 and the reference target box 202 is (1 – 0.6) + |0.6 – 0.5| = 0.5. The detection target box 103 successfully matches the reference target box 203, with a corresponding IU / R of 0.9; therefore, the first inconsistency between the detection target box 103 and the reference target box 203 is (1 – 0.9) + |0.3 – 0.5| = 0.3.

[0070] If the detection target box 104 is found to be unmatched, then the second inconsistency of the detection target box 104 is characterized by its corresponding first confidence level, that is, the second inconsistency of the detection target box 104 is 0.3. If the reference target box 204 is found to be unmatched, then the third inconsistency of the reference target box 204 is characterized by its corresponding second confidence level, that is, the third inconsistency of the reference target box 204 is 0.4.

[0071] As described above, the inconsistency between at least one detected target and at least one reference target in any frame of point cloud data includes a first inconsistency, a second inconsistency, and a third inconsistency. In one embodiment of this application, the inconsistency corresponding to any frame of point cloud data is the sum of the first inconsistency, the second inconsistency, and the third inconsistency.

[0072] The inconsistency corresponding to any frame of point cloud data is the sum of the first inconsistency, the second inconsistency, and the third inconsistency. Understandably, performing a first detection operation on any frame of point cloud data from multiple frames yields at least one detection target, each of which has a first confidence level. Performing a second detection operation on any frame yields at least one reference target, each of which has a second confidence level. The first inconsistency between successfully paired detection targets and reference targets, the second inconsistency of unpaired detection targets, and the third inconsistency of unpaired reference targets are calculated. The number of first, second, and third inconsistencies can be one or more. Adding the first, second, and third inconsistencies corresponding to any frame of point cloud data yields the inconsistency corresponding to that frame, thus allowing us to obtain the inconsistency corresponding to each frame of point cloud data across multiple frames.

[0073] For example, please refer to again Figure 2 Continuing with the above embodiments, the detection target and reference target are represented in the form of boxes. The first inconsistency between the detection target box 101 and the reference target box 201 is 0.5, the first inconsistency between the detection target box 102 and the reference target box 202 is 0.5, the first inconsistency between the detection target box 103 and the reference target box 203 is 0.3, the second inconsistency between the detection target box 104 is 0.3, and the third inconsistency between the reference target box 204 is 0.4. Therefore, all the calculated first, second, and third inconsistencies are added together, that is, 0.5, 0.5, 0.3, 0.3, and 0.4 are added together to obtain the inconsistency corresponding to the point cloud data as 2.

[0074] As described above, a first detection operation is performed on any frame of point cloud data to obtain at least one detected target; a second detection operation is performed on any frame of point cloud data to obtain at least one reference target. In one embodiment of this application, one of the first and second detection operations includes: performing a folding operation on any frame of point cloud data to obtain folded point cloud data; inputting the folded point cloud data into a preset target detection model to obtain at least one initial target; and performing a folding operation on the at least one initial target to obtain at least one target. The other of the first and second detection operations includes: inputting any frame of point cloud data into a preset target detection model to obtain at least one target.

[0075] Understandably, performing a folding process on any frame of point cloud data means performing a folding process on each frame of point cloud data in a multi-frame point cloud dataset to obtain corresponding multi-frame folded point cloud data. Inputting any frame of folded point cloud data into a preset target detection model will yield at least one initial target. The preset target detection model can be selected based on actual usage requirements and is not specifically limited. Then, performing a folding process on each of the at least one initial target will yield at least one target; that is, performing a first detection operation / second detection operation on any frame of point cloud data will correspondingly yield at least one target.

[0076] The first detection operation and the second detection operation, one of which includes: inputting any frame of point cloud data into a preset target detection model to obtain at least one target. It is understood that any frame of point cloud data from the acquired multi-frame point cloud data can be directly input into the preset target detection model without any other processing, and at least one target can be obtained. That is, performing the first detection operation / second detection operation on any frame of point cloud data will correspondingly obtain at least one target. The preset target detection model can be selected according to actual usage requirements and is not specifically limited.

[0077] As described above, any frame of point cloud data is flipped to obtain flipped point cloud data; at least one initial target is flipped to obtain at least one target. In one embodiment of this application, flipping any frame of point cloud data includes: obtaining the coordinates of each point in the point cloud data, and flipping the coordinates of each point based on the horizontal and vertical axes of the coordinate system in which the coordinates of each point are located, thereby obtaining the corresponding flipped point cloud data; and / or, flipping at least one initial target includes: obtaining the coordinates of each initial target in the at least one initial target, and flipping the coordinates of each initial target based on the horizontal and vertical axes of the coordinate system in which the coordinates of each initial target are located, thereby obtaining the corresponding at least one target.

[0078] Understandably, for each point in any frame of point cloud data, the coordinates of each point are obtained. Based on the horizontal and vertical axes of the coordinate system in which each point's coordinates lie, the coordinates of each point are flipped to obtain the corresponding flipped point cloud data. The order in which each point is flipped along the horizontal and vertical axes is not restricted; for example, each point can be flipped along the horizontal axis first and then along the vertical axis, or vice versa.

[0079] For example, Figure 3 This is a schematic diagram of a point cloud data flipping scenario in an embodiment of this application, such as... Figure 3 As shown, point A is any point in any frame of point cloud data. To obtain the coordinates of point A in the first quadrant, first, point A is flipped along the horizontal axis to obtain point A' in the fourth quadrant. Then, point A' in the fourth quadrant is flipped along the vertical axis to obtain point A'" in the third quadrant. Alternatively, point A is first flipped along the vertical axis to obtain point A' in the second quadrant. Then, point A' in the second quadrant is flipped along the horizontal axis to obtain point A'" in the third quadrant. This completes the flipping of point A, resulting in the flipped point A'". In this way, any frame of point cloud data can be flipped to obtain the corresponding flipped point cloud data.

[0080] Understandably, obtaining the coordinates of each initial target within at least one initial target is acceptable. This could be the vertex coordinates of each initial target, the midpoint coordinates of each edge of the initial target, or any other achievable positional coordinate value; no specific limitation is imposed. Based on the horizontal and vertical axes of the coordinate system containing each initial target's coordinates, the coordinates of each initial target are flipped to obtain the corresponding target. The order in which the coordinates of each initial target are flipped along the horizontal and vertical axes is not restricted; for example, the coordinates of each initial target can be flipped first along the horizontal axis and then along the vertical axis, or vice versa.

[0081] For example, Figure 4 This is a schematic diagram of the initial target folding scenario in an embodiment of this application, such as... Figure 4 As shown, for any initial target B among at least one initial target, obtain the coordinates of the initial target B in the first quadrant. First, flip the initial target B based on the horizontal axis to obtain the initial target B' in the fourth quadrant. Then, flip the initial target B' in the fourth quadrant based on the vertical axis to obtain the target B” in the third quadrant. Alternatively, first, flip the initial target B based on the vertical axis to obtain the initial target B' in the second quadrant. Then, flip the initial target B' in the second quadrant based on the horizontal axis to obtain the target B” in the third quadrant. This completes the flipping of the initial target B, resulting in target B”. In this way, at least one initial target can be flipped to obtain at least one corresponding target.

[0082] Please see Figure 5 , Figure 5 This is a flowchart illustrating the model training method in an embodiment of this application. The model training method includes the following steps:

[0083] S21. Obtain the point cloud annotation dataset, and perform annotation operations on the point cloud annotation dataset to obtain the annotation results; wherein, the point cloud annotation dataset is obtained by using the point cloud annotation dataset construction method in the above embodiment.

[0084] A point cloud annotation dataset is obtained, which is constructed using the point cloud annotation dataset construction method described in the above embodiments. For example, multiple frames of point cloud data within a preset time period are acquired; a first detection operation is performed on any frame of point cloud data to obtain at least one detection target for that frame; based on the inconsistency between at least one detection target and at least one reference target of that frame, corresponding point cloud data is selected from the multiple frames to obtain the point cloud annotation dataset; wherein, at least one reference target is obtained by performing a second detection operation on any frame of point cloud data.

[0085] The point cloud annotation dataset is processed to perform annotation operations to obtain annotation results. The specific annotation operation methods are not specifically limited.

[0086] S22. The model is trained using the labeled results to obtain the target model after training.

[0087] The model is trained by processing the labeled results to obtain the target model, thereby achieving the training of the model.

[0088] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0089] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device in an embodiment of this application. The electronic device 600 includes a memory 601 and a processor 602 coupled to each other. The processor 602 executes program instructions stored in the memory 601 to implement the steps in the above-described embodiment of the point cloud annotation dataset construction method or the steps in the above-described embodiment of the model training method. In a specific implementation scenario, the electronic device 600 may include, but is not limited to, a microcomputer or a server.

[0090] Specifically, processor 602 controls itself and memory 601 to implement the steps in the above-described method embodiments for constructing point cloud annotation datasets or the above-described method embodiments for model training. Processor 602 can also be called a CPU (Central Processing Unit), and may be an integrated circuit chip with signal processing capabilities. Processor 602 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 602 can be implemented using integrated circuit chips.

[0091] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium in an embodiment of this application. The computer-readable storage medium 700 is used to store program instructions 701. When executed by the processor 602, the program instructions 701 are used to implement the steps in the above-described embodiment of the point cloud annotation dataset construction method or the steps in the above-described embodiment of the model training method.

[0092] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed methods and related devices can be implemented in other ways. For example, the related device implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication disconnection shown or discussed may be indirect coupling or communication disconnection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0094] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0095] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for constructing a point cloud annotation dataset, characterized in that, include: Acquire multiple frames of point cloud data within a preset time period; A first detection operation is performed on any frame of point cloud data in the multi-frame point cloud data to obtain at least one detection target of the point cloud data in any frame of point cloud data. The first detection operation is used to obtain at least one category probability corresponding to each detection target. Each of the at least one detection target has a first confidence level. Based on the inconsistency between at least one detected target and at least one reference target in any frame of point cloud data, corresponding point cloud data are selected from the multi-frame point cloud data to obtain a point cloud annotation dataset. Wherein, the at least one reference target is obtained by performing a second detection operation on any frame of point cloud data, the second detection operation is used to obtain at least one category probability corresponding to each reference target, and each of the at least one reference target has a corresponding second confidence level; The inconsistency includes a first inconsistency between the successfully paired detection target and the reference target, a second inconsistency between the unpaired detection targets, and a third inconsistency between the unpaired reference targets, wherein the successfully paired detection targets and the reference targets belong to the same category; The formula for calculating the first inconsistency is: incon RB =(1–IOU RB )+|score R –score B | Among them, incon RB Indicates the first inconsistency, IOU RB The score represents the intersection-union ratio (IoU) between the detected targets and the reference target of the same category. R The score represents the first confidence level corresponding to the detected target. B This represents the second confidence level corresponding to the reference target; The second inconsistency is characterized by the first confidence level corresponding to the detection target; The third inconsistency is characterized by the second confidence level corresponding to the reference target.

2. The method according to claim 1, characterized in that, Successful pairing of any of the detection targets with any of the reference targets includes: Determine the cross-union ratio (CUI) between any of the detected targets and each of the at least one reference target to obtain at least one CUI corresponding to any of the detected targets; Obtain at least one cross-union ratio (CUP) greater than a preset threshold for any of the detected targets, wherein the reference target and the detected target corresponding to the maximum CUP of the same category are characterized as successfully paired.

3. The method according to claim 1, characterized in that, The inconsistency corresponding to any frame of point cloud data is the sum of the first inconsistency, the second inconsistency, and the third inconsistency.

4. The method according to claim 1, characterized in that, One of the first detection operation processing and the second detection operation processing includes: Fold any frame of point cloud data to obtain folded point cloud data; The folded point cloud data is input into a preset target detection model to obtain at least one initial target; The at least one initial target is folded to obtain at least one target; The other of the first detection operation processing and the second detection operation processing includes: Input any frame of point cloud data into the preset target detection model to obtain at least one target.

5. The method according to claim 4, characterized in that, Folding the point cloud data in any frame includes: Obtain the coordinates of each point in any frame of point cloud data; based on the horizontal and vertical axes of the coordinate system in which each point's coordinates lie, flip the coordinates of each point to obtain the corresponding flipped point cloud data; and / or Folding the at least one initial target includes: Obtain the coordinates of each initial target among the at least one initial targets, and based on the horizontal and vertical axes of the coordinate system in which the coordinates of each initial target are located, flip the coordinates of each initial target to obtain at least one corresponding target.

6. A model training method, characterized in that, include: A point cloud annotation dataset is obtained, and annotation operations are performed on the point cloud annotation dataset to obtain annotation results; wherein, the point cloud annotation dataset is obtained using the point cloud annotation dataset construction method according to any one of claims 1-5; The model is trained using the labeled results to obtain the trained target model.

7. An electronic device, characterized in that, It includes a memory and a processor that are coupled to each other, the processor being used to execute program instructions stored in the memory to implement the method for constructing a point cloud annotation dataset as described in any one of claims 1-5 or the model training method as described in claim 6.

8. A non-volatile computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions, which, when executed by a processor, are used to implement the method for constructing a point cloud annotation dataset as described in any one of claims 1-5 or the model training method as described in claim 6.

Images