Point cloud data segmentation model training method, device, storage medium and electronic device

Abstract

The present application proposes a point cloud data segmentation model training method, device, storage medium and electronic device, which obtain the annotation result of the initial model on the first point cloud data, and the first point cloud data is the point cloud data that has not been manually annotated; determine the sample point cloud data used as the training sample from the first point cloud data according to the annotation result; perform weighted training on the initial model based on the sample point cloud data so that the initial model converges, and the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence. In the present application, the sample point cloud data used as the training sample is annotated by the initial model, and new training samples can be obtained without manual annotation, thereby solving the problems of difficulty, long cycle and high cost of manual annotation of three-dimensional point clouds.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and more specifically, to a point cloud data segmentation model training method, device, storage medium and electronic device. Background Art

[0002] As various types of automated equipment are increasingly used in agriculture, they can provide more and more information, including two-dimensional image information and three-dimensional point cloud information. The three-dimensional point cloud segmentation task (or process) is an important step in the application process of three-dimensional point cloud information. The three-dimensional point cloud segmentation task is more dependent on the annotation data of the three-dimensional point cloud data.

[0003] Compared with the annotation of two-dimensional images, the annotation of three-dimensional point cloud data is more difficult. At present, the demand for three-dimensional point cloud segmentation tasks is increasing. The existing three-dimensional point cloud segmentation algorithms rely heavily on annotation data, but the annotation of three-dimensional point cloud data is difficult, time-consuming and costly. As a result, the implementation of three-dimensional point cloud segmentation is difficult, time-consuming and costly. Summary of the invention

[0004] The purpose of the present application is to provide a point cloud data segmentation model training method, device, storage medium and electronic device to at least partially improve the above-mentioned problems.

[0005] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a point cloud data segmentation model training method, the method comprising:

[0007] Obtaining a labeling result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually labeled, and the labeling result includes a prediction category and a confidence level corresponding to each point of the first point cloud data;

[0008] Determining sample point cloud data used as training samples from the first point cloud data according to the labeling result;

[0009] The initial model is weightedly trained based on the sample point cloud data to converge the initial model, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence.

[0010] In a second aspect, an embodiment of the present application provides a point cloud data segmentation model training device, the device comprising:

[0011] A processing unit, configured to obtain a labeling result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually labeled, and the labeling result includes a prediction category and a confidence level corresponding to each point of the first point cloud data;

[0012] The processing unit is further used to determine sample point cloud data used as training samples from the first point cloud data according to the labeling result;

[0013] A training unit is used to perform weighted training on the initial model based on the sample point cloud data to make the initial model converge, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence.

[0014] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.

[0016] Compared with the prior art, the point cloud data segmentation model training method, device, storage medium and electronic device provided in the embodiment of the present application obtain the annotation result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually annotated, and the annotation result includes the prediction category and confidence corresponding to each point of the first point cloud data; determine the sample point cloud data used as a training sample from the first point cloud data according to the annotation result; and perform weighted training on the initial model based on the sample point cloud data to make the initial model converge, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence. In this solution, the sample point cloud data used as training samples are annotated by the initial model, and new training samples can be obtained without manual annotation, thereby solving the problems of difficulty, long cycle and high cost of manual annotation of three-dimensional point clouds. The sample point cloud data is determined from the first point cloud data according to the annotation results. Therefore, the annotation results of the sample point cloud data have a high degree of credibility. On this basis, the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence level, so that the point cloud data segmentation model can focus on generalizing to data with poor performance, and the model has better generalization ability.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a process for training a point cloud data segmentation model provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of sub-steps of S107 provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of sub-steps of S103 provided in an embodiment of the present application;

[0023] Figure 5 One of the sub-step schematic diagrams of S103 provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of a preset range centered on a medium confidence point is provided for an embodiment of the present application;

[0025] Figure 7 One of the sub-step schematic diagrams of S103 provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of a preset range centered on a high confidence point is provided for an embodiment of the present application;

[0027] Fig. 9 One of the flow charts of the point cloud data segmentation model training method provided in the embodiment of the present application;

[0028] Fig.10 One of the flow charts of the point cloud data segmentation model training method provided in the embodiment of the present application;

[0029] Fig.11 A unit diagram of the point cloud data segmentation model training method provided in an embodiment of the present application.

[0030] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 201 - processing unit; 202 - training unit. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0032] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0034] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0035] In the description of the present application, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product of the application is usually placed when in use. They are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0036] In the description of this application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0037] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0038] The training of point cloud data segmentation models requires a large amount of labeled data. However, manual labeling of 3D point clouds is difficult, time-consuming, and costly. How to complete the training of point cloud data segmentation models while reducing the cost of manual labeling is exactly the problem that this application solution aims to overcome.

[0039] The present application embodiment provides an electronic device, which may be a computer device or a server. Figure 1 , a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12, and the processor 10 is used to execute an executable module stored in the memory 11, such as a computer program.

[0040] The processor 10 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the point cloud data segmentation model training method can be completed by the hardware integrated logic circuit in the processor 10 or the instructions in the form of software. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0041] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0042] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Although only one bidirectional arrow is used in the figure, it does not mean that there is only one bus 12 or only one type of bus 12 .

[0043] The memory 11 is used to store programs, such as programs corresponding to the point cloud data segmentation model training device. The point cloud data segmentation model training device includes at least one software function module that can be stored in the memory 11 in the form of software or firmware or fixed in the operating system (OS) of the electronic device. After receiving the execution instruction, the processor 10 executes the program to implement the point cloud data segmentation model training method.

[0044] Possibly, the electronic device provided in the embodiment of the present application further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus. The electronic device can communicate with other terminals via the communication interface 13.

[0045] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0046] The point cloud data segmentation model training method provided in the embodiment of the present application can be applied to but not limited to Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 , point cloud data segmentation model training methods include:

[0047] S102, obtaining the annotation result of the initial model on the first point cloud data.

[0048] The first point cloud data is point cloud data that has not been manually labeled, and the labeling result includes the prediction category and confidence level corresponding to each point of the first point cloud data.

[0049] It can be understood that the initial model is a point cloud data segmentation model (neural network model) that has completed preliminary training. The initial model can perform semantic segmentation on the first point cloud data to obtain the annotation results of the first point cloud data, and the annotation results include the prediction category and confidence corresponding to each point of the first point cloud data. For example, the prediction category of a certain point is a telephone pole with a confidence of 90; or the prediction category is a wire with a confidence of 80, and so on. Of course, the prediction category can also be a pond, a field, a road, a house, and so on. In one possible implementation, the prediction category can be empty.

[0050] S103: Determine sample point cloud data used as training samples from the first point cloud data according to the labeling result.

[0051] It can be understood that in a possible implementation, the confidence corresponding to the predicted category of a portion of the points in the first point cloud data is higher than the confidence qualification line, indicating that the confidence of the predicted category of the portion of points is high, and the portion of points can be used as training samples to participate in the training of the subsequent point cloud data segmentation model. For the specific process of determining the sample point cloud data, please refer to the contents of S103-1 to S103-3 in the following text.

[0052] S107, performing weighted training on the initial model based on the sample point cloud data to make the initial model converge.

[0053] Among them, the weight of each point in the sample point cloud data is negatively correlated with its corresponding confidence.

[0054] In the present application, the sample point cloud data used as training samples are annotated by the initial model, and new training samples can be obtained without manual annotation, thereby solving the problems of difficulty, long cycle and high cost of manual annotation of three-dimensional point clouds. The sample point cloud data is determined from the first point cloud data according to the annotation result, so the annotation result of the sample point cloud data has a high credibility. On this basis, the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence, so that the point cloud data segmentation model can focus on generalizing to data with poor performance, so that the model has better generalization ability.

[0055] It should be noted that although the sample point cloud data carries non-manually annotated pseudo-labels (the predicted category of each point), it is also composed of predicted categories with higher credibility after being screened by confidence. Therefore, the pseudo-labels carried by the sample point cloud data are accurate and will not cause the point cloud data segmentation model to learn in the wrong direction. In addition, the weight of each point in the sample point cloud data is negatively correlated with its corresponding confidence, that is, the higher the confidence, the lower the weight. The purpose is to allow the point cloud data segmentation model to focus on generalizing to data with poor performance (low confidence data).

[0056] In summary, the embodiment of the present application provides a point cloud data segmentation model training method, obtaining the annotation result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually annotated, and the annotation result includes the prediction category and confidence corresponding to each point of the first point cloud data; determining the sample point cloud data used as a training sample from the first point cloud data according to the annotation result; performing weighted training on the initial model based on the sample point cloud data to make the initial model converge, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence. In this scheme, the sample point cloud data used as the training sample is annotated by the initial model, and new training samples can be obtained without manual annotation, thereby solving the problems of difficulty, long cycle and high cost of manual annotation of three-dimensional point clouds. The sample point cloud data is determined from the first point cloud data according to the annotation result, so the annotation result of the sample point cloud data has a high degree of credibility. On this basis, the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence, so that the point cloud data segmentation model can focus on generalizing to data with poor performance, and the model has better generalization ability.

[0057] exist Figure 2 Based on the content in S107, the present application embodiment also provides a possible implementation method, please refer to Figure 3 , S107 includes S107-1.

[0058] S107-1, performing weighted training on the initial model based on the second point cloud data and the sample point cloud data.

[0059] The second point cloud data is point cloud data that has been manually annotated, and the weights corresponding to the points of the second point cloud data are higher than the weights corresponding to the points of the sample point cloud data.

[0060] In a possible implementation manner, the second point cloud data and the sample point cloud data may be acquired according to a preset ratio.

[0061] It can be understood that by using multiple groups of second point cloud data and multiple groups of sample point cloud data as training sets for the point cloud data segmentation model at the same time, compared with using only the second point cloud data as the training set for the point cloud data segmentation model, the present application solution requires less data to be manually labeled, thereby reducing the difficulty and cost of labeling.

[0062] It can be understood that the label accuracy of the manually annotated point cloud data is guaranteed, so the weights corresponding to the points of the second point cloud data are higher than the weights corresponding to the points of the sample point cloud data, thereby ensuring the training effect of the model.

[0063] In a possible implementation manner, the weight corresponding to each point in the second point cloud data is 1.

[0064] In a possible implementation, after executing S107-1, the above S102-S107 are repeatedly executed until the point cloud data segmentation model can be stably generalized on the sample point cloud data, the point cloud data segmentation model converges, and the various indicators of the point cloud data segmentation model no longer change.

[0065] exist Figure 2 Based on the content in S103, the present application embodiment also provides a possible implementation method, please refer to Figure 4 , S103 includes S103-1.

[0066] S103-1, determining high confidence points in the first point cloud data as sample point cloud data.

[0067] Among them, the high confidence point is a point where the confidence corresponding to the predicted category is greater than or equal to the first confidence threshold.

[0068] In one possible implementation, the prediction category and confidence corresponding to each point of the first point cloud data in the annotation result are traversed, and points whose confidence of the corresponding prediction category is greater than or equal to the first confidence threshold are screened out and determined as high confidence points. The first confidence threshold can be understood as a high threshold TH. It can be understood that a point whose confidence is greater than or equal to the first confidence threshold indicates that the confidence of the point is high, the corresponding prediction category can be trusted, and the point can be used as sample point cloud data. Therefore, using sample point cloud data to train the point cloud data segmentation model will not cause the point cloud data segmentation model to learn in the wrong direction.

[0069] In a possible implementation, the first confidence thresholds corresponding to different prediction categories may not be completely the same. For example, the prediction category of point A is electric wire, the corresponding confidence is 85, and the first confidence threshold corresponding to the electric wire is 80, then point A can be determined as a high-confidence point, and the electric wire can be determined as the pseudo-label of point A; the prediction category of point B is road, the corresponding confidence is 85, and the first confidence threshold corresponding to the road is 90, then point B cannot be determined as a high-confidence point.

[0070] exist Figure 4 On the basis of, when the number of high confidence points is small, the sample point cloud data carrying pseudo labels in the training set is not rich enough. In order to enrich the sample point cloud data, the embodiment of the present application also provides a possible implementation method, please refer to Figure 5 , S103 also includes S103-3.

[0071] S103-3, determining the medium confidence points in the first point cloud data as sample point cloud data.

[0072] Among them, the medium confidence point is a point whose confidence corresponding to the prediction category is less than the first confidence threshold and greater than the second confidence threshold, and there is at least one high confidence point with the same prediction category as the medium confidence point within a preset range centered on the medium confidence point.

[0073] Specifically, please refer to Figure 6 , Figure 6 Point A in is a high confidence point. Figure 6 It can be seen that within the preset range centered at point A1 ( Figure 6 ), there is a point A with the same prediction category as point A1. That is, point A has the same prediction category as point A1, the confidence of point A is greater than the corresponding first confidence threshold, the confidence of point A1 is greater than the second confidence threshold and less than the first confidence threshold, and point A is within the preset range centered on point A1, that is, point A1 can be determined as a medium confidence point.

[0074] When the confidence of the corresponding prediction category is less than the first confidence threshold and greater than the second confidence threshold, by verifying that there is at least one high confidence point with the same prediction category within the preset range centered on it, it is determined whether the corresponding prediction category is accurate. If there is, it means that the accuracy of the prediction category corresponding to the point is high, and the corresponding prediction category can be trusted. The medium confidence point can be determined as the sample point cloud data, and the prediction category corresponding to the medium confidence point is used as the pseudo label of the sample point cloud data. In this way, the sample point cloud data is enriched under the premise of ensuring the accuracy of the pseudo label carried by the sample point cloud data.

[0075] Optionally, when a point in the first point cloud data is neither a high confidence point nor a medium confidence point, the predicted category of the point may be set to empty.

[0076] exist Figure 5 On the basis of how to determine the medium confidence point, the embodiment of the present application also provides a possible implementation method, please refer to the following, S103 also includes:

[0077] After S103-1, all points whose confidences are less than the first confidence threshold and greater than the second confidence threshold are screened out as points to be confirmed;

[0078] It is determined whether there is at least one high confidence point with the same prediction category as the point to be confirmed within the preset range of each point to be confirmed. If so, the point to be confirmed is regarded as a medium confidence point.

[0079] Please continue to refer to Figure 6 , assuming that point A1 is the point to be confirmed, its confidence is less than the first confidence threshold and greater than the second confidence threshold; the preset range centered on point A1 is Figure 6 Assume that Figure 6 There are two high confidence points in the gray area, namely point A and point B. At this time, it is determined whether the predicted category of point A1 is the same as the predicted category of point A or point B. When the predicted category of point A1 is the same as the predicted category of either point A or point B, point A1 can be determined as a medium confidence point, and the predicted category of point A1 is used as the pseudo label of point A1. Figure 8 The corresponding approach can avoid the situation where some points are simultaneously within the preset range of multiple high-confidence points and the process of repeatedly determining whether they are medium-confidence points is avoided.

[0080] exist Figure 4 On the basis of how to determine the medium confidence point, the present application embodiment also provides a possible implementation method, please refer to Figure 7 , S103 also includes S103-2 and S103-3.

[0081] S103-2, selecting points with the same prediction category as the high confidence point and corresponding confidence less than the first confidence threshold and greater than the second confidence threshold from among the points in the preset area centered on the high confidence point as medium confidence points.

[0082] Optionally, a preset range centered on a high confidence point is searched according to a K-nearest neighbor algorithm to obtain a point to be confirmed.

[0083] Specifically, when high-confidence points have been screened out in S103-1, the K-nearest neighbor algorithm searches a preset range with the high-confidence point as the center to obtain points to be confirmed, and all the points to be confirmed are combined into a point set P. Optionally, the point set P does not include other high-confidence points except the high-confidence point at the center. Please refer to Figure 8 , assuming that point B is a high confidence point, the preset range centered on point B is Figure 8 The area corresponding to the gray part. Figure 8 It can be seen that points B1, B2, B3, B4, C1 and M1 are all in the area corresponding to the gray part, that is, points B1, B2, B3, B4, C1 and M1 are all points to be confirmed. The point set P corresponding to point B includes: points B1, B2, B3, B4, C1 and M1.

[0084] Then, the points in the to-be-confirmed point set P whose corresponding prediction categories are the same as the prediction categories of the high-confidence points and whose confidence of the prediction categories is less than the first confidence threshold and greater than the second confidence threshold are determined as medium-confidence points.

[0085] Continue to refer Figure 8 , respectively determine whether the prediction categories corresponding to point B1, point B2, point B3, point B4, point C1 and point M1 are the same as the prediction category of point B, and select point B1, point B2, point B3 and point B4 with the same prediction category as point B. Then determine whether the confidence of the prediction category of point B1, point B2, point B3 and point B4 is less than the first confidence threshold and greater than the second confidence threshold, and determine the points among point B1, point B2, point B3 and point B4 whose confidence meets the above range as medium confidence points.

[0086] S103-3, determining the medium confidence points in the first point cloud data as sample point cloud data.

[0087] In a possible implementation, the predicted categories corresponding to the high-confidence points and the medium-confidence points in the first point cloud data can be determined as their pseudo labels, and the pseudo labels of other types of points in the first point cloud data can be set to empty. Then, the first point cloud data with pseudo labels added is used as the sample point cloud data for weighted training.

[0088] exist Figure 2 On the basis of, in the case where the initial weight of each point in the sample point cloud data is negatively correlated with its corresponding confidence, regarding how to determine the final weight corresponding to each point in the sample point cloud data, the embodiment of the present application also provides a possible implementation method, please refer to Fig. 9 Before S107, the point cloud data segmentation model training method also includes: S104, S105 and S106.

[0089] S104, determining an initial weight according to the confidence level corresponding to each point in the sample point cloud data.

[0090] Among them, the initial weight is negatively correlated with the corresponding confidence.

[0091] Optionally, the corresponding initial weight can be determined by substituting the confidence into the weight-confidence linear expression, or the confidence can be converted into the initial weight through other conversion relationships.

[0092] S105, obtaining the accurate evaluation score corresponding to the sample point cloud data.

[0093] Among them, the accurate evaluation points represent the accuracy of each prediction category in the sample point cloud data.

[0094] For example, in the annotation results output by the initial model, the first point cloud data involves a total of five prediction categories, namely, electric poles, electric wires, fields, fruit trees, and houses. Electric poles, electric wires, fields, fruit trees, and houses each correspond to an accuracy, and the accuracy of any two different prediction categories can be the same or different.

[0095] For example, the accuracy for electric poles is 0.5, the accuracy for electric wires is 0.25, the accuracy for fields is 0.75, the accuracy for fruit trees is 0.75, and the accuracy for houses is 1.00. Of course, there may be some prediction categories with an accuracy of 0. Understandably, 1.00 means completely accurate, 0.75 means mostly accurate (basically accurate), 0.5 means generally accurate, 0.25 means basically inaccurate, and 0 means completely inaccurate.

[0096] In a possible implementation, the accuracy assessment score corresponding to the sample point cloud data may be manually assessed and uploaded to the electronic device, or obtained by the electronic device's autonomous assessment, for example, the corresponding accuracy may be determined by identifying a network model. The corresponding accuracy may also be obtained based on the confidence statistics of all points in the category.

[0097] S106, determining the final weight of each point according to the corresponding accurate evaluation score and the initial weight of each point in the sample point cloud data.

[0098] Specifically, after determining the initial weight according to the confidence level corresponding to each point in the sample point cloud data, the accurate evaluation score corresponding to the predicted category is determined by the predicted category corresponding to the point. The product of the accurate evaluation score and the initial weight is determined as the final weight of each point.

[0099] Understandably, the range of values ​​for the accurate assessment score and the initial weight are both 0-1. Multiplication requires that both scores must be high to ultimately obtain a high weight, and the differences in the high weight intervals are widened, while ensuring that the final weight range is also 0-1.

[0100] exist Figure 2 On the basis of how to obtain the initial model, the present application embodiment also provides a possible implementation method, please refer to Fig.10 , the point cloud data segmentation model training method also includes S101.

[0101] S101, training the initial model according to the second point cloud data until the initial model converges.

[0102] The second point cloud data is point cloud data that has been manually annotated.

[0103] Optionally, three-dimensional point cloud data is obtained by using drones and three-dimensional reconstruction technology, and the point cloud data is manually annotated to generate three-dimensional point cloud data (second point cloud data) with category labels as training set D1; multiple trainings are performed based on training data D1 to obtain a preliminary converged point cloud data segmentation model M1, and the performance of the model on the test set will not improve with the results of re-training. Training with a small amount of annotated data can shorten the overall training process and save training time.

[0104] In the point cloud data segmentation model training method provided in the embodiment of the present application, for each category generated by the initial model segmentation, different confidence threshold screening methods are used to screen out high-confidence pseudo-label data, and a proximity search method is used to search and update low-confidence label data categories to generate high-quality pseudo-label data, and the existing annotated data and high-quality pseudo-label data are used to realize the autonomous iterative evolution of the segmentation model.

[0105] See also Fig.11 , Fig.11 A point cloud data segmentation model training device is provided in an embodiment of the present application. Optionally, the point cloud data segmentation model training device is applied to the electronic device described above.

[0106] The point cloud data segmentation model training device includes: a processing unit 201 and a training unit 202.

[0107] The processing unit 201 is used to obtain the labeling result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually labeled, and the labeling result includes the prediction category and confidence corresponding to each point of the first point cloud data.

[0108] Optionally, the processing unit 201 may execute the above-mentioned S102.

[0109] The processing unit 201 is further configured to determine sample point cloud data used as training samples from the first point cloud data according to the labeling result.

[0110] Optionally, the processing unit 201 may execute the above-mentioned S103.

[0111] The training unit 202 is used to perform weighted training on the initial model based on the sample point cloud data to make the initial model converge, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence.

[0112] Optionally, the training unit 202 may execute the above-mentioned S107.

[0113] In a possible implementation, the training unit 202 is further configured to perform weighted training on the initial model based on the second point cloud data and the sample point cloud data, wherein the second point cloud data is point cloud data that has been manually annotated, and the weights corresponding to the points of the second point cloud data are higher than the weights corresponding to the points of the sample point cloud data. Optionally, the training unit 202 may perform the above-mentioned S107-1.

[0114] In a possible implementation manner, the weight of each point in the second point cloud data is 1.

[0115] In a possible implementation, the processing unit 201 is further configured to determine high-confidence points in the first point cloud data as sample point cloud data, wherein the high-confidence points are points whose confidence corresponding to the predicted category is greater than or equal to a first confidence threshold.

[0116] In one possible implementation, the processing unit 201 is also used to determine a medium confidence point in the first point cloud data as sample point cloud data, wherein the medium confidence point is a point whose confidence corresponding to a predicted category is less than a first confidence threshold and greater than a second confidence threshold, and there is at least one high confidence point with the same predicted category as the medium confidence point within a preset range centered on the medium confidence point.

[0117] In a possible implementation, the processing unit 201 is also used to filter out points with the same prediction category as the high confidence point and corresponding confidence less than the first confidence threshold and greater than the second confidence threshold from each point within a preset area centered on the high confidence point as medium confidence points, and determine the medium confidence points as sample point cloud data.

[0118] Optionally, the processing unit 201 may also execute the above-mentioned S103 - 1 to S103 - 3 .

[0119] In one possible implementation, the initial weight of each point in the sample point cloud data is negatively correlated with its corresponding confidence level, and the processing unit 201 is further used to determine the initial weight based on the confidence level corresponding to each point in the sample point cloud data; obtain the accuracy evaluation score corresponding to the sample point cloud data, wherein the accuracy evaluation score represents the accuracy of each prediction category in the sample point cloud data; and determine the final weight of each point in the sample point cloud data based on the corresponding accuracy evaluation score and the initial weight.

[0120] Optionally, the processing unit 201 may also execute the above-mentioned S104 to S106.

[0121] In a possible implementation, the training unit 202 is further configured to train the initial model based on the second point cloud data until the initial model converges; wherein the second point cloud data is point cloud data that has been manually annotated. Optionally, the training unit 202 may also perform the above-mentioned S101.

[0122] It should be noted that the point cloud data segmentation model training device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effect. For the sake of brief description, for parts not mentioned in this embodiment, please refer to the corresponding content in the above embodiment.

[0123] The present application also provides a storage medium storing computer instructions and programs, which, when read and executed, execute the point cloud data segmentation model training method of the above embodiment. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0124] An electronic device is provided below, which may be a computer device or a server device. Figure 1 As shown, the above-mentioned point cloud data segmentation model training method can be implemented; specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 can be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the point cloud data segmentation model training method of the above-mentioned embodiment is executed.

[0125] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0127] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0129] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A point cloud data segmentation model training method, characterized in that: The method comprises: Obtaining a labeling result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually labeled, and the labeling result includes a prediction category and a confidence level corresponding to each point of the first point cloud data; Determining sample point cloud data used as training samples from the first point cloud data according to the labeling result; The initial model is weightedly trained based on the sample point cloud data to converge the initial model, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence.

2. The point cloud data segmentation model training method according to claim 1, characterized in that: The step of performing weighted training on the initial model based on the sample point cloud data comprises: The initial model is weightedly trained based on the second point cloud data and the sample point cloud data, wherein the second point cloud data is point cloud data that has been manually annotated, and the weights corresponding to the points of the second point cloud data are higher than the weights corresponding to the points of the sample point cloud data.

3. The point cloud data segmentation model training method according to claim 2, characterized in that: The weight of each point in the second point cloud data is 1.

4. The point cloud data segmentation model training method according to claim 1, characterized in that: The step of determining sample point cloud data used as training samples from the first point cloud data according to the labeling result includes: The high confidence points in the first point cloud data are determined as the sample point cloud data, wherein the high confidence points are points whose confidence corresponding to the predicted category is greater than or equal to a first confidence threshold.

5. The point cloud data segmentation model training method according to claim 4, characterized in that: Also includes: The medium confidence point in the first point cloud data is determined as the sample point cloud data, wherein the medium confidence point is a point whose confidence corresponding to the predicted category is less than the first confidence threshold and greater than the second confidence threshold, and there is at least one high confidence point with the same predicted category as the medium confidence point within a preset range centered on the medium confidence point.

6. The point cloud data segmentation model training method according to claim 4, characterized in that: Also includes: From the points within the preset area centered on the high confidence point, select points with the same prediction category as the high confidence point and corresponding confidence less than the first confidence threshold and greater than the second confidence threshold as medium confidence points, and determine the medium confidence points as the sample point cloud data.

7. The point cloud data segmentation model training method according to claim 1, characterized in that: The initial weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence. Before weighted training is performed on the initial model based on the sample point cloud data, the method further includes: Determining the initial weight according to the confidence level corresponding to each point in the sample point cloud data; Obtaining an accuracy evaluation score corresponding to the sample point cloud data, wherein the accuracy evaluation score represents the accuracy of each prediction category in the sample point cloud data; The final weight of each point in the sample point cloud data is determined according to the corresponding accurate evaluation score and the initial weight of each point.

8. The point cloud data segmentation model training method according to claim 1, characterized in that: Before obtaining the labeling result of the initial model on the first point cloud data, the method further includes: Training the initial model according to the second point cloud data until the initial model converges; The second point cloud data is point cloud data that has been manually annotated.

9. A point cloud data segmentation model training device, characterized in that: The device comprises: A processing unit, configured to obtain a labeling result of the initial model on the first point cloud data, wherein the first point cloud data is point cloud data that has not been manually labeled, and the labeling result includes a prediction category and a confidence level corresponding to each point of the first point cloud data; The processing unit is further used to determine sample point cloud data used as training samples from the first point cloud data according to the labeling result; A training unit is used to perform weighted training on the initial model based on the sample point cloud data to make the initial model converge, wherein the weight of each point of the sample point cloud data is negatively correlated with its corresponding confidence.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images