Target detection model sample optimization method and device, and model training method and device

Abstract

The application relates to the technical field of target detection, and provides a target detection model sample optimization method, a model training method and device, the target detection model sample optimization method comprises the following steps: acquiring a sample image with at least one label frame; determining a target area in the label frame according to the category of a target object in the label frame; setting the confidence of sample pixels in the target area to be greater than the confidence of sample pixels in a non-target area in the label frame, the model training method comprises the following steps: the target detection model receives a sample image, and outputs a prediction result predicted according to the sample image; the prediction result and a label with different confidences corresponding to the sample image are brought into a preset loss function, and the training is completed when the loss function converges. The target detection model trained according to the optimized sample image can detect targets more accurately.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a target detection model sample optimization method, model training method, and apparatus. Background Technology

[0002] External obstacle detection is a crucial component of Advanced Driving Assistance Systems (ADAS) and is essential for the overall safety of the system. Obstacles typically include passenger cars, vans, buses, pedestrians, animals, and traffic cones. During data annotation, bounding boxes are usually created for these objects, and the annotated images are then fed into the network for training. In the annotated images, the information within the bounding boxes is usually considered entirely positive, while the rest is considered negative. However, in general, the bounding boxes contain a large amount of negative sample information (everything outside the main object is considered negative). Figure 1 As shown, the upper two sides of the bounding box of the traffic cone contain negative samples, but the confidence level of the entire bounding box is set to 1. Because the bounding box contains a large amount of negative sample information, the model's detection performance is poor, leading to both false positives and false negatives in ADAS. Summary of the Invention

[0003] This invention provides a sample optimization method, a model training method, and an apparatus for target detection models, which solves the problem in the prior art where the model detection performance is poor due to a large amount of negative sample information within the bounding boxes during model training.

[0004] In a first aspect, the present invention provides a method for optimizing sample data of a target detection model, comprising:

[0005] Obtain a sample image with at least one bounding box;

[0006] The target area is determined within the annotation box based on the category of the target object within the annotation box;

[0007] The confidence level of sample pixels in the target region is set to be greater than the confidence level of sample pixels in the non-target region within the annotation box.

[0008] According to a target detection model sample optimization method provided by the present invention, a target region is determined within the annotation box based on the category of the target object within the annotation box, including:

[0009] The target area is calculated using a preset shape formula based on the category of the target object.

[0010] According to the target detection model sample optimization method provided by the present invention, the preset shape formula includes: ellipse formula, triangle formula and rectangle formula.

[0011] According to the target detection model sample optimization method provided by the application, the confidence of sample pixels in the target region is 1, and the confidence of sample pixels in the non-target region in the label box is 0.5-0.8.

[0012] In a second aspect, the application provides a target detection model training method, comprising:

[0013] The target detection model receives a sample image and outputs a prediction result predicted according to the sample image, the sample image has at least one label box, each label box is provided with a target region determined according to the category of a target object, and the confidence of sample pixels in the target region is higher than that of sample pixels in the non-target region in the label box.

[0014] The prediction result and the label with different confidences corresponding to the sample image are brought into a preset loss function, and the training is completed when the loss function converges.

[0015] According to the target detection model training method provided by the application, in the loss function, the classification loss function is obtained by weighting the sub-loss functions corresponding to different confidences, and the weight of the sub-loss function with high confidence is greater than that of the sub-loss function with low confidence.

[0016] In a third aspect, the application provides a target detection model sample optimization device, comprising:

[0017] A sample image acquisition module is configured to acquire a sample image with at least one label box.

[0018] A target region determination module is configured to determine a target region in the label box according to the category of a target object in the label box.

[0019] A confidence setting module is configured to set the confidence of sample pixels in the target region to be higher than that of sample pixels in the non-target region in the label box.

[0020] In a fourth aspect, the application provides a target detection model training device, comprising:

[0021] A model prediction module is configured to receive a sample image by the target detection model and output a prediction result predicted according to the sample image, the sample image has at least one label box, each label box is provided with a target region determined according to the category of a target object, and the confidence of sample pixels in the target region is higher than that of sample pixels in the non-target region in the label box.

[0022] A loss function operation module is configured to bring the prediction result and the label with different confidences corresponding to the sample image into a preset loss function, and the training is completed when the loss function converges.

[0023] In a fifth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the target detection model sample optimization method of any one of the above aspects, or implements the target detection model training method of any one of the above aspects.

[0024] In a sixth aspect, the present application provides a non-transitory computer readable storage medium, having stored thereon a computer program, wherein the computer program is executable on a processor to implement the target detection model sample optimization method of any one of the above aspects, or implement the target detection model training method of any one of the above aspects.

[0025] The target detection model sample optimization method, model training method and device provided by the present application, by obtaining a sample image with at least one annotation box; determining a target region in the annotation box according to the category of the target object in the annotation box; setting the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box, that is, distinguishing the confidence of the positive samples in the annotation box according to the target region, and setting the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box, so that the target detection model trained according to the optimized sample image detects the target more accurately. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0027] Figure 1 is a schematic diagram of the sample annotation box of the target detection model in the prior art;

[0028] Figure 2 is a schematic diagram of the flow of the target detection model sample optimization method provided by the present application;

[0029] Figure 3 is a schematic diagram of the determination of the target region in the target detection model sample optimization method provided by the present application;

[0030] Figure 4 is a schematic diagram of the flow of the target detection model training method provided by the present application;

[0031] Figure 5 is a schematic diagram of the structure of the target detection model sample optimization device provided by the present application;

[0032] Figure 6 is a structural schematic diagram of a target detection model training device provided by the present application.

[0033] Figure 7 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0034] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0035] The target detection model sample optimization method provided by the first embodiment of the present application is as shown in Figure 2 , which comprises:

[0036] Step S210, a sample image with at least one annotation box is obtained. The target in the sample image can have one or more. For an automatic driving system, the targets to be recognized include vehicles, pedestrians, animals, traffic cones and the like. As long as there is the above target in the sample image, it will be marked with an annotation box. The sample pixels in the annotation box are positive samples, and the sample pixels outside the annotation box are negative samples.

[0037] Step S220, the target region is determined in the annotation box according to the category of the target object in the annotation box. This step aims to mark the more accurate region of the target object from the annotation box.

[0038] Step S230, the confidence of the sample pixels in the target region is set to be greater than the confidence of the sample pixels in the non-target region in the annotation box. Specifically, the confidence of the sample pixels in the target region can be set to 1, and the confidence of the sample pixels in the non-target region in the annotation box is greater than 0 and less than 1.

[0039] In the target detection model sample optimization method of the present embodiment, the positive samples in the annotation box are distinguished by confidence according to the target region, that is, the region where the target object is further subdivided, and the confidence of the sample pixels in the target region is set to be greater than the confidence of the sample pixels in the non-target region in the annotation box, so that the target detection model trained according to the optimized sample image detects the target more accurately.

[0040] In the present embodiment, step S220 comprises: calculating the target region according to the pre-set shape formula according to the category of the target object. Since the outlines of target objects of different categories are different, the target region needs to be calculated according to different shape formulas, for example, an elliptical formula, a triangular formula and a rectangular formula are used to calculate the target region.

[0041] As shown in Figure 3 the target object is a traffic cone, an ellipse formula can be used, y 2 / a 2 +x 2 / b 2 =1 to calculate the target area, taking the center of the target frame as the coordinate origin, the coordinates of the target area on the x and y axes, that is, the expression of the ellipse formula, and a and b are the length and width of the labeling frame respectively. A large number of test results show that for traffic cones, using an elliptical area as the target area will have better results.

[0042] Different types of target objects are set to different target areas. For a target object that is a pedestrian, the target area can also be an elliptical area, and in the elliptical formula, a is the height of the pedestrian or the height of the labeling frame (because the labeling frame is drawn close to the target object), and b is the shoulder width of the pedestrian or the width of the labeling frame.

[0043] Since the labeling frame is drawn close to the target object, the parameters in the shape-related formula of the target area are all referenced to the maximum size of the labeling frame, such as a and b in the above formula. Of course, the parameters in the shape-related formula of the target area can also be adjusted slightly, for example, the upper end of the traffic cone is smaller, and the b in the ellipse formula can be adjusted slightly to make b slightly smaller than the width of the labeling frame.

[0044] For cases where the outer contour of the target object itself is or approaches a rectangle, for example, the target object is a car, the target area obtained is likely to be the labeling frame itself. In order to reduce the calculation amount of sample optimization, for cases where the outer contour of the target object itself is or approaches a rectangle, the labeling frame is directly determined as the target area, and the rectangular formula is not used to determine the rectangular area.

[0045] For irregular target objects, the target area can be formed by connecting the points on the recognized outer contour of the target object.

[0046] Since the pixels in the target area are almost 100% objects, the confidence of the sample pixels in the target area is 1, and the pixels near the target area may be 50% to 80% objects, so the confidence of the sample pixels in the non-target area in the labeling frame is 0.5 to 0.8, that is Figure 3 the confidence of the sample pixels in the elliptical area is 1, and the confidence of the sample pixels in the area outside the elliptical area in the labeling frame is 0.5 to 0.8, for example, 0.8.

[0047] The target detection model training method provided by the second embodiment of the present application, as shown in Figure 4 includes:

[0048] Step S410, the target detection model receives a sample image, and outputs a prediction result predicted according to the sample image, wherein at least one label frame is arranged in the sample image, each label frame is provided with a target region determined according to the category of a target object, and the confidence of sample pixels in the target region is greater than the confidence of sample pixels in a non-target region in the label frame. Specifically, the confidence of sample pixels in the target region is 1, and the confidence of sample pixels in the non-target region in the label frame is greater than 0 and less than 1.

[0049] Step S420, the prediction result and the label with different confidences corresponding to the sample image are brought into a preset loss function, and the training is completed when the loss function converges.

[0050] In the embodiment, the label frame in the sample image is further subdivided according to the target object in the label frame, and the label with different confidences is brought into the loss function for training when the model is trained, so that the target detection model after training can detect targets more accurately.

[0051] Specifically, Loss_total=w1×loss_cls+w2×loss_reg, wherein w1 and w2 are respectively the weights of the classification loss function loss_cls and the label frame position loss function loss_reg, and in the embodiment, w1=1 and w2=2 can be used, and loss_reg can be a traditional regression loss function IOU loss.

[0052] In the target detection model training method of the embodiment, in the loss function, the classification loss function is obtained by weighting sub-loss functions corresponding to different confidences, and the weight of a sub-loss function with high confidence is greater than the weight of a sub-loss function with low confidence. Specifically, the sub-loss function can be a binary classification cross-entropy loss function BCE, and the loss function of the model is as follows:

[0053]

[0054] Wherein, β and γ are respectively the weights of two sub-loss functions, pred is the prediction result of the target detection model, target is the label (i.e. the confidence), β=2 and γ=0.5.

[0055] The third embodiment of the application provides a target detection model sample optimization device, as shown in the figure, the device comprises: Figure 5

[0056] A sample image acquisition module 510 is configured to acquire a sample image with at least one label frame.

[0057] A target region determination module 520 is configured to determine a target region in the label frame according to the category of a target object in the label frame. ​

[0058] The confidence setting module 530 is configured to set the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region within the label frame.

[0059] In the target detection model sample optimization device, the confidence of the positive samples in the label frame is divided according to the target region, and the confidence of the sample pixels in the target region is set to be greater than the confidence of the sample pixels in the non-target region within the label frame, so that the target detection model trained according to the optimized sample image can detect the target more accurately.

[0060] Optionally, the target region determination module 520 is specifically configured to calculate the target region according to a preset shape formula selected according to the category of the target object.

[0061] Optionally, the preset shape formula includes an elliptical formula, a triangular formula and a rectangular formula.

[0062] Optionally, the confidence of the sample pixels in the target region is 1, and the confidence of the sample pixels in the non-target region within the label frame is 0.5-0.8.

[0063] The fourth embodiment of the present application provides a target detection model training device, as shown in the accompanying drawings, which comprises: Figure 6

[0064] The model prediction module is configured to receive a sample image by the target detection model, and output a prediction result predicted according to the sample image, wherein the sample image comprises at least one label frame, each label frame is provided with a target region determined according to the category of the target object, and the confidence of the sample pixels in the target region is greater than the confidence of the sample pixels in the non-target region within the label frame.

[0065] The loss function operation module is configured to bring the prediction result and the label with different confidences corresponding to the sample image into a preset loss function, and the training is completed when the loss function converges.

[0066] In the embodiment, the target region in the label frame in the sample image is further subdivided according to the target object, and the label with different confidences is brought into the loss function for training during model training, so that the target detection model trained is more accurate in detecting the target.

[0067] Optionally, in the loss function, the classification loss function is obtained by weighting the sub-loss functions corresponding to different confidences, and the weight of the sub-loss function with high confidence is greater than the weight of the sub-loss function with low confidence.

[0068] Figure 7 An example of an entity structure diagram of an electronic device is shown in the accompanying drawings, Figure 7 ​As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a target detection model sample optimization method, which includes:

[0069] Obtaining a sample image with at least one annotation box.

[0070] Determining a target region in the annotation box according to the category of the target object in the annotation box.

[0071] Setting the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box.

[0072] Alternatively, a target detection model training method is executed, which includes:

[0073] The target detection model receives a sample image and outputs a prediction result predicted according to the sample image, wherein the sample image has at least one annotation box, each annotation box is provided with a target region determined according to the category of the target object, and the confidence of the sample pixels in the target region is greater than the confidence of the sample pixels in the non-target region in the annotation box.

[0074] Bringing the prediction result and the label with different confidences corresponding to the sample image into a preset loss function, and training is completed when the loss function converges.

[0075] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0076] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being executable by a processor to enable a computer to perform the target detection model sample optimization method provided by the above method, which comprises:

[0077] obtaining a sample image with at least one annotation box.

[0078] determining a target region in the annotation box according to the category of the target object in the annotation box.

[0079] setting the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box.

[0080] Alternatively, a target detection model training method is performed, which comprises:

[0081] The target detection model receives a sample image and outputs a prediction result predicted according to the sample image, the sample image has at least one annotation box, each annotation box is provided with a target region determined according to the category of the target object, and the confidence of the sample pixels in the target region is greater than the confidence of the sample pixels in the non-target region in the annotation box.

[0082] The prediction result and the label with different confidences corresponding to the sample image are brought into a preset loss function, and the training is completed when the loss function converges.

[0083] In still another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, the computer program being executable by a processor to perform the target detection model sample optimization method provided by the above method, which comprises:

[0084] obtaining a sample image with at least one annotation box.

[0085] determining a target region in the annotation box according to the category of the target object in the annotation box.

[0086] setting the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box.

[0087] Alternatively, a target detection model training method is performed, which comprises:

[0088] The target detection model receives a sample image and outputs a prediction result predicted according to the sample image, the sample image has at least one annotation box, each annotation box is provided with a target region determined according to the category of the target object, and the confidence of the sample pixels in the target region is greater than the confidence of the sample pixels in the non-target region in the annotation box.

[0089] The prediction result and the label with different confidence degrees corresponding to the sample image are brought into a preset loss function, and the training is completed when the loss function converges.

[0090] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0091] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0092] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for optimizing a target detection model sample, characterized in that, The method comprises the following steps: acquiring a sample image with at least one annotation box; determining a target region in the annotation box according to the category of the target object in the annotation box, the target region being a more accurate region of the target object in the annotation box; setting the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box; wherein the step of determining the target region in the annotation box according to the category of the target object in the annotation box comprises: selecting a preset shape formula according to the category of the target object to calculate the target region, the preset shape formula comprising an elliptical formula, a triangular formula and a rectangular formula.

2. The target detection model sample optimization method according to claim 1, characterized in that, The confidence of the sample pixels in the target region is 1, and the confidence of the sample pixels in the non-target region in the annotation box is 0.5-0.

8.

3. A target detection model training method, characterized in that, The method comprises the following steps: the target detection model receives a sample image and outputs a prediction result predicted according to the sample image, the sample image having at least one annotation box, each annotation box having a target region determined according to the category of the target object, the target region being a more accurate region of the target object in the annotation box, the confidence of the sample pixels in the target region being greater than the confidence of the sample pixels in the non-target region in the annotation box, the target region being calculated according to a preset shape formula selected according to the category of the target object, the preset shape formula comprising an elliptical formula, a triangular formula and a rectangular formula; the prediction result and the labels with different confidences corresponding to the sample image are input into a preset loss function, and the training is completed when the loss function converges.

4. The target detection model training method of claim 3, wherein, In the loss function, the classification loss function is obtained by weighting the sub-loss functions corresponding to different confidences, and the weight of the sub-loss function with high confidence is greater than the weight of the sub-loss function with low confidence.

5. An object detection model sample optimization apparatus, characterized by, The method comprises the following steps: a sample image acquisition module is configured to acquire a sample image with at least one annotation box; a target region determination module is configured to determine a target region in the annotation box according to the category of the target object in the annotation box, the target region being a more accurate region of the target object in the annotation box; a confidence setting module is configured to set the confidence of the sample pixels in the target region to be greater than the confidence of the sample pixels in the non-target region in the annotation box; the target region determination module is specifically configured to select a preset shape formula according to the category of the target object to calculate the target region, the preset shape formula comprising an elliptical formula, a triangular formula and a rectangular formula. 6.A device for training a target detection model, characterized in that, The method comprises the following steps: a model prediction module is configured to cause the target detection model to receive a sample image and output a prediction result predicted according to the sample image, the sample image having at least one annotation box, each annotation box having a target region determined according to the category of the target object, the target region being a more accurate region of the target object in the annotation box, the confidence of the sample pixels in the target region being greater than the confidence of the sample pixels in the non-target region in the annotation box, the target region being calculated according to a preset shape formula selected according to the category of the target object, the preset shape formula comprising an elliptical formula, a triangular formula and a rectangular formula; The loss function operation module is configured to input the prediction result and labels with different confidences corresponding to the sample image into a preset loss function, and the training is completed when the loss function converges.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the target detection model sample optimization method in any one of claims 1-2 or the target detection model training method in claim 3 or 4 when executing the program.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the target detection model sample optimization method in any one of claims 1-2 or the target detection model training method in claim 3 or 4.

Images