A method and system for assigning labels to training samples of a neural network

Abstract

The present invention discloses a training sample label distribution method and system for a neural network, and the distribution method comprises the following steps: S1, inputting the image to be trained into the anchor-free neural network after preprocessing; S2, outputting a feature map after the image to be trained transmitted by the anchor-free neural network is processed by the anchor-free neural network; S3, obtaining the anchor of each point on the feature map according to the target frame; S4, calculating the IOU value between the anchor obtained in step S3 and the target frame, and then determining the positive sample point, the negative sample point and / or ignoring the sample point according to the IOU value. The present invention solves the problems that the distribution method of positive and negative samples of targets with relatively large length and width ratios in the prior art is obviously poor, the detection effect is poor, and the like.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and system for assigning labels to training samples of a neural network. Background Art

[0002] In the development of computer vision, target detection has always been the focus of researchers. It mainly focuses on detecting the bounding box and category of the target in the image. In recent years, with the development of convolutional neural networks, anchor-based detection networks have dominated target detection for many years. It mainly sets a large number of anchors on the image manually, and then predicts the bounding box and category of the target in the image based on the position and size of each anchor. Since anchor-based detection is generally intensive prediction, its post-processing is time-consuming. In recent years, anchor-free has developed rapidly. However, since it has no preset anchor, its positive and negative samples cannot use the IOU (Intersection over Union) between the anchor and the target box to determine the positive and negative samples for training like anchor-base. Instead, it generally determines the positive and negative samples for training based on whether the anchor-point falls in the target box. This method is more effective for general detection targets, but for targets with a relatively large length and width such as zebra crossings, this method of allocating positive and negative samples is obviously not effective. Therefore, it is necessary to design a training label allocation strategy that is effective for targets with a small aspect ratio and has a better effect on targets with a relatively large length and width such as zebra crossings. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides a method and system for assigning labels to training samples of a neural network, which solves the problems in the prior art of obviously poor effect of allocating positive and negative samples for targets with relatively large length and width, and poor detection effect.

[0004] The technical solution adopted by the present invention to solve the above problems is:

[0005] A method for assigning labels to training samples of a neural network comprises the following steps:

[0006] S1, input the image to be trained into the anchor-free neural network after preprocessing;

[0007] S2, processing the image to be trained transmitted by the anchor-free neural network through the anchor-free neural network and outputting a feature map;

[0008] S3, get the anchor of each point on the feature map according to the target box;

[0009] S4, calculating the IOU value between the anchor obtained in step S3 and the target box, and then determining the positive sample points, negative sample points and / or ignored sample points according to the IOU value.

[0010] As a preferred technical solution, in step S3, each point of the feature map is taken as the center, and the length and width of the target box are used to sequentially set the anchors at various positions on the feature map, thereby obtaining the anchors of each point on the feature map.

[0011] As a preferred technical solution, the anchor size is determined according to the aspect ratio of each target on the feature map, so as to obtain the anchor of each point on the feature map.

[0012] As a preferred technical solution, in step S3, when a feature map has N targets, N corresponding anchors are obtained, and the size of the anchors is consistent with the size of the targets.

[0013] As a preferred technical solution, points with IOU values ​​greater than a are set as positive sample points, points with IOU values ​​less than b are set as negative sample points, and areas with IOU values ​​in the range of [b, a] are ignored sample points; where the range of b is (0.39, 0.41), and the range of a is (0.49, 0.61).

[0014] As a preferred technical solution, if there is a situation where the IOU values ​​between multiple anchors and the target box on a feature point on the feature map are greater than a, the target box with the smallest area is used to match the feature point.

[0015] As a preferred technical solution, b=0.4 and a=0.5 are set.

[0016] As a preferred technical solution, in step S1, preprocessing includes data enhancement processing.

[0017] As a preferred technical solution, in step S2, processing by an anchor-free neural network includes processing by network forward calculation.

[0018] A training sample label assignment system for a neural network includes the following modules electrically connected in sequence:

[0019] The input module is used to input the image to be trained into the anchor-free neural network after preprocessing, and then transmit it to the feature map output module;

[0020] The feature map output module is used to process the image to be trained transmitted by the anchor-free neural network through the anchor-free neural network to output the feature map, and send the feature map to the anchor acquisition module;

[0021] The anchor acquisition module is used to obtain the anchor of each point on the feature map according to the target frame, and transmit the obtained anchor to the sample acquisition module;

[0022] The sample acquisition module is used to calculate the IOU value between the anchor transmitted by the anchor acquisition module and the target box, and then determine the positive sample points, negative sample points and / or ignore the sample points according to the IOU value.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] (1) The present invention does not need to consider the aspect ratio of the detection target, and does not need to preset multiple anchors on the feature map for dense prediction. The positive and negative relationship of each point on the feature map can be adaptively determined according to the size of the detection target;

[0025] (2) The present invention has good detection effect not only for targets with normal aspect ratios, but also for targets with large aspect ratios such as zebra crossings;

[0026] (3) The present invention effectively solves the anchor-free sampling method for positive and negative samples during training for targets with relatively large aspect ratios. At the same time, according to the area size of the target frame, it can ensure that each feature map position is associated with only one target frame. In actual tests, its effect achieves the detection effect of anchor-base without presetting the anchor ratio.

[0027] (4) The present invention has a better distribution method for positive and negative samples for targets with relatively large length and width, and has a good detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the steps of a method for assigning labels to training samples of a neural network according to the present invention;

[0029] Figure 2 A structural block diagram of a training sample label distribution system for a neural network according to the present invention;

[0030] Figure 3 This is a flowchart of a method for assigning labels to training samples of a neural network according to Embodiment 3 of the present invention;

[0031] Figure 4 It is a schematic diagram of the anchor and target box of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0033] Example 1

[0034] like Figures 1 to 4 As shown, a method for assigning labels to training samples of a neural network comprises the following steps:

[0035] S1, input the image to be trained into the anchor-free neural network after preprocessing;

[0036] S2, processing the image to be trained transmitted by the anchor-free neural network through the anchor-free neural network and outputting a feature map;

[0037] S3, get the anchor of each point on the feature map according to the target box;

[0038] S4, calculating the IOU value between the anchor obtained in step S3 and the target box, and then determining the positive sample points, negative sample points and / or ignored sample points according to the IOU value.

[0039] The present invention does not need to consider the aspect ratio of the detection target, does not need to preset multiple anchors on the feature map for dense prediction, and can adaptively determine the positive and negative relationship of points on each feature map according to the size of the detection target; the present invention not only has a good detection effect on targets with ordinary aspect ratios, but also has a good detection effect on targets with relatively large aspect ratios such as zebra crossings; it effectively solves the sampling method of positive and negative samples during training based on anchor-free for targets with relatively large aspect ratios, and at the same time, according to the area size of the target frame, it can ensure that each feature map position is associated with only one target frame. In actual tests, its effect achieves the detection effect of anchor-base, and there is no need to pre-set the ratio of anchors; thus, the distribution method of positive and negative samples for targets with relatively large aspect ratios is better, and the detection effect is good.

[0040] As a preferred technical solution, in step S3, each point of the feature map is taken as the center, and the length and width of the target box are used to sequentially set the anchors at various positions on the feature map, thereby obtaining the anchors of each point on the feature map.

[0041] This makes it easier to get the anchors for each point on the feature map.

[0042] As a preferred technical solution, the anchor size is determined according to the aspect ratio of each target on the feature map, so as to obtain the anchor of each point on the feature map.

[0043] This method obtains the anchor of each point on the feature map based on the aspect ratio of the target, which is a dynamic allocation method.

[0044] As a preferred technical solution, in step S3, when a feature map has N targets, N corresponding anchors are obtained, and the size of the anchors is consistent with the size of the targets.

[0045] Through the above steps, the anchor on the feature map is obtained.

[0046] As a preferred technical solution, points with IOU values ​​greater than a are set as positive sample points, points with IOU values ​​less than b are set as negative sample points, and areas with IOU values ​​in the range of [b, a] are ignored sample points; where the range of b is (0.39, 0.41), and the range of a is (0.49, 0.61).

[0047] This makes it easier to obtain positive sample points, negative sample points, and ignored sample points during training.

[0048] As a preferred technical solution, if there is a situation where the IOU values ​​between multiple anchors and the target box on a feature point on the feature map are greater than a, the target box with the smallest area is used to match the feature point.

[0049] This makes sample acquisition more precise and reduces errors.

[0050] As a preferred technical solution, b=0.4 and a=0.5 are set.

[0051] This makes it easier to select appropriate thresholds to obtain positive sample points, negative sample points, and ignored sample points during training.

[0052] As a preferred technical solution, in step S1, preprocessing includes data enhancement processing.

[0053] Data enhancement processing is helpful to increase the data volume of training samples and improve the generalization ability of the model.

[0054] As a preferred technical solution, in step S2, processing by an anchor-free neural network includes processing by network forward calculation.

[0055] The network forward calculation process facilitates the acquisition of feature maps.

[0056] Example 2

[0057] like Figures 1 to 4 As shown, as a further optimization of Example 1, this embodiment includes all the technical features of Example 1. In addition, this embodiment also includes the following technical features:

[0058] A training sample label assignment system for a neural network includes the following modules electrically connected in sequence:

[0059] The input module is used to input the image to be trained into the anchor-free neural network after preprocessing, and then transmit it to the feature map output module;

[0060] The feature map output module is used to process the image to be trained transmitted by the anchor-free neural network through the anchor-free neural network to output the feature map, and send the feature map to the anchor acquisition module;

[0061] The anchor acquisition module is used to obtain the anchor of each point on the feature map according to the target frame, and transmit the obtained anchor to the sample acquisition module;

[0062] The sample acquisition module is used to calculate the IOU value between the anchor transmitted by the anchor acquisition module and the target box, and then determine the positive sample points, negative sample points and / or ignore the sample points according to the IOU value.

[0063] The present invention does not need to consider the aspect ratio of the detection target, does not need to preset multiple anchors on the feature map for dense prediction, and can adaptively determine the positive and negative relationship of points on each feature map according to the size of the detection target; the present invention not only has a good detection effect on targets with ordinary aspect ratios, but also has a good detection effect on targets with relatively large aspect ratios such as zebra crossings; it effectively solves the anchor-free label allocation method for positive and negative samples during training for targets with relatively large aspect ratios, and at the same time, according to the area size of the target frame, it can ensure that each feature map position is associated with only one target frame. In actual tests, its effect achieves the detection effect of anchor-base, and there is no need to pre-set the anchor ratio; thus, the allocation method of positive and negative samples for targets with relatively large aspect ratios is better, and the detection effect is good.

[0064] Example 3

[0065] like Figures 1 to 4 As shown, this embodiment includes all the technical features of Embodiment 1 and Embodiment 2. Based on Embodiment 1 and Embodiment 2, this embodiment provides a more detailed implementation method.

[0066] The purpose of this embodiment is to provide a label assignment method for positive and negative samples in the network model retraining stage, which not only has good detection for targets with normal aspect ratios, but also has good detection effects for targets with larger aspect ratios such as zebra crossings. Since the size of the external frame of the target to be detected is unknown, the anchor-base detection network presets anchors of different scales in the feature map, and then determines the positive and negative samples based on the IOU of the preset anchor and the target frame. This method achieves good detection results in target detection. Therefore, considering that anchor-free does not have a preset anchor, in order to use IOU to determine the positive and negative sample assignment method, it is necessary to design a method to simulate the preset frame of anchor-base.

[0067] To solve the above technical problems, the following steps are included:

[0068] 1) Input the image to be trained, pass it into the neural network after preprocessing, and output the feature map through the network;

[0069] 2) With each point of the feature map as the center, the length and width of the target box are used to sequentially set the anchors at each feature map position. In this way, when there are N targets in a picture, each position on the feature map will have corresponding N anchors, and the size of the anchor is consistent with the size of the corresponding target. The difference from the anchor-base method is that the anchor-base scheme pre-sets the aspect ratio of each anchor. This method may not cover some special targets, such as targets with a large length and width ratio like zebra crossings. The method of the present invention is based on the actual length and width ratio of the target, and is not limited by the length and width ratio of the target. It is a dynamic allocation method;

[0070] 3) Calculate the IOU value between the anchor generated in 2) and the target box, set the value greater than 0.5 as a positive sample point, less than 0.4 as a negative sample point, and ignore the sample point between [0.4, 0.5]. There may be a feature point whose IOU value with multiple target boxes is greater than 0.5. If this happens, the point will be matched with the smaller area of ​​the target box.

[0071] The beneficial effects of this embodiment are as follows:

[0072] By adopting the above technical solution, the label allocation method of positive and negative samples in the training of targets with relatively large length and width is effectively solved based on anchor-free. At the same time, according to the area size of the target box, it can be ensured that each feature map position is associated with only one target box. In actual tests, its effect can achieve the detection effect of anchor-base, without pre-setting the anchor ratio.

[0073] Through the above method, this embodiment mainly proposes a method for assigning positive and negative sample labels to points on a feature map when training an anchor-free detector. It does not need to consider the aspect ratio of the detection target, does not need to preset multiple anchors on the feature map for dense prediction, and can adaptively determine the positive and negative relationship of points on each feature map according to the size of the detection target.

[0074] It is worth noting that: 1. In the present invention, the target frame refers to the annotation frame in the training image; 2. In the present invention, the corresponding quantitative relationship between feature points, target frames, and anchors is: in step S3, initially, 1 feature point-N target frames (N frames enclose 1 feature point)-N anchors; after S3 processing, the most suitable 1 target frame and 1 anchor are selected, 1 feature point-1 frame-1 anchor. Preferably, N targets will have N circumscribed rectangular frames, and then with each feature point as the center, the anchor size is the size of the N targets, and one feature point will correspond to N anchors, and then these N anchors are used to calculate the IOU with the circumscribed rectangular frames of the N targets, and the target corresponding to the maximum value is found. The category of this feature point is the category of the corresponding target.

[0075] like Figure 4 As shown in the figure: the outermost rectangular area is the size of the network input image, with width W and height H. The target box is the outer rectangular box of the target. Since the feature map size is obtained after the input image is downsampled twice, that is, the size is 1 / 4 of the input image size; the number of feature points is (W / 4*H / 4), so the spacing between the feature points mapped to the original image is 4. With each feature point as the center, use the size of each target box to generate an anchor. If there are N target boxes, there are N*W / 4*H / 4 anchors on the feature map. Then use all the anchors to calculate the IOU with the target box, and then decide which target box each feature point matches based on the IOU and the size of the target box, and which feature points are assigned as positive sample points, negative sample points, and ignored sample points. As shown in the figure, the solid points represent positive sample feature points, the hollow points represent ignored sample feature points, and the others not drawn are negative sample feature points.

[0076] As described above, the present invention can be preferably implemented.

[0077] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0078] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. According to the technical essence of the present invention, within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for assigning labels to training samples of a neural network, characterized in that: The following steps are involved: S1, input the image to be trained into the anchor-free neural network after preprocessing; S2, processing the image to be trained transmitted by the anchor-free neural network through the anchor-free neural network and outputting a feature map; S3, get the anchor of each point on the feature map according to the target box; S4, calculating the IOU value between the anchor obtained in step S3 and the target box, and then determining the positive sample points, negative sample points and / or ignored sample points according to the IOU value; In step S3, with each point of the feature map as the center, the length and width of the target box are used to sequentially set the anchors at various positions on the feature map, thereby obtaining the anchors of each point on the feature map.

2. The method for assigning labels to training samples of a neural network according to claim 1, characterized in that: The anchor size is determined according to the length-width ratio of each target on the feature map, so as to obtain the anchor of each point on the feature map.

3. The method for assigning labels to training samples of a neural network according to claim 2, characterized in that: In step S3, when a feature map has N targets, N corresponding anchors are obtained, and the size of the anchors is consistent with the size of the targets.

4. The method for assigning labels to training samples of a neural network according to claim 3, characterized in that: Set the points with IOU values ​​greater than a as positive sample points, the points with IOU values ​​less than b as negative sample points, and the areas with IOU values ​​in the range of [b, a] as ignored sample points; where the range of b is (0.39, 0.41) and the range of a is (0.49, 0.61).

5. The method for assigning labels to training samples of a neural network according to claim 4, characterized in that: If the IOU value between multiple anchors and the target box on a feature point on the feature map is greater than a, the target box with the smallest area is used to match the feature point.

6. The method for assigning labels to training samples of a neural network according to claim 5, characterized in that: Set b=0.4, a=0.

5.

7. A method for assigning labels to training samples of a neural network according to any one of claims 1 to 6, characterized in that: In step S1, preprocessing includes data enhancement processing.

8. The method for assigning labels to training samples of a neural network according to claim 7, characterized in that: In step S2, the anchor-free neural network processing includes network forward calculation processing.

9. A neural network training sample label assignment system, characterized in that: A method for assigning labels to training samples of a neural network according to any one of claims 1 to 8, comprising the following modules electrically connected in sequence: The input module is used to input the image to be trained into the anchor-free neural network after preprocessing, and then transmit it to the feature map output module; The feature map output module is used to process the image to be trained transmitted by the anchor-free neural network through the anchor-free neural network to output the feature map, and send the feature map to the anchor acquisition module; The anchor acquisition module is used to obtain the anchor of each point on the feature map according to the target frame, and transmit the obtained anchor to the sample acquisition module; The sample acquisition module is used to calculate the IOU value between the anchor transmitted by the anchor acquisition module and the target box, and then determine the positive sample points, negative sample points and / or ignore the sample points according to the IOU value.

Images