Direction perception and target detection method based on small sample learning

Abstract

The invention discloses a direction perception and target detection method based on small sample learning. The method comprises the following steps: marking a target bounding box, a direction and a category for each picture in a training set; training a detection model based on the marked training set; and identifying a to-be-identified picture by using the trained detection model to obtain a target detection result and a target direction. According to the method disclosed by the invention, the accuracy of small sample data detection can be greatly improved, and meanwhile, the target direction is accurately predicted.

Description

Technical Field

[0001] The present invention relates to a method for identifying key parts of a target based on knowledge graph reasoning, and belongs to the technical field of image recognition. Background Art

[0002] The complexity and difficulty of image acquisition in the real world often leads to a small number of image samples of a certain category or an imbalance in the number of image samples between different categories, which is called the small sample problem of images.

[0003] Unbalanced data distribution, such as a long-tail distribution (where a large number of samples are concentrated in a small number of categories), causes the model to prefer learning image categories that contain a large number of samples, resulting in low accuracy and poor robustness of small-sample image recognition algorithms. Furthermore, the model's accuracy in judging images of unknown categories is low, and its generalization ability is weak. The algorithm's ability to process cross-modal data is also limited, making it difficult to effectively learn from multimodal data.

[0004] Therefore, it is necessary to conduct more in-depth research on existing target detection methods to solve the above problems. Summary of the Invention

[0005] To overcome the above problems, we conducted in-depth research and proposed a direction perception and target detection method based on small sample learning, which includes the following steps:

[0006] S1. Label each image in the training set with its target bounding box, direction, and category.

[0007] S2. Train the detection model based on the labeled training set;

[0008] S3. Use the trained detection model to identify the image to be identified and obtain the target detection result and target direction.

[0009] In a preferred embodiment, in S2, the detection model includes an MPSR checker and a rotating box object detector.

[0010] In a preferred embodiment, a margin loss function is provided in the MPSR checker, wherein the margin loss function Expressed as:

[0011]

[0012] Among them, i represents different target types, is the intra-class distance of class i, is the distance between classes of class i, N b is the number of base classes, and N is the number of novel classes.

[0013] In a preferred embodiment, the intra-class distance is expressed as:

[0014]

[0015] Among them, j represents different eigenvectors, K is the total number of eigenvectors, and v ij Represented as the j-th eigenvector of class i, μ i is the average eigenvector of class i, and ‖·‖2 is the calculated 2-norm.

[0016] In a preferred embodiment, the class spacing is expressed as:

[0017]

[0018] Among them, w uj is a weight vector used to measure the similarity between each class and other classes.

[0019] In a preferred embodiment, the weight vector is obtained by taking the pseudo-inverse of the following formula:

[0020]

[0021] In a preferred embodiment, a disturbance module is further provided in the MPSR checker, and the disturbance module is represented as follows:

[0022]

[0023] β′ ij =β ij ·T(G(β ij ))

[0024] Among them, β ij represents the j-th eigenvector of class i, G(β ij ) is the characteristic β ij The corresponding gradient, τ() represents the threshold function, T(G(β ij )) is the intermediate function, β′ ij Represents feature β ij The perturbed eigenvector.

[0025] In a preferred embodiment, the gradient G(β ij ) is expressed as:

[0026]

[0027] in, is the inter-class loss function, and ‖·‖ is the normalization operation.

[0028] The present invention also provides an electronic device, comprising:

[0029] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the above methods.

[0030] The present invention also provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the above methods.

[0031] The beneficial effects of the present invention include:

[0032] (1) Significantly improve the accuracy of small sample data detection;

[0033] (2) Ability to accurately predict target direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The figure is a flow chart of a direction perception and target detection method based on small sample learning according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be described in further detail below with reference to the accompanying drawings and examples, through which the features and advantages of the present invention will become more clearly understood.

[0036] The word "exemplary" is used exclusively herein to mean "serving as an example, example, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0037] According to the present invention, a direction perception and target detection method based on small sample learning is provided. Figure 1 As shown, the following steps are included:

[0038] S1. Label each image in the training set with its target bounding box, direction, and category.

[0039] S2. Train the detection model based on the labeled training set;

[0040] S3. Use the trained detection model to identify the image to be identified and obtain the target detection result and target direction.

[0041] In traditional training sets, only the target bounding box and category are annotated. Different from the annotation in traditional training sets, in S1 of the present invention, the target direction is also annotated to provide support for subsequent direction recognition.

[0042] Preferably, the target direction is represented by the angle of the target forward direction.

[0043] In S2, the detection model is based on the MPSR checker. MPSR (Multi-Scale Positive Sample Refinement) is a method for few-shot object detection (FSOD), which was proposed by Jiaxi Wu, Songtao Liu, Di Huang and Yunhong Wang and first published at the ECCV 2020 conference.

[0044] Furthermore, different from the traditional MPSR checker, in the present invention, a margin loss function is provided in the MPSR checker.

[0045] The margin loss function Expressed as:

[0046]

[0047] Among them, i represents different target types, is the intra-class distance of class i, is the distance between classes of class i, N b is the number of base classes, and N is the number of novel classes.

[0048] According to the present invention, the margin loss function includes two parts, the first part is the intra-class distance calculation, and the second part is the inter-class distance calculation. By dividing the sum of the intra-class distances of each class by the sum of the inter-class distances of each class as the model training loss function, the intra-class distances of each class can be reduced and the inter-class distances can be increased, so that the prototype vector of each class needs to be as close to the class average vector as possible and as far away from the class average vectors of other classes as possible.

[0049] Furthermore, the intra-class distance is expressed as:

[0050]

[0051] Among them, j represents different eigenvectors, K is the total number of eigenvectors, and v ij Represented as the j-th eigenvector of class i, μ i is the average eigenvector of class i, and ‖·‖2 is the calculated 2-norm.

[0052] The class spacing is expressed as:

[0053]

[0054] Among them, w ijis a weight vector used to measure the similarity between each class and other classes.

[0055] Preferably, the weight vector is obtained by taking the pseudo-inverse of the following formula:

[0056]

[0057] That is, it is believed that the features of the novel class can be linearly represented by other classes, where other classes select features v ij The weight vector is calculated for the nearest class.

[0058] Experiments have shown that by setting the margin loss function, the aliasing between different classes can be well alleviated.

[0059] In a preferred embodiment, a disturbance module is further provided in the MPSR checker, and the disturbance module is represented as follows:

[0060]

[0061] β′ ij =β ij ·T(G(β ij ))

[0062] Among them, β ij represents the j-th eigenvector of class i, G(β ij ) is the characteristic β ij The corresponding gradient, τ() represents the threshold function, T(G(β ij )) is the intermediate function, β′ ij Represents feature β ij The perturbed eigenvector.

[0063] The threshold function τ can be used to control the degree of weight perturbation. When the category weight is greater than or equal to the threshold, it is set to 0, and when it is less than the threshold, it remains unchanged.

[0064] Preferably, the gradient G(β ij ) is expressed as:

[0065]

[0066] in, is the inter-class loss function, and ‖·‖ is the normalization operation.

[0067] In the present invention, by setting a perturbation module, the image or feature can be perturbed through gradients. Since the perturbed features introduce new data compared to the current novel class data, the perturbed features will shift toward the margin of the class distribution, and the class margin loss function will make the perturbed features gather toward the center of the distribution. We use such a training method to achieve a balanced state of novel class distance in small sample target detection, thereby solving the data bias problem caused by the small amount of novel class data.

[0068] In a preferred embodiment, the backbone network of the MPSR inspector adopts a Resnet network to extract features from the image and obtain a four-layer feature map containing small-scale target information to large-scale target information;

[0069] In a preferred embodiment, the multi-scale feature pyramid network in the MPSR checker adopts an FPN network to perform feature fusion on the output of the backbone network.

[0070] In a preferred embodiment, the detection model further includes a rotating box object detector, which can use R3Det rotating box object detector, Oriented RCNN rotating box object detector, or any other object detector. Preferably, the rotating box object detector uses Oriented RCNN rotating box object detector, which is a general two-stage oriented detector with good accuracy and efficiency. Its first stage is the oriented region proposal network (orientation RPN), which directly generates high-quality oriented proposals; the second stage is the oriented R-CNN head, which is used to refine the oriented regions of interest and identify them.

[0071] Oriented R-CNN is trained in an end-to-end manner by jointly optimizing the ordered RPN and the ordered R-CNN head. During inference, the ordered proposals generated by the ordered RPN usually have high overlap. To reduce redundancy, in the present invention, 2000 proposals of each FPN level are retained in the first stage, and then non-maximum suppression (NMS) is performed; the remaining proposals of all levels are merged, and the top 1000 are selected as the input of the second stage based on their classification scores.

[0072] Preferably, in the second stage, polygon NMS is performed on those predicted ordered bounding boxes whose class probabilities are greater than 0.05.

[0073] Taking into account the inference speed, preferably, in the present invention, in the first stage, horizontal NMS with an IoU threshold of 0.8 is adopted; in the second stage, the IoU threshold of polygon NMS is 0.1.

[0074] According to the present invention, the output of the rotating box object detector is the BBOX box, center point and angle Theta of the object in the image.

[0075] For most targets, their structures do not have obvious differences, such as ship hulls and cars. The direction of the target can be obtained through simple geometric analysis based on the angle Theta.

[0076] Various embodiments of the methods described above in the present invention may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0078] Example

[0079] Example 1

[0080] A target recognition experiment is conducted using a dataset with sufficient sample images for base class samples and only 10 images for novel samples. The target recognition experiment includes the following steps:

[0081] S1. Label each image in the training set with its target bounding box, direction, and category.

[0082] S2. Train the detection model based on the labeled training set;

[0083] S3. Use the trained detection model to identify the image to be identified and obtain the target detection result and target direction.

[0084] In S2, the detection model is based on an MPSR checker, and a margin loss function is set in the MPSR checker.

[0085] The margin loss function Expressed as:

[0086]

[0087] The intra-class distance is expressed as:

[0088]

[0089] The class spacing is expressed as:

[0090]

[0091] The weight vector is obtained by taking the pseudo-inverse of the following formula:

[0092]

[0093] A disturbance module is also provided in the MPSR checker, and the disturbance module is expressed as:

[0094]

[0095] β′ ij =β ij ·T(G(β ij ))

[0096] Gradient G(β ij ) is expressed as:

[0097]

[0098] The backbone network of the MPSR inspector adopts the Resnet network to extract features from the image and obtain a four-layer feature map containing small-scale target information to large-scale target information; the multi-scale feature pyramid network of the MPSR inspector adopts the FPN network to perform feature fusion on the output of the backbone network.

[0099] The detection model also includes a rotating box target detector, which adopts an OrientedRCNN rotating box target detector.

[0100] Comparative Example

[0101] Comparative Example 1

[0102] The same data set as in Example 1 was used to conduct the same experiment as in Example 1, except that Matching Network, Neural Statistician and Prototype network were used respectively.

[0103] Wherein, the Matching Network refers to the literature Vinyals O, Blundell C, Lillicrap T, et al. Matching networks for one shot learning [J]. Advances in neural information processing systems, 2016, 29;

[0104] The Neural Statistician is described in Edwards H, Storkey A. Towards a neural statistician [J]. arXiv preprint arXiv: 1606.02185, 2016.

[0105] The Prototype Network can be found in the literature Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning [J]. Advances in neural information processing systems, 2017, 30.

[0106] The results of Example 1 and Comparative Example 1 are shown in Table 1.

[0107] Table 1

[0108] Model Distance representation Accuracy Comparative Example 1-Matching Network Cosine distance 98.9 Comparative Example 1-Neural Statistician - 98.7 Comparative Example 1-Prototype Network Euclidean distance 99.5 Example 1 Two-norm 99.9

[0109] As can be seen from Table 1, the method in Example 1 has a higher accuracy rate than other methods. The present invention has been described above in conjunction with preferred embodiments, but these embodiments are only exemplary and serve only as an illustration. On this basis, various replacements and improvements can be made to the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A direction perception and target detection method based on small sample learning, characterized in that: The following steps are involved: S1. Label each image in the training set with its target bounding box, direction, and category. S2. Train the detection model based on the labeled training set; S3. Use the trained detection model to identify the image to be identified and obtain the target detection result and target direction.

2. The direction perception and target detection method based on small sample learning according to claim 1, characterized in that: In S2, the detection model includes an MPSR checker and a rotation box object detector.

3. The direction perception and target detection method based on small sample learning according to claim 2, characterized in that: A margin loss function is set in the MPSR checker. Expressed as: Among them, i represents different target types, is the intra-class distance of class i, is the distance between classes of class i, N b is the number of base classes, and N is the number of novel classes.

4. The direction perception and target detection method based on small sample learning according to claim 3 is characterized in that: The intra-class distance is expressed as: Among them, j represents different eigenvectors, K is the total number of eigenvectors, and v ij Represented as the hth eigenvector of class i, μ i is the average eigenvector of class i, and ‖·‖2 is the calculated 2-norm.

5. The direction perception and target detection method based on small sample learning according to claim 4 is characterized in that: The class spacing is expressed as: Among them, w ij is a weight vector used to measure the similarity between each class and other classes.

6. The direction perception and target detection method based on small sample learning according to claim 5, characterized in that: The weight vector is obtained by taking the pseudo-inverse of the following formula:

7. The direction perception and target detection method based on small sample learning according to claim 3, characterized in that: A disturbance module is also provided in the MPSR checker, and the disturbance module is expressed as: b′ ij =b ij ·T(G(β ij )) Among them, β ij represents the j-th eigenvector of class i, G(β ij ) is the characteristic β ij The corresponding gradient, τ( ) represents the threshold function, T(G(β ij )) is the intermediate function, β′ ij Represents feature β ij The perturbed eigenvector.

8. The direction perception and target detection method based on small sample learning according to claim 7, characterized in that: Gradient G(β ij ) is expressed as: in, is the inter-class loss function, and ‖·‖ is the normalization operation.

9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

Images