Multi-stage target detection method and system suitable for small sample training and storage medium thereof

Abstract

The invention discloses a multi-stage target detection method and system suitable for small sample training and a storage medium thereof. The method comprises the following steps: adding a branch to a feature map of each anchor after a neck network structure, wherein the branch is used for outputting a vector while a model outputs a target frame; constructing triple loss based on positive samples and negative samples of targets in training samples, weighting the triple loss according to the proportion of the positive samples and the negative samples during training, and performing spatial visualization on each target frame output feature vector output by the model during each round of training verification; separating the positive samples from the negative samples by using a knn algorithm, calculating the separation accuracy of the positive samples and the negative samples in the training samples after each round of knn algorithm separation, and recording the average Euclidean distance of the two types; obtaining a positive sample Euclidean distance threshold value on the basis of the data of the training optimal round; according to the invention, the problem of excessive false detection caused by too large difference between an application scene and training data is solved.

Description

Technical Field

[0001] The present invention relates to deep learning, and in particular to a multi-stage target detection method and system suitable for small sample training and a storage medium thereof. Background Art

[0002] As a core task in the field of computer vision, target detection aims to accurately identify and locate various target objects in images. This technology plays a vital role in many application scenarios, including but not limited to autonomous driving, security monitoring, medical image analysis, etc. In recent years, with the vigorous development of deep learning technology, especially the widespread application of convolutional neural networks (CNN), the performance of target detection algorithms has been significantly improved.

[0003] Although CNN-based object detection algorithms have made great progress in both theory and practice, their performance still depends heavily on the quality and diversity of training datasets. A high-quality and diverse training dataset can cover more possible scenes and object forms, thereby improving the generalization ability and accuracy of the algorithm. However, in practical applications, since the training dataset is often difficult to fully cover all situations in actual usage scenarios, especially when these scenarios are significantly different, the performance of the algorithm is often affected.

[0004] Specifically, when there is a large difference between the training data set and the actual usage scenario, the object detection algorithm may have too many false detections. These false detections will not only reduce the accuracy of the algorithm, but may also cause a series of work risks, especially in application scenarios with extremely high accuracy requirements.

[0005] To address this problem, existing solutions mainly include data enhancement and feature fusion, but the effects are minimal. Summary of the invention

[0006] Therefore, in order to solve the above-mentioned shortcomings, the present invention provides a multi-stage target detection method, system and storage medium suitable for small sample training. The present invention solves the problem existing in the prior art, that is, the problem of too many false detections caused by the large difference between the application scenario and the training data.

[0007] In a first aspect, the present invention provides a multi-stage target detection method suitable for small sample training, based on a convolutional neural network model, the convolutional neural network model includes a backbone network, a neck network and a head network; the method includes: A branch is added to the feature map of each anchor after the neck network structure, so that the model can output a vector at the same time when outputting the target box; Based on the positive and negative samples of the target in the training sample, a triplet loss can be constructed. During training, it is weighted according to the ratio of positive and negative samples. At the same time, the output feature vector of each target box output by the model during each round of training verification is spatially visualized, where the positive sample is with the target and the negative sample is without the target; Use the knn algorithm to separate positive samples and negative samples, and calculate the separation accuracy of positive and negative samples in the training samples after each round of knn algorithm separation, and record the average Euclidean distance between the two categories; Based on the data of the optimal training round, the Euclidean distance threshold of the positive sample is obtained.

[0008] Optionally, the multi-stage target detection method suitable for small sample training also includes: The model output is obtained by inferring the input data, the target frame is screened out, and the output feature vector corresponding to each target frame is taken out according to the target frame index, wherein the input data is a picture, a video, or a video stream; Calculate each corresponding Euclidean distance between each of the output feature vectors and the basic positive sample feature vector; Substitute the calculated Euclidean distance into the positive sample Euclidean distance threshold to determine whether the input data is a positive sample.

[0009] Optionally, based on the data of the optimal training round, the method to obtain the Euclidean distance threshold of the positive sample is: The output feature vector of each anchor of each image in the model inference training set is spatially visualized, and the vector with the most central spatial position of the positive sample feature vector is obtained as the positive sample basic feature vector. The spatial Euclidean distance between each positive sample feature vector and its basic feature vector is calculated, and the average and variance of the Euclidean distance are obtained. The positive sample Euclidean distance threshold is obtained based on the average and variance.

[0010] Specifically, the optimal training round is the training round with the lowest triple loss value.

[0011] In one embodiment, the branch is composed of multiple computing blocks. For example, the branch is a block composed of convolutional neural network + batch normalization + activation function.

[0012] In a second aspect, the present invention provides a multi-stage target detection system suitable for small sample training, comprising The component module is used to add a branch to the feature map of each anchor after the neck network structure, so that the model can output a vector when outputting the target box; The training module is used to construct triplet loss based on the positive and negative samples of the target in the training samples. During training, they are weighted according to the ratio of positive and negative samples. At the same time, the output feature vector of each target box output by the model during each round of training verification is spatially visualized; A separation module is used to separate positive samples and negative samples using the knn algorithm, and calculate the separation accuracy of positive and negative samples in the training samples after each round of knn algorithm separation, and record the average Euclidean distance between the two categories; and The threshold module is used to obtain the Euclidean distance threshold of the positive sample based on the data of the optimal training round.

[0013] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which is executed by a processor to implement the method for detecting small targets of drones based on a small number of samples.

[0014] The present invention has the following advantages: The present invention can solve the problem that in many objective situations, it is impossible to collect sufficient data and fully cover all application scenarios, resulting in the trained algorithm model generating more false detections when the application scenario and the training data are too different. This greatly reduces the false detection rate of small sample data training in generalization scenarios, which not only reduces the data collection cost, but also can adapt to the implementation of algorithms for some application scenarios where data collection cannot be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the multi-stage target detection method suitable for small sample training according to the present invention; Figure 2 It is a systematic block diagram of the multi-stage target detection system suitable for small sample training according to the present invention; In the figure: 100, component module; 200, training module; 300, separation module; 400, threshold module; 500, reasoning module; 600, calculation module; 700, judgment module. DETAILED DESCRIPTION

[0016] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0017] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises", or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0018] As described in the background technology, although CNN-based object detection algorithms have made great progress in both theory and practice, their performance still depends largely on the quality and diversity of training datasets. A high-quality and diverse training dataset can cover more possible scenes and object forms, thereby improving the generalization ability and accuracy of the algorithm. However, in practical applications, since the training dataset is often difficult to fully cover all situations in actual usage scenarios, especially when these scenarios are significantly different, the performance of the algorithm is often affected.

[0019] Specifically, when there is a large difference between the training data set and the actual usage scenario, the object detection algorithm may have too many false detections. These false detections will not only reduce the accuracy of the algorithm, but may also cause a series of work risks, especially in application scenarios with extremely high accuracy requirements.

[0020] To address this problem, existing solutions mainly include data enhancement and feature fusion, but the effects are minimal.

[0021] Based on the above reasons, the present invention provides the following embodiments: like Figure 1 As shown, this embodiment provides a multi-stage target detection method suitable for small sample training, based on a convolutional neural network model, the convolutional neural network model includes a backbone network, a neck network and a head network; the method includes: Step S100, adding a branch to the feature map of each anchor after the neck network structure, so that the model can output a vector at the same time when outputting the target frame; Exemplarily, a branch is added to the feature map of each auchor after the model neck structure to calculate and output a feature vector (generally a one-dimensional 16-bit vector), and then the obtained output is concatenated with the model output, with the aim of outputting a vector (a one-dimensional 16-bit vector) at the same time as the model outputs the target box.

[0022] In this step, the branch is composed of multiple computing blocks. Exemplarily, the branch is a block composed of convolutional neural network + batch normalization + activation function; residual structure, bottleneck structure, heavy parameter structure, etc. can also be further adopted. At the same time, when using ternary loss, a larger ternary loss threshold can be set.

[0023] Step S200: A triplet loss (triplet loss) can be constructed based on the positive and negative samples of the target in the training sample. During training, the positive and negative samples are weighted according to the ratio of positive and negative samples. At the same time, the output feature vector of each target box output by the model during each round of training verification is spatially visualized, where the positive sample is with the target and the negative sample is without the target; Step S300, using the knn algorithm to separate positive samples and negative samples, and calculating the separation accuracy of positive and negative samples in the training samples after each round of knn algorithm separation, and recording the average Euclidean distance between the two categories; Step S400: Based on the data of the optimal training round, a positive sample Euclidean distance threshold is obtained. Preferably, the optimal training round is the training round with the lowest ternary loss value.

[0024] Exemplarily, the output feature vector of each auchor (anchor) of each image in the model inference training set is spatially visualized, and the vector with the most central spatial position of the positive sample feature vector is obtained as the positive sample basic feature vector, and the spatial Euclidean distance between each positive sample feature vector and its basic feature vector is calculated, and the average and variance of the Euclidean distance are obtained, and the positive sample Euclidean distance threshold is obtained based on the average and variance.

[0025] Step S500, using input data to infer the model output, first filter out the target box according to nms (non-maximum suppression), and extract the output feature vector corresponding to each target box according to the target box index, wherein the input data is a picture or a video or a video stream; Step S600, calculating each corresponding Euclidean distance between each of the output feature vectors and the basic positive sample feature vector; Step S700: Substitute the calculated Euclidean distance into the positive sample Euclidean distance threshold to determine whether the input data is a positive sample.

[0026] The above technical features can solve the problem of excessive false detections caused by large differences between application scenarios (such as traffic light or road sign turn recognition) and training data.

[0027] In another embodiment, if Figure 2 As shown, a multi-stage target detection system suitable for small sample training is also provided, including The component module 100 is used to add a branch to the feature map of each anchor after the neck network structure, so as to output a vector when the model outputs the target box; A training module 200 is used to construct a triplet loss based on the positive and negative samples of the target in the training sample. During training, the positive and negative samples are weighted according to the ratio of the positive and negative samples, and the feature vector output of each target box output by the model during each round of training verification is spatially visualized; The separation module 300 is used to separate positive samples and negative samples using the knn algorithm, and calculate the separation accuracy of positive and negative samples in the training samples after each round of knn algorithm separation, and record the average Euclidean distance between the two categories; The threshold module 400 is used to obtain the positive sample Euclidean distance threshold based on the data of the optimal training round. An inference module 500 is used to infer the model output using the input data, filter out the target frame, and extract the output feature vector corresponding to each target frame according to the target frame index, wherein the input data is a picture, a video, or a video stream; A calculation module 600 calculates each corresponding Euclidean distance between each of the output feature vectors and the basic positive sample feature vectors; The judgment module 700 brings the calculated Euclidean distance into the positive sample Euclidean distance threshold to determine whether the input data is a positive sample.

[0028] It should be noted that, in the above embodiments, the description of each embodiment has its own focus, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0029] It should be understood by those skilled in the art that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program codes.

[0030] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0031] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0032] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0033] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the above-mentioned multi-stage target detection method suitable for small sample training.

[0034] The module / unit integrated in the multi-stage target detection system / terminal device suitable for small sample training can be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as an independent product. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0035] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0036] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0037] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0038] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0039] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application. The above embodiments are only used to illustrate the technical solution of the present application, not to limit it; although the present application is described in detail with reference to the above embodiments, ordinary technicians in this field should understand that: it is still possible to modify the technical solutions recorded in the above embodiments, or to replace some of the technical features therein by equivalent; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

[0040] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-stage target detection method suitable for small sample training, characterized by: Based on a convolutional neural network model, the convolutional neural network model includes a backbone network, a neck network and a head network; the method includes: A branch is added to the feature map of each anchor after the neck network structure, so that the model can output a vector at the same time when outputting the target box; A triplet loss is constructed based on the positive and negative samples of the target in the training sample. During training, it is weighted according to the ratio of positive and negative samples. At the same time, the output feature vector of each target box output by the model during each round of training verification is spatially visualized, where positive samples are with targets and negative samples are without targets. Use the knn algorithm to separate positive samples and negative samples, and calculate the separation accuracy of positive and negative samples in the training samples after each round of knn algorithm separation, and record the average Euclidean distance between the two categories; Based on the data of the optimal training round, the Euclidean distance threshold of the positive sample is obtained.

2. A multi-stage target detection method suitable for small sample training according to claim 1, characterized in that: Also includes The model output is obtained by inferring the input data, the target frame is screened out, and the output feature vector corresponding to each target frame is taken out according to the target frame index, wherein the input data is a picture, a video, or a video stream; Calculate each corresponding Euclidean distance between each of the output feature vectors and the basic positive sample feature vector; Substitute the calculated Euclidean distance into the positive sample Euclidean distance threshold to determine whether the input data is a positive sample.

3. The multi-stage target detection method suitable for small sample training according to claim 1, characterized in that: Based on the data of the optimal training round, the method to obtain the Euclidean distance threshold of the positive sample is: The output feature vector of each anchor of each image in the model inference training set is spatially visualized, and the vector with the most central spatial position of the positive sample feature vector is obtained as the positive sample basic feature vector. The spatial Euclidean distance between each positive sample feature vector and its basic feature vector is calculated, and the average and variance of the Euclidean distance are obtained. The positive sample Euclidean distance threshold is obtained based on the average and variance.

4. A multi-stage target detection method suitable for small sample training according to claim 1 or 3, characterized in that: The best training epoch is the one with the lowest triplet loss.

5. The multi-stage target detection method suitable for small sample training according to claim 1, characterized in that: The branch is composed of multiple computing blocks.

6. A multi-stage target detection method suitable for small sample training according to claim 5, characterized in that: The branch is a block composed of convolutional neural network + batch normalization + activation function.

7. A multi-stage target detection system suitable for small sample training, characterized by: include The component module is used to add a branch to the feature map of each anchor after the neck network structure, so that the model can output a vector when outputting the target box; The training module is used to construct triplet loss based on the positive and negative samples of the target in the training samples. During training, they are weighted according to the ratio of positive and negative samples. At the same time, the output feature vector of each target box output by the model during each round of training verification is spatially visualized; The separation module is used to separate positive samples and negative samples using the knn algorithm, and calculate the separation accuracy of positive and negative samples in the training samples after each round of knn algorithm separation, and record the average Euclidean distance between the two categories; as well as The threshold module is used to obtain the Euclidean distance threshold of the positive sample based on the data of the optimal training round.

8. A storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method for detecting small targets of unmanned aerial vehicles based on a small number of samples as described in any one of claims 1 to 6.

Images