A few-sample detection model generation method, device, equipment and storage medium

Abstract

The application discloses a few-sample detection model generation method and device, equipment and storage medium, including: obtaining a first training set and training a first detection model; the first training set includes a first sample image and a first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image; a second sample image is obtained, and the trained first detection model is used to detect the base class target in the second sample image to determine the non-base class target in the second sample image; after the non-base class target in the second sample image is annotated, a second training set containing the second sample image and a second sample label is obtained, and the second detection model is trained by using the second training set; the second detection model is a model obtained by adding a preset number of first detection head networks for classifying non-base class targets on the output side of the first detection model. A robust detection model can be built with few samples, and the training efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a few-sample detection model generation method and device, equipment and storage medium. BACKGROUND

[0002] With the in-depth development of artificial intelligence, detection models are used in more and more complex scenarios to identify various targets. For example, hard hat detection on construction sites. For construction sites with complex environments, work hazards, and casualties caused by on-site workers not wearing safety hats or not wearing reflective vests, it is imperative to detect hard hat wearing based on construction sites to reduce management difficulty, improve on-site personnel safety awareness, ensure safety in production and thus protect enterprise interests. However, the actual situation of the construction site is complex and variable, with a variety of safety hats, reflective vests, etc., which are not practical to list and cost huge, and if not listed, the unlisted target types cannot be identified.

[0003] Therefore, for different scenarios, how to build a robust detection model with a small number of samples is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a few-sample detection model generation method, device, equipment and storage medium, which can build a robust detection model with a small number of samples. The specific scheme is as follows:

[0005] The first aspect of the present application provides a few-sample detection model generation method, comprising:

[0006] obtaining a first training set and training a first detection model using the first training set to obtain a trained first detection model; wherein the first training set comprises a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image;

[0007] obtaining a second sample image and detecting the base class target in the second sample image using the trained first detection model to determine a non-base class target in the second sample image according to the detected base class target;

[0008] annotating the non-base class target in the second sample image containing the non-base class target to obtain a second training set containing the second sample image and a corresponding second sample label, and training a second detection model using the second training set to obtain a trained second detection model; wherein the second detection model is a model obtained by adding a preset number of first detection head networks for classifying the non-base class target on the output side of the first detection model.

[0009] Optionally, after the first training set is obtained, the method further comprises:

[0010] performing a deduplication operation on the first training set, and determining a target attribute category from each attribute category of the labeled information of the deduplicated first training set by using principal component analysis;

[0011] obtaining a filtered first training set by filtering the first sample images of the target attribute type that meet a preset condition from the deduplicated first training set, so as to train the first detection model by using the filtered first training set.

[0012] Optionally, the deduplication operation on the first training set comprises:

[0013] calculating the similarity of the first sample images by using a perceptual hashing algorithm, and removing the first sample images and the corresponding first sample labels with a similarity less than a preset threshold from the first training set.

[0014] Optionally, before the first training set is used to train the first detection model, the method further comprises:

[0015] improving the backbone network structure in the traditional YOLO model into a network structure composed of a first convolutional block and a second convolutional block, and improving the neck network structure in the traditional YOLO model into a feature pyramid network structure, to obtain the first detection model; wherein the first convolutional block is a 3*3 convolutional kernel connected in parallel with a 1*1 convolutional kernel, and the outputs of the two convolutional kernels are stacked in features, and the second convolutional block is a 3*3 convolutional kernel connected in parallel with a 1*1 convolutional kernel, and the outputs of the two convolutional kernels and the input of the second convolutional block are stacked in features.

[0016] Optionally, the improvement of the backbone network structure in the traditional YOLO model into a network structure composed of a first convolutional block and a second convolutional block, and the improvement of the neck network structure in the traditional YOLO model into a feature pyramid network structure, comprises:

[0017] improving the backbone network structure in the traditional YOLO model into a network structure composed of two first convolutional blocks, one second convolutional block, one first convolutional block, three second convolutional blocks, one first convolutional block, thirteen second convolutional blocks, and one first convolutional block connected in series;

[0018] The feature map output by the last first convolutional block of the improved backbone network structure is subjected to a 1*1 convolution kernel to obtain a first neck layer output, the first neck layer output is superimposed with the outputs of the thirteen second convolutional blocks to obtain a second neck layer output, and the second neck layer output is superimposed with the outputs of the three second convolutional blocks to obtain a third neck layer output; wherein each neck layer output corresponds to a second detection head network for classifying the base class target.

[0019] Optionally, before the training of the second detection model using the second training set, the method further comprises:

[0020] A first detection head network is added in parallel to each second detection head network in the first detection model, so that each neck layer output corresponds to the first detection head network and the second detection head network, respectively.

[0021] Optionally, the loss function L of the second detection model ft is:

[0022]

[0023]

[0024] wherein, is a non-base class target loss, is a non-base class category loss, is a non-base class regression box loss, λ is a parameter factor, and L con is a joint loss, C b is a second sample image set, is a non-base class target probability in the second sample image set, is a base class target probability in the second sample image set.

[0025] The second aspect of the present application provides a few-shot detection model generation device, comprising:

[0026] A first training module is configured to obtain a first training set and train a first detection model using the first training set to obtain a trained first detection model; wherein the first training set comprises a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image;

[0027] A detection module is configured to obtain a second sample image and detect the base class target in the second sample image using the trained first detection model to determine a non-base class target in the second sample image according to the detected base class target.

[0028] a second training module configured to label the non-base-class targets in the second sample images containing the non-base-class targets to obtain a second training set containing the second sample images and corresponding second sample labels, and train a second detection model using the second training set to obtain the trained second detection model; wherein the second detection model is a model obtained by adding a preset number of output layers on the output side of the first detection model.

[0029] A third aspect of the present application provides an electronic device, comprising a processor and a memory; wherein the memory is configured to store a computer program, which is loaded and executed by the processor to implement the foregoing few-shot detection model generation method.

[0030] A fourth aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and the computer executable instructions are loaded and executed by a processor to implement the foregoing few-shot detection model generation method.

[0031] In the present application, a first training set is first obtained and a first detection model is trained using the first training set to obtain the trained first detection model; wherein the first training set includes first sample images and corresponding first sample labels, and the first sample labels are annotation information for labeling base-class targets in the first sample images; then a second sample image is obtained and the base-class targets in the second sample image are detected using the trained first detection model to determine non-base-class targets in the second sample image according to the detected base-class targets; finally, the non-base-class targets in the second sample image containing the non-base-class targets are labeled to obtain a second training set containing the second sample images and corresponding second sample labels, and a second detection model is trained using the second training set to obtain the trained second detection model; wherein the second detection model is a model obtained by adding a preset number of first detection head networks for classifying the non-base-class targets on the output side of the first detection model. In the present application, the base-class targets in the second sample image are detected using the model obtained by training the first detection model using the first training set containing the base-class targets to determine the non-base-class targets in the second sample image, and then the second detection model obtained by adding a preset number of detection head networks on the output side of the first detection model using the second sample image containing the non-base-class targets is trained again. The above process can build a robust detection model with few samples for different scenarios, reduce model iteration training time and human annotation cost, and improve training efficiency and model detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only only the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0033] Figure 1 A few sample detection model generation method flowchart provided for the present application;

[0034] Figure 2 A specific few sample detection model generation method schematic diagram provided for the present application;

[0035] Figure 3 A first detection model structure diagram provided for the present application;

[0036] Figure 4 A first convolutional block structure diagram provided for the present application;

[0037] Figure 5 A second convolutional block structure diagram provided for the present application;

[0038] Figure 6 A second detection model structure diagram provided for the present application;

[0039] Figure 7 A second detection model output result diagram provided for the present application;

[0040] Figure 8 A few sample detection model generation device structure schematic diagram provided for the present application;

[0041] Figure 9 A few sample detection model generation electronic equipment structure diagram provided for the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] At present, most of the target detection algorithms based on deep learning rely on large-scale data labeling, but large-scale manual labeling data makes the sample quality low, and there are problems such as a large number of repeated pictures, non-standard labeling, etc., resulting in low recognition rate and high false alarm rate. In view of the above technical defects, the present application provides a kind of few sample detection model generation scheme, which can build a robust detection model with few samples for different scenes, reduce model iteration training time and manpower labeling cost, improve training efficiency and model detection accuracy.

[0044] Figure 1 A flow chart of a few sample detection model generation method provided by an embodiment of the present application is shown in Figure 1 The few sample detection model generation method comprises:

[0045] S11: obtaining a first training set and training a first detection model using the first training set to obtain a trained first detection model; wherein the first training set comprises a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image.

[0046] In this embodiment, the first training set is obtained and the first detection model is trained using the first training set to obtain the trained first detection model. The first training set comprises a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image. The first detection model can also be referred to as a base class model, and the first training set can also be referred to as a base class data set.

[0047] In order to ensure the detection accuracy of the model, the base class data set needs to be cleaned, which is described in detail in Figure 2The similar pictures are first cleaned up, and the first training set is deduplicated. For example, the similarity of the first sample image can be calculated using a perceptual hashing algorithm, and the first sample image and the corresponding first sample label with a similarity less than a preset threshold can be removed from the first training set. The perceptual hashing algorithm is used to calculate the picture similarity, and the distance threshold is set to 5. If it is less than the threshold, the picture and the label file are removed. This measure can remove a large number of similar labeled samples and maintain the universality of the labeled samples. Then, statistical analysis is performed on the labeled data. The target attribute category can be determined from each attribute category of the labeled information of the first training set after deduplication by using principal component analysis. The first sample image of the target attribute type that meets the preset condition is filtered from the first training set after deduplication to obtain a filtered first training set. The first detection model is trained using the filtered first training set. For example, when each attribute category of the labeled information is the picture size, the aspect ratio of the labeled box, the area, the proportion of the area to the original picture, and the number of labeled targets in the picture, principal component analysis PCA is performed on the attribute categories to find the most important information elements. For example, the aspect ratio of the labeled box and the proportion of the area to the original picture are selected after analysis, and the data distribution is observed on a visual interface according to the two factors. The abnormal values are removed, and the unqualified labeled samples are removed. Generally, for a safety helmet detection model, the aspect ratio is in the range of 0.5-1.5, and the proportion of the area to the original picture is in the range of 1%-20%.

[0048] In this embodiment, the first detection model adopts a variant based on YOLO, the backbone network structure in the traditional YOLO model is improved into a network structure composed of a first convolution block and a second convolution block, and the neck network structure in the traditional YOLO model is improved into a feature pyramid network structure, so as to obtain the first detection model; wherein the first convolution block is to connect a 3*3 convolution kernel with a 1*1 convolution kernel and to perform feature superposition on the outputs of the two convolution kernels, and the second convolution block is to connect a 3*3 convolution kernel with a 1*1 convolution kernel and to perform feature superposition on the outputs of the two convolution kernels and the input of the second convolution block. Specifically, on the one hand, the backbone network structure in the traditional YOLO model is improved into a network structure composed of two first convolution blocks, one second convolution block, one first convolution block, three second convolution blocks, one first convolution block, thirteen second convolution blocks, and one first convolution block connected in sequence. On the other hand, the feature map output by the last first convolution block of the improved backbone network structure is subjected to a 1*1 convolution kernel to obtain a first neck layer output, and the first neck layer output is subjected to feature superposition with the outputs of the thirteen second convolution blocks to obtain a second neck layer output, and the second neck layer output is subjected to feature superposition with the outputs of the three second convolution blocks to obtain a third neck layer output; wherein each neck layer output corresponds to a second detection head network for classifying the base class target.

[0049] Figure 3 The first detection model obtained based on the variant of yolov5 is shown, in which the original backbone is replaced by a 3x3 network structure. The first detection model is composed of block1 and block2, wherein block1 is composed of 3x3 convolution and 1x1 convolution, as shown in Figure 4 . Block2 is composed of 3x3 convolution, 1x1 convolution, and itself, as shown in Figure 5 . Because the operation efficiency of "large and whole" is much higher than that of "small and fragmented", the performance is significantly improved. In addition, the neck structure is replaced by FPN, which can fully extract feature information, especially for small targets, so that the entire network is very light and effective. Among them, the meaning of block2*3 is that there are three repeated block2 modules. During training, data enhancement adopts mosaic enhancement, the classification loss function adopts BCEWithLogitsLoss, the loss function of regression adopts CIoU loss, and 200 epochs are trained to obtain the trained base class model.

[0050] S12: acquire a second sample image and detect the base-class targets in the second sample image by using the trained first detection model to determine non-base-class targets in the second sample image according to the detected base-class targets.

[0051] In this embodiment, a second sample image is acquired and the base-class targets in the second sample image are detected by using the trained first detection model to determine non-base-class targets in the second sample image according to the detected base-class targets. The second sample image is mainly used for non-base-class detection training. The second sample image is mainly a real-time scene image on a construction site. After the base-class model is trained, the base-class model is deployed to the actual construction site for real-time detection, that is, real-time detection is performed on the second sample image. The false positive and false negative data on the project are collected back for re-labeling, that is, the determined non-base-class targets are re-labeled. Figure 2 The alarm picture in the second sample image of the non-base-class targets is uploaded to the cloud after being labeled, and the model is continuously iteratively optimized.

[0052] S13: after labeling the non-base-class targets in the second sample image containing the non-base-class targets, a second training set containing the second sample image and corresponding second sample labels is obtained, and a second detection model is trained by using the second training set to obtain a trained second detection model; wherein the second detection model is a model obtained by adding a preset number of first detection head networks for classifying the non-base-class targets on the output side of the first detection model.

[0053] In this embodiment, after labeling the non-base-class targets in the second sample image containing the non-base-class targets, a second training set containing the second sample image and corresponding second sample labels is obtained, and a second detection model is trained by using the second training set to obtain a trained second detection model; wherein the second detection model is a model obtained by adding a preset number of first detection head networks for classifying the non-base-class targets on the output side of the first detection model. That is, a few-shot training model structure and a loss function are designed and retrained, wherein the few-shot training model structure is specifically as shown in Figure 6 As shown in FIG. 13, one first detection head network is added in parallel to each second detection head network in the first detection model to obtain the second detection model, and each neck layer output in the second detection model respectively corresponds to the first detection head network and the second detection head network.

[0054] The second detection model structure mainly fixes the backbone and fpn layers and the detection output layer out network parameters, adds new class detection heads NewOut1, NewOut2, and NewOut3. This design can ensure that the model fully extracts features while maintaining the detection effect of the base class. Meanwhile, the base class and new class joint loss function is used for backpropagation training to improve the classification effect of the new class. The loss function L of the second detection model is ft

[0055]

[0056]

[0057] wherein, is the non-base class target loss, is the non-base class category loss, is the non-base class regression box loss, and λ is a parameter factor (optimal value 0.1), L con is the joint loss, C b is a second sample image set, is the non-base class target probability in the second sample image set, is the base class target probability in the second sample image set.

[0058] The safety helmet detection of a construction site is taken as an example to illustrate the embodiment. The base class safety helmet model is obtained by using a base class safety helmet data set of 17W training samples, and is deployed on the construction site. The safety helmet types that cannot be recognized or misrecognized by the model, i.e., non-base class safety helmet types (such as safety helmets with protective masks in the steel welding area or heat preservation safety helmets suitable for winter, which are different from ordinary base class safety helmet types), are selected. The new network model branch and the loss function are iteratively trained to obtain a safety helmet detection model that can detect base class and new class at the same time. The effect is as shown in Figure 7 Compared with the full quantity model training, the new class only uses 200, the training time is 3 minutes, and the final base class plus the new class recognition rate reaches more than 98%.

[0059] ​It can be seen that the embodiment of the application first acquires a first training set and trains a first detection model by using the first training set to obtain a trained first detection model; wherein the first training set comprises a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image; then a second sample image is acquired and the base class target in the second sample image is detected by using the trained first detection model to determine a non-base class target in the second sample image according to the detected base class target; finally, the non-base class target in the second sample image containing the non-base class target is annotated to obtain a second training set containing the second sample image and a corresponding second sample label, and a second detection model is trained by using the second training set to obtain a trained second detection model; wherein the second detection model is a model obtained by adding a preset number of first detection head networks for classifying the non-base class target on the output side of the first detection model. The embodiment of the application determines the non-base class target in the second sample image by detecting the base class target in the second sample image by using the model trained by the first detection model by using the first training set containing the base class target, and then trains the second detection model obtained by adding a preset number of detection head networks on the output side of the first detection model by using the second sample image containing the non-base class target. The above process can build a robust detection model with few samples for different scenarios, reduce model iteration training time and human annotation cost, improve training efficiency and model detection accuracy.

[0060] Referring to Figure 8 The embodiment of the application also discloses a few-sample detection model generation device, comprising:

[0061] The first training module 11 is configured to acquire a first training set and train a first detection model by using the first training set to obtain a trained first detection model; wherein the first training set comprises a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating a base class target in the first sample image;

[0062] The detection module 12 is configured to acquire a second sample image and detect the base class target in the second sample image by using the trained first detection model to determine a non-base class target in the second sample image according to the detected base class target;

[0063] The second training module 13 is configured to: obtain a second training set comprising the second sample image and a corresponding second sample label after labeling the non-base-class target in the second sample image containing the non-base-class target, and train a second detection model by using the second training set to obtain the trained second detection model; wherein the second detection model is a model obtained after adding a preset number of output layers on the output side of the first detection model.

[0064] It can be seen that, in the embodiments of the present application, the first training set is obtained first, and the first detection model is trained by using the first training set to obtain the trained first detection model; wherein the first training set comprises a first sample image and a corresponding first sample label, and the first sample label is labeling information for labeling a base-class target in the first sample image; then the second sample image is obtained, and the base-class target in the second sample image is detected by using the trained first detection model to determine the non-base-class target in the second sample image according to the detected base-class target; finally, the second training set comprising the second sample image and a corresponding second sample label is obtained after labeling the non-base-class target in the second sample image containing the non-base-class target, and the second detection model is trained by using the second training set to obtain the trained second detection model; wherein the second detection model is a model obtained after adding a preset number of first detection head networks for classifying the non-base-class target on the output side of the first detection model. In the embodiments of the present application, the base-class target in the second sample image is detected by using the model trained by the first detection model by using the first training set containing the base-class target, to determine the non-base-class target in the second sample image, and then the second detection model obtained after adding a preset number of detection head networks on the output side of the first detection model by using the second sample image containing the non-base-class target is trained again. The above process can build a robust detection model with few samples for different scenarios, reduce model iteration training time and human labeling cost, improve training efficiency and model detection accuracy.

[0065] In some specific embodiments, the first training module 11 further comprises:

[0066] The deduplication unit is configured to perform a deduplication operation on the first training set.

[0067] The determination unit is configured to determine a target attribute category from each attribute category of the labeling information of the deduplicated first training set by using principal component analysis.

[0068] a screening unit configured to screen the first sample images satisfying a preset condition of the target attribute type from the first training set after deduplication to obtain a screened first training set, so as to train the first detection model by using the screened first training set;

[0069] The deduplication unit is further configured to calculate the similarity of the first sample images by using a perceptual hashing algorithm, and remove the first sample images and the corresponding first sample labels with a similarity less than a preset threshold from the first training set.

[0070] In some specific embodiments, the few-sample detection model generation apparatus further comprises:

[0071] The first improvement module is configured to improve a backbone network structure in a traditional YOLO model into a network structure composed of a first convolutional block and a second convolutional block, and improve a neck network structure in the traditional YOLO model into a feature pyramid network structure, to obtain the first detection model; the first convolutional block is a 3*3 convolutional kernel connected in parallel with a 1*1 convolutional kernel, and the outputs of the two convolutional kernels are stacked in features; and the second convolutional block is a 3*3 convolutional kernel connected in parallel with a 1*1 convolutional kernel, and the outputs of the two convolutional kernels and the input of the second convolutional block are stacked in features.

[0072] The second improvement module is configured to add one first detection head network in parallel to each of the second detection head networks in the first detection model, so that each neck layer output corresponds to the first detection head network and the second detection head network respectively.

[0073] In some specific embodiments, the first improvement module specifically comprises:

[0074] The first improvement unit is configured to improve the backbone network structure in the traditional YOLO model into a network structure composed of two first convolutional blocks, one second convolutional block, one first convolutional block, three second convolutional blocks, one first convolutional block, thirteen second convolutional blocks, and one first convolutional block connected in series.

[0075] The second improved unit is configured to obtain a first neck layer output by performing a 1*1 convolution kernel on the feature map output by the last first convolution block of the improved backbone network structure, perform feature superposition on the first neck layer output and outputs of the thirteen second convolution blocks to obtain a second neck layer output, and perform feature superposition on the second neck layer output and outputs of the three second convolution blocks to obtain a third neck layer output; each neck layer output corresponds to a second detection head network for classifying the base class target.

[0076] Further, the embodiment of the present application further provides an electronic device. Figure 9 The electronic device 20 structure diagram shown in the figure is according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the use range of the present application.

[0077] Figure 9 The electronic device 20 structure diagram provided by the embodiment of the present application. The electronic device 20, specifically can include: at least one processor 21, at least one memory 22, power supply 23, communication interface 24, input output interface 25 and communication bus 26. Wherein, the memory 22 is used for storing computer program, the computer program is loaded and executed by the processor 21, to realize the related steps in the few sample detection model generation method disclosed by any preceding embodiment.

[0078] In the embodiment, the power supply 23 is used for providing working voltage for each hardware device on the electronic device 20; the communication interface 24 can create data transmission channel between the electronic device 20 and external device, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited here; the input output interface 25 is used for obtaining external input data or outputting data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not limited here.

[0079] In addition, the memory 22 as the carrier of resource storage can be read-only memory, random access memory, disk or optical disk, etc., and the resources stored thereon can include operating system 221, computer program 222 and data 223, etc., and the storage mode can be temporary storage or permanent storage.

[0080] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the mass data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. The computer program 222 can further include computer programs capable of completing other specific work in addition to the computer programs capable of completing the few-sample detection model generation method disclosed by any one of the foregoing embodiments and executed by the electronic device 20. The data 223 can include sample images collected by the electronic device 20.

[0081] Further, the embodiment of the present application further discloses a storage medium, wherein the storage medium stores a computer program, and the computer program is loaded and executed by a processor to realize the few-sample detection model generation method disclosed by any one of the foregoing embodiments.

[0082] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0083] Finally, it should be noted that, in this document, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0084] The few-sample detection model generation method, device, equipment and storage medium provided by the present application are introduced in detail above, and the principle and implementation manner of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method and core idea of the present application; at the same time, for the general technical personnel in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as the limitation of the present application.

Claims

1. A method for generating a few-sample detection model, characterized in that, include: The backbone network structure in the traditional YOLO model is improved into a network structure consisting of a first convolutional block and a second convolutional block, and the neck network structure in the traditional YOLO model is improved into a feature pyramid network structure to obtain a first detection model. The first convolutional block is formed by connecting a 3*3 convolutional kernel in parallel with a 1*1 convolutional kernel and then summing the features of the outputs of the two convolutional kernels. The second convolutional block is formed by connecting a 3*3 convolutional kernel in parallel with a 1*1 convolutional kernel and then summing the features of the outputs of the two convolutional kernels and the input of the second convolutional block. The improvement of the backbone network structure in the traditional YOLO model to a network structure composed of a first convolutional block and a second convolutional block, and the improvement of the neck network structure in the traditional YOLO model to a feature pyramid network structure, includes: The backbone network structure in the traditional YOLO model is improved into a network structure composed of two first convolutional blocks, one second convolutional block, one first convolutional block, three second convolutional blocks, one first convolutional block, thirteen second convolutional blocks, and one first convolutional block connected in sequence; The feature map output from the last first convolutional block of the improved backbone network structure is passed through a 1*1 convolutional kernel to obtain the first neck layer output. The first neck layer output is then superimposed with the features of the outputs of the thirteen second convolutional blocks to obtain the second neck layer output. The second neck layer output is then superimposed with the features of the outputs of the three second convolutional blocks to obtain the third neck layer output. Each neck layer output corresponds to a second detection head network used for classifying base class targets. A first training set is obtained and the first detection model is trained using the first training set to obtain the trained first detection model; wherein, the first training set includes a first sample image and a corresponding first sample label, and the first sample label is annotation information for the base class target in the first sample image; A second sample image is acquired, and the trained first detection model is used to detect the base class targets in the second sample image, so as to determine the non-base class targets in the second sample image based on the detected base class targets; the second sample image is used for non-base class detection training. After labeling the non-base class targets in the second sample image containing the non-base class targets, a second training set containing the second sample image and the corresponding second sample labels is obtained. The second training set is then used to train the second detection model to obtain the trained second detection model. The second detection model is a model obtained by adding a preset number of first detection head networks for classifying the non-base class targets to the output side of the first detection model.

2. The method for generating a few-shot detection model according to claim 1, characterized in that, After obtaining the first training set, the process also includes: The first training set is deduplicated, and the target attribute category is determined from the attribute categories of the labeled information of the first training set after deduplication using principal component analysis. The first training set is obtained by selecting the first sample images whose target attribute type meets the preset conditions from the deduplicated first training set, and then using the selected first training set to train the first detection model.

3. The method for generating a few-sample detection model according to claim 2, characterized in that, The deduplication operation on the first training set includes: The similarity of the first sample images is calculated using a perceptual hash algorithm, and the first sample images and their corresponding first sample labels with similarity less than a preset threshold are removed from the first training set.

4. The method for generating a few-shot detection model according to claim 1, characterized in that, Before training the second detection model using the second training set, the method further includes: One of the first detection head networks is added in parallel to each of the second detection head networks in the first detection model, so that the output of each neck layer corresponds to the first detection head network and the second detection head network respectively.

5. The method for generating a few-sample detection model according to any one of claims 1 to 4, characterized in that, The loss function of the second detection model for: ； ； in, For non-base class target loss, For non-base class category loss, Non-base class regression box loss, For parameter factors, For joint losses, For the second set of sample images, The probability of non-base class targets in the second sample image set. denoted as the base class target probability in the second set of sample images.

6. A device for generating a few-sample detection model, characterized in that, include: The first improvement module is used to improve the backbone network structure in the traditional YOLO model into a network structure composed of a first convolutional block and a second convolutional block, and to improve the neck network structure in the traditional YOLO model into a feature pyramid network structure, so as to obtain a first detection model; wherein, the first convolutional block is composed of a 3*3 convolutional kernel connected in parallel with a 1*1 convolutional kernel and the outputs of the two convolutional kernels are superimposed as features; the second convolutional block is composed of a 3*3 convolutional kernel connected in parallel with a 1*1 convolutional kernel and the outputs of the two convolutional kernels and the input of the second convolutional block are superimposed as features. The first improvement module specifically includes: a first improvement unit, used to improve the backbone network structure in the traditional YOLO model into a network structure composed of two first convolutional blocks, one second convolutional block, one first convolutional block, three second convolutional blocks, one first convolutional block, thirteen second convolutional blocks, and one first convolutional block connected in sequence; a second improvement unit, used to pass the feature map output of the last first convolutional block of the improved backbone network structure through a 1*1 convolutional kernel to obtain the first neck layer output, and to superimpose the first neck layer output with the outputs of the thirteen second convolutional blocks to obtain the second neck layer output, and to superimpose the second neck layer output with the outputs of the three second convolutional blocks to obtain the third neck layer output; wherein each neck layer output corresponds to a second detection head network for classifying base class targets; The first training module is used to acquire a first training set and use the first training set to train the first detection model to obtain the trained first detection model; wherein, the first training set includes a first sample image and a corresponding first sample label, and the first sample label is annotation information for annotating the base class target in the first sample image; A detection module is used to acquire a second sample image and use the trained first detection model to detect the base class targets in the second sample image, so as to determine the non-base class targets in the second sample image based on the detected base class targets; the second sample image is used for non-base class detection training. The second training module is used to annotate the non-base class targets in the second sample image containing the non-base class targets to obtain a second training set containing the second sample image and the corresponding second sample labels, and to train the second detection model using the second training set to obtain the trained second detection model; wherein, the second detection model is a model obtained by adding a preset number of output layers to the output side of the first detection model.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the few-sample detection model generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store computer-executable instructions, which, when loaded and executed by a processor, implement the few-sample detection model generation method as described in any one of claims 1 to 5.

Images