An unknown target detection method based on feature sharing and a storage medium

Abstract

The application provides a feature sharing-based unknown target detection method and a storage medium, wherein first, sample image data is obtained and classified; then, attribute decomposition is performed on the sample image data according to the category of the sample image data, a plurality of attribute features are obtained, and different attribute labels are assigned to each attribute feature; then, all attribute features and attribute labels are input into a detection model for training to obtain a trained detection model; and then, a to-be-detected image is input into the trained detection model to output a detection result. Since the sample image data is decomposed according to attributes, each sample image data adaptively generates one or more attribute features, thereby effectively widening the sample quantity. Meanwhile, each attribute feature is subjected to recognition training, so that the trained detection model judges the category of the to-be-detected image, thereby effectively improving the recognition speed of target detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image recognition, and particularly relates to an unknown target detection method based on feature sharing and a storage medium. BACKGROUND

[0002] Image recognition is to compare stored information (information stored in memory) with current information (information entering the senses at the time) to achieve image recognition. Target detection is to input the to-be-detected image into a trained model, output a recognition result according to the detection judgment of the trained model, and realize tracking of a target object in the to-be-detected image.

[0003] A large amount of sample image data is needed when training a detection model. Existing target detection and recognition based on a large amount of sample image data are very popular, and many good methods have emerged. However, there are few target detection and classification methods for a small amount of samples or zero samples. A deep network has a million-level parameter to fit the training data, so it can learn a huge function space (formally, it has a high VC dimension, which is part of the reason why it can learn from complex high-dimensional data). Unfortunately, the advantage of neural networks has become a major obstacle to their small sample learning. SUMMARY

[0004] In order to overcome the defects in the prior art, the application provides a technical scheme of unknown target detection based on feature sharing, to solve the problem that the neural network model cannot be fully trained when the sample quantity is small, resulting in low accuracy of image target recognition.

[0005] In a first aspect, the application provides an unknown target detection method based on feature sharing, which comprises the following steps:

[0006] Obtaining sample image data, classifying the sample image data;

[0007] According to the category to which the sample image data belongs, attribute decomposition is performed on the sample image data, a plurality of attribute features are obtained, and different attribute labels are assigned to each attribute feature;

[0008] All attribute features and attribute labels are input into a detection model for training, and a trained detection model is obtained;

[0009] The to-be-detected image is input into the trained detection model, and a detection result is output.

[0010] As an optional embodiment, the method comprises the following steps:

[0011] The sample image data is data-augmented, and the augmented sample image data is input into the detection model for training.

[0012] As an optional embodiment, the data augmentation of the sample image data comprises:

[0013] The sample image data is subjected to any one of the following operations to obtain augmented sample image data: flipping, cropping, scaling, adding noise, adjusting brightness.

[0014] As an optional embodiment, the data augmentation of the sample image data comprises:

[0015] The sample image data is identified to extract key information and first background information.

[0016] Second background information adapted to the first background information is obtained from a background database, and the key information and the second background information are combined to obtain augmented sample image data.

[0017] As an optional embodiment, the obtaining of the second background information adapted to the first background information comprises:

[0018] The background information adapted to the color of the first background information is obtained, and the obtained background information is adjusted according to the brightness information of the first background information to obtain the second background information.

[0019] As an optional embodiment, the inputting of the to-be-detected image into the trained detection model and outputting of the detection result comprises:

[0020] The to-be-detected image and attribute description information corresponding to the to-be-detected image are input into the trained detection model, the trained detection model identifies corresponding attribute features in the to-be-detected image according to the attribute description information, and determines the types of each target contained in the to-be-detected image according to the identified attribute features and outputs.

[0021] As an optional embodiment, the attribute description information comprises text information or local image information.

[0022] As an optional embodiment, the inputting of the to-be-detected image into the trained detection model and outputting of the detection result comprises:

[0023] The attribute features contained in the trained detection model are matched with the to-be-detected image, and the category to which the to-be-detected image belongs is determined according to the attribute features identified from the to-be-detected image and outputted.

[0024] As an optional embodiment, the determining the category to which the image to be detected belongs and the outputting further comprise:

[0025] determining whether the image to be detected is contained in the sample image data of the same category in the trained detection model, and if not, storing the image to be detected in the position of the sample image data of the same category and adding identification information when outputting the category to which the image to be detected belongs.

[0026] In a second aspect, the present application provides a storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method steps of the first aspect of the present application.

[0027] The present application can achieve the following technical effects:

[0028] The present application first acquires sample image data, classifies the sample image data, then performs attribute decomposition on the sample image data according to the category to which the sample image data belongs, obtains a plurality of attribute features, and assigns different attribute labels to each attribute feature, then inputs all attribute features and attribute labels into a detection model for training to obtain a trained detection model, and then inputs the image to be detected into the trained detection model to output a detection result. Since the sample image data is decomposed according to attributes, each sample image data adaptively generates one or more attribute features, thereby effectively widening the sample quantity. At the same time, each attribute feature is identified and trained, so that the trained detection model can quickly determine the category of the image to be detected by analyzing whether the image to be detected contains the corresponding attribute feature, thereby effectively improving the recognition speed and accuracy of target detection. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of an unknown target detection method based on feature sharing related to an embodiment of the present application;

[0030] Figure 2 is a flowchart of an unknown target detection method based on feature sharing related to another embodiment of the present application;

[0031] Figure 3 is a module schematic diagram of an electronic device related to an embodiment of the present application;

[0032] Figure 4 is a module schematic diagram of an electronic device related to another embodiment of the present application;

[0033] Figure 5 is a schematic diagram of attribute decomposition related to an embodiment of the present application;

[0034] Figure 6is a schematic diagram of target recognition based on attribute description information according to an embodiment of the present application;

[0035] Figure 7 is a schematic diagram of target recognition without attribute description information according to an embodiment of the present application;

[0036] Figure 8 is a schematic diagram of output result of target recognition without attribute description information according to an embodiment of the present application.

[0037] Reference signs:

[0038] 20, electronic device; 201, processor; 202, storage medium. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not constitute a limitation on the present application.

[0040] As shown in Figure 1 , a flowchart of an unknown target detection method based on feature sharing according to an embodiment of the present application. The method comprises the following steps:

[0041] Firstly, enter step S101 to obtain sample image data, and classify the sample image data;

[0042] Then, enter step S102 to perform attribute decomposition on the sample image data according to the category to which the sample image data belongs, obtain a plurality of attribute features, and assign different attribute labels to each attribute feature;

[0043] Then, enter step S103 to input all attribute features and attribute labels into a detection model for training, and obtain a trained detection model;

[0044] Then, enter step S104 to input the image to be detected into the trained detection model, and output a detection result.

[0045] In the present application, the sample image data is data to be input into the detection model for training. The sample image data is image data as the name implies, which can be obtained directly from relevant images or can be obtained by cutting from video stream data.

[0046] The category to which the sample image data belongs refers to a category name corresponding to a target object contained in the sample image data. For example, a certain sample image data is a carrier image, the carrier image includes fighter, radar, bridge and other categories, and then after attribute decomposition based on the category to which the sample image data belongs, the fighter, radar, bridge and other local images, i.e. attribute features, can be obtained separately. The attribute label refers to identification information corresponding to each attribute feature, such as "fighter", "radar", "bridge", etc.

[0047] The detection model refers to a neural network model with image target recognition function, mainly including two categories of candidate region-based target recognition model and end-to-end-based target recognition model, such as R-CNN, SPP-net, Fast R-CNN, Faster R-CNN, R-FCN, etc. End-to-end-based target recognition model such as YOLO, SSD, etc.

[0048] Through the scheme of the present application, the sample image data is decomposed according to the attributes, and one or more attribute features are adaptively generated for each sample image data, thereby effectively widening the sample quantity. At the same time, each attribute feature is identified and trained, so that the trained detection model can quickly judge the category of the to-be-detected image by analyzing whether the to-be-detected image contains the corresponding attribute feature, thereby effectively improving the recognition speed and accuracy of target detection.

[0049] In some embodiments, the method of the present application comprises the following steps: data augmentation is performed on the sample image data, and the augmented sample image data is input into the detection model for training. Preferably, the data augmentation on the sample image data comprises: performing any one of the following operations on the sample image data to obtain the augmented sample image data: flipping, cropping, scaling, adding noise, adjusting brightness. By data augmentation on the sample image data, when the original sample image data is relatively small, the sample data quantity for training can be greatly widened, thereby improving the accuracy of detection model recognition.

[0050] As shown in FIG. 2, in some embodiments, the data augmentation on the sample image data comprises: first entering step S201 to identify the sample image data and extract key information and first background information; and then entering step S202 to obtain second background information adapted to the first background information from the background database, and combining the key information and the second background information to obtain the augmented sample image data. Figure 2 Preferably, the second background information adapted to the first background information comprises: obtaining background information adapted to the color of the first background information, and adjusting the obtained background information according to the brightness information of the first background information, thereby obtaining the second background information.

[0051] The key information refers to the main part of the sample image data, and the background information refers to the background part of the sample image data other than the main part. The background database is a database storing a plurality of different types of background information, which can be a background image taking mountains, oceans, lands, islands, cities, villages, skies, etc. as the background, or a background image taking mixed images of different colors as the background, or a photo input by a user.

[0052] Taking three warships sailing in the ocean as an example of sample image data, the three warships are the key information corresponding to the sample image data, and the ocean and other parts are the first background information corresponding to the sample image data. When the sample image data is obtained, the key information and the first background information are extracted first, and then the background database is searched for a background image related to the ocean, and the obtained background image related to the ocean is added to the three warships extracted from the sample image data to obtain new sample image data, thereby realizing the conversion of one sample image data into multiple sample image data.

[0053] In order to improve the training depth of the detection model for a certain sample image data, after obtaining the background information from the background database, the obtained background information is adjusted according to the brightness information of the first background information to obtain the second background information. For example, after obtaining the ocean image from the background database, the obtained ocean image is adjusted according to the background ocean tone of the sample image data. The second background information obtained in this way is closer to the original sample image data after being fused with the key information, and after repeated training and recognition by the detection model, the accuracy of image recognition of the original sample image data of this type can be effectively improved.

[0054] In some embodiments, inputting the to-be-detected image into the trained detection model outputs the detection result, including: inputting the to-be-detected image and the attribute description information corresponding to the to-be-detected image into the trained detection model, the trained detection model identifying the corresponding attribute features in the to-be-detected image according to the attribute description information, and determining the types of each target contained in the to-be-detected image according to the identified attribute features and outputting.

[0055] The attribute description information refers to information describing the target object on the image. Preferably, the attribute description information includes text information or local image information. For example, after obtaining a sample image data, attribute description information of a target object possessed by the sample image data can be received synchronously by a user. Taking three warships sailing in the sea as an example, the attribute description information can be that an aircraft runway and the bow of different warships are input by text. The trained detection model identifies the corresponding attribute features in the to-be-detected image according to the attribute description information, determines the types of the targets contained in the to-be-detected image according to the identified attribute features, and outputs the types, that is, in the output result, the aircraft runway and the bow of the warships are identified in addition to the warships.

[0056] Specifically, the detection model can summarize the features contained in different warships. For example, the necessary attribute features of an aircraft carrier include an aircraft runway and a fighter. When all the necessary attribute features of the aircraft carrier are found in the to-be-detected image, the position of the aircraft carrier can be framed in the output result, and the positions of the attribute features such as the aircraft runway and the fighter can be marked.

[0057] In some embodiments, the to-be-detected image is input into the trained detection model, and the output detection result includes: matching the attribute features contained in the trained detection model with the to-be-detected image, determining the category of the to-be-detected image according to the attribute features identified from the to-be-detected image, and outputting the category. Preferably, determining the category of the to-be-detected image and outputting the category further include: determining whether the to-be-detected image is contained in the sample image data of the same category in the trained detection model, if not, storing the to-be-detected image in the position of the sample image data of the same category, and adding identification information when outputting the category of the to-be-detected image.

[0058] For example, the trained detection model stores sample image data of an A-type fighter and a B-type fighter. Through attribute decomposition, it is found that different types of fighters contain a striking device and a pod attribute. However, through the learning and generalization of the previous attribute features and attribute labels, when the to-be-detected image is identified to contain the striking device and the pod attribute features, but the identified target is not the A-type fighter or the B-type fighter stored in the detection model database, the detection model defines the identification result as a fighter C and displays the result, where the fighter is the category of the identified target, and C is the added identification information.

[0059] In a second aspect, the present application also provides a storage medium 202, which stores a computer program. When the computer program is executed by a processor, the steps of the method of the first aspect of the present application are implemented.

[0060] like Figure 3 As shown, in a third aspect, this application also provides an electronic device 20, which includes a processor 201 and a storage medium 202. The processor 201 is used to execute a computer program stored in the storage medium 202 to implement the method steps described in the first aspect of this application.

[0061] Preferably, the electronic device 20 includes, but is not limited to: personal computer, server, general-purpose computer, special-purpose computer, network equipment, embedded device, programmable device, smart mobile terminal, smart home device, wearable smart device, vehicle-mounted smart device, etc.

[0062] Processor 201 is an electronic component with data processing capabilities, including but not limited to: central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), etc.

[0063] Storage media 202 include, but are not limited to: RAM, ROM, magnetic disk, magnetic tape, optical disk, flash memory, USB flash drive, portable hard drive, memory card, memory stick, network server storage, network cloud storage, etc.

[0064] like Figure 4 As shown, the electronic device of this application includes a learning module and a real-time monitoring and recognition module. When recognizing target objects in an image, it handles three scenarios: a large number of samples, a small number of samples, and zero samples. For image data with a large number of samples, these samples can be directly input into the learning module for training. The trained real-time monitoring and recognition module then processes the real-time image (i.e., the image to be detected). For image data with a small number of samples, preprocessing data augmentation techniques can be used to increase the sample data from few to many before inputting it into the learning module for training. The trained real-time monitoring and recognition module then processes the real-time image (i.e., the image to be detected). For image data with zero samples, preprocessing generative model techniques can be used to transform the sample data from zero to many before inputting it into the learning module for training. The trained real-time monitoring and recognition module then processes the real-time image (i.e., the image to be detected).

[0065] In this application, the learning module generates feature representations with strong generalization capabilities by learning from a large number of samples of known categories, and decomposes and learns the target of interest, such as learning the segmentation labels of various parts of a warship, such as the bridge, landing pad, and radome. In this way, the entire detection system not only contains feature information of typical known targets, but also some local features. When we only know the local features and not the actual image and all features, we can still perform detection and recognition by interactively inputting the description of the local features.

[0066] For the target detection and recognition of both few-shot and zero-shot, the following is combined with Figures 5-8 Detailed expansion:

[0067] ① For few-shot or one-shot recognition, first, make full use of the few available samples, i.e. data augmentation technology, such as changing the brightness of sample image data, adding noise, flipping, random cropping, scaling, etc. to create a large number of sample data, which is a pre-processing method to increase samples; second, attribute decomposition and learning of a large number of heterogeneous labeled samples with similar attributes (these sample data are usually easy to obtain), attribute features obtained by decomposition are assigned different attribute labels, and these attribute labels and decomposed samples are learned with other labeled samples to generate a new normalized model, which is used for real-time detection and recognition. The real-time detection and recognition module can detect all types of samples in the complete sample library and can also detect and recognize local feature attributes. This process is shown in Figure 5 Figure 5 Few-shot in the middle refers to target samples of the same category as the detected target.

[0068] ② For zero-shot target detection and recognition, it usually contains the following two cases:

[0069] a. When we know the characteristics of the target to be recognized, we can input attribute description information, use the feature attribute training results of the learning module trained in Figure 4 and zero-shot feature model generation technology to finally generate a series of detected target feature attribute sets in the learning classifier. When the real-time detection and recognition module detects a target that meets the description, it will be marked out, as shown in Figure 6

[0070] b. When we have no prior knowledge about the detected target and do not know how to input attribute description information, the zero-shot image information learning module will output similar categories to the target it contains and be labeled, such as A and B fighter samples in the database, but not known fighter A and B, which are defined as fighter C. Through attribute decomposition, it is found that different types of fighters contain strike devices and pod attributes, but through the previous feature and attribute learning normalization, when a fighter that is not in the database appears in the real-time image, the detection and recognition module can recognize it as a fighter and mark it out, as shown in Figures 7-8

[0071] ​​​In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the specification and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0072] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

[0073] The above specific embodiments of the present application do not constitute a limitation on the scope of protection of the present application. Any various other corresponding changes and modifications made in accordance with the technical concept of the present application shall be included in the scope of protection of the claims of the present application.

Claims

1. A feature sharing based unknown target detection method, characterized in that, The method comprises the following steps: Obtaining sample image data, data augmentation of the sample image data, and classification of the sample image data; the data augmentation of the sample image data comprises: recognizing the sample image data, extracting key information and first background information; obtaining second background information that is adapted to the first background information from a background database according to the brightness information and the hue information of the first background information, combining the key information and the second background information to obtain augmented sample image data, so that the second background information is closer to the original sample image data after being fused with the key information; Attribute decomposition of the sample image data according to the category to which the sample image data belongs is performed to obtain local images of a plurality of attribute features, and different attribute labels are assigned to each attribute feature; All attribute features and attribute labels are input into a detection model for training to obtain a trained detection model; A to-be-detected image is input into the trained detection model, and a detection result is output, comprising: inputting the to-be-detected image and attribute description information corresponding to the to-be-detected image into the trained detection model, the trained detection model identifying corresponding attribute features in the to-be-detected image according to the attribute description information; matching the attribute features contained in the trained detection model with the to-be-detected image, determining the category to which the to-be-detected image belongs according to the attribute features identified from the to-be-detected image, and outputting; the attribute description information comprises text information or local image information.

2. The feature sharing based unknown target detection method of claim 1, wherein, The data augmentation of the sample image data comprises: Any one of the following operations is performed on the sample image data to obtain augmented sample image data: flipping, cropping, scaling, adding noise, and adjusting brightness.

3. The feature sharing based unknown target detection method of claim 1, wherein, The second background information that is adapted to the first background information comprises: Obtaining background information that is adapted to the color of the first background information, and adjusting the obtained background information according to the brightness information of the first background information to obtain the second background information.

4. The feature sharing based unknown target detection method of claim 1, wherein, Determining the category to which the to-be-detected image belongs and outputting further comprises: Judging whether the to-be-detected image is contained in sample image data of the same category in the trained detection model, and if not, storing the to-be-detected image to the position of the sample image data of the same category and adding identification information when outputting the category to which the to-be-detected image belongs.

5. A storage medium, characterized by The storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps of any one of claims 1-4. The storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps of any one of claims 1-4.

Images