Small sample target detection method and device and computer readable storage medium

Abstract

Embodiments of the invention provide a small sample target detection method and apparatus, and a computer readable storage medium. The method comprises the following steps of obtaining a to-be-detected picture; performing feature extraction on the to-be-detected picture by adopting an image feature extraction model obtained by pre-training to obtain an actual feature map; analyzing the to-be-detected picture by adopting an object candidate model obtained by pre-training to obtain a plurality of actual detection frames respectively containing corresponding detection objects; and extracting actual detection frame features from the actual feature map by adopting each actual detection frame, and performing similarity matching on each actual detection frame feature and each detection prototype pre-stored in a detection prototype library, and marking the actual detection frame corresponding to the actual detection frame feature with the highest similarity and higher than a preset similarity threshold as a target detection frame, and determining the category of the correspondingly matched detection prototype as the category of the target detection frame. According to the embodiment, the detection efficiency is effectively improved, and the computing power cost is reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of target detection technology, and in particular, to a small sample target detection method, device, and computer-readable storage medium. Background Art

[0002] At present, in computer image vision processing technology, target detection is a commonly used image recognition technology. Target detection is to find all targets of interest in the image to be detected, determine the position of the target in the picture and the category of the target; and small sample target detection is a technology for small target detection based on traditional target detection. It forms a detection model after pre-training on a large-scale data set, and further forms a support set after a small number of instance annotations for new types of targets. It can quickly enable the detection model to support the detection of the above-mentioned new types of targets, thereby making the small sample target detection method have general detection capabilities.

[0003] An existing small sample target detection method is a small sample target detection method based on transfer learning. In the process of transfer training on the support set, the neural network inevitably only focuses on the image features in the support set containing a smaller number of categories, and forgets most of the model features in the base class in the pre-training dataset, resulting in a decrease in the target detection accuracy of the base class in the pre-training dataset, and ultimately causing a decrease in the general detection ability of the method; moreover, the method uses transfer learning, which involves transfer training on the support set, specifically involving cumbersome contents such as optimization method selection, hyperparameter setting, and the use of hardware accelerators, resulting in complex processing methods and high hardware computing power costs. Summary of the invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a small sample target detection method to effectively improve the detection efficiency and reduce the computing power cost.

[0005] A further technical problem to be solved by the embodiments of the present invention is to provide a small sample target detection device to effectively improve detection efficiency and reduce computing power costs.

[0006] A further technical problem to be solved by an embodiment of the present invention is to provide a computer-readable storage medium capable of storing a computer program that effectively improves detection efficiency and reduces computing power costs.

[0007] In order to solve the above technical problems, the embodiment of the present invention first provides the following technical solution: a small sample target detection method, comprising the following steps: Get the image to be detected; Using a pre-trained image feature extraction model to extract features from the image to be detected to obtain an actual feature map; Using a pre-trained object candidate model to analyze the image to be detected to obtain a number of actual detection frames that respectively contain corresponding detection objects; and Each of the actual detection frames is used to extract actual detection frame features from the actual feature map, and each of the actual detection frame features is matched with each detection prototype pre-stored in the detection prototype library for similarity, and the actual detection frame corresponding to the actual detection frame feature with the highest similarity and higher than a preset similarity threshold is marked as a target detection frame, and the category of the corresponding matched detection prototype is determined as the category of the target detection frame, wherein the detection prototype is obtained in the following manner: the image feature extraction model is used to extract features from each template detection frame on each template image in a preset support set to obtain the corresponding template detection frame features, and the template detection frame features of the same category are averaged and stored in the detection prototype library as a detection prototype, and each template image in the support set is pre-framed with a variety of detection objects of different categories with the template detection frame.

[0008] Furthermore, the image feature extraction model is obtained by performing expression learning training on a public large-scale image dataset.

[0009] Furthermore, the object candidate model is obtained by performing object candidate training on a public large-scale image dataset.

[0010] Furthermore, after the corresponding template detection frame features are obtained by using the image feature extraction model to extract features from each template detection frame on each template image in the preset support set, the feature lengths of the template detection frame features of the same category are first aligned using the region of interest alignment algorithm model, and then the template detection frame features of the same category are averaged and stored in the detection prototype library as a detection prototype.

[0011] Furthermore, the method further comprises: Non-maximum suppression optimization is performed on each of the target detection frames to obtain an optimized detection frame, and the category corresponding to each of the optimized detection frames is output.

[0012] On the other hand, in order to solve the above-mentioned technical problems, an embodiment of the present invention further provides the following technical solution: a small sample target detection device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the small sample target detection method as described in any one of the above items is implemented.

[0013] On the other hand, in order to solve the above-mentioned technical problems, an embodiment of the present invention further provides the following technical solution: a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the small sample target detection method as described in any one of the above-mentioned methods.

[0014] After adopting the above technical scheme, the embodiment of the present invention has at least the following beneficial effects: the embodiment of the present invention adopts the pre-trained image feature extraction model and the object candidate model to perform corresponding image processing on the to-be-detected picture to obtain the actual feature map and the actual detection frames of several detection objects; then, each of the actual detection frames is used to perform feature extraction on the actual feature map to obtain the actual detection frame features of the detection target in the corresponding actual detection frame, and then, the principle of similarity matching is adopted to perform similarity matching on each of the actual detection frame features with each of the detection prototypes pre-stored in the detection prototype library. Since each detection prototype in the detection prototype library uses the image feature extraction model to perform feature extraction on each template detection frame on each template picture in the preset support set to obtain the corresponding template detection frame The feature is then obtained by averaging the features of each template detection frame of the same category, so that the actual detection frame corresponding to the actual detection frame feature with the highest similarity and higher than the preset similarity threshold can be marked as the target detection frame, and finally the category corresponding to the matched detection prototype is determined as the category of the target detection frame, which can very quickly determine the category of the detection object and has high detection efficiency; moreover, it is only necessary to perform the same processing method on the support set to regenerate the detection prototype library and obtain the detection capability of the newly added category target, which is more concise in usage method and only needs to use similarity matching to realize target detection, and the hardware computing power cost is low; in addition, the method obtains the detection of new category targets by making a new detection prototype library, while the detection accuracy on the base category targets remains unchanged, thereby improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The following is a flowchart of an optional embodiment of the small sample target detection method of the present invention.

[0016] Figure 2 This is a principle block diagram of an optional embodiment of the small sample target detection device of the present invention.

[0017] Figure 3 This is a functional module diagram of an optional embodiment of the small sample target detection device of the present invention. DETAILED DESCRIPTION

[0018] The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention. In addition, the embodiments and features in the embodiments of the present application may be combined with each other without conflict.

[0019] like Figure 1 As shown, an optional embodiment of the present invention provides a small sample target detection method, comprising the following steps: S1: Get the image to be detected; S2: Using a pre-trained image feature extraction model to extract features from the image to be detected to obtain an actual feature map; S3: using the pre-trained object candidate model to analyze the image to be detected to obtain a number of actual detection frames that respectively contain corresponding detection objects; and S4: Using each of the actual detection frames to extract actual detection frame features from the actual feature map, respectively, performing similarity matching on each of the actual detection frame features with each of the detection prototypes pre-stored in the detection prototype library, marking the actual detection frame corresponding to the actual detection frame feature with the highest similarity and higher than a preset similarity threshold as a target detection frame, and determining the category of the corresponding matched detection prototype as the category of the target detection frame, wherein the detection prototype is obtained in the following manner: using the image feature extraction model to perform feature extraction on each template detection frame on each template image in a preset support set to obtain the corresponding template detection frame features, averaging the features of each template detection frame of the same category and storing them in the detection prototype library as a detection prototype, and each template image in the support set is pre-framed with a plurality of detection objects of different categories with a template detection frame.

[0020] The embodiment of the present invention uses a pre-trained image feature extraction model and an object candidate model to perform corresponding image processing on the image to be detected to obtain an actual feature map and actual detection frames of several detection objects; then uses each of the actual detection frames to perform feature extraction on the actual feature map to obtain actual detection frame features of the detection target in the corresponding actual detection frame, and then uses the principle of similarity matching to perform similarity matching on each of the actual detection frame features with each of the detection prototypes pre-stored in the detection prototype library. Since each detection prototype in the detection prototype library uses the image feature extraction model to perform feature extraction on each template detection frame on each template image in the preset support set to obtain the corresponding template detection frame features, each template of the same category is then used to extract the features of the template detection frame. The detection frame features are averaged to obtain the actual detection frame corresponding to the actual detection frame features with the highest similarity and higher than the preset similarity threshold, so that the actual detection frame corresponding to the actual detection frame features can be marked as the target detection frame, and finally the category corresponding to the matched detection prototype is determined as the category of the target detection frame, which can very quickly determine the category of the detection object and has high detection efficiency; moreover, it is only necessary to perform the same processing method on the support set to regenerate the detection prototype library and obtain the detection capability of the newly added category target, which is more concise in usage method and only needs to use similarity matching to achieve target detection, and the hardware computing power cost is low; in addition, the method obtains the detection of new category targets by making a new detection prototype library, while the detection accuracy on the base category targets remains unchanged, thereby improving the detection efficiency.

[0021] In the specific implementation, it can be understood that step S3 can be performed synchronously with step S2 (such as Figure 1 As shown), or executed after step S2; in addition, the detection prototype library is pre-made, and the detection task of this time has been determined when the detection prototype library is made. Therefore, the image to be detected obtained should match the detection prototype library in this detection task; in addition, the template detection boxes on each template image in the support set are usually pre-framed manually.

[0022] In an optional embodiment of the present invention, the image feature extraction model is obtained by performing expression learning training on a public large-scale image dataset. In this embodiment, the image feature extraction model is directly obtained by performing expression learning training on a public large-scale image dataset, and image learning is simple, which ensures that image feature extraction can be effectively realized.

[0023] In an optional embodiment of the present invention, the object candidate model is obtained by training object candidates on a public large-scale image dataset. In this embodiment, similarly, the object candidate model is directly obtained by training object candidates on a public large-scale image dataset, and the image learning and training is simple, ensuring that the selection of potential targets can be effectively achieved.

[0024] In an optional embodiment of the present invention, after the image feature extraction model is used to extract the corresponding template detection frame features from each template detection frame on each template image in the preset support set, the feature lengths of the template detection frame features of the same category are first aligned using the region of interest alignment algorithm model (Roi Align), and then the template detection frame features of the same category are averaged and stored in the detection prototype library as a detection prototype. In this embodiment, when making each detection prototype of the detection prototype library, after extracting the template detection frame features corresponding to each template detection frame, the feature lengths of the template detection frame features of the same category are aligned using the region of interest alignment algorithm model to ensure the consistency and accuracy of the template detection frame features, so as to perform average calculation.

[0025] In a specific implementation, the image feature extraction model is set to FM, the object candidate model is set to OPM, the support set is set to S, and the detection prototype library is set to Pr. In the support set S, at least k detection objects of each category are pre-framed by a template detection frame. The specific number of categories is usually determined by the detection task. For example, for a motor vehicle scene detection task, the detection categories can be 30 categories, including people, cars, roads, guardrails, traffic signs, etc. For the i-th category of detection objects, the detection prototype can be expressed by the following formula: Pr i =avg(FM(B i )); Among them, B i It indicates that the features of the actual detection frame in the actual detection frame B on the actual feature map extracted by the image feature extraction model FM are aligned one by one through the region of interest alignment algorithm model, and avg is the average of the aligned feature lengths; The size of the image to be detected I is set to W×H, and the block size output by the image feature extraction model FM is set to P, and the feature dimension is set to D; Then the actual feature map can be expressed as: F=FM(I); Where F∈R(H / P)×(W / P)×D); Then the n actual detection frames calculated from the image to be detected are expressed as: B=OPM(I); Among them, B∈R(n×4).

[0026] In an optional embodiment of the present invention, the method further includes: S5: Perform non-maximum suppression (NMS) optimization on each of the target detection frames to obtain an optimized detection frame, and output a category corresponding to each of the optimized detection frames.

[0027] In this embodiment, non-maximum suppression optimization is performed on each target detection frame to reduce repeated and redundant detection frames and improve the accuracy and efficiency of detection.

[0028] On the other hand, Figure 2 As shown, an embodiment of the present invention further provides a small sample target detection device 3, including a processor 30, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 30. When the processor 30 executes the computer program, the small sample target detection method described in the above embodiment is implemented.

[0029] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the small sample target detection device 3. For example, the computer program may be divided into Figure 3 The functional modules in the small sample target detection device 3, wherein the image acquisition module 41, the actual feature extraction module 42, the detection frame generation module 43, the prototype matching module 44 and the detection frame optimization module 45 respectively execute the above steps S1 to S5.

[0030] The small sample target detection device 3 can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The small sample target detection device 3 can include, but is not limited to, a processor 30 and a memory 32. Those skilled in the art can understand that the schematic diagram is only an example of the small sample target detection device 3 and does not constitute a limitation on the small sample target detection device 3. It can include more or less components than shown in the figure, or combine certain components, or different components. For example, the small sample target detection device 3 can also include input and output devices, network access devices, buses, etc.

[0031] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 30 is the control center of the small sample target detection device 3, and uses various interfaces and lines to connect various parts of the entire small sample target detection device 3.

[0032] The memory 32 can be used to store the computer program and / or module. The processor 30 realizes various functions of the small sample target detection device 3 by running or executing the computer program and / or module stored in the memory 32 and calling the data stored in the memory 32. The memory 32 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a graphic recognition function, a graphic overlay function, etc.); the data storage area can store data (such as graphic data, etc.) created according to the use of the control device. In addition, the memory 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0033] If the functions described in the embodiments of the present invention are implemented in the form of software function modules or units and sold or used as independent products, they can be stored in a storage medium readable by a computing device. Based on this understanding, the embodiments of the present invention implement all or part of the processes in the above-mentioned embodiments, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 30, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0034] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the small sample target detection method described in the above embodiment.

[0035] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0036] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, which all belong to the protection scope of the present invention.

Claims

1. A small sample target detection method, characterized in that: The method comprises the following steps: Get the image to be detected; Using a pre-trained image feature extraction model to extract features from the image to be detected to obtain an actual feature map; Using a pre-trained object candidate model to analyze the image to be detected to obtain a number of actual detection frames that respectively contain corresponding detection objects; and Each of the actual detection frames is used to extract actual detection frame features from the actual feature map, and each of the actual detection frame features is matched with each detection prototype pre-stored in the detection prototype library for similarity, and the actual detection frame corresponding to the actual detection frame feature with the highest similarity and higher than a preset similarity threshold is marked as a target detection frame, and the category of the corresponding matched detection prototype is determined as the category of the target detection frame, wherein the detection prototype is obtained in the following manner: the image feature extraction model is used to extract features from each template detection frame on each template image in a preset support set to obtain the corresponding template detection frame features, and the template detection frame features of the same category are averaged and stored in the detection prototype library as a detection prototype, and each template image in the support set is pre-framed with a variety of detection objects of different categories with the template detection frame.

2. The small sample target detection method according to claim 1, characterized in that: The image feature extraction model is obtained by performing expression learning training on a public large-scale image dataset.

3. The small sample target detection method according to claim 1 or 2, characterized in that: The object candidate model is obtained by performing object candidate training on a public large-scale image dataset.

4. The small sample target detection method according to claim 1, characterized in that: After the image feature extraction model is used to extract features from each template detection frame on each template image in the preset support set and the corresponding template detection frame features are obtained, the feature lengths of the template detection frame features of the same category are first aligned using the region of interest alignment algorithm model, and then the template detection frame features of the same category are averaged and stored in the detection prototype library as a detection prototype.

5. The small sample target detection method according to claim 1, characterized in that: The method further comprises: Non-maximum suppression optimization is performed on each of the target detection frames to obtain an optimized detection frame, and the category corresponding to each of the optimized detection frames is output.

6. A small sample target detection device, characterized in that: The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the small sample target detection method according to any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the small sample target detection method according to any one of claims 1 to 5.

Images