High-resolution picture association small target detection method and system
By combining the YOLOv9n and YOLOv9l models, the secondary detection is performed using the attention mechanism adaptive selection mechanism, which solves the problem of loss of detailed information during small and medium-sized objects in high-resolution pictures, and improves the detection accuracy and accuracy.
Patent Information
- Application Number
- CN202411232577.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional small object detection methods will lose some of the detailed information of the image when processing high-resolution pictures, resulting in poor detection results.
YOLOv9n is used as the first detection model for preliminary screening to reduce background information. Then, the YOLOv9l model is used to perform secondary detection in combination with the attention mechanism adaptive selection mechanism to ensure the accuracy of feature extraction.
By reducing redundant calculations and retaining native image information, the recognition accuracy of small targets is improved, and the problem of pixel information loss caused by downsampling is alleviated.
Smart Images

Figure CN119091124A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a high-resolution image-associated small target detection method and system. Background Art
[0002] Since small targets usually have low resolution, weak edge information and severe occlusion, traditional target detection methods have difficulties in dealing with small-sized targets. Currently, deep learning-based methods are mainly used to extract richer feature representations and reduce the spatial invariance of features by using technologies such as deep convolutional neural networks (CNN) and region proposal networks (RPN), thereby improving the detection performance of small targets.
[0003] The current small target detection technology based on deep learning mainly focuses on multi-scale fusion and attention mechanism. A common problem is that the original feature map will lose some detailed information of the image itself after downsampling. This problem is particularly prominent in the task of detecting small targets in high-resolution images. Summary of the invention
[0004] In view of this, in order to solve the technical problem that some detail information of the image itself is lost in the existing small target detection method, thereby resulting in low detection effect, the present invention proposes a high-resolution image associated small target detection method, the method comprising the following steps:
[0005] Using YOLOv9n as the framework, build the first detection model;
[0006] Taking YOLOv91 as the framework, the attention mechanism adaptive selection mechanism is introduced to build the second detection model;
[0007] Inputting the image to be tested into the first detection model and outputting the associated subject image;
[0008] The associated subject image is input into the second detection model, and a detection result is output.
[0009] Among them, YOLOv9n is the lightest version of YOLOv9.
[0010] In some embodiments, it also includes:
[0011] Identify the task relevance of the image to be tested.
[0012] The calculation method of the task relevance is as follows:
[0013]
[0014] Among them, Q represents task relevance, A represents the associated subject area, and B represents the associated target area.
[0015] Through this preferred step, the task relevance is identified in advance, and whether the detection task is suitable for the detection method is determined based on the task relevance, which can help improve the detection accuracy.
[0016] In some embodiments, the step of inputting the image to be tested into the first detection model and outputting the associated subject image specifically includes:
[0017] Inputting the image to be tested into the first detection model;
[0018] Identify and locate the associated subject in the image to be tested to obtain position information;
[0019] Convert the position information into a mask form, and perform a similar mapping on the input original image to obtain a marked position;
[0020] The image is captured according to the marked position to obtain an associated subject image.
[0021] Through this preferred step, the lightest YOLOv9 model is used for preliminary detection, which can filter out most of the background information in one detection and reduce a lot of redundant calculations.
[0022] In some embodiments, the second detection model includes a backbone network, a neck network, and a head network, wherein:
[0023] The backbone network includes CBS convolutional layer, RepNCSPELAN4 cross-level fusion layer and attention mechanism adaptive black box;
[0024] The neck network RepNCSPELAN4 has a cross-level fusion layer and an adaptive efficient layer aggregation pyramid structure.
[0025] Through this preferred step, an attention selection mechanism is introduced, which is set to adaptively select the attention mechanism according to different scenarios to improve the detection accuracy in the corresponding scenarios.
[0026] In some embodiments, the step of inputting the associated subject image into the second detection model and outputting the detection result specifically includes:
[0027] inputting the associated subject image into the second detection model;
[0028] Extracting features of the associated subject image based on the backbone network to obtain semantic features;
[0029] Enhance the representation capability of the semantic feature based on the neck network to obtain the final feature;
[0030] The head network outputs the location and category information of the target according to the central feature.
[0031] Through this preferred step, the second detection model with higher accuracy is used to perform depth detection of the associated small target, thereby achieving secondary detection.
[0032] In some embodiments, the adaptive high-efficiency layer aggregation pyramid structure is provided with a first branch, a second branch and a decision parameter, wherein:
[0033] When the determination parameter is within a first threshold interval, directly fusing the feature graphs of the first branch and the second branch;
[0034] When the determination parameter is in the second threshold interval, the spatial attention mechanism is enabled to focus on the spatial information of the feature maps of each layer in the first branch and the second branch and then fuse them.
[0035] Through this optimization step, an adaptive selection mechanism of the attention mechanism is introduced, which enables the model to better detect different target detection tasks, improve the overall fault tolerance of the model, increase the learnability of the model itself, and reduce human intervention.
[0036] In some embodiments, it also includes:
[0037] The first detection model and the second detection model are pre-trained.
[0038] The present invention also proposes a high-resolution image-associated small target detection system, the system comprising:
[0039] The first model building unit uses YOLOv9n as a framework to build a first detection model;
[0040] The second model building unit uses YOLOv91 as the framework, introduces the attention mechanism adaptive selection mechanism, and builds the second detection model;
[0041] A primary inspection unit, used for inputting the image to be tested into the first detection model and outputting an associated subject image;
[0042] The final detection unit is used to input the associated subject image into the second detection model and output the detection result.
[0043] Based on the above scheme, the present invention provides a high-resolution image associated small target detection method and system. Compared with the existing scheme that uses YOLOv9x to directly detect small targets, in terms of speed, the preliminary screening of most background information by YOLOv9n can greatly reduce redundant calculations, and the combination of YOLOv9n and YOLOv9l has fewer parameters than YOLOv9x; in terms of accuracy, the use of a secondary detection strategy can ensure sufficient native image information for feature extraction under the premise of locking the approximate positioning of the target, alleviate the problem of loss of only a small amount of pixel information of small targets due to downsampling, and ensure the recognition accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a flowchart of the steps of a high-resolution image-associated small target detection method of the present invention;
[0045] Figure 2 is a schematic diagram of task relevance analysis of a specific embodiment of the present invention;
[0046] Figure 3 It is a schematic diagram of the principle of the first detection model of a specific embodiment of the present invention;
[0047] Figure 4 It is a structural block diagram of a high-resolution image-associated small target detection system of the present invention. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0049] It should be noted that, for the convenience of description, only the parts related to the relevant invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.
[0050] It should be understood that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.
[0051] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.
[0052] In the description of the embodiments of the present application, "plurality" means two or more than two. The following terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.
[0053] In addition, flow charts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.
[0054] Reference Figure 1 , is a flow chart of an optional example of a high-resolution image associated small target detection method proposed in the present invention. The method can be applied to a computer device. The detection method proposed in this embodiment may include but is not limited to the following steps:
[0055] Step S1, using YOLOv9n as a framework, constructing a first detection model;
[0056] Step S2: Using YOLOv91 as a framework, introducing the attention mechanism adaptive selection mechanism, and constructing the second detection model;
[0057] Among them, the main detection model is YOLOv9. According to the accuracy and speed of target detection, YOLOv9 is divided into five versions: YOLOv9n, YOLOv9s, YOLOv9m, YOLOv9l, and YOLOv9x. Among them, n is the lightest version and x is the highest accuracy version.
[0058] Step S3, inputting the image to be tested into the first detection model, and outputting the associated subject image;
[0059] Step S4: input the associated subject image into the second detection model and output the detection result.
[0060] In some feasible embodiments, before inputting the image to be detected into the first detection model, the method further includes:
[0061] Identify the task relevance of the image to be tested.
[0062] The calculation method of the task relevance is as follows:
[0063]
[0064] Considering the scalability of the task, a judgment formula is designed to analyze whether the task meets the conditions of the detection method of this technical solution, see Figure 2 , set the associated subject area to A, the associated target area to B, and the intersection area of the associated subject and the associated target to C. When the ratio of the intersection area C to the associated target area B is greater than 0.9 (the judgment value of the task scalability evaluation formula is 1), the detection task is considered to be applicable to this technical solution. When the judgment value is 0, it means that the current task does not meet the detection conditions of the association method, and the judgment data source is the training data set of the task.
[0065] In some feasible embodiments, the step S3 specifically includes:
[0066] like Figure 3 As shown, the function of the module is to locate the associated subject, that is, to complete a detection through YOLOv9n, identify and locate the associated subject, and convert the output position information into a mask form, and perform a year-on-year mapping on the input original image, intercept the original image of the marked position area, and use the intercepted single or multiple associated subject images as input for the secondary detection, so that most of the background information can be screened out by a single detection, reducing a large amount of redundant calculations.
[0067] In some feasible embodiments, the second detection model in step S4 includes three parts: a backbone network, a neck network, and a head network, wherein:
[0068] The backbone network is responsible for extracting high-level semantic features from the input image. It gradually reduces the size of the feature map through a series of convolution and pooling layers, and gradually increases the number of channels, thereby achieving multi-scale representation of the image and extraction of semantic information. The neck module is located between the backbone network and the head, and is used to increase the representation ability of the features and the richness of the semantic information to improve the accuracy of target detection. The head is responsible for predicting the location and category information of the target. It consists of a series of convolutional layers and fully connected layers, which processes and compresses the features from the neck to generate the output of target detection.
[0069] In addition, the overall structure of the first detection model is similar to or identical to that of the second detection model.
[0070] The Backbone network module consists of a CBS convolutional layer, a RepNCSPELAN4 cross-level fusion layer, and an attention mechanism adaptive black box. The convolutional layer is responsible for transforming the number of channels of the input image, extracting features, and transforming the size of the feature map. The size transformation of the feature map is achieved by adjusting the stride step parameter. The RepNCSPELAN4 cross-level fusion layer connects the features of the front part with the features of the back part after a series of convolution operations to achieve feature fusion at different stages.
[0071] The Neck network and the Head network are the neck and head of YOLOv9. The feature fusion part of the neck adopts the GELAN (generalized efficient layer aggregation network) sampling fusion structure. Secondly, it also includes the RepNCSPELAN4 cross-level fusion layer and the SPPFELAN-A adaptive efficient layer aggregation pyramid structure, which solves the problem of fixed-size feature map output caused by input images of different scales during the feature extraction process.
[0072] In the backbone network, the attention mechanism adaptively selects the black box. Considering that the feature extraction process needs to pay attention to the channel information and spatial information of the feature map, a total of four types of attention mechanisms are designed, namely independent channel attention mechanism, independent spatial attention mechanism, serial channel and spatial attention mechanism, and parallel channel and spatial attention mechanism. Previously, a parameter α for model training was set, and four parameter thresholds were set for it, representing four attention mechanism modes. When the parameter α value learned by the model is within the threshold set by a certain mode, the model will move the input feature map to the attention mechanism of the mode for processing and output. The independent channel attention mechanism focuses on the channel information of the feature map; the independent spatial attention mechanism focuses on the spatial information of the feature map; the serial mode of the channel and spatial attention mechanism mainly focuses on obtaining better performance; the parallel mode of the channel and spatial attention mechanism mainly focuses on the comprehensive feature information.
[0073] In the neck network, SPPFELAN-A (Adaptive High-Efficiency Layer Aggregation Pyramid Structure) mainly introduces the attention mechanism adaptive selection mechanism on the SPPFELAN structure. The feature map passes through the first branch with only one convolution module to retain the original feature information of the feature map. The feature map passes through the second multi-branch structure composed of three maximum pooling operations. Each branch from the pooling kernel to the γ module is used to extract the feature information of small, medium and large scale targets. Considering the task of having a tendency to select the target scale, a learnable parameter γ is set to select whether to use the attention mechanism. Finally, all feature maps are fused and output under the channel transformation of the multi-layer convolution structure. The γ parameter is learned through repeated iterative training of the model. In order to allow the model to independently decide whether to use the spatial attention mechanism STN (SpatialTransformer Network), this technology designs a γ determination formula, that is, when the γ training value is between 0 and 0.5, the ELAN structure in the model will directly access the concat fusion operation to fuse the feature map, and when the γ value is between 0.5-1, the model will enable the spatial attention mechanism STN to focus on the spatial information of each layer of feature map before output. The introduced γ parameter can enable the model to better detect different target detection tasks, improve the overall fault tolerance of the model, increase the learnability of the model itself, and reduce human intervention.
[0074] Based on the above scheme, the specific implementation process of the present invention is that the picture to be detected is input into the lightweight target detection model YOLOv9n, and the target detection result is obtained therefrom. Then, the position of the associated subject is determined by screening the output information of YOLOv9n. Then, according to the associated subject position information, a mask image is created, in which the associated subject position is marked as the foreground and the other areas are the background. Next, the image area corresponding to the associated subject position is intercepted from the input original image. These image areas are input one by one into the larger YOLOv9l model for small target detection. For the small target position information output by the YOLOv9l model, a mask image with the same size as the input original image is generated, in which the small target position is marked as the foreground. Finally, the mask image of the small target is merged with the mask image of the associated subject to obtain the final mask image. The final mask image can be matched with the input original image to obtain the final output result, which includes the associated subject position information and the detected small target position information.
[0075] The beneficial effects of the overall solution of the present invention include but are not limited to: (1) Using the associated subject of the small target as a carrier, the associated subject with obvious features and easy to detect is identified and located through a single detection, the background area of the input image is filtered out, and redundant calculations are reduced. Then, the relationship between the associated subject and the associated target is used to perform a secondary detection on the intercepted associated subject image to identify the small target, and finally the output information is mapped back to the original image. The detection range is narrowed by the association characteristics of the object, and the native pixels of the small target are retained as much as possible to ensure that more feature information of the small target can be extracted; (2) In order to better analyze the applicability of the detection task to the technical solution, a set of judgment formulas for measuring the degree of use of the task and the technical solution is designed. By calculating the ratio of the intersection of the associated subject and the associated target to the associated target, the degree of association between the two objects is analyzed. Finally, within the allowable error range, the applicability of the task and the solution is felt by human experience. The judgment formula is used as an auxiliary tool for subjectively measuring applicability to assist users in making decisions. (3) The improved YOLOv9 model mainly proposes an adaptive attention mechanism black box. The model continuously updates the designed α parameter during the training process, so that it can autonomously select the best attention mechanism. This method can not only improve the model's autonomous learning ability, but also improve the model's detection ability to a certain extent. At the same time, an adaptive efficient layer aggregation pyramid structure (SPPFELAN-A) is proposed. Based on SPPFELAN, it introduces the characteristics of adaptive selection of spatial attention mechanism, so that the model can decide whether to enable the spatial attention mechanism according to different detection tasks to obtain better performance.
[0076] like Figure 4 As shown, a high-resolution image-associated small target detection system comprises:
[0077] The first model building unit uses YOLOv9n as a framework to build a first detection model;
[0078] The second model building unit uses YOLOv91 as the framework, introduces the attention mechanism adaptive selection mechanism, and builds the second detection model;
[0079] A primary inspection unit, used for inputting the image to be tested into the first detection model and outputting an associated subject image;
[0080] The final detection unit is used to input the associated subject image into the second detection model and output the detection result.
[0081] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0082] A high-resolution image-associated small target detection device:
[0083] at least one processor;
[0084] at least one memory for storing at least one program;
[0085] When the at least one program is executed by the at least one processor, the at least one processor implements the high-resolution image associated small target detection method as described above.
[0086] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0087] A storage medium stores instructions executable by a processor, wherein the instructions executable by the processor are used to implement a high-resolution image-associated small target detection method as described above when executed by the processor.
[0088] The contents of the above method embodiments are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0089] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A high-resolution image-associated small target detection method, characterized in that: The following steps are involved: Using YOLOv9n as the framework, build the first detection model; Using YOLOv91 as the framework, the attention mechanism adaptive selection mechanism is introduced to build the second detection model; Inputting the image to be tested into the first detection model and outputting the associated subject image; The associated subject image is input into the second detection model, and a detection result is output.
2. According to claim 1, a high-resolution image-associated small target detection method is characterized in that: Also includes: Identify the task relevance of the image to be tested.
3. According to claim 2, a high-resolution image-associated small target detection method is characterized in that: The task relevance is calculated as follows: Among them, Q represents task relevance, A represents the associated subject area, and B represents the associated target area.
4. According to claim 2, a high-resolution image-associated small target detection method is characterized in that: The step of inputting the image to be tested into the first detection model and outputting the associated subject image specifically includes: Inputting the image to be tested into the first detection model; Identify and locate the associated subject in the image to be tested to obtain position information; Convert the position information into a mask form, and perform a similar mapping on the input original image to obtain a marked position; The image is captured according to the marked position to obtain an associated subject image.
5. According to claim 2, a high-resolution image-associated small target detection method is characterized in that: The second detection model includes a backbone network, a neck network and a head network, wherein: The backbone network includes CBS convolutional layer, RepNCSPELAN4 cross-level fusion layer and attention mechanism adaptive black box; The neck network RepNCSPELAN4 has a cross-level fusion layer and an adaptive efficient layer aggregation pyramid structure.
6. According to claim 5, a high-resolution image-associated small target detection method is characterized in that: The step of inputting the associated subject image into the second detection model and outputting the detection result specifically includes: inputting the associated subject image into the second detection model; Extracting features of the associated subject image based on the backbone network to obtain semantic features; Enhance the representation capability of the semantic feature based on the neck network to obtain the final feature; The head network outputs the location and category information of the target according to the central feature.
7. A high-resolution image-associated small target detection method according to claim 6, characterized in that: The adaptive high-efficiency layer aggregation pyramid structure is provided with a first branch, a second branch and a determination parameter, wherein: When the determination parameter is within a first threshold interval, directly fusing the feature graphs of the first branch and the second branch; When the determination parameter is in the second threshold interval, the spatial attention mechanism is enabled to focus on the spatial information of the feature maps of each layer in the first branch and the second branch and then fuse them.
8. According to claim 1, a high-resolution image-associated small target detection method is characterized in that: Also includes: The first detection model and the second detection model are pre-trained.
9. A high-resolution image-associated small target detection system, characterized in that: include: The first model building unit uses YOLOv9n as a framework to build a first detection model; The second model building unit uses YOLOv91 as the framework, introduces the attention mechanism adaptive selection mechanism, and builds the second detection model; A preliminary inspection unit, used for inputting the image to be tested into the first detection model and outputting an associated subject image; The final detection unit is used to input the associated subject image into the second detection model and output the detection result.
