Small target detection method and device and medium thereof
By introducing SPPF-MAA structure and adaptive weight loss function in YOLOv8 model, the problems of unreasonable anchor settings and feature loss in small object detection are solved, and the accuracy and loss optimization effect of small object detection are improved.
Patent Information
- Application Number
- CN202411351549.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-12-27
AI Technical Summary
In the existing technology, in small object detection, there are problems such as unreasonable anchor settings, resulting in missed detection, loss of features in deep networks, and insufficient loss optimization, resulting in low detection accuracy of small object.
The improved YOLOv8 model is adopted, and the SPPF structure is replaced with the SPPF-MAA structure, and an adaptive weight loss function is introduced. Multi-scale features and detailed information are retained through the CBS layer, the Split layer, the maximum pooling layer and the average pooling layer, combined with channel attention Force mechanism, using adaptive weight loss function to enhance the contribution of small target loss.
The accuracy of small object detection is improved, multi-scale features and context information in the deep network is retained, the ability to characterize details is enhanced, and the detection accuracy and loss optimization effect of small object objects is improved.
Smart Images
Figure CN119206186A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a small target detection method, device and medium thereof. Background Art
[0002] With the development of advanced driver assistance system (ADAS) technology, ADAS has been integrated into daily driving, improving the driving experience of drivers. As an important part of ADAS, the target detection task affects the subsequent regulation and decision-making, and plays a key role in optimizing ADAS effects and improving user driving experience.
[0003] With the development of deep learning technology, target detection has achieved good results in some simple scenarios. However, in ADAS application scenarios, there are a large number of small targets, such as pedestrians, traffic lights, etc. The main reasons for the low accuracy of small target detection are:
[0004] 1. Small targets are small in size. In the detection algorithm in anchor_base, it is necessary to set a reasonable number and size of anchors. If the anchor size is unreasonable, the anchor cannot match the small target's annotation box to a positive sample, which will lead to a large number of missed detections of small targets. If the number is unreasonable, the clustered anchors cannot fully cover all the annotation box sizes, which will also lead to missed detections of small targets. Therefore, how to reasonably set the anchor is a difficulty in small target detection.
[0005] 2. Small targets are small in size. After multiple convolutions and downsampling in the deep learning model, the features of small targets are easily lost in the deep network layer, affecting the detection accuracy of small targets.
[0006] 3. Although there are a large number of small targets in ADAS images, the number of large, medium and small targets is still unbalanced overall. When calculating the loss, the loss of large and medium targets dominates. Therefore, when optimizing the model along the gradient direction, the model will mainly optimize the model along the gradient direction of optimizing the loss of large targets, thereby limiting the degree of optimization of the loss of small targets.
[0007] In view of the above reasons, an anchor-free detection algorithm is adopted based on the Yolov8 detection model, which does not require preset anchors and can eliminate the influence of anchors on small target detection. However, the existing Yolov8-based detection model still has problems such as loss of small target features and insufficient optimization of small target loss, which affects the accuracy of small target detection. Summary of the invention
[0008] The present invention aims to solve at least one of the technical problems existing in the prior art.
[0009] To this end, the present invention provides a small target detection method, device and medium thereof, which can significantly improve the accuracy of small target detection.
[0010] According to a small target detection method according to an embodiment of the present invention, the method includes:
[0011] Step S1, collecting original images and preprocessing the original images to obtain a data set;
[0012] Step S2, constructing an original YOLOv8 model, improving the original YOLOv8 model, replacing the SPPF structure of the original YOLOv8 model with the SPPF-MAA structure, and obtaining an improved YOLOv8 model;
[0013] Step S3, inputting the data set into the improved YOLOv8 model, and training the improved YOLOv8 model using an adaptive weight loss function;
[0014] Step S4, use the trained YOLOv8 model to perform target detection.
[0015] The beneficial effect of the present invention is that, compared with the existing YOLOv8 model, the improved YOLOv8 model can retain multi-scale features and rich context information in the deep network layer, while increasing the representation of details in the deep network layer, and the detection accuracy of small targets is high. In addition, the contribution of small target loss to the overall loss is improved through the adaptive weight loss function, and more attention is paid to small target loss when optimizing the loss function, further improving the detection accuracy.
[0016] Further, specifically, the SPPF-MAA structure includes:
[0017] The CBS layer is configured to extract the feature map output by the upper network;
[0018] The Split layer is configured to split the channel into halves and transmit the target object feature map separately;
[0019] Multiple stacked maximum pooling layers are configured to obtain the local maximum features of the target object feature map;
[0020] Multiple stacked average pooling layers are configured to obtain detailed features of the target object feature map;
[0021] The Concat cov layer combines the local maximum feature and detail feature of the target object feature map to obtain a fused feature map.
[0022] Further, specifically, the SPPF-MAA structure also introduces a channel attention mechanism, which is configured to learn the weights of each channel in the fusion feature map.
[0023] Further, specifically, the maximum pooling layer convolution kernel size is 5*5, and the average pooling layer convolution kernel size is 5*5.
[0024] Further, specifically, the adaptive weight loss function includes position loss and classification loss;
[0025] The position loss includes CIOU loss and DFL loss;
[0026] Further, specifically, the calculation formula of the CIOU loss is:
[0027]
[0028] Among them, ρ 2 is the Euclidean distance, b is the center point coordinate of the prediction box, and b gt is the coordinate of the center point of the annotation box, c is the diagonal length of the minimum circumscribed rectangle formed by the annotation box and the prediction box, a is the weight coefficient, and v is used to measure the consistency of the aspect ratio of the prediction box and the annotation box;
[0029] The calculation formula of DFL loss is:
[0030] l DFL =-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))
[0031] Among them, y is the distance from the center point to the bounding box, y i+1 is the integer rounded upwards, y i is the integer of y rounded down, S i and S i+1 Yes i+1 and i The corresponding model prediction value.
[0032] Further, specifically, the adaptive weight loss function includes a position loss function and a classification loss function, and the calculation formula of the position loss function is:
[0033] l bbox =w·(l ciou +l DFL )
[0034] The calculation formula of the classification loss function is:
[0035] l cls =w·BCE(y p ,y g )
[0036] in, d g is the diagonal length of the annotation box, d fm is the diagonal length of the feature map of the output layer. The result range of the sigmoid function is between 0 and 1. From the formula, we can see that the size of the weight w is related to d g Inversely proportional to the weight of the small target, the weight w ranges from 0.5 to 2, y p is the predicted value, y g is true value.
[0037] Further, specifically, in step S1, the preprocessing step includes cropping, mirror reversal and labeling steps.
[0038] A computer device comprising:
[0039] processor;
[0040] A memory for storing executable instructions;
[0041] The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the small target detection method as described above.
[0042] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the small target detection method as described above.
[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0046] Figure 1 It is a schematic diagram of the method flow of embodiment 1 of the present invention.
[0047] Figure 2 It is a structural diagram of the SPPF-MAA structure of the first embodiment of the present invention.
[0048] Figure 3 It is a structural diagram of a computer device according to Embodiment 1 of the present invention.
[0049] In the figure, 10 is a computer device; 1002 is a processor; 1004 is a memory; and 1006 is a transmission device. DETAILED DESCRIPTION
[0050] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.
[0051] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.
[0052] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0053] Example 1
[0054] The present application embodiment provides a small target detection method, such as Figure 1 As shown, the method includes:
[0055] Step S1, collecting original images and preprocessing the original images to obtain a data set;
[0056] Step S2, constructing an original YOLOv8 model, improving the original YOLOv8 model, replacing the SPPF structure of the original YOLOv8 model with the SPPF-MAA structure, and obtaining an improved YOLOv8 model;
[0057] Step S3, inputting the data set into the improved YOLOv8 model, and training the improved YOLOv8 model using an adaptive weight loss function;
[0058] Step S4, use the trained YOLOv8 model to perform target detection.
[0059] In this embodiment, the preprocessing step in step S1 includes cropping, mirror reversal and annotation steps. Specifically, an industrial camera is used to shoot the road environment to obtain an original image with a size of 4043 pixels * 643 pixels. Since the original image is rectangular and has a long length, and there are areas in the original image that do not need to be detected, the image is manually cropped to crop the target area that needs to be detected separately, and the cropped size is unified to 600 pixels * 600 pixels. The cropped image is enhanced by a mirror flip algorithm, and the defects in the enhanced image are annotated to form a data set, and then the annotated data set is divided to obtain training and test data sets.
[0060] For example, use the LabelImg tool to label the enhanced image and mark the target object in the image. Through the above method, the 1203 original images are increased to 2000 after mirroring, and divided into training set and test set in a ratio of 8:2, resulting in 1600 training sets and 400 test sets.
[0061] In this embodiment, if Figure 2 As shown, the SPPF-MAA structure includes:
[0062] The CBS layer is configured to extract the feature map output by the upper network;
[0063] The Split layer is configured to split the channel into halves and transmit the target object feature map separately;
[0064] Multiple stacked max pooling layers (MaxPool) are configured to obtain the local maximum features of the target object feature map;
[0065] Multiple stacked average pooling layers (AvgPool) are configured to obtain detailed features of the target object feature map;
[0066] Concat cov layer combines the local maximum feature and detail feature of the target object feature map to obtain a fused feature map.
[0067] Furthermore, the maximum pooling layer convolution kernel size is 5*5, and the average pooling layer convolution kernel size is 5*5.
[0068] It should be noted that the original SPPF structure stacks multiple maximum pooling layers and splices multiple target feature maps that have passed through the maximum pooling layer to enhance the network's ability to process features of different scales. However, the maximum pooling layer only uses the maximum value in the 5*5 area to represent the features of the entire 5*5 area, which will cause the loss of detailed features. The average pooling layer is set to retain detailed features. The average pooling layer uses the average value of all features in the 5*5 area to represent the features of the entire 5*5 area. Compared with the maximum pooling layer, the average pooling layer can retain more detailed features. Splicing the features that have passed through multiple maximum pooling layers and multiple average pooling layers can not only retain features of different scales, but also increase detailed features and enhance the representation ability of the model. After multiple maximum pooling layers and multiple average pooling layers, the number of channels of the fused feature map obtained after splicing becomes twice as much as the original. The increase in the number of channels will lead to an increase in the amount of model calculation, which will increase and reduce the real-time performance of the model prediction. The set Split layer splits the original target feature map sent to the maximum pooling layer into two parts. The target feature map of each part has 1 / 2 of the original number of channels. At this time, the number of channels of the target feature map obtained after multiple maximum pooling layers and average pooling layers is the same as the original feature map, and the amount of calculation is reduced, and the target detection efficiency is fast.
[0069] The SPPF-MAA structure also introduces a channel attention mechanism, which is configured to learn the weights of each channel in the fusion feature map. The learned channel weights strengthen the attention to important channels while suppressing the interference of unimportant channels, thereby improving the representation ability and performance of the YOLOv8 model.
[0070] The adaptive weight loss function includes position loss and classification loss; the position loss includes CIOU loss and DFL loss; the calculation formula of CIOU loss is:
[0071]
[0072] Among them, ρ 2 is the Euclidean distance, b is the center point coordinate of the prediction box, and b gt is the coordinate of the center point of the annotation box, c is the diagonal length of the minimum circumscribed rectangle formed by the annotation box and the prediction box, a is the weight coefficient, and v is used to measure the consistency of the aspect ratio of the prediction box and the annotation box;
[0073] The calculation formula of DFL loss is:
[0074] l DFL =-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))
[0075] Among them, y is the distance from the center point to the bounding box, y i+1 is the integer rounded upwards, y i is the integer of y rounded down, S i and S i+1 Yes i+1 and i The corresponding model prediction value.
[0076] The adaptive weight loss function includes the position loss function and the classification loss function. The calculation formula of the position loss function is:
[0077] l bbox =w·(l ciou +l DFL )
[0078] The calculation formula of the classification loss function is:
[0079] l cls =w·BCE(y p ,y g )
[0080] in, d g is the diagonal length of the annotation box, d fm is the diagonal length of the feature map of the output layer. The result range of the sigmoid function is between 0 and 1. From the formula, we can see that the size of the weight w is related to d g Inversely proportional to the weight of the small target, the weight w ranges from 0.5 to 2, y p is the predicted value, y g is true value.
[0081] It should be noted that the original loss function did not take into account the imbalance in the number of small targets and medium and large targets, resulting in a small contribution of the loss of small targets to the overall loss. The degree of optimization of the loss of small targets in the optimization loss function is limited, which ultimately leads to poor detection of small targets by the model. The adaptive weight loss function of this embodiment adaptively assigns a weight to each target based on the diagonal length of the annotation box. The weight of each target is inversely proportional to the diagonal length of its annotation box. The smaller the diagonal length of the target, the greater the weight, thereby increasing the loss weight of small target detection and thus improving detection accuracy.
[0082] In summary, the small target detection method of the present invention is relatively improved compared to the existing YOLOv8 model. The YOLOv8 model can retain multi-scale features and rich context information in the deep network layer, while increasing the representation of details in the deep network layer, and the detection accuracy of small targets is high. In addition, the contribution of small target loss to the overall loss is improved through the adaptive weight loss function, and more attention is paid to small target loss when optimizing the loss function, which further improves the detection accuracy.
[0083] Example 2
[0084] An embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a small target detection method provided in the above method embodiment.
[0085] Figure 3 The hardware structure diagram of a device for implementing a small target detection method provided in an embodiment of the present application is shown. The device may participate in constituting or include the apparatus or system provided in an embodiment of the present application. Figure 3 As shown, the computer device 10 may include one or more processors 1002 (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown.
[0086] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer device 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0087] The memory 1004 can be used to store software programs and modules of application software, such as a program instruction / data storage device corresponding to a small target detection method in an embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, that is, implementing the above-mentioned method. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0088] The transmission device 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer device 10. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 1006 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0089] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer device 10 (or mobile device).
[0090] Example 3
[0091] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction or at least one program related to a small target detection method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement a small target detection method provided in the above method embodiment.
[0092] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.
[0093] Example 4
[0094] The embodiment of the present invention further provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes a small target detection method provided in the above various optional implementations.
[0095] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0096] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0097] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0098] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. A small target detection method, characterized in that: The method comprises: Step S1, collecting original images and preprocessing the original images to obtain a data set; Step S2, constructing an original YOLOv8 model, improving the original YOLOv8 model, replacing the SPPF structure of the original YOLOv8 model with the SPPF-MAA structure, and obtaining an improved YOLOv8 model; Step S3, inputting the data set into the improved YOLOv8 model, and training the improved YOLOv8 model using an adaptive weight loss function; Step S4, use the trained YOLOv8 model to perform target detection.
2. The small target detection method according to claim 1, characterized in that: The SPPF-MAA structure includes: The CBS layer is configured to extract the feature map output by the upper network; The Split layer is configured to split the channel into halves and transmit the target object feature map separately; Multiple stacked maximum pooling layers are configured to obtain the local maximum features of the target object feature map; Multiple stacked average pooling layers are configured to obtain detailed features of the target object feature map; The Concat cov layer combines the local maximum feature and detail feature of the target object feature map to obtain a fused feature map.
3. The small target detection method according to claim 2, characterized in that: The SPPF-MAA structure also introduces a channel attention mechanism, which is configured to learn the weights of each channel in the fusion feature map.
4. The small target detection method according to claim 2, characterized in that: The maximum pooling layer convolution kernel size is 5*5, and the average pooling layer convolution kernel size is 5*5.
5. The small target detection method according to claim 1, characterized in that: The adaptive weight loss function includes position loss and classification loss; The position loss includes CIOU loss and DFL loss.
6. The small target detection method according to claim 5, characterized in that: The calculation formula of the CIOU loss is: Among them, ρ 2 is the Euclidean distance, b is the center point coordinate of the prediction box, and b gt is the coordinate of the center point of the annotation box, c is the diagonal length of the minimum circumscribed rectangle formed by the annotation box and the prediction box, a is the weight coefficient, and v is used to measure the consistency of the aspect ratio of the prediction box and the annotation box; The calculation formula of DFL loss is: l DFL =-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))Where y is the distance from the center point to the bounding box, y i +1 is the integer rounded up to the nearest integer. i is the integer of y rounded down, S i and S i +1 for y i +1 and y i The corresponding model prediction value.
7. The small target detection method according to claim 6, characterized in that: The adaptive weight loss function includes a position loss function and a classification loss function. The calculation formula of the position loss function is: l pbox ＝w·(l ciou +l DFL ) The calculation formula of the classification loss function is: l cls =w·BCE(y p ,y g ) in, d g is the diagonal length of the annotation box, d fm is the diagonal length of the feature map of the output layer. The result of the sigmoid function ranges from 0 to 1. From the formula, we can see that the size of the weight w is related to d g Inversely proportional to the weight of the small target, the weight w ranges from 0.5 to 2, y p is the predicted value, y g is true value.
8. The small target detection method according to claim 1, characterized in that: In step S1, the preprocessing step includes cropping, mirror reversal and labeling steps.
9. A computer device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the small target detection method as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the small target detection method according to any one of claims 1 to 8.