An improved YOLOv8n small target detection method based on feature fusion

The significance and matching data of small targets are obtained through feature fusion technology, and the target recognition window is resplit, which solves the problem of loss of feature information of small targets in the YOLOv8n model, and improves the accuracy and efficiency of small target detection.

CN118657927BActive Publication Date: 2024-11-29BEIJING DINGXING TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410908903.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-29
Estimated Expiration
2044-07-08

AI Technical Summary

Technical Problem

In the small object detection, the existing YOLOv8n model is easily lost during the convolution process, resulting in a decrease in detection efficiency and accuracy.

Method used

Through an improved method based on feature fusion, the significance data and matching degree data of small targets are obtained, the target recognition window is resplit, and the feature signal extraction and retention of small targets in the convolutional layer are enhanced.

Benefits of technology

It improves the accuracy and efficiency of small object detection, reduces the loss of feature information, and improves the small object recognition ability in high-resolution large-format images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657927B_ABST
    Figure CN118657927B_ABST
Patent Text Reader

Abstract

The present application discloses an improved YOLOv8n small target detection method based on feature fusion, which relates to the technical field of image feature extraction. The method comprises: based on the original image, obtaining the first feature map of target detection; based on the first feature map, obtaining the first significance data of the small target; based on the first significance data of the small target, obtaining the target recognition window of the small target; based on the target recognition window of the small target, obtaining the detection result corresponding to the small target. The present application extracts the characteristic signal concentration formed by the local characteristic signal intensity of the signal formed by the single-layer feature extraction of the small target in the feature map, and comprehensively extracts the characteristic signal concentration formed by the deepening of the convolution layer as the characteristic signal intensity distribution of the small target area, and re-splits the window according to the distribution of high-value small targets in the window, thereby increasing the accuracy of small target recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of image feature extraction, and specifically to an improved YOLOv8n small target detection method based on feature fusion. Background Art

[0002] YOLO (You Only Look Once) is a real-time target detection algorithm that can provide an algorithm that balances detection quality and detection speed. Therefore, the YOLO model can achieve real-time and efficient target detection and has a wide range of applications in visual detection.

[0003] YOLOv8n is an updated version of the YOLO model, which is optimized for the detection of tiny objects and improves the accuracy and robustness of small target detection. In the target detection process, small target detection often plays an important role. For example, in the actual process of small target detection, the number of pixels at the small target position in the image is small, resulting in low feature information content, and it is difficult to identify in high-resolution and large-format monitoring images.

[0004] For small targets of various shapes, the existing fixed feature window size method requires cutting and deforming the selected target range, which further interferes with the low feature information of the small target itself, resulting in a further decrease in monitoring efficiency. Therefore, the existing technology uses the Spatial Pyramid Pooling (SPP) layer to select targets of different sizes. However, in the process of extracting feature signals from small targets, due to the limited strength of the feature information of small targets, it is easy to lose signals in deeper convolutional layers. Summary of the invention

[0005] The purpose of this application is to provide an improved YOLOv8n small target detection method based on feature fusion to solve the technical problem that the feature information of small targets in the prior art is easily lost in the convolution process.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] This application proposes a technical solution for improving the YOLOv8n small target detection method based on feature fusion, and the method includes:

[0008] Based on the original image, obtaining a first feature map for target detection; the original image is obtained in advance;

[0009] Based on the first feature map, obtaining first saliency data of the small object;

[0010] Based on the first saliency data of the small target, obtaining a target recognition window of the small target;

[0011] Based on the target recognition window of the small target, obtaining a detection result corresponding to the small target;

[0012] The detection result is output visually.

[0013] As a specific solution in the technical solution of the present application, the step of obtaining a first feature map for target detection based on the original image includes:

[0014] Based on a collection device, video data is acquired; the collection device is pre-installed;

[0015] Based on the video data, obtaining an original image;

[0016] The original image is input into the YOLOv8n model, and based on the network convolution layer of the YOLOv8n model, a first feature map of target detection is obtained.

[0017] As a specific solution in the technical solution of the present application, the obtaining of the first saliency data of the small target based on the first feature map includes:

[0018] Based on the first feature map, obtaining the signal strength of the small target;

[0019] Based on the first feature map, obtaining position information data of the small target;

[0020] Based on the signal strength and the position information data, first saliency data of the small target is acquired.

[0021] As a specific solution in the technical solution of the present application, the calculation formula for obtaining the first saliency data of the small target based on the signal strength and the position information data is as follows:

[0022] ;

[0023] in, is the first saliency data of the target at position i; The signal strength of the target at position i; is the mean value of the significant signal strength of all position targets in the first feature map; is the standard deviation of the significant signal strength of all position targets in the first feature map; is the number of locations of targets with significant signal strength that are connected to the target at location i; is the position interval of the farthest pair of targets among all the significant signal strength positions in the first feature map; is the distance between the target at position i and its nearest significant signal strength position.

[0024] As a specific solution in the technical solution of the present application, the step of obtaining the target recognition window of the small target based on the first saliency data of the small target includes:

[0025] Based on the YOLOv8n model and the first feature map, obtain a second feature map;

[0026] Based on the second feature map, obtaining second saliency data of the small object;

[0027] Taking the position of the small target as the center and based on the first feature map, obtaining a first convolution window;

[0028] Taking the center of the first convolution window as the center and based on the second feature map, obtaining a second convolution window;

[0029] Based on the first convolution window and the second convolution window, obtaining small target matching degree data;

[0030] Based on the small target matching degree data, a target recognition window of the small target is obtained.

[0031] As a specific solution in the technical solution of the present application, the obtaining of small target matching degree data based on the first convolution window and the second convolution window includes:

[0032] Based on the first convolution window, searching for a target position with significant signal strength from the center of the first convolution window in a direction where the signal strength at the center of the first convolution window decreases the most, to form a first target position sequence;

[0033] Based on the second convolution window, searching for a target position with a significant signal strength from the center of the second convolution window in a direction where the signal strength at the center of the second convolution window decreases the most, to form a second target position sequence;

[0034] Based on the first target position sequence and the second target position sequence, small target matching degree data is acquired.

[0035] As a specific solution in the technical solution of the present application, the calculation formula for obtaining the small target matching degree data based on the first target position sequence and the second target position sequence is as follows:

[0036] ;

[0037] Among them, C is the small target matching degree data, is the number of elements in the first target position sequence; is the number of elements in the second target position sequence.

[0038] As a specific solution in the technical solution of the present application, the step of obtaining the target recognition window of the small target based on the small target matching degree data includes:

[0039] Based on the second feature map, obtaining a current recognition window of the small target;

[0040] Based on the current identification window, obtaining a collection of target positions of all significant signal strengths in the current identification window;

[0041] Based on the target position collection, obtaining a small target matching degree data collection;

[0042] Based on the small target matching degree data set, obtaining split evaluation data;

[0043] Based on the split evaluation data, the current recognition window of the small target is split to obtain the target recognition window of the small target.

[0044] As a specific solution in the technical solution of this application, the calculation formula for obtaining the split evaluation data based on the small target matching degree data set is as follows:

[0045] ;

[0046] in, is the number of data in the small target matching degree data set; is the number of data in the small target matching degree data set after excluding the small target matching degree data of the j-th target position; is the matching degree of the small target corresponding to the j-th target position; is the matching degree of the small target corresponding to the h-th target position; It is the mean of the small target matching degree in the small target matching degree data collection.

[0047] As a specific solution in the technical solution of the present application, the current recognition window of the small target is split based on the split evaluation data to obtain the target recognition window of the small target, including:

[0048] Based on the position of the small target, a first distance is acquired; the first distance is the shortest distance between the small target and the edge of the current recognition window;

[0049] Based on the position of the small target, a target line is obtained; the target line is a line connecting the position of the small target and other target positions in the current recognition window;

[0050] Based on the target line, obtaining the target midpoint;

[0051] Based on the positions of the target midpoint and the small target, obtaining a second distance;

[0052] Based on the first distance and the second distance, obtaining a mean distance;

[0053] Based on the position of the small target and the mean distance, a target recognition window of the small target is obtained.

[0054] Compared with the prior art, the beneficial effects of this application are:

[0055] This application addresses the technical problem that the existing Yolov8n model used for small target recognition has poor small target position recognition effect in the process of small target detection only through network layer changes and feature recognition windows of different preset sizes. This application extracts the characteristic signal concentration formed by the significance of the local characteristic signal strength of the signal formed by the small target in the single-layer feature extraction in the feature map, comprehensively analyzes the distribution of characteristic signal strength in the small target area as the convolution layer goes deeper, and re-splits the window according to the distribution of high-value small targets in the window, thereby increasing the accuracy of small target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of an improved YOLOv8n small target detection method based on feature fusion proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] The terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, such as the first significant data and the second significant data proposed below, which belong to different data. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in an order other than the content illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules that appear in the embodiments of the present application is only a logical division. There may be other division methods when implemented in actual applications, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0060] In order to solve the technical problem in the prior art that the feature information of small targets is easily lost during the convolution process, the present application proposes an improved YOLOv8n small target detection method based on feature fusion, and the method includes steps S100 to S500.

[0061] Step S100: Based on the original image, a first feature map for target detection is obtained.

[0062] It should be clear that in the embodiments of the present application, the original image is acquired in advance. In the present embodiment, there is no restriction on the acquisition method of the original image. For example, the original image can be an image taken by a camera, an image downloaded from the network, an image taken by a remote sensing aircraft, or image frame data acquired based on video data. In other words, in the embodiments of the present application, there is no restriction on the application scenario of the method.

[0063] Since the small target detection method is mainly used in large-format long-distance monitoring application scenarios, in a specific embodiment of the present application, step S100, based on the original image, obtains the first feature map of target detection, including steps S110 to S130.

[0064] Step S110: Acquire video data based on the acquisition device.

[0065] It should be clear that in the embodiments of the present application, the acquisition device is pre-installed. In this embodiment, the acquisition device can use a high-resolution monitoring device. And the high-resolution monitoring device is installed in a higher position area (for example, a watchtower on the beach or a telephone pole in a square, etc.), and the monitoring device is set to video mode, and the frame rate is set to 60fps (frames per second). Of course, in other embodiments of the present application, the monitoring device can also be used to only obtain image data, or obtain video data at other frame rates.

[0066] Step S120: Acquire an original image based on the video data.

[0067] It should be clear that obtaining the original image (i.e., image data) based on the video data is a mature technology and will not be described in detail here. In a specific embodiment of the present application, the video data can be transmitted to a computer via the H.264 / AVC encoding format, and after the computer decodes the above video data, continuous frame images in the video data are extracted as the original image.

[0068] Step S130: input the original image into the YOLOv8n model, and obtain a first feature map of target detection based on the network convolution layer of the YOLOv8n model.

[0069] It should be clear that obtaining the feature map of the original image through the YOLOv8n model is a mature technology and will not be described in detail here.

[0070] Step S200: Based on the first feature map, obtain first saliency data of the small object.

[0071] It should be clear that, since the feature information of a small target is easily lost in multiple convolution processes, in the embodiment of the present application, the first saliency data of the small target refers to the saliency of the small target among all targets in the first feature map. It is easy to understand that the higher the saliency of the small target, the more it is necessary to avoid the loss of the feature information of the small target in the subsequent convolution process.

[0072] In one embodiment of the present application, the first saliency data of the small target may be any data that can express the saliency of the small target in the first feature map. For example, in one embodiment of the present application, the first saliency data may be the signal strength of the small target. The greater the signal strength, the higher the saliency of the small target, and the less likely the feature information of the small target is to be lost.

[0073] It is easy to understand that in the first feature map, the target positions occupied by the large target are relatively large. In other words, a large target has multiple different signal strengths. If only the signal strength is used as the first saliency data of the small target, the first saliency data of multiple small targets can be obtained based on a large target, and a large number of target recognition windows will be obtained in the subsequent step S300. A large number of target recognition windows will result in too much data, which is not conducive to subsequent data processing.

[0074] It should be understood that, assuming that there is a small target in the first feature map, the stronger the signal strength of the target position corresponding to the small target (that is, the pixel value of the target position), the more obvious the difference between the signal strength of the target position and the signal strength of other nearby positions. Assuming that there is a large target in the first feature map, although the signal strength of the target position corresponding to the large target is stronger, the difference between the signal strength of the target position and the signal strength of other nearby positions is small. Therefore, in an embodiment of the present application, the first significance data of the small target can be obtained in this way. It is easy to understand that if there is a position point with a strong signal strength in the above-mentioned first feature map, and the difference between the signal strength of the position point and the signal strength of other nearby positions is relatively obvious, it can be considered that the position point belongs to the position point of the small target, that is, the significance of the position point is higher.

[0075] In one embodiment of the present application, step S200, obtaining first saliency data of the small target based on the first feature map, includes steps S210 to S230.

[0076] Step S210: Based on the first feature map, obtain the signal strength of the small target.

[0077] It should be clear that, as can be seen from the foregoing, the signal strength of the small target is also the pixel value of the target position corresponding to the small target. Based on the first feature map, the signal strength of the small target is obtained, that is, based on the feature map, the pixel value of the target position is obtained. Since it is a mature technology to obtain the pixel value of the target position in the feature map based on the feature map, it will not be described here.

[0078] Step S220: Based on the first feature map, obtain the position information data of the small target.

[0079] It should be understood that, in this embodiment, the position information data of the small target refers to the position of the small target in the first feature map.

[0080] Step S230: Based on the signal strength and the position information data, obtain first saliency data of the small target.

[0081] As can be seen from the foregoing, in this embodiment, the first significant data refers to data that can characterize the strength of the signal at the target location and the difference in signal strength between the target location and other adjacent locations. In other words, the first significant data can be any data that can characterize the strength of the signal at the target location and the difference in signal strength between the target location and other adjacent locations.

[0082] In order to obtain the quantified first saliency data of the small target, in one embodiment of the present application, in step S230, based on the signal strength and the position information data, a calculation formula for obtaining the first saliency data of the small target is as follows:

[0083] ;

[0084] in, is the first saliency data of the target at position i; is the signal strength of the target at position i; is the mean value of the significant signal strength of all position targets in the first feature map; is the standard deviation of the significant signal strength of all position targets in the first feature map; is the number of locations of targets with significant signal strength that are connected to the target at position i (that is, the number of locations of targets with significant signal strength that are adjacent to the target at position i); is the position interval of the farthest pair of targets among all the significant signal strength positions in the first feature map; is the distance between the target at position i and its nearest significant signal strength position.

[0085] In this embodiment, based on the first feature map, the signal strength of each position is arranged in descending order to obtain a signal strength descending sequence. The sequence is detected by the OTSU algorithm to obtain the position of the maximum inter-class variance of the signal strength, and the signal strength in the category where the maximum signal strength is located is recorded as the significant signal strength.

[0086] It should be clear that in the embodiments of the present application As the significance relationship of the signal strength of the other positions in the first feature map of the target at position i, it is easy to understand that the higher the value, the higher the significance of the target at position i. As the proportional relationship between the area of ​​the high characteristic signal region where the target at position i is located and the characteristic signal distribution distance, the larger the formula is, the more likely the target at position i is to be in a small and independent characteristic signal region. In other words, the target at position i is more likely to be an independent small target. It is easy to understand that in the embodiment of the present application, the first saliency data The larger the value of , the greater the probability that the target at position i is a small target, that is, the more the feature information of the target at position i cannot be lost.

[0087] Step S300: Based on the first saliency data of the small target, an object recognition window of the small target is obtained.

[0088] It is easy to understand that in the prior art, since the convolution kernel will superimpose the low-intensity signal around the high-intensity signal with the high-intensity signal at the same time during the convolution process, the intensity of the high-intensity signal in the result will be forced to be averaged, and the small target often occupies a smaller area, and the location of the high-intensity signal is limited. Therefore, after the signal is extracted through multiple convolution layers, the high-intensity signal position is gradually aggregated into a smaller area, and the signal intensity level of the low-signal intensity position around is lower than that of the high-intensity signal position. Therefore, after multiple convolution operations, the feature information of the small target is easily lost. It should be clear that since the feature information of the small target is easily lost in multiple convolution processes, in the embodiments of the present application, a target recognition window suitable for the small target can be set separately for the small target to avoid the feature information of the small target from being lost in the next convolution layer.

[0089] In an embodiment of the present application, we can form a target recognition window for a small target in each convolution process to avoid the loss of feature information of the small target during multiple convolutions. As can be seen from the foregoing, since the convolution kernel will simultaneously superimpose the low-intensity signal around the high-intensity signal with the high-intensity signal during the convolution process, the intensity of the high-intensity signal in the result is forced to be averaged, which easily causes the feature information of the small target to be lost. It is easy to understand that if a target recognition window is formed for all small targets, if the number of small targets in the first feature map is large, it is very easy to cause a sharp increase in the number of subsequent convolutions, which leads to a surge in calculations. It should be clear that in multiple convolutions, not all feature information of small targets will be lost, but the feature information of some small targets close to the large target is easily lost. In order to reduce the amount of data processed in subsequent convolutions, in a specific embodiment of the present application, step S300, based on the first saliency data of the small target, obtains the target recognition window of the small target, including steps S310 to S360.

[0090] Step S310: Based on the YOLOv8n model and the first feature map, obtain a second feature map.

[0091] It should be clear that the general YOLOv8n model has multiple convolutional layers. For example, in a specific embodiment of the present application, a YOLOv8n with 5 convolutional layers is used, which are respectively recorded as convolutional layer 1 to convolutional layer 5, where 1 to 5 are the order of convolutional layers, and the convolution kernel size of each convolutional layer is 3*3. In other embodiments of the present application, the YOLOv8n model may also have other numbers of convolutional layers.

[0092] In an embodiment of the present application, the first feature map is obtained by the current convolutional layer in the YOLOv8n model, and the second feature map is obtained by the next convolutional layer of the current convolutional layer. For example, if the current convolutional layer is convolutional layer 1, the next convolutional layer of the current convolutional layer is convolutional layer 2; if the current convolutional layer is convolutional layer 2, the next convolutional layer of the current convolutional layer is convolutional layer 3, and so on. It should be clear that based on the YOLOv8n model and the first feature map, obtaining the second feature map is a mature technology, which will not be described in detail here.

[0093] Step S320: Based on the second feature map, obtain second saliency data of the small object.

[0094] It should be noted that, in this embodiment, the specific operation of step S320 may be the same as the operation of step S200, except that the input and output of step S320 are different from those of step S200.

[0095] Step S330: Taking the position of the small target as the center and based on the first feature map, obtain a first convolution window.

[0096] It should be clear that obtaining a convolution window in a feature map with a fixed center is a mature technology and will not be described in detail here.

[0097] Step S340: Taking the center of the first convolution window as the center and based on the second feature map, obtain a second convolution window.

[0098] It should be clear that obtaining a convolution window in a feature map with a fixed center is a mature technology and will not be described in detail here.

[0099] Step S350: Acquire small target matching degree data based on the first convolution window and the second convolution window.

[0100] It should be clear that the first convolution window is centered on the position of the small target, and the second convolution window is centered on the center of the first convolution window. In other words, we can use the signal strength at the center of the second convolution window as the signal strength of the small target in the second feature map. It is easy to understand that the signal strength at the position centered on the small target is strong, and the signal strength around the position is weak. If the feature information of the small target in the second feature map obtained based on the first feature map is not lost, the first convolution window and the second convolution window must have similar features, for example, the signal strength attenuation at the center of the first convolution window and the second convolution window is basically similar, or the position points of significant signal strength in the first convolution window and the second convolution window are similar. In other words, if the feature information of the small target in the second feature map obtained based on the first feature map has been lost, the features of the first convolution window and the second convolution window must be quite different.

[0101] In an embodiment of the present application, the small target matching degree data refers to the degree of difference between the first convolution window and the second convolution window. It is easy to understand that if the degree of difference between the first convolution window and the second convolution window is large, it can be considered that in the process of obtaining the second feature map based on the first feature map, the feature information of the small target is lost. In other words, if the degree of difference between the first convolution window and the second convolution window is small, it can be considered that in the process of obtaining the second feature map based on the first feature map, the feature information of the small target is not lost. If the feature information of the small target is lost, it is necessary to set a target recognition window for the small target to avoid the small target being lost during the convolution process; if the feature information of the small target is not lost, there is no need to set a target recognition window for the small target to avoid increasing the computational complexity of subsequent convolution operations.

[0102] In an embodiment of the present application, the small target matching degree can be obtained based on any difference between the first convolution window and the second convolution window to determine whether the small target is lost during the convolution process. For example, all target positions corresponding to the significant signal strengths of the first convolution window and the second convolution window are traversed, and the small target matching degree (that is, the difference between the first convolution window and the second convolution window) is obtained based on the number of target positions in the first convolution window and the second convolution window.

[0103] In order to improve the accuracy of the obtained small target matching degree, in one embodiment of the present application, step S350, based on the first convolution window and the second convolution window, obtains small target matching degree data, including steps S351 to S353.

[0104] Step S351: Based on the first convolution window, with the center of the first convolution window, search for a target position with a significant signal strength in a direction where the signal strength at the center of the first convolution window decreases the most, to form a first target position sequence.

[0105] As can be seen from the previous text, the signal strength of the position point corresponding to the small target is significant, and the signal strength of other position points around the position point is weak, so we can determine whether the feature information of the small target is lost by the attenuation of the signal strength of the position point corresponding to the small target. Compared with traversing all the target positions corresponding to the significant signal strength in the first convolution window, this method processes less data and has higher accuracy.

[0106] Step S352: Based on the second convolution window, search for a target position with significant signal strength from the center of the second convolution window in the direction where the signal strength at the center of the second convolution window decreases the most, to form a second target position sequence.

[0107] It should be clear that the operation of this step is the same as step S351, the only difference is that the input and output of the step are different.

[0108] Step S353: Based on the first target position sequence and the second target position sequence, small target matching degree data is obtained.

[0109] It should be clear that if the feature information of the small target is not lost, the number of elements in the first target position sequence and the second target position sequence is similar. In an embodiment of the present application, the small target matching degree data can be obtained in an arbitrary manner based on the first target position sequence and the second target position sequence. For example, in one embodiment of the present application, the small target matching degree data can be obtained based on the difference in the number of elements in the first target position sequence and the second target position sequence. When used, if the difference in the number of elements exceeds the first threshold, it is considered that the small target matching degree is poor (that is, the difference between the first convolution window and the second convolution window is large); if the difference in the number of elements is less than the first threshold, it is considered that the small target matching degree is good (that is, the difference between the first convolution window and the second convolution window is small).

[0110] It should be clear that the order of magnitude of the elements in the first convolution window and the second convolution window formed by different first feature maps and second feature maps may be inconsistent. In other words, it is difficult to accurately determine the difference between all the first convolution windows and the second convolution windows based on the difference in the number of elements in the first target position sequence and the second target position sequence, that is, it is impossible to accurately obtain the matching degree of small targets.

[0111] In order to obtain accurate small target matching degree, in one embodiment of the present application, in step S353, based on the first target position sequence and the second target position sequence, a calculation formula for obtaining small target matching degree data is as follows:

[0112] ;

[0113] Among them, C is the small target matching degree data, is the number of elements in the first target position sequence; is the number of elements in the second target position sequence.

[0114] It should be clear that the deeper convolution layer has a more obvious effect on the extraction of high-intensity signal positions, which will cause the signal strength difference between the high-intensity signal position and the surrounding signal position to further expand. Therefore, the concentration of high-intensity feature signals generated by the small target extracted by the convolution layer can be judged through the signal attenuation ratio formed between the two convolution layers. The degree of small target matching between the two convolution layers at the current significant signal position can be judged. If the small target matching degree data (that is, C) is larger, it means that the characteristic signal generated by the proximity of the significant signal position output by the second feature map is more obviously weakened, thereby more reflecting the characteristics of the small target compared to the large target through the convolution layer. The stronger the characteristic signal aggregation. In other words, it is necessary to add a target recognition window for the small target to avoid the loss of feature information of the small target.

[0115] Step S360: Based on the small target matching degree data, obtain a target recognition window of the small target.

[0116] In an embodiment of the present application, Yolov8n can be triggered to add a target recognition window for a small target in any way. For example, in one embodiment of the present application, if the small target matching degree data is greater than the second threshold, Yolov8n can be directly triggered to add a target recognition window. In order to accurately determine whether it is necessary to increase the target recognition window for a small target, in a specific embodiment of the present application, step S360, based on the small target matching degree data, obtains the target recognition window for the small target, including steps S361 to S365.

[0117] Step S361: Based on the second feature map, obtain the current recognition window of the small target.

[0118] It should be clear that the second feature map is retrieved through the selective search algorithm (Selective Search) to obtain K feature windows (i.e., the current recognition window), where K is a preset value, i.e., the number of windows that the algorithm needs to obtain. Since obtaining the current recognition window through the selective search algorithm based on the feature map is a mature technology, it will not be described here.

[0119] Step S362: Based on the current recognition window, a collection of target positions with all significant signal strengths in the current recognition window is obtained.

[0120] It should be clear that the fewer significant feature signal positions there are in the window, the more the selective search algorithm can determine that the feature signal distribution in the window is different from that outside the window, reflecting that the significant feature signal positions in the area within the window are more similar. However, there are still cases where the significant feature signal positions belong to different objects in the original image but the algorithm mistakenly identifies them as the same window. It is necessary to determine whether the window should change its size based on the distribution of different significant feature signal positions in the window to ensure that small targets can be accurately identified.

[0121] Step S363: Based on the target position collection, a small target matching degree data collection is obtained.

[0122] It should be clear that, as can be seen from the foregoing, each target position corresponds to a small target matching degree data, so a corresponding small target matching degree data collection can be obtained based on the target position collection.

[0123] Step S364: Based on the small target matching degree data set, obtain split evaluation data.

[0124] It should be clear that the split evaluation data is used to determine whether it is necessary to add a target recognition window for small targets.

[0125] In a specific embodiment of the present application, in step S364, based on the small target matching degree data set, the calculation formula for obtaining the split evaluation data is as follows:

[0126] ;

[0127] in, is the number of data in the small target matching degree data set; is the number of data in the small target matching degree data set after excluding the small target matching degree data of the j-th target position; is the matching degree of the small target corresponding to the j-th target position; is the matching degree of the small target corresponding to the h-th target position; It is the mean of the small target matching degree in the small target matching degree data collection.

[0128] It should be clear that As the difference in the matching of small targets between a single significant feature signal position in the current window and other positions, the larger the value is, the more feature positions with different signal strengths appear in the current window. This shows that there are more inconsistent feature positions in the current window, which is even more inconsistent with the situation that after the small targets are extracted through the convolution window, the feature signals formed by the area with a smaller area ratio in the original image are concentrated and similar. In other words, it is necessary to add a target recognition window to the small targets in the current window to avoid the loss of feature information of subsequent small targets.

[0129] Step S365: Based on the split evaluation data, split the current recognition window of the small target to obtain the target recognition window of the small target.

[0130] In the embodiment of the present application, the split evaluation data of the current window is obtained After that, the split evaluation data can be normalized by the sigmoid layer. Normalize to the interval [0,1] to obtain the normalized split evaluation , and take the normalized split evaluation The window of the third threshold is split. In other embodiments of the present application, the third threshold may be 0.6, or any other suitable data. In this embodiment, the window is split for different normalized split evaluations to obtain a real target recognition window for a small target, and the window is split to avoid a decrease in the accuracy of small target recognition caused by different small targets being recognized by the same window.

[0131] In an embodiment of the present application, the current recognition window can be split in any manner to obtain the target recognition window of the small target. For example, the target recognition window of the small target can be obtained at a fixed distance with the small target as the center. In a specific embodiment of the present application, step S365, based on the split evaluation data, splits the current recognition window of the small target to obtain the target recognition window of the small target, including steps S370 to S375.

[0132] Step S370: Based on the position of the small target, a first distance is acquired, where the first distance is the shortest distance between the small target and the edge of the current recognition window.

[0133] It should be clear that if the position of the small target is determined and the current recognition window is determined, the first distance can be acquired based on the position of the small target and the current recognition window.

[0134] Step S371: based on the position of the small target, obtaining a target line; the target line is a line connecting the position of the small target and other target positions in the current recognition window.

[0135] It should be clear that if two position points are determined, then the line between the two position points can also be determined.

[0136] Step S372: Based on the target line, obtain the target midpoint.

[0137] It should be clear that if the target line is determined, the midpoint of the target line can be determined.

[0138] Step S373: Acquire a second distance based on the positions of the target midpoint and the small target.

[0139] It should be clear that if two points are determined, the distance between the two points (ie, the second distance) can be determined.

[0140] Step S374: Based on the first distance and the second distance, obtain a mean distance.

[0141] In an embodiment of the present application, for the significant feature position (i.e., small target) existing in the current identification window, the point closest to the window edge is determined, and then the midpoint of the line connecting the current significant feature position and other positions in the window is obtained, and the average of the shortest distance from the current position to the window edge and the distance to the midpoint of the line connecting each position is taken as the signal strength coverage distance of the current position.

[0142] Step S375: Based on the position of the small target and the mean distance, obtain a target recognition window of the small target.

[0143] In an embodiment of the present application, the intersection of the area covered by one rotation of the corresponding signal strength coverage distance and the area of ​​the current window area is taken with the current significant feature position (i.e., the small target) as the center, and the minimum circumscribed rectangle of the intersection is taken as the target recognition window for the small target.

[0144] Step S400: Based on the target recognition window of the small target, a detection result corresponding to the small target is obtained.

[0145] In an embodiment of the present application, all significant feature positions in the current recognition window are traversed to obtain target recognition windows for each significant feature position. After the traversal is completed, the current recognition window is discarded and no longer used. After the current recognition window of the second feature map is split again, the signal map with the window mark is input into the SPP layer. The pooling window group used by the SPP layer in this embodiment is (4x4, 2x2, 1x1). For a single window, a 1x21 feature vector can be detected, and the feature vector is input into the subsequent FC layer (fully connected layer), and then the small target detection output result of the current original image is output by the FC layer.

[0146] Step S500: Visually output the detection result.

[0147] The recognition result of the current frame output by YOLOv8n needs to be visualized. Visual display requires the use of a computer and a display, where the computer: a computing device used to run the YOLOv8n model and image processing, needs to be equipped with a high-performance GPU (graphics processor). Display: a display device used to display the processed image and recognition results.

[0148] The embodiments of the present application are directed to the technical problem that the existing Yolov8n model used for small target recognition has poor small target position recognition effect in the process of small target detection only through changes in network hierarchy and feature recognition windows of different preset sizes. The embodiments of the present application extract the significance of the local characteristic signal strength of the signal formed by the small target in the single-layer feature extraction in the feature map, comprehensively analyzes the distribution of the characteristic signal strength in the small target area as the convolution layer goes deeper, and re-splits the window according to the distribution of high-value small targets in the window, thereby increasing the accuracy of small target recognition.

[0149] After introducing the embodiment of the improved YOLOv8n small target detection method based on feature fusion proposed in the present application, a computer-readable storage medium proposed in the present application is introduced below. A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, the improved YOLOv8n small target detection method based on feature fusion as described in any of the above embodiments is implemented.

[0150] It should be clear that the computer-readable storage media in this application include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0151] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the methods, devices and equipment described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0153] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0154] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0156] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0157] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0158] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. An improved YOLOv8n small target detection method based on feature fusion, characterized in that: The method comprises: Based on the original image, obtaining a first feature map for target detection; the original image is obtained in advance; Based on the first feature map, obtaining first saliency data of the small object; Based on the first saliency data of the small target, obtaining a target recognition window of the small target; Based on the target recognition window of the small target, obtaining a detection result corresponding to the small target; Visually output the detection result; The acquiring first saliency data of the small object based on the first feature map includes: Based on the first feature map, obtaining the signal strength of the small target; Based on the first feature map, obtaining position information data of the small target; Based on the signal strength and the position information data, acquiring first saliency data of the small target; The calculation formula for obtaining the first saliency data of the small target based on the signal strength and the position information data is as follows: ; in, is the first saliency data of the target at position i; The signal strength of the target at position i; is the mean value of the significant signal strength of all position targets in the first feature map; is the standard deviation of the significant signal strength of all position targets in the first feature map; is the number of locations of targets with significant signal strength that are connected to the target at location i; is the position interval of the farthest pair of targets among all the significant signal strength positions in the first feature map; is the distance between the target at position i and its nearest significant signal strength position; The step of obtaining a target recognition window for the small target based on the first saliency data of the small target includes: Based on the YOLOv8n model and the first feature map, obtain a second feature map; Based on the second feature map, obtaining second saliency data of the small object; Taking the position of the small target as the center and based on the first feature map, obtaining a first convolution window; Taking the center of the first convolution window as the center and based on the second feature map, obtaining a second convolution window; Based on the first convolution window and the second convolution window, obtaining small target matching degree data; Based on the small target matching degree data, obtaining a target recognition window of the small target; The step of obtaining a target recognition window for the small target based on the small target matching degree data includes: Based on the second feature map, obtaining a current recognition window of the small target; Based on the current identification window, obtaining a collection of target positions of all significant signal strengths in the current identification window; Based on the target position collection, obtaining a small target matching degree data collection; Based on the small target matching degree data set, obtaining split evaluation data; Based on the split evaluation data, the current recognition window of the small target is split to obtain the target recognition window of the small target.

2. The improved YOLOv8n small target detection method based on feature fusion according to claim 1, characterized in that: The step of acquiring a first feature map for target detection based on the original image includes: Based on a collection device, video data is acquired; the collection device is pre-installed; Based on the video data, obtaining an original image; The original image is input into the YOLOv8n model, and based on the network convolution layer of the YOLOv8n model, a first feature map of target detection is obtained.

3. The improved YOLOv8n small target detection method based on feature fusion according to claim 1, characterized in that: The acquiring small target matching degree data based on the first convolution window and the second convolution window includes: Based on the first convolution window, searching for a target position with significant signal strength from the center of the first convolution window in a direction where the signal strength at the center of the first convolution window decreases the most, to form a first target position sequence; Based on the second convolution window, searching for a target position with a significant signal strength from the center of the second convolution window in a direction where the signal strength at the center of the second convolution window decreases the most, to form a second target position sequence; Based on the first target position sequence and the second target position sequence, small target matching degree data is acquired.

4. The improved YOLOv8n small target detection method based on feature fusion according to claim 3, characterized in that: The calculation formula for obtaining the small target matching degree data based on the first target position sequence and the second target position sequence is as follows: ; Among them, C is the small target matching degree data, is the number of elements in the first target position sequence; is the number of elements in the second target position sequence.

5. The improved YOLOv8n small target detection method based on feature fusion according to claim 1, characterized in that: The calculation formula for obtaining the split evaluation data based on the small target matching degree data set is as follows: ; in, is the number of data in the small target matching degree data set; is the number of data in the small target matching degree data set after excluding the small target matching degree data of the j-th target position; is the matching degree of the small target corresponding to the j-th target position; is the matching degree of the small target corresponding to the h-th target position; It is the mean of the small target matching degree in the small target matching degree data collection.

6. The improved YOLOv8n small target detection method based on feature fusion according to claim 1, characterized in that: The step of splitting the current recognition window of the small target based on the split evaluation data to obtain the target recognition window of the small target includes: Based on the position of the small target, a first distance is acquired; the first distance is the shortest distance between the small target and the edge of the current recognition window; Based on the position of the small target, a target line is obtained; the target line is a line connecting the position of the small target and other target positions in the current recognition window; Based on the target line, obtaining the target midpoint; Based on the positions of the target midpoint and the small target, obtaining a second distance; Based on the first distance and the second distance, obtaining a mean distance; Based on the position of the small target and the mean distance, a target recognition window of the small target is obtained.

Citation Information

Patent Citations

  • Image main target detection method and device based on convolutional neural network

    CN110188766A