A small target detection method and system based on context information and global attention, a storage medium and a terminal

Abstract

The application discloses a small target detection method and system based on context information and global attention, a storage medium and a terminal, and relates to the technical field of image processing.The method comprises the following steps: inputting an image and extracting image features by combining a backbone extraction network with context information to obtain feature maps of different depths; combining a global attention mechanism with the context information to perform feature fusion on the feature maps of different depths to obtain fused feature maps; and sending the fused feature maps to a detection head for detection to obtain a target detection frame.The application can effectively analyze small targets with missing information through context information, balance information of small targets in feature maps of different depths, improve the detection rate of small targets, and construct a more accurate cost matrix to complete a matching task of target tracking.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method, system, storage medium, and terminal for small target detection based on contextual information and global attention. Background Technology

[0002] With the gradual maturation of target detection and tracking technology, target detection and tracking algorithms have played an indispensable role in image and video perception and analysis tasks on various terminal devices such as satellites and drones. Unlike general scenarios, small targets typically refer to targets smaller than 32*32 pixels in an image. Due to the large background image and small target size, small target detection and tracking technology can more comprehensively analyze the behavior and trajectory of targets in videos and images, and can even meet the requirements for perceiving the aggregation and diffusion of small targets. The advantage of small target detection and tracking technology lies in its wide applicability and scalability, providing an accurate and efficient foundation for target data analysis in most complex scenarios, making it more feasible for devices to perform complex behavior analysis tasks on small targets.

[0003] Since the target of the small target detection and tracking task occupies fewer pixels in the image and lacks positional diversity, common target detection and tracking algorithms also have many defects: (1) It is difficult to balance the input image resolution and the computational load of the algorithm. If the input resolution is too small, the downsampling ratio of the original image will be too large, causing the feature information of the target to disappear in the feature map of the deeper layer, and the feature of the small target cannot be captured; if the input resolution is too large, the convolution operation will increase quadratically, which is difficult to meet the real-time requirements; (2) The receptive field of view lacks universality. Small targets usually have large size, shape and displacement changes during the movement. Therefore, a single receptive field of view is usually difficult to adapt to small targets with highly dynamic change characteristics; (3) For the information extraction strategy of small targets in complex scenes, when there are occluded and densely arranged small targets, the target itself can no longer provide enough feature information, but no strategy similar to combining the context is adopted for analysis, resulting in a serious drop in accuracy in the scene of occlusion and dense small targets. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing small target detection technologies and to provide a small target detection method, system, storage medium, and terminal based on contextual information and global attention.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] In a first aspect, a small target detection method based on contextual information and global attention is provided, the method comprising the following steps:

[0007] S1. Input the image and extract image features by combining the backbone extraction network with contextual information to obtain feature maps of different depths;

[0008] S2. Combine global attention mechanism and context information to fuse feature maps of different depths to obtain fused feature maps;

[0009] S3. The fused feature map is fed into the detection head for detection to obtain the target detection box.

[0010] As a preferred option, a small target detection method based on contextual information and global attention, wherein the image feature extraction is performed by combining a backbone extraction network with contextual information, including:

[0011] We employ a network structure combining CNN and Swin Transformer for image feature extraction, and introduce a residual module to preserve and combine contextual information and original feature information.

[0012] As a preferred option, a small target detection method based on contextual information and global attention, wherein the method combines global attention mechanism and contextual information to perform feature fusion on feature maps of different depths, including:

[0013] S21. Combine upsampling, global attention mechanism and stitching to perform upward feature fusion on feature maps of adjacent depths;

[0014] S22. Downward feature fusion based on contextual information and global attention mechanism.

[0015] As a preferred option, a small target detection method based on contextual information and global attention is provided, wherein the global attention mechanism includes a channel attention module and a spatial attention module.

[0016] As a preferred option, a small target detection method based on contextual information and global attention, wherein the fused feature map is fed into a detection head for detection, includes:

[0017] The feature map from each detection head is moved up one level, allowing target prediction to be performed from a higher resolution feature map.

[0018] As a preferred option, a small target detection method based on contextual information and global attention further includes the following steps:

[0019] S4. The target detection box is tracked using the deepsort algorithm.

[0020] As a preferred embodiment, a small target detection method based on contextual information and global attention, wherein step S4 includes the following steps:

[0021] S41. Enlarge the center of the target detection box by 15% to 20%;

[0022] S42. Describe the location information;

[0023] S43. Fuzzy motion prediction of the target.

[0024] In a second aspect, a small target detection system based on contextual information and global attention is provided, the system comprising:

[0025] The feature map extraction module is configured to receive the input image and extract image features by combining the backbone extraction network with contextual information to obtain feature maps of different depths.

[0026] The feature fusion module is configured to combine global attention mechanism and context information to fuse feature maps of different depths to obtain a fused feature map.

[0027] The detection head is configured to detect the fused feature map to obtain the target detection box.

[0028] In a third aspect, a computer storage medium is provided, on which computer instructions are stored, wherein the computer instructions, when executed, perform any of the relevant steps in the small target detection method based on context information and global attention described above.

[0029] In a fourth aspect, a terminal is provided, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the relevant steps in any one of the small target detection methods based on context information and global attention when executing the computer instructions.

[0030] It should be further noted that the technical features corresponding to the above options can be combined or substituted to form new technical solutions if there is no conflict.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] (1) This invention extracts image features by combining the backbone extraction network with contextual information. It can effectively analyze the missing information of small targets through contextual information, balance the information of small targets in each depth feature map, increase the localization information of small targets, reduce the loss of target feature information in the transmission process, and improve the detection rate of small targets.

[0033] (2) In one example, the network structures of CNN and Swing Transformer are combined to extract features from the image. By introducing a residual module, the context information and the original feature information are preserved and combined. The receptive field can be expanded well by dividing the window. Furthermore, the multi-head self-attention mechanism based on window and sliding window does not increase the amount of computation too much.

[0034] (3) In one example, a global attention mechanism is introduced to balance the feature information of feature maps at different depths. At the same time, a context information extraction module is added before the global attention mechanism module to increase the model's ability to perceive small targets.

[0035] (4) In one example, a strategy of shallowing the detection head is proposed, which advances the feature map from each detection head by one level and predicts the target from the higher resolution feature map, which can be highly targeted to small targets.

[0036] (5) In one example, a small target tracking algorithm based on multi-dimensional information is proposed. By using appearance diffusion strategy, dilation distance index and fuzzy motion prediction strategy, the similarity of small targets between frames is measured from three dimensions: appearance, location and motion, respectively, so as to construct a more accurate cost matrix to complete the target tracking matching task. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating a small target detection method based on contextual information and global attention, as shown in an embodiment of the present invention.

[0038] Figure 2 This is a model structure diagram illustrating an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the context information extraction module shown in an embodiment of the present invention;

[0040] Figure 4 This is a flowchart illustrating the global attention module in an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram illustrating the principle of expansion distance in an embodiment of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0044] Example 1

[0045] Reference Figure 1 In one exemplary embodiment, a small target detection method based on contextual information and global attention is provided, the method comprising the following steps:

[0046] S1. Input the image and extract image features by combining the backbone extraction network with contextual information to obtain feature maps of different depths;

[0047] S2. Combine global attention mechanism and context information to fuse feature maps of different depths to obtain fused feature maps;

[0048] S3. The fused feature map is fed into the detection head for detection to obtain the target detection box.

[0049] Generally, object detection algorithms are divided into two main categories: two-stage detection and one-stage detection. Due to the existence of candidate box prediction algorithms, two-stage detection algorithms are usually relatively inefficient. Therefore, we consider one-stage detection algorithms. One-stage detection algorithms are further divided into anchor-based and anchor-free algorithms based on the selection of candidate boxes. The former is represented by the YOLO series, while the latter is represented by CornerNet, CenterNet, etc. Considering the problem of limited datasets in small object detection tasks, we need a series of data augmentation strategies to increase the robustness of the model and adapt it to targets of different sizes. Therefore, we choose YOLOv5 as the basic framework of the entire algorithm.

[0050] This invention extracts image features by combining a backbone extraction network with contextual information. It can effectively analyze the missing information of small targets through contextual information, balance the information of small targets in various depth feature maps, increase the localization information of small targets, reduce the loss of target feature information during transmission, and improve the detection rate of small targets.

[0051] Preferably, the step of extracting image features by combining the backbone extraction network with contextual information includes:

[0052] This paper employs a network structure combining CNN and Swin Transformer for image feature extraction, and introduces a residual module to preserve and combine contextual and original feature information. Specifically, among numerous object detection algorithms, convolutional neural networks (CNNs) have proven their effectiveness in downstream computer vision tasks as the backbone extraction network for most algorithms. However, because single-pass convolution cannot acquire domain information within a wide receptive field, CNNs struggle to meet the requirements for contextual information extraction. Swin Transformer, as a network structure based on a global self-attention mechanism, can effectively expand the receptive field by dividing the window, and the multi-head self-attention mechanism based on windows and sliding windows does not significantly increase computation. Therefore, we consider using a network structure combining CNN and Swin Transformer for image feature extraction, and introduce a residual module to preserve and combine contextual and original feature information.

[0053] Preferably, the feature fusion of feature maps of different depths by combining global attention mechanism and contextual information includes:

[0054] S21. Combine upsampling, global attention mechanism and stitching to perform upward feature fusion on feature maps of adjacent depths;

[0055] S22. Downward feature fusion based on contextual information and global attention mechanism.

[0056] Specifically, in target detection tasks, since the motion state of the target cannot be known in advance, especially the size and shape changes of small targets during motion, a feature fusion module is needed to detect targets on feature maps of different depths in order to adapt to targets with motion change characteristics. Furthermore, in the backbone extraction network, target information is usually distributed across feature maps of different depths. Shallow feature maps often contain more localization information, while deeper feature maps contain more target feature information. Different information weights should be assigned to targets of different sizes at different feature fusion stages. Therefore, we introduce a global attention mechanism (GAM) to guide the feature fusion module in balancing feature map information across different depths. This global attention mechanism includes a channel attention module and a spatial attention module. Simultaneously, we note that the context information extraction module also uses residual structures to fuse context information and original feature information. Therefore, a context information extraction module is added before the global attention mechanism module to enhance the model's ability to perceive small targets. Thus, the entire fusion module is divided into an upward-merging feature fusion module and a downward-merging feature information fusion module based on the attention mechanism and context information.

[0057] Preferably, the step of feeding the fused feature map into the detection head for detection includes:

[0058] We advance the feature maps from each detection head by one level, allowing target prediction from higher-resolution feature maps. Specifically, after the feature fusion module, the feature maps already retain more contextual information and contain feature maps of multiple depths. To enable the model to be highly targeted to small targets, we propose a strategy of shallowing the detection heads, advancing the feature maps from each detection head by one level, and performing target prediction from higher-resolution feature maps to better adapt to the morphological changes of small targets.

[0059] Example 2

[0060] Based on Embodiment 1, and with specific examples, the small target detection method of this application is described below, with reference to... Figure 2 The entire algorithm model structure consists of a backbone network, a feature fusion network, and a detection head. The specific algorithm flow includes the following steps:

[0061] (1) First, in order to balance the resolution size and the computational cost of the algorithm, we set the input resolution of the image to 1280*1280. The backbone extraction network uses Darknet53 to extract feature maps of different depths (assumed to be C2, C3, C4, C5). Considering that the context needs to have sufficient semantic information, we added a context extraction module after the 16x downsampled feature map C4, and added a feature fusion path at the 4x downsampled feature map C2. This increases the localization information of small targets while reducing the loss of target feature information during the transmission process. Then, we start to perform feature fusion upwards.

[0062] (2) When fusing features upwards, we choose the method of "upsampling + global attention mechanism + concatenation" to fuse feature maps of adjacent depths, assuming P i This indicates that the downsampling factor is 2. i The upward fusion feature maps are calculated using the following formula:

[0063]

[0064] After 32x downsampling, an SPPF layer was added to normalize the resolution of the feature map. Then, feature fusion was performed by concatenating adjacent feature map channels.

[0065] (3) During the downward feature fusion based on contextual information and global attention mechanism, since the detection head depth is advanced, we hope that the shallow feature map has sufficient small target localization information. Therefore, the C2 feature map is added to P2 to supplement the information through the 4x downsampling fusion path in step (1). Assume Ni This indicates that the downsampling factor is 2. i The feature maps are then fused downwards. Next, feature fusion based on contextual information and a global attention mechanism is performed, using the following formula:

[0066]

[0067] Finally, the feature maps (N2, N3, N4) are obtained and sent to the detection head for small target detection.

[0068] Furthermore, Figure 3 This is a simplified diagram of the context information extraction module. The entire context information extraction module is divided into two sub-modules: window-based multi-head self-attention and sliding window-based multi-head self-attention. Apart from the window division, the two sub-modules are structurally identical, and their workflow is mainly as follows:

[0069] S1: After obtaining the feature map, the window-based multi-head attention module first puts it into the NormalizationLayer for normalization.

[0070] S2: Divide the obtained feature map into windows with a side length of M to obtain N×N windows (i.e., the width of the feature map is W=MN). Then, when performing self-attention calculation, the receptive field of each window is only within its own single window, and the windows do not interfere with each other.

[0071] S3: Perform a residual calculation between the calculated feature map and the original feature map, and then put the feature map into the Normalization Layer again for normalization.

[0072] S4: The normalized result is fed into an MLP with a compression ratio of r for calculation.

[0073] S5: Complete the operation from S1 to S4 again, but in step S2 of this round, the window division needs to be adjusted. Using a mask, the window division of the entire image is shifted to the lower right corner by M / 2 pixels. Then, the self-attention within each window is calculated again after the sliding.

[0074] Furthermore, Figure 4 This is a flowchart of the Global Attention Module (GAM). The Global Attention Module is mainly divided into two parts: the channel attention module and the spatial attention module. Its internal flow is as follows:

[0075] S1: First, GAM's channel attention module transforms the dimensions of the input feature map, using a 3D arrangement to preserve information across the three dimensions. Next, it uses a two-layer MLP with a compression ratio of r. Finally, the dimensions are restored and output via a sigmoid function. The purpose of the channel attention module is to learn the weights of different channels, rearrange the 3D information of the image, and then use an MLP to amplify the cross-dimensional channel space, performing multiple partitions of the different channels of the feature map.

[0076] S2: In the spatial attention module, to focus on more spatial information, GAM uses two 7×7 convolutional layers for spatial information fusion. The compression ratio r used in the two convolutional layers is the same as that in the channel attention module. The spatial attention module mainly focuses on the position of the target in the image, using convolution to fuse spatial information. By weighting the spatial features, the features of each space are expanded, making the channel more attentive to spatial information.

[0077] As can be seen, the size and dimensions of the feature map do not change after completing one GAM, whether it is the channel attention module or the spatial attention module. Therefore, it will not affect the subsequent process of the original structure at all and can be easily inserted into any module.

[0078] Example 3

[0079] Based on Example 1, a small target detection and tracking method based on contextual information and global attention is provided, including the following steps:

[0080] S4. The target detection box is tracked using the deepsort algorithm.

[0081] Specifically, the tracking module uses a deepsort algorithm cascaded matching framework. An optimization strategy is employed to improve the accuracy of the cost matrix during construction. Then, for trackers that fail to match, fuzzy motion prediction is used to simulate their motion, optimizing the ID switching problem under occlusion conditions. Addressing the challenges of large displacement of small targets and fuzzy appearance and positioning information, a multi-dimensional information-integrated tracking and matching algorithm is proposed. Its internal process is as follows:

[0082] S41: After obtaining the target detection bounding box provided by the target detection module, the detection bounding box is enlarged by 15% to 20% (a custom parameter that can be set as needed). This enlargement allows the bounding box to include more information about adjacent regions, i.e., contextual information, thus aiding in the description of the target's features. It's important to note that during tracker updates, the original size detection bounding box is still used; the enlarged bounding box is only used in the appearance feature descriptor extraction network (RE-ID). This vector is used to measure the cosine distance between different targets.

[0083] S42: Describing location information, this invention proposes an index for describing the location similarity between distant targets, called dilated distance, the principle of which is as follows: Figure 5 As shown, because traditional IOU relies heavily on the intersection size between bounding boxes, it is difficult for IOU to describe the localization differences between small targets that may be far apart between frames. Distance dilation uses two bounding boxes R... i R j The area of ​​the rectangle formed by connecting the centers, when summed with their respective intersections and divided by the area of ​​the connection matrix, shows that even if the target distance is very far, the expansion distance ED... ij It will not be 0. Furthermore, to include the target's size in the measurement of location similarity differences, the dilation distance also adds an aspect ratio similarity factor, calculated as follows:

[0084]

[0085]

[0086]

[0087] After simplification, it can be seen that it is similar to the checkmark function. The similarity factor can only reach its maximum value of 1 when the width and height ratios of the two rectangles are both 1. By combining the dilated distance with IOU, the localization similarity between targets can be better measured, and the cost matrix can be constructed.

[0088] S43: Fuzzy motion prediction of the target. After Hungarian matching is completed, some trackers may fail to match the detection box. If the target is not updated at this time, it may reappear due to occlusion and fail to match successfully because the displacement is too large. The motion fuzzy prediction proposed in this invention can improve this situation. Its principle is as follows:

[0089]

[0090] Using a decreasing scaling factor p i (Initial value 0.9) Predict the motion of the target. Even if the target tracker fails to match, the target position is still updated using a linear model. The update method is that the tracked predicted box moves along the center line by a scaling factor. Over time, the scaling factor will decrease due to the decay factor γ. i The scaling factor becomes smaller and smaller to simulate the target's trajectory under occlusion. In the extreme case, the scaling factor will approach 1 / 3, indicating that the target is still moving linearly at 1 / 3 speed. When the target is successfully matched, the scaling factor and the attenuation factor will be reset to their initial state. When the target's life cycle ends, the blurred motion will also stop.

[0091] Example 4

[0092] In another exemplary embodiment, a small target detection system based on contextual information and global attention is provided, the system comprising:

[0093] The feature map extraction module is configured to receive the input image and extract image features by combining contextual information with the backbone extraction network to obtain feature maps of different depths. The backbone extraction network uses the Swing Transformer, which is a modification of the Vision Transformer using shifted windows (SW). It divides the fixed-size sampling blocks of the Vision Transformer into blocks of different sizes according to hierarchy. The information between each block is not shared, and the computation is independent, greatly improving computational efficiency. The architecture of the Swing Transformer is very similar to that of the Vision Transformer, the difference being that its Transformer Block consists of two consecutive Swing Transformer Blocks. As a network structure based on a global self-attention mechanism, the Swing Transformer can effectively expand the receptive field by dividing the window, and the multi-head self-attention mechanism based on windows and shifted windows does not increase the computational load too much. Therefore, we consider using a network structure combining CNN and Swing Transformer for image feature extraction, and introduce a residual module to preserve and combine contextual information and original feature information.

[0094] The feature fusion module is configured to combine global attention and contextual information to fuse feature maps of different depths, resulting in a fused feature map. Specifically, in object detection tasks, since the motion state of the target cannot be known in advance, especially the size and shape changes of small targets during motion, a feature fusion module is needed to detect targets with motion-changing characteristics. Furthermore, in the backbone extraction network, target information is typically distributed across feature maps of different depths. Shallow feature maps often contain more localization information, while deeper feature maps contain more target feature information. Different information weights should be assigned to targets of different sizes at different feature fusion stages. Therefore, we introduce a global attention mechanism (GAM) to guide the feature fusion module in balancing feature map information across different depths. Simultaneously, we note that the contextual information extraction module also uses residual structures to fuse contextual information and original feature information. Therefore, a contextual information extraction module is added before the global attention mechanism module to enhance the model's ability to perceive small targets. Thus, the entire fusion module is divided into an upward-merging feature fusion module and a downward-merging feature fusion module based on attention and contextual information.

[0095] The detection head is configured to detect targets from the fused feature maps. Specifically, after the feature fusion module, the feature maps retain a significant amount of contextual information and contain feature maps of multiple depths. To enable the model to be highly targeted at small targets, we propose a strategy of shallowing the detection head. This involves moving the feature maps from each detection head one level forward, predicting targets from higher-resolution feature maps to better adapt to the morphological changes of small targets.

[0096] Example 5

[0097] In another exemplary embodiment, the present invention provides a computer storage medium storing computer instructions that, when executed, perform relevant steps in the small target detection method based on context information and global attention.

[0098] Based on this understanding, the technical solution of this embodiment, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] Example 6

[0100] In another exemplary embodiment, the present invention provides a terminal including a memory and a processor, wherein the memory stores computer instructions executable on the processor, and the processor executes relevant steps in the small target detection method based on context information and global attention when executing the computer instructions.

[0101] The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0102] The embodiments of the subject matter and functional operation described in this specification can be implemented in: tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing device or for controlling the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing device.

[0103] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0104] Suitable processors for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0105] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0106] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0107] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. A small object detection method based on context information and global attention, characterized in that, The method comprises the following steps: S1, inputting an image and extracting image features by combining a backbone extraction network with context information to obtain feature maps at different depths; the step of extracting image features by combining the backbone extraction network with the context information comprises: The network structure combining CNN and Swin Transformer is adopted for image feature extraction, and a residual module is introduced for reservation and combination of context information and original feature information; the specific process is as follows: After obtaining the feature map, the window-based multi-head self-attention first performs a normalization operation; The obtained feature map is divided into N×N windows with a window size of M, and then the receptive field of each window is only within the single window during self-attention calculation; The calculated feature map is subjected to a residual calculation with the original feature map, and then the feature map is normalized again; The normalized result is put into an MLP with a compression ratio of r for calculation; The above operation is completed again, and in this round, the window division is adjusted, the window division of the entire image is translated to the lower right corner by M / 2 pixels using a mask, and the self-attention in each window after sliding is calculated again; S2, combining a global attention mechanism with context information to perform feature fusion on feature maps at different depths to obtain a fused feature map; S3, feeding the fused feature map into a detection head for detection to obtain a target detection frame; S4, tracking the target detection frame using a deepsort algorithm, measuring the similarity of inter-frame small targets from the aspects of appearance, positioning and motion through an appearance diffusion strategy, an inflation distance index and a fuzzy motion prediction strategy; the appearance diffusion strategy comprises: After obtaining the target detection frame provided by the target detection module, the center of the detection frame is enlarged by a certain proportion; The inflation distance index is used to describe the positioning similarity between distant targets, and by combining the inflation distance with a positioning loss function, the positioning similarity between targets can be better measured to complete the construction of a cost matrix; The fuzzy motion prediction strategy comprises: A decreasing proportion factor is used to predict the motion of the target, a linear model is used to update the position of the target, and the update mode is that the predicted frame of the tracking moves on the center line by the proportion factor, and as time goes on, the proportion becomes smaller and smaller due to the decay factor, thereby simulating the motion trajectory of the target in the occlusion situation.

2. The small object detection method based on context information and global attention according to claim 1, characterized in that, The step of combining the global attention mechanism with the context information to perform feature fusion on feature maps at different depths comprises: S21, performing upward feature fusion on feature maps at adjacent depths by combining upsampling, a global attention mechanism and splicing; S22, performing downward feature fusion based on context information and a global attention mechanism.

3. The small object detection method based on context information and global attention according to claim 1, characterized in that, The global attention mechanism comprises a channel attention module and a spatial attention module.

4. The small object detection method based on context information and global attention according to claim 1, characterized in that, The step of feeding the fused feature map into a detection head for detection comprises: The feature map from each detection head is extracted one level in advance, and the target is predicted from the feature map with higher resolution.

5. The small object detection method based on context information and global attention according to claim 1, characterized in that, The step S4 comprises the following steps: S41, center magnifying the target detection frame by using a ratio of 15% to 20%; S42, describing the positioning information; S43, fuzzy motion prediction of the target.

6. A small target detection system based on context information and global attention, used for the small target detection method based on context information and global attention in any one of claims 1-5, characterized in that, The system comprises: a feature map extraction module configured to receive an input image and extract image features by a backbone extraction network combined with context information to obtain feature maps of different depths; a feature fusion module configured to combine a global attention mechanism and context information to perform feature fusion on the feature maps of different depths to obtain fused feature maps; a detection head configured to detect the fused feature maps to obtain a target detection frame.

7. A computer storage medium having stored thereon computer instructions, wherein the computer instructions, when executed by a computer, cause the computer to perform the method of any one of claims 1-6. The computer instructions perform the steps of the small target detection method based on context information and global attention in any one of claims 1-5 when running.

8. A terminal comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein, The processor performs the steps of the small target detection method based on context information and global attention in any one of claims 1-5 when running computer instructions.

Images