Small target detection method, device, equipment and storage medium based on feature fusion

Abstract

The present invention discloses a method, apparatus, device and storage medium for small target detection based on feature fusion, wherein the method comprises: performing size transformation on a low-dimensional input feature map acquired in advance to obtain a high-dimensional feature map; fusing the high-dimensional semantic information of the high-dimensional feature map to the spatial domain according to the weight sharing of the channel in the channel domain, and re-sampling to obtain a resampled feature map; fusing the input feature map and the resampled feature map to obtain a fused feature map; inputting the fused feature map into a pre-trained target detection model to obtain a small target detection result. The present invention fuses the low-dimensional input feature map with the high-dimensional feature map obtained by size transformation and resampling by means of size transformation and resampling, which helps to retain the spatial information of the small target object, thereby improving the accuracy of the target detection model in detecting and identifying the small target object, and the processing parameters are small, the amount of calculation is small, and the processing efficiency is higher.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a small target detection method, device, equipment and storage medium based on feature fusion. Background Art

[0002] Object detection is a fundamental problem in computer vision. Its primary goal is to determine whether an object instance of a given class (such as a person, car, bicycle, dog, or cat) exists in an image. If so, the spatial location and extent of each object instance are returned. Object detection can mimic human vision in its ability to judge the external environment and is widely used in pedestrian detection, face detection, and smart transportation. With the rapid advancement of deep learning in recent years, object detection has been combined with it to achieve improved detection accuracy, faster efficiency, and a wider range of applications, such as autonomous driving and human-computer interaction. However, small objects are common in image scenes and are difficult to detect due to their low resolution, limited information content, and high noise. Current object detection algorithms only achieve half the accuracy of large object detection. Therefore, to improve the universality and multi-scenario applicability of object detection algorithms, an efficient small object detection method is urgently needed.

[0003] Currently, to address the challenges of small object detection, such as difficulty in detection and missed detection, deconvolution can be used to fuse high-level feature maps with low-level feature maps, enriching the semantic information of the previous layers and ultimately generating a single high-level feature map with high resolution and semantic information to detect small objects in images. Alternatively, the YOLOv5 algorithm can be employed, with a coordinated attention mechanism added to the YOLOv5 backbone extraction network, allowing the mobile network to obtain information from a larger area without introducing large overhead. Varifocal Loss is used to train densely packed, mutually occluded small objects, enabling the network model to accurately identify large clusters of overlapping objects. However, in both of these approaches, the large number of parameters required for deconvolution or transposed convolution affects overall training efficiency. Furthermore, transposed convolution has lower feature extraction capabilities than ordinary convolution, impacting overall detection accuracy. While incorporating an attention mechanism into the backbone network improves the network's ability to summarize semantic information, the channel-wise attention calculation requires global average pooling of the feature maps, which obscures the spatial information of small objects and hinders the final detection performance. Summary of the Invention

[0004] In view of this, the present application provides a video optimization method, apparatus, terminal device and storage medium to solve the problems of low detection accuracy and low efficiency of existing small target detection models.

[0005] To solve the above technical problems, a technical solution adopted in this application is: to provide a small target detection method based on feature fusion, including: resizing the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map; fusing the high-dimensional semantic information of the high-dimensional feature map to the spatial domain according to the channel weight sharing in the channel domain, and re-sampling to obtain a resampled feature map; fusing the input feature map and the resampled feature map to obtain a fused feature map; inputting the fused feature map into a pre-trained target detection model to obtain a small target detection result.

[0006] As a further improvement of the present application, the low-dimensional input feature map obtained in advance is resized to obtain a high-dimensional feature map, including: obtaining a preset upsampling factor and the size of the input feature map (C*W*H), where C is the number of channels of the input feature map, W is the width value of the input feature map, and H is the height value of the input feature map; dividing the number of channels into γ based on the upsampling factor 2 groups, γ is the upsampling factor; the input feature map is upsampled to place the parameter values at the same position in each group in the corresponding position of the generated upsampled map in order, and the high-dimensional feature map after size transformation is obtained. The size of the high-dimensional feature map is (C'*2W*2H), where C'=C / γ 2 , C′ is the number of channels of the high-dimensional feature map.

[0007] As a further improvement of the present application, the high-dimensional feature map is integrated into the spatial domain by sharing the high-dimensional semantic information according to the channel weights in the channel domain, and is sampled again to obtain a resampled feature map, including: based on preset rules, generating a resampling kernel for each spatial position according to the number of channels of the high-dimensional feature map; dividing the number of channels of the high-dimensional feature map into a preset number of groups; multiplying each group by the resampling kernel of the corresponding spatial position and then accumulating the values in the spatial domain to obtain the parameter value of each spatial position of the resampled feature map to be generated; and performing two-dimensional expansion on the parameter value of each spatial position to obtain a resampled feature map.

[0008] As a further improvement of this application, the calculation formula of the resampling kernel is:

[0009]

[0010] φ(X i，j )=W1σ(W0X i，j );

[0011] Among them, K i，j is the resampling kernel, In the high-dimensional feature map, X i，j The set of regions centered on i，j is the neighborhood of coordinate (i, j), φ() is the kernel operation formula, is a fully connected operation, C′ is the number of channels of the high-dimensional feature map, r is the preset channel reduction rate, σ is the activation function, G is the preset number, and K is expressed in X i，j is the number of central spatial domain ranges;

[0012] The transformation process of the parameter value at each spatial position is expressed as:

[0013]

[0014] Among them, FC represents the fully connected operation;

[0015] The two-dimensional expansion process of the parameter value at each spatial position is expressed as:

[0016]

[0017] Among them, reshape represents a two-dimensional expansion operation.

[0018] As a further improvement of the present application, the input feature map and the resampled feature map are fused to obtain a fused feature map, including: expanding the size of the input feature map based on a bilinear interpolation algorithm to obtain a first-size feature map; performing a convolution operation on the resampled feature map so that the number of channels of the resampled feature map is the same as the number of channels of the first-size feature map; adding parameter values of the resampled feature map after dimensionality increase and the first-size feature map according to the position correspondence to obtain a fused feature map.

[0019] As a further improvement of this application, the calculation formula of the fused feature map is expressed as:

[0020] X out =f up (X in )+f conv (X SIR );

[0021] in, is the input feature map, is the fusion feature map, X SIR is the resampled feature map, f up is the bilinear interpolation algorithm; f conv is the convolution operation, C is the number of channels of the input feature map, W is the width of the input feature map, and H is the height of the input feature map.

[0022] As a further improvement of the present application, training a target detection model includes: resizing a low-dimensional sample feature map to obtain a high-dimensional sample feature map; fusing the high-dimensional semantic information of the high-dimensional sample feature map to the spatial domain according to the channel weight sharing in the channel domain, and re-sampling to obtain a resampled sample feature map; fusing the low-dimensional sample feature map and the resampled sample feature map to obtain a fused sample feature map; and using the fused sample feature map to construct a training set and train the target detection model.

[0023] To solve the above technical problems, another technical solution adopted in this application is: to provide a small target detection device based on feature fusion, including: a feature shuffling module, used to resize the low-dimensional input feature map obtained in advance to obtain a high-dimensional feature map; a feature resampling module, used to fuse the high-dimensional semantic information of the high-dimensional feature map into the spatial domain according to the channel weight sharing in the channel domain, and re-sample it to obtain a resampled feature map; a feature fusion module, used to fuse the input feature map and the resampled feature map to obtain a fused feature map; a target detection module, used to input the fused feature map into a pre-trained target detection model to obtain a small target detection result.

[0024] To solve the above technical problems, another technical solution adopted in this application is: to provide a computer device, which includes a processor and a memory coupled to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the steps of the small target detection method based on feature fusion as any one of the above items.

[0025] In order to solve the above technical problems, another technical solution adopted in this application is: to provide a storage medium storing program instructions that can implement any of the above-mentioned small target detection methods based on feature fusion or small target detection methods based on feature fusion.

[0026] The beneficial effect of the present application is: the small target detection method based on feature fusion of the present application obtains a high-dimensional feature map by resizing the acquired low-dimensional input feature map, and then fuses the high-dimensional semantic information of the high-dimensional feature map into the spatial domain according to the channel weight sharing, and then resamples the high-dimensional feature map to achieve the combination between the channel domain and the spatial domain. Finally, the input feature map is fused with the resampled feature map to obtain a fused feature map, which realizes the combination of high-level semantics and spatial information, which helps to retain the spatial information of small target objects, so that the target detection model can accurately detect and identify small targets according to the fused feature map. At the same time, it has fewer parameter settings, fast calculation speed and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1This is a flow chart of a small target detection method based on feature fusion according to an embodiment of the present invention;

[0028] Figure 2 is a schematic diagram of the structure of a feature fusion network according to an embodiment of the present invention;

[0029] Figure 3 1 is a schematic diagram of the functional modules of a small target detection device based on feature fusion according to an embodiment of the present invention;

[0030] Figure 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention;

[0031] Figure 5 It is a schematic structural diagram of a storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] The terms "first," "second," and "third" in this application are used only for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of such features. In the description of this application, "multiple" means at least two, for example, two, three, etc., unless otherwise specifically defined. All directional indications in the embodiments of this application (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship, movement, etc. between the components under a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications also change accordingly. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products, or devices.

[0034] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] Figure 1 It is a flow chart of a small target detection method based on feature fusion according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present invention is not based on Figure 1 It should be noted that, in this embodiment, the small target detection method based on feature fusion is implemented based on a feature fusion network and a target detection network, wherein the feature fusion network is used to realize the fusion of low-dimensional feature maps and high-dimensional feature maps to retain the spatial information of small target objects, and the target detection network detects and identifies small target objects based on the fused feature maps. Figure 2 As shown in the figure, the feature fusion network includes a feature shuffling module, a feature resampling module and a feature fusion module. The feature shuffling module is used to convert the size of the feature map. The feature resampling module is used to combine the channel domain and spatial domain of the feature map and resample it. The feature fusion module is used to fuse the resampled feature map with the low-dimensional feature map, thereby realizing the combination of high-level semantics and spatial information.

[0036] like Figure 1 As shown, the small target detection method based on feature fusion includes the following steps:

[0037] Step S101: resizing the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map.

[0038] Specifically, after obtaining the input feature map required for small target object recognition, a size transformation operation is performed on the input feature map to transform the input feature map from low dimension to high dimension.

[0039] Furthermore, in some embodiments, step S101 specifically includes:

[0040] 1.1. Get the preset upsampling factor and the size of the input feature map (C*W*H), where C is the number of channels of the input feature map, W is the width of the input feature map, and H is the height of the input feature map.

[0041] The upsampling factor is preset. After the input feature map is obtained, the size (C*W*H) of the input feature map is obtained.

[0042] 1.2. Divide the number of channels into γ based on the upsampling factor 2groups, and γ is the upsampling factor.

[0043] Specifically, in this embodiment, the upsampling factor is described by taking 2 as an example, that is, the number of channels of the input feature map is divided into 4 groups.

[0044] 1.3. Perform an upsampling operation on the input feature map to place the parameter values at the same position in each group in the corresponding position of the generated upsampled map in order to obtain a high-dimensional feature map after size transformation. The size of the high-dimensional feature map is (C′*2W*2H), where C′=C / γ 2 , C′ is the number of channels of the high-dimensional feature map.

[0045] Specifically, the parameter values at the same position in each group are placed in order at the corresponding positions of the generated upsampling image, that is, the spatial coordinates (i, j) of the low-dimensional input feature map of the first group correspond to the spatial coordinates (2i, 2j) of the first group of high-dimensional feature maps, and the spatial coordinates (i, j) of the low-dimensional input feature map of the second group correspond to the spatial coordinates (2i+1, 2j) of the first group of high-dimensional feature maps. Along the two-dimensional order, the spatial coordinates (i, j) of the third and fourth groups correspond to (2i, 2j+1) and (2i+1, 2j+1), respectively, and so on, so as to obtain the spatial coordinates of the high-dimensional feature map corresponding to the spatial coordinates of each group of low-dimensional input feature maps.

[0046] Step S102: The high-dimensional feature map is fused into the spatial domain by sharing the high-dimensional semantic information according to the channel weights in the channel domain, and is sampled again to obtain a resampled feature map.

[0047] Specifically, after the input feature map is resized into a high-dimensional feature map, it is transformed in the channel domain. In order to avoid data distortion, the high-dimensional feature map needs to be resampled to retrieve the original structure of the feature map. Therefore, after obtaining the high-dimensional feature map, this embodiment fuses the high-dimensional semantic information of the high-dimensional feature map into the spatial domain according to the channel weight sharing in the channel domain, and samples it again to obtain a resampled feature map. It should be understood that the traditional way is to directly achieve the resampling of the high-dimensional feature map through a convolutional neural network. However, this embodiment is different from the traditional convolution operation in sharing the convolution kernel parameters in the spatial domain. This embodiment shares parameters in the high-dimensional speech information.

[0048] Furthermore, in some embodiments, step S102 specifically includes:

[0049] 2.1. Based on preset rules, a resampling kernel is generated for each spatial position according to the number of channels of the high-dimensional feature map.

[0050] The calculation formula of the resampling kernel is:

[0051]

[0052] φ(X i，j )=W1σ(W0X i，j );

[0053] Among them, K i，j is the resampling kernel, In the high-dimensional feature map, X i，j The set of regions centered on i，j is the neighborhood of coordinate (i, j), φ() is the kernel operation formula, is a fully connected operation, C′ is the number of channels of the high-dimensional feature map, r is the preset channel reduction rate, σ is the activation function, G is the preset number, and K is expressed in X i，j is the number of central spatial domain ranges.

[0054] 2.2. Divide the number of channels of the high-dimensional feature map into a preset number of groups.

[0055] Specifically, the number of channels of the high-dimensional feature map is divided into G groups. In this embodiment, the value of G is 1.

[0056] 2.3. Multiply each group by the resampling kernel of the corresponding spatial position and then accumulate the values in the spatial domain to obtain the parameter value of each spatial position of the resampling feature map to be generated.

[0057] Specifically, after multiplying each group with a resampling kernel of size K*K*1 and accumulating the values in the spatial domain, the parameter value of 1*1*C at the coordinate (i, j) of the resampled feature map can be obtained.

[0058] Among them, the transformation process of the parameter value of each spatial position is expressed as:

[0059]

[0060] Among them, FC represents the fully connected operation.

[0061] 2.4. Expand the parameter value of each spatial position in two dimensions to obtain the resampled feature map.

[0062] Among them, the two-dimensional expansion process of the parameter value of each spatial position is expressed as:

[0063]

[0064] Among them, reshape represents a two-dimensional expansion operation.

[0065] The resampling kernel contains high-dimensional semantic information. Covering it with spatial information can preserve both semantic information and position information. i，jThe parameter value at the spatial coordinate (i, j) of the resampled feature map is obtained by multiplying the neighboring parameters of the center coordinate with the corresponding resampling kernel and adding them together. The data size is 1*1*C′. For different spatial coordinates, the resampling kernel at the corresponding position is used to obtain the resampled feature map.

[0066] Step S103: Fusing the input feature map and the resampled feature map to obtain a fused feature map.

[0067] Inspired by the ResNet model, this embodiment fuses the original low-dimensional input feature map information into the resampled feature map to accelerate network convergence. After obtaining the low-dimensional input feature map and the high-dimensional resampled feature map, the two are resized and then fused to obtain a fused feature map.

[0068] Furthermore, in some embodiments, step S103 specifically includes:

[0069] 3.1. Expand the size of the input feature map based on the bilinear interpolation algorithm to obtain a first-size feature map.

[0070] Specifically, the size of the input feature map is expanded using a bilinear interpolation algorithm, and the size of the input feature map is converted from (C*W*H) to (C*2W*2H).

[0071] 3.2. Perform a convolution operation on the resampled feature map so that the number of channels of the resampled feature map is the same as the number of channels of the first size feature map.

[0072] Specifically, the resampled feature map is dimensionally upgraded through a convolution operation so that the number of channels of the resampled feature map after dimension upgrade is the same as the number of channels of the input feature map after size transformation.

[0073] 3.3. Add the parameter values of the resampled feature map after dimensionality increase and the first size feature map according to the position correspondence to obtain the fused feature map.

[0074] Among them, the calculation formula of the fusion feature map is expressed as:

[0075] X out =f up (X in )+f conv (X SIR );

[0076] in, is the input feature map, is the fusion feature map, X SIR is the resampled feature map, f up is the bilinear interpolation algorithm; f convis the convolution operation, C is the number of channels of the input feature map, W is the width of the input feature map, and H is the height of the input feature map.

[0077] Step S104: Input the fused feature map into a pre-trained target detection model to obtain a small target detection result.

[0078] Specifically, after obtaining the fused feature map, the fused feature map is input into the trained target detection model to detect and identify small target objects.

[0079] Furthermore, the target detection model is pre-trained, and the steps of training the target detection model include:

[0080] 4.1. Perform size transformation on the low-dimensional sample feature map to obtain a high-dimensional sample feature map.

[0081] 4.2. The high-dimensional sample feature map is fused into the spatial domain by sharing the high-dimensional semantic information according to the channel weights in the channel domain, and then sampled again to obtain the resampled sample feature map.

[0082] 4.3. Fuse the low-dimensional sample feature map and the resampled sample feature map to obtain a fused sample feature map.

[0083] 4.4. Use the fused sample feature map to construct a training set and train the target detection model.

[0084] It should be understood that in order to ensure that the target detection model has a better detection and recognition effect on the fused features, after obtaining the sample image, the sample image is fused with low-dimensional features and high-dimensional features in the above manner, and then the sample image with fused features is used to train the target detection model.

[0085] The small target detection method based on feature fusion of the embodiment of the present invention obtains a high-dimensional feature map by resizing the acquired low-dimensional input feature map, and then fuses the high-dimensional semantic information of the high-dimensional feature map into the spatial domain according to the channel weight sharing, and then resamples the high-dimensional feature map to achieve the combination between the channel domain and the spatial domain. Finally, the input feature map is fused with the resampled feature map to obtain a fused feature map, which realizes the combination of high-level semantics and spatial information. It helps to retain the spatial information of small target objects, so that the target detection model can accurately detect and identify small targets according to the fused feature map. At the same time, it has fewer parameter settings, fast calculation speed and high efficiency.

[0086] Figure 3 FIG is a functional module diagram of a small target detection device based on feature fusion according to an embodiment of the present invention. Figure 3As shown, the small target detection device 30 based on feature fusion includes a feature shuffling module 31 , a feature resampling module 32 , a feature fusion module 33 and a target detection module 34 .

[0087] A feature shuffling module 31 is used to resize the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map;

[0088] The feature resampling module 32 is used to fuse the high-dimensional semantic information of the high-dimensional feature map into the spatial domain according to the channel weight sharing in the channel domain, and resample it to obtain a resampled feature map;

[0089] A feature fusion module 33 is used to fuse the input feature map and the resampled feature map to obtain a fused feature map;

[0090] The target detection module 34 is used to input the fused feature map into a pre-trained target detection model to obtain small target detection results.

[0091] Optionally, the feature shuffling module 31 performs a size transformation on the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map. Specifically, the operation includes: obtaining a preset upsampling factor and the size of the input feature map (C*W*H), where C is the number of channels of the input feature map, W is the width value of the input feature map, and H is the height value of the input feature map; dividing the number of channels into γ based on the upsampling factor; 2 groups, γ is the upsampling factor; the input feature map is upsampled to place the parameter values at the same position in each group in the corresponding position of the generated upsampled map in order, and the high-dimensional feature map after size transformation is obtained. The size of the high-dimensional feature map is (C'*2W*2H), where C'=C / γ 2 , C′ is the number of channels of the high-dimensional feature map.

[0092] Optionally, the feature resampling module 32 fuses the high-dimensional semantic information of the high-dimensional feature map to the spatial domain according to the channel weight sharing in the channel domain, and samples again to obtain the resampled feature map. The operations specifically include: based on preset rules, generating a resampling kernel for each spatial position according to the number of channels of the high-dimensional feature map; dividing the number of channels of the high-dimensional feature map into a preset number of groups; multiplying each group with the resampling kernel of the corresponding spatial position and then accumulating the values in the spatial domain to obtain the parameter value of each spatial position of the resampled feature map to be generated; and performing two-dimensional expansion on the parameter value of each spatial position to obtain the resampled feature map.

[0093] Optionally, the resampling kernel is calculated as:

[0094]

[0095] φ(X i，j)=W1σ(W0X i，j );

[0096] Among them, K i，j is the resampling kernel, In the high-dimensional feature map, X i，j The set of regions centered on i，j is the neighborhood of coordinate (i, j), φ() is the kernel operation formula, is a fully connected operation, C′ is the number of channels of the high-dimensional feature map, r is the preset channel reduction rate, σ is the activation function, G is the preset number, and K is expressed in X i，j is the number of central spatial domain ranges;

[0097] The transformation process of the parameter value at each spatial position is expressed as:

[0098]

[0099] Among them, FC represents the fully connected operation;

[0100] The two-dimensional expansion process of the parameter value at each spatial position is expressed as:

[0101]

[0102] Among them, reshape represents a two-dimensional expansion operation.

[0103] Optionally, the feature fusion module 33 performs the operation of fusing the input feature map and the resampled feature map to obtain the fused feature map, which specifically includes: expanding the size of the input feature map based on a bilinear interpolation algorithm to obtain a first-size feature map; performing a convolution operation on the resampled feature map so that the number of channels of the resampled feature map is the same as the number of channels of the first-size feature map; adding the parameter values of the resampled feature map after dimensionality increase and the first-size feature map according to the position correspondence to obtain a fused feature map.

[0104] Optionally, the calculation formula of the fused feature map is expressed as:

[0105] X out =f up (X in )+f conv (X SIR );

[0106] in, is the input feature map, is the fusion feature map, X SIR is the resampled feature map, f up is the bilinear interpolation algorithm; f convis the convolution operation, C is the number of channels of the input feature map, W is the width of the input feature map, and H is the height of the input feature map.

[0107] Optionally, the small target detection device 30 based on feature fusion also includes a training module for training the target detection model, including: resizing the low-dimensional sample feature map to obtain a high-dimensional sample feature map; fusing the high-dimensional semantic information of the high-dimensional sample feature map to the spatial domain according to the channel weight sharing in the channel domain, and re-sampling to obtain a resampled sample feature map; fusing the low-dimensional sample feature map and the resampled sample feature map to obtain a fused sample feature map; and using the fused sample feature map to construct a training set and train the target detection model.

[0108] For other details about the technical solutions for implementing each module in the small target detection device based on feature fusion in the above embodiment, please refer to the description of the small target detection method based on feature fusion in the above embodiment, which will not be repeated here.

[0109] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similarities between the various embodiments can be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simple, and for relevant details, reference can be made to the description of the method embodiments.

[0110] See also Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Figure 4 As shown, the computer device 30 includes a processor 31 and a memory 32 coupled to the processor 31. The memory 32 stores program instructions. When the program instructions are executed by the processor 31, the processor 31 executes the steps of the small target detection method based on feature fusion described in any of the above embodiments.

[0111] The processor 31 may also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip having signal processing capabilities. The processor 31 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0112] See Figure 5 , Figure 5Schematic diagram of the structure of the storage medium of an embodiment of the present invention. The storage medium of an embodiment of the present invention stores program instructions 41 that can implement the above-mentioned small target detection method based on feature fusion, wherein the program instructions 41 can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer device such as a computer, a server, a mobile phone, or a tablet.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed computer devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0114] In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A small target detection method based on feature fusion, characterized in that: The method comprises: Resize the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map; The high-dimensional feature map is fused into the spatial domain by sharing the high-dimensional semantic information according to the channel weights in the channel domain, and is sampled again to obtain a resampled feature map; Fusing the input feature map and the resampled feature map to obtain a fused feature map; Input the fused feature map into a pre-trained target detection model to obtain small target detection results; where: The high-dimensional feature map is fused into the spatial domain by sharing the high-dimensional semantic information according to the channel weights in the channel domain, and is sampled again to obtain a resampled feature map, including: Based on a preset rule, generating a resampling kernel for each spatial position according to the number of channels of the high-dimensional feature map; Dividing the number of channels of the high-dimensional feature map into a preset number of groups; Multiply each group by the resampling kernel of the corresponding spatial position and then accumulate the values in the spatial domain to obtain the parameter value of each spatial position of the resampling feature map to be generated; The parameter value of each spatial position is two-dimensionally expanded to obtain the resampled feature map.

2. The small target detection method based on feature fusion according to claim 1, characterized in that: The step of performing a size transformation on the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map includes: Get the preset upsampling factor and the size of the input feature map (C*W*H), where C is the number of channels of the input feature map, W is the width of the input feature map, and H is the height of the input feature map; The number of channels is divided into γ based on the upsampling factor 2 groups, γ is the upsampling factor; An upsampling operation is performed on the input feature map to place the parameter values at the same position in each group in the corresponding positions of the generated upsampled map in order to obtain a high-dimensional feature map after size transformation. The size of the high-dimensional feature map is (C′*2W*2H), where C′=C / γ 2 , C′ is the number of channels of the high-dimensional feature map.

3. The small target detection method based on feature fusion according to claim 2, characterized in that: The calculation formula of the resampling kernel is: φ(X i,j )=W1σ(W0X i,j ); Among them, K i,j is the resampling kernel, In the high-dimensional feature map, X i,j The set of regions centered on i,j is the neighborhood of coordinate (i, j), φ() is the kernel operation formula, is a fully connected operation, C′ is the number of channels of the high-dimensional feature map, r is the preset channel reduction rate, σ is the activation function, G is the preset number, and K is expressed in X i,j is the number of central spatial domain ranges; The transformation process of the parameter value at each spatial position is expressed as: Among them, FC represents the fully connected operation; The two-dimensional expansion process of the parameter value of each spatial position is expressed as: Among them, reshape represents a two-dimensional expansion operation.

4. The small target detection method based on feature fusion according to claim 1, characterized in that: The fusing the input feature map and the resampled feature map to obtain a fused feature map includes: Enlarging the size of the input feature map based on a bilinear interpolation algorithm to obtain a first-size feature map; Performing a convolution operation on the resampled feature map so that the number of channels of the resampled feature map is the same as the number of channels of the first size feature map; The parameter values of the resampled feature map after dimensionality increase and the first size feature map are added according to the position correspondence to obtain the fused feature map.

5. The small target detection method based on feature fusion according to claim 4, characterized in that: The calculation formula of the fusion feature map is expressed as: X out =f up (X in )+f conv (X SIR ); in, is the input feature map, is the fusion feature map, X SIR is the resampled feature map, f up is the bilinear interpolation algorithm; f conv is a convolution operation, C is the number of channels of the input feature map, W is the width of the input feature map, and H is the height of the input feature map.

6. The small target detection method based on feature fusion according to claim 1, characterized in that: Training the target detection model includes: Perform size transformation on the low-dimensional sample feature map to obtain a high-dimensional sample feature map; The high-dimensional sample feature map is fused into the spatial domain by sharing the high-dimensional semantic information according to the channel weights in the channel domain, and is sampled again to obtain a resampled sample feature map; Fusing the low-dimensional sample feature map and the resampled sample feature map to obtain a fused sample feature map; A training set is constructed using the fused sample feature map and the target detection model is trained.

7. A small target detection device based on feature fusion using the small target detection method based on feature fusion according to claim 1, characterized in that: include: The feature shuffling module is used to resize the pre-acquired low-dimensional input feature map to obtain a high-dimensional feature map; A feature resampling module is used to fuse the high-dimensional semantic information of the high-dimensional feature map into the spatial domain according to the channel weight sharing in the channel domain, and resample it to obtain a resampled feature map; A feature fusion module, configured to fuse the input feature map and the resampled feature map to obtain a fused feature map; The target detection module is used to input the fused feature map into a pre-trained target detection model to obtain small target detection results.

8. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein program instructions are stored in the memory. When the program instructions are executed by the processor, the processor performs the steps of the small target detection method based on feature fusion as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The device stores program instructions capable of implementing the small target detection method based on feature fusion as described in any one of claims 1 to 6.

Images