An image target detection method and system based on multi-scale feature fusion
By combining local information entropy analysis and implicit semantic displacement compensation with weighted mask matrix and topological consistency monitoring, the problem of centroid shift in multi-scale feature fusion is solved, achieving high accuracy and stability in image target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA THREE GORGES UNIV
- Filing Date
- 2026-05-27
- Publication Date
- 2026-07-28
AI Technical Summary
In image target detection, existing technologies suffer from centroid shifts between shallow and deep feature layers and inaccurate overlap of cross-scale features during multi-scale feature fusion, leading to target edge localization deviations. Furthermore, the lack of effective monitoring of the consistency of the geometric structure of the fused features affects detection accuracy and stability.
By calculating the sub-pixel-level offset vector of the deep feature layer relative to the shallow feature layer through local information entropy analysis, a hidden displacement field is constructed for spatial resampling. Then, a weighted fusion is performed pixel by pixel using a weighted mask matrix. Combined with the topology consistency monitoring module, the displacement prediction is corrected in real time, and a closed-loop feedback training mechanism is constructed.
It achieves precise alignment of features across scales, improves the accuracy and robustness of bounding box localization in object detection, and enhances the model's detection capability in complex backgrounds.
Smart Images
Figure CN122473601A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically to an image target detection method and system based on multi-scale feature fusion. Background Technology
[0002] Image target detection systems are typically deployed in computing devices that integrate processors and memory, and are connected to image acquisition devices such as cameras. These systems rely on a feature extraction backbone network to perform multi-level convolution operations on the input image signal. They utilize computing units of different depths to obtain feature layers containing spatial detail and category semantic information, and then transmit the processed feature maps to the detection module via a data interface, thereby achieving the identification and localization of target objects.
[0003] In handling multi-scale feature fusion, existing technologies generally employ a feature pyramid structure to combine deep and shallow feature maps. The specific implementation typically involves upsampling the deep feature layers with a large downsampling ratio (e.g., using bilinear interpolation or transposed convolution) to enlarge their spatial dimensions to match those of the shallow feature layers. Then, the two are superimposed through pixel-wise addition or channel-axis concatenation to construct a fused feature with both spatial resolution and semantic expressive power. However, after multiple convolution and pooling operations, the center position of the semantic features of deep features often experiences sub-pixel displacement relative to the spatial pixel coordinates of the original image. Existing upsampling alignment methods primarily focus on matching the spatial dimensions of feature maps, neglecting the centroid shift between deep and shallow feature layers. This results in the inability to achieve precise pixel-level overlap during cross-scale feature fusion. This mismatch in spatial correspondence interferes with the model's perception of target edges, causing localization errors in the target bounding boxes output by the detection head module, thus affecting the accuracy of the target detection task. Furthermore, existing methods typically employ fixed fusion weights or channel attention mechanisms during the fusion process, making it difficult to dynamically adjust the injection ratio of spatial details and semantic information based on the geometric complexity of local regions. This can easily lead to missed or false detections in complex backgrounds or densely populated areas with small targets. Simultaneously, existing technologies lack effective means to monitor the consistency of the geometric structure of the fused features, failing to promptly correct manifold distortions caused by inaccurate displacement predictions during training, further limiting the stability and robustness of the detection system. Summary of the Invention
[0004] The purpose of this invention is to provide an image target detection method and system based on multi-scale feature fusion to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: An image target detection method based on multi-scale feature fusion, applied to an image processing device, includes the following steps: S1. Receive the image to be detected, perform convolution operation to output a shallow feature layer containing spatial distribution features and a deep feature layer containing category semantic features; S2. Calculate the local information entropy of the shallow feature layer to generate an entropy distribution map with the same spatial resolution as the shallow feature layer. S3. Determine the sampling density based on the entropy value in the entropy distribution map, and calculate the sub-pixel level offset vector of the deep feature layer relative to the shallow feature layer based on the sampling density to construct the hidden displacement field; S4. Perform spatial resampling on the deep feature layer based on the implicit displacement field to align the deep feature layer with the shallow feature layer in the pixel coordinate system, and output the aligned feature layer. S5. Calculate the feature norm distribution of the aligned feature layer, and construct a weight mask matrix in combination with the entropy distribution map. Use the weight mask matrix to perform pixel-wise weighted fusion of the shallow feature layer and the aligned feature layer to generate a fused feature map. S6. Calculate the local topological correlation matrix of the fused feature map and the original topological correlation matrix of the shallow feature layer respectively, and compare them to obtain the topological deviation value; S7. When the topology deviation value is within the preset threshold range, perform classification and regression operations on the fused feature map and output the target's category probability and bounding box coordinates; when the topology deviation value exceeds the preset threshold, adjust the weight parameters of the displacement vector prediction branch according to the topology deviation value and return to step S3.
[0006] Furthermore, the generation of the entropy distribution map includes: performing channel averaging on the shallow feature layer to convert the three-dimensional feature tensor into a two-dimensional single-channel feature map; calculating the maximum and minimum feature intensity values in the two-dimensional single-channel feature map and performing linear normalization; performing discretization mapping on the normalized data according to a preset quantization level to generate a quantized feature map; setting a sliding window of size k×k on the quantized feature map, counting the frequency of each quantization level pixel within the window, calculating the probability of occurrence based on the ratio of the frequency to the total number of pixels in the window, and then calculating the local information entropy using a logarithmic operation with the natural constant as the base based on the probability of occurrence, and traversing the entire quantized feature map to obtain the entropy distribution map.
[0007] Furthermore, the method for determining the sampling density based on the entropy value in the entropy distribution map is as follows: the entropy distribution map is converted into a sampling density mapping matrix using a sampling density mapping function. The sampling density mapping function is in the form of an exponential S-shaped growth curve with the natural constant e as the base, wherein the sampling density factor is positively correlated with the entropy value.
[0008] Furthermore, the construction of the hidden displacement field includes: performing concatenation on the shallow feature layer, the upsampled deep feature layer, and the sampling density mapping matrix along the channel axis to generate a multidimensional associated feature tensor; performing linear transformation through two sets of parallel 1×1 convolutional layers to calculate the offset component in the horizontal direction and the offset component in the vertical direction of each pixel coordinate point, thereby constructing the hidden displacement field.
[0009] Furthermore, the spatial resampling includes: calculating the sub-pixel coordinates to be sampled in the deep feature layer based on the implicit displacement field, determining four adjacent integer pixel points surrounding the sub-pixel coordinates to be sampled, and performing bilinear interpolation operation based on the horizontal interpolation coefficients and the vertical interpolation coefficients to obtain the aligned feature layer.
[0010] Furthermore, the weight mask matrix is constructed as follows: the entropy distribution map is multiplied by a preset structural weight parameter, the eigenvector L2 norm of the aligned feature layer is multiplied by a preset semantic weight parameter, the sum of the two products is input into a Sigmoid activation function to obtain pixel-wise weight values, thus forming the weight mask matrix; wherein the structural weight parameter and the semantic weight parameter are trainable parameters.
[0011] Furthermore, the local topological correlation matrix is calculated as follows: for each pixel position, a local search window is defined, and the negative of the square of the second norm of the feature vector of the center pixel and the adjacent pixels in the window is calculated by the Gaussian kernel function and divided by the exponent value of twice the preset topological smoothing coefficient. This exponent is used as the topological correlation weight between the two pixel positions. The local topological correlation matrix is obtained by traversing all pixels.
[0012] Furthermore, adjusting the weight parameters of the displacement vector prediction branch includes: constructing a joint optimization objective function, which is a weighted sum of the detection task loss function and the topology loss function, wherein the topology loss function is calculated based on the difference between the local topological correlation matrix of the fused feature map and the shallow feature layer; and using the backpropagation algorithm to backpropagate the gradient of the joint optimization objective function to the displacement vector prediction branch to perform weight parameter updates.
[0013] An image target detection system based on multi-scale feature fusion includes a processor and a memory storing computer execution instructions, wherein the processor executes by calling the computer execution instructions. The feature extraction backbone network is used to output shallow and deep feature maps; The local information entropy analysis module is used to perform statistical analysis on the shallow feature map and generate an entropy distribution map. The implicit semantic displacement compensation module is used to determine the sampling density based on the entropy distribution map, calculate the sub-pixel level offset vector of the deep feature map relative to the shallow feature map and construct the implicit displacement field, and perform spatial resampling on the deep feature map to output an aligned feature map; The dynamic mask fusion module is used to construct a weighted mask matrix based on the feature norm of the entropy distribution map and the aligned feature map, and to perform pixel-level weighted fusion of the shallow feature map and the aligned feature map to generate a fused feature map. The topology consistency monitoring module is used to calculate the local topology correlation matrix between the fused feature map and the shallow feature map and compare them to obtain the topology deviation value. When the topology deviation value exceeds a preset threshold, a correction signal is sent to the implicit semantic displacement compensation module. The detection head module is used to receive the fused feature map and output the target category probability and bounding box coordinates.
[0014] Furthermore, the implicit semantic displacement compensation module includes: a feature concatenation unit, used to concatenate the shallow feature map, the upsampled deep feature map, and the sampling density mapping matrix generated according to the entropy distribution map along the channel axis to generate a multidimensional associated feature tensor; an offset vector output unit, used to calculate the offset components of each pixel coordinate point in the horizontal and vertical directions through two sets of parallel 1×1 convolutional layers to form the implicit displacement field; and a sampling execution unit, used to calculate the sub-pixel coordinates to be sampled in the deep feature map according to the implicit displacement field, and obtain the aligned feature map through bilinear interpolation.
[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention utilizes local information entropy analysis and implicit semantic displacement compensation to adaptively determine the sampling density based on the local structural complexity of the image, and calculates the sub-pixel-level offset vector of deep features relative to shallow features. This feature effectively solves the spatial centroid shift problem during upsampling alignment of multi-scale features, achieving accurate alignment of cross-scale features at the pixel level, thereby significantly improving the accuracy of bounding box localization in object detection tasks.
[0016] 2. By constructing a dynamic mask fusion module, the system can generate a weighted mask matrix by combining the norm distribution of aligned features and local information entropy, dynamically adjusting the fusion ratio of spatial details and semantic information. This pixel-wise weighted fusion mechanism enables the generated fused feature map to retain key geometric contours while also possessing stronger class discrimination performance, enhancing the model's robustness in detecting targets of different sizes against complex backgrounds.
[0017] 3. A topology consistency monitoring module is introduced and a feedback correction path is established. By calculating the local topological correlation differences between the fused feature map and the original shallow features, the geometric deformation in the displacement prediction process is effectively constrained. This module can detect and correct manifold distortions that may occur during feature alignment in real time, ensuring that the fused feature layer still follows the spatial topological logic of the original image, and guaranteeing the output stability of the detection results in structurally complex regions.
[0018] 4. This invention jointly optimizes the detection loss and topology loss to form a closed-loop feedback training mechanism, which makes the displacement prediction network subject to the dual constraints of alignment accuracy and geometric consistency when updating parameters, thereby further improving the model's generalization ability and detection accuracy in cross-scale fusion scenarios. Attached Figure Description
[0019] Figure 1 A schematic diagram of the system structure of an image target detection system based on multi-scale feature fusion provided in an embodiment of the present invention; Figure 2 A flowchart illustrating an image target detection method based on multi-scale feature fusion provided in an embodiment of the present invention; Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] The method in this embodiment is executed by a terminal, which can be a mobile phone, computer, PDA, laptop or desktop computer, etc. Of course, it can also be other devices with similar functions, and this embodiment does not limit them.
[0021] Reference Figure 1 As shown, this embodiment provides an image target detection system based on multi-scale feature fusion. The system runs on a computing device, which includes a processor and a memory. The memory stores computer execution instructions, and the processor executes these instructions by calling them to run a feature extraction backbone network 101, a local information entropy analysis module 102, a latent semantic displacement compensation module 103, a dynamic mask fusion module 104, a topology consistency monitoring module 105, and a detection head module 106. An image acquisition device is connected to the computing device to acquire raw image data and transmit it to the computing device.
[0022] (I) System Module Composition The feature extraction backbone network 101 performs convolution operations on the original image to generate feature layers with different downsampling ratios. The feature layers output by the feature extraction backbone network 101 include shallow feature layers. and deep feature layers Among them, the shallow feature layer It preserves the spatial pixel details of the original image, deep feature layers It has a large receptive field and extracts global semantic features. In this embodiment, the feature extraction backbone network 101 can adopt a convolutional neural network structure such as ResNet-50 or VGG16, and those skilled in the art can choose according to the actual accuracy and computing resource requirements.
[0023] The local information entropy analysis module 102 is connected to the feature extraction backbone network 101. The local information entropy analysis module 102 receives shallow feature layers. and shallow feature layers The feature intensity within a preset-sized sliding window region is statistically analyzed using probability distribution. The local information entropy analysis module 102 calculates the local information entropy at each pixel location based on the statistical results, generating a sequence similar to that of the shallow feature layer. Entropy distribution diagram E with consistent spatial dimensions.
[0024] The implicit semantic displacement compensation module 103 is connected to the feature extraction backbone network 101 and the local information entropy analysis module 102, respectively. The implicit semantic displacement compensation module 103 receives deep feature layers. Shallow feature layer And the entropy distribution map E. The implicit semantic displacement compensation module 103 uses the entropy distribution map E as the sampling density factor to calculate the deep feature layer. Compared to shallow feature layers The sub-pixel level offset is used to generate a hidden displacement field, and this hidden displacement field is then used to apply to deeper feature layers. Perform spatial resampling and output aligned feature layers. '.
[0025] The dynamic mask fusion module 104 is connected to the local information entropy analysis module 102 and the implicit semantic displacement compensation module 103, respectively. The dynamic mask fusion module 104 utilizes the entropy distribution map E and the aligned feature layer... The feature response intensity is used to construct a weight mask matrix. The dynamic mask fusion module 104 then applies the weight mask matrix to the shallow feature layer. Alignment feature layer Perform pixel-level weighted summation to obtain the fused feature map. The weight mask matrix determines the fused feature map. The injection ratio of shallow to deep features.
[0026] The topology consistency monitoring module 105 is connected to the dynamic mask fusion module 104. The topology consistency monitoring module 105 extracts the fused feature map. The local manifold structure features are obtained and then compared with the shallow feature layers. The original topology is compared with the original topology, and the difference in the topological correlation matrix between the two is calculated. If the difference exceeds a preset threshold, the topology consistency monitoring module 105 generates a correction signal and feeds it back to the implicit semantic displacement compensation module 103. The implicit semantic displacement compensation module 103 adjusts the bias parameters of the displacement vector calculation branch according to the correction signal and recalculates the implicit displacement field.
[0027] The detection head module 106 is connected to the topology consistency monitoring module 105. The detection head module 106 receives the fused feature map. It performs classification and regression calculations, outputting the target category probability and bounding box coordinates. These modules interact through a data interface to complete the process from image input to detection result output.
[0028] (II) Methods and Procedures Reference Figure 2 As shown, this embodiment also provides an image target detection method based on multi-scale feature fusion, including the following steps: In step S101, the feature extraction backbone network 101 receives the image to be detected and performs convolution operations, outputting a multi-scale feature layer. The multi-scale feature layer includes a shallow feature layer. and deep feature layers Shallow feature layer The downsampling rate is lower than that of deep feature layers. downsampling ratio, shallow feature layer Includes spatial distribution features of the image, deep feature layer Includes the semantic features of the image category.
[0029] Step S102, the local information entropy analysis module 102 receives the shallow feature layer and shallow feature layers The feature activation intensity distribution is statistically analyzed. The local information entropy analysis module 102 calculates the information entropy value based on the probability of feature activation intensity occurrence within a preset local window, generating a value corresponding to the shallow feature layer. Entropy distribution map E with consistent spatial resolution. Entropy distribution map E is used to characterize the structural complexity of different coordinate regions of the image.
[0030] Specifically, the process of generating the entropy distribution map E includes: First, processing the shallow feature layer... Channel averaging is performed to convert the 3D feature tensor into a 2D single-channel feature map. The maximum and minimum feature intensities in this 2D single-channel feature map are then calculated, followed by linear normalization. The normalized data is then discretized according to a preset quantization level L (e.g., L = 16) to generate a quantized feature map. A sliding window of size k×k is then set on the quantized feature map, where k is an odd number (e.g., k = 3). When the sliding window extends beyond the feature map's boundaries near the edge, zero-padding is used to fill the excess portion with zeros. The values within the window are then statistically analyzed. Frequency of pixel occurrence According to the formula Calculate the probability of occurrence P_i, and use logarithmic operations with the natural logarithm e as the base to calculate the information entropy. , If This term is then ignored. Traversing the entire quantized feature map yields the entropy distribution map E.
[0031] Step S103: The implicit semantic displacement compensation module 103 obtains the shallow feature layer. Deep feature layer And the entropy distribution map E. The implicit semantic displacement compensation module 103 determines the sampling density of different coordinate regions based on the entropy values in the entropy distribution map E, and the sampling density is positively correlated with the entropy value. Specifically, it utilizes the sampling density mapping function
[0032] The entropy distribution map E is converted into a sampling density mapping matrix, where γ is a preset positive real number with a value ranging from 5 to 15. In this embodiment, γ is set to 10. This value ensures that when the entropy value E(x,y) is within the effective range, the sampling density factor ρ(x,y) can cover the interval from 0.5 to 0.95, thereby achieving more refined alignment calculations in structurally complex regions.
[0033] The implicit semantic displacement compensation module 103 calculates the deep feature layer. Compared to shallow feature layers The sub-pixel level offset vector is used to construct the latent displacement field. The process of constructing the latent displacement field includes: the feature stitching unit 1031 first processes the deep feature layer... Perform bilinear interpolation upsampling to make its spatial dimensions match those of the shallow feature layer. Consistent, then along the channel axis towards the shallow feature layer Upsampled deep feature layer The sampling density mapping matrix is then concatenated to generate a multidimensional correlated feature tensor. The upsampled deep feature layer... This refers to the deep feature layer Upsampled to the shallow feature layer using bilinear interpolation. The same spatial resolution is maintained. The feature association extraction unit 1032 consists of several sets of cascaded residual convolutional layers, which perform convolution operations on the multidimensional associated feature tensor and perform element-wise multiplication operations between the intermediate feature map and the sampling density mapping matrix. The offset vector output unit 1033 performs linear transformations through two sets of parallel 1×1 convolutional layers, calculating the offset component in the horizontal direction for each pixel coordinate point. and the offset component in the vertical direction This constitutes an implicit displacement field.
[0034] Step S104, the implicit semantic displacement compensation module 103 adjusts the deep feature layer according to the implicit displacement field. Perform spatial resampling to achieve deep feature layers With shallow feature layer Alignment in pixel coordinate system, output alignment feature layer The spatial resampling process includes: the sampling execution unit 1034 traversing each spatial coordinate point (x, y) of the output feature map and extracting the corresponding normalized displacement vector from the latent displacement field. Multiply by the deep feature layer The width and height are used to obtain the pixel-scale displacement components. , Calculate deep feature layers Subpixel coordinates to be sampled in If the calculated result or If the coordinates exceed the feature map boundary range of 0 to W-1 or 0 to H-1, then boundary truncation is performed, and the nearest valid integer coordinates are taken. Determine the bounding box. Four adjacent integer pixels Calculate the horizontal interpolation coefficients and vertical interpolation coefficients ,in .
[0035] Perform bilinear interpolation: .
[0036] Arrange all the calculated output vectors in spatial coordinate order to generate an aligned feature layer. '.
[0037] Step S105, the dynamic mask fusion module 104 calculates the aligned feature layer. The feature norm distribution is calculated, and a weight mask matrix is constructed by combining it with the entropy distribution map E. The mask computation unit 1041 receives the entropy distribution map E and the aligned feature layers. For the aligned feature layer A L2 norm operation is performed along the channel dimension to generate a two-dimensional feature response intensity map. The mask calculation unit 1041 has trainable structural weight parameters α and semantic weight parameters β, which are automatically updated through backpropagation during model training. In this embodiment, initial settings α=0.5 and β=0.5 are given; in practical applications, these can be dynamically adjusted based on validation set performance. The entropy distribution map E is multiplied pixel-by-pixel by α, and the feature response intensity map is multiplied pixel-by-pixel by β. The two results are summed at the pixel level, and then a Sigmoid nonlinear activation operation is performed on the summed result to generate a fused mask M. The feature synthesis unit 1042 acquires the aligned feature layer. Shallow feature layer And the fusion mask M, which is then processed by a 1×1 convolutional layer. and Mapped to the same number of channels Calculate the complementary weight matrix (1-M), and then perform pixel-level weighted synthesis: Generate fused feature maps .
[0038] In step S106, the topology consistency monitoring module 105 calculates the fused feature map respectively. Local topological correlation matrix and shallow feature layer The original topological correlation matrix is obtained. For each pixel position i in the feature map, a local search window of size p×p is defined with i as the center. For example, p is 3. The value of p is independent of the spatial resolution of the feature map and is usually an odd number between 3 and 7. The set of adjacent pixels j within the window is extracted. The position is calculated using the Gaussian kernel function. and Topological association weights between , in The preset topology smoothing coefficient ranges from 0.5 to 2.0. In this embodiment... =1.0. If j is not within the local search window of i, then Traverse all coordinate points and generate local topological correlation matrices for the fused feature maps. Original topological correlation matrix of shallow feature layers Topology loss calculation unit 1051 pairs and Perform row normalization, then calculate the topology loss function.
[0039] in The coordinates in the original local topological incidence matrix are elements, Local topological incidence matrix The median coordinate is Element. This is a preset constant, with a range of values, for example, . to .
[0040] Step S107, when the topology deviation value is within the preset threshold range (i.e. When fusing feature maps, Input detection head module 106; where A preset threshold is used, for example, a value between 0.05 and 0.2, which can be adjusted based on the validation set. The detection head module 106 pairs of fused feature maps. The detection head module 106 performs classification and regression operations, outputting the target's class probability and bounding box coordinates. Internally, it includes a classification branch unit 1061, a regression branch unit 1062, and a post-processing unit 1063. The classification branch unit 1061 consists of two 3×3 convolutional layers and one 1×1 convolutional layer with N output channels, where N equals the total number of target classes, outputting a class prediction tensor. The regression branch unit 1062 consists of two 3×3 convolutional layers and one 1×1 convolutional layer with 4 output channels, outputting a coordinate regression tensor and four offset variables. These represent the horizontal and vertical coordinate offsets of the center point, the width scaling factor, and the height scaling factor, respectively. The post-processing unit 1063 performs threshold filtering on the category prediction tensor and calculates the physical coordinates of the bounding box based on the original coordinates of the preserved position and the offset variables: the center point coordinates are... Width is The height is Then, non-maximum suppression processing is performed (e.g., using an IoU threshold of 0.5), and the final detection result is output.
[0041] Step S108, when the topology deviation value exceeds the preset threshold (i.e. When the topology consistency monitoring module 105 feeds back the topology error signal to the implicit semantic displacement compensation module 103, the implicit semantic displacement compensation module 103 adjusts the weight parameters of the displacement vector prediction branch according to the feedback signal and returns to execution step S103 until the topology deviation value meets the preset requirements. Specifically, the implicit semantic displacement compensation module 103 constructs a joint optimization objective function. ,in The loss function for the detection task is a weighted average of classification loss and regression loss, for example... Binary cross-entropy loss or Focal Loss can be used. Smooth L1 loss or The loss is represented by μ, a balancing coefficient, preferably μ = 1.0. λ is the weight balancing coefficient, which in this embodiment is λ = 0.5. The backpropagation algorithm is used to... The gradient is backpropagated to the displacement vector prediction branch to update the weight parameters, so that the implicit displacement field generated in subsequent iterations is subject to topological consistency constraints while satisfying the spatial alignment requirements.
[0042] Through the above steps, this embodiment achieves sub-pixel-level precise alignment of cross-scale features, dynamic weighted fusion, and topological consistency closed-loop feedback, significantly improving the bounding box localization accuracy and system robustness of target detection.
[0043] Example 2 The difference between this embodiment and embodiment 1 is that the quantization parameters for generating the entropy distribution map, the γ parameter of the sampling density mapping function, and the α and β parameters in the weight mask matrix in the local information entropy analysis module 102 are preferred specific values, and the size of the local search window and the topology smoothing coefficient in the topology consistency monitoring module 105 are further limited.
[0044] Specifically, when generating the entropy distribution map E, the preset quantization level L is set to 16. (Sliding window size) In this context, k is set to 3. The preset positive real number γ in the sampling density mapping function is set to 10. The structural weight parameter α in the weight mask matrix is initially set to 0.8, and the semantic weight parameter β is initially set to 0.3; both are trainable parameters and are automatically optimized during training. In the topology consistency monitoring module, the local search window size... In the figure, p takes a value of 5, which is the topological smoothing coefficient. The value is set to 1.5. The minimum constant ε in the topological loss function is set to 1×10^{-5}. The weight balancing coefficient λ in the joint optimization objective function is set to 0.8. This embodiment, by adjusting the above parameters, further enhances the model's structural sensitivity to complex texture regions, making it suitable for scenarios with densely distributed small targets or highly complex backgrounds, such as pedestrian detection in aerial images and surveillance videos.
[0045] Except for the different parameter values mentioned above, the system structure, module connection relationship and method flow of this embodiment are the same as those of Embodiment 1, and will not be repeated here.
[0046] Example 3 The difference between this embodiment and Embodiment 1 is that the feature extraction backbone network 101 adopts a different structure, and the displacement prediction network in the implicit semantic displacement compensation module 103 adopts a deeper convolutional structure.
[0047] Specifically, the feature extraction backbone network 101 uses ResNet-101 with a feature pyramid structure (FPN) as its base network. Among these, the shallow feature layers... Taken from the P2 layer of FPN (downsampled 4 times), deep feature layer The data is taken from the P5 layer of the FPN (downsampled by 32 times). In the implicit semantic displacement compensation module 103, the feature association extraction unit 1032 consists of four sets of cascaded residual convolutional layers, each set containing two 3×3 convolutional layers and one skip connection. The offset vector output unit 1033 outputs... and Previously, a batch normalization layer and a ReLU activation layer were added.
[0048] In the dynamic mask fusion module 104, the feature synthesis unit 1042 performs a process on the shallow feature layer. and Alignment Feature Layer After performing 1×1 convolution mapping, a channel attention module, namely the Squeeze-and-Excitation module, is added to adaptively recalibrate the response weights of each channel before performing pixel-level weighted summation.
[0049] In the topology consistency monitoring module 105, before calculating the local topology association matrix, the fused feature map is first processed. and shallow feature layer Global average pooling downsampling is performed to reduce the spatial resolution to half that of the original image, thereby reducing computational cost.
[0050] The other modules and steps in this embodiment are the same as in Embodiment 1. By employing a deeper backbone network and a more complex displacement prediction structure, this embodiment can achieve higher detection accuracy within the limits of computing resources, making it particularly suitable for offline detection scenarios with high accuracy requirements and relatively relaxed real-time requirements.
[0051] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0052] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, equipment, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0053] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0054] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program instructions, such as USB flash drives, portable hard drives, read-only storage servers, random access storage servers, magnetic disks, or optical disks.
[0055] Furthermore, it should be noted that the combination of the various technical features in this case is not limited to the combination methods described in the claims of this case or the combination methods described in the specific embodiments. All technical features described in this case can be freely combined or combined in any way, unless they contradict each other.
[0056] It should be noted that the above examples are merely specific embodiments of the present invention, and the present invention is obviously not limited to the above embodiments, with many similar variations. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in this invention should fall within the protection scope of this invention.
[0057] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An image target detection method based on multi-scale feature fusion, characterized in that, Applied to image processing devices, it includes the following steps: S1. Receive the image to be detected, perform convolution operation to output a shallow feature layer containing spatial distribution features and a deep feature layer containing category semantic features; S2. Calculate the local information entropy of the shallow feature layer to generate an entropy distribution map with the same spatial resolution as the shallow feature layer. S3. Determine the sampling density based on the entropy value in the entropy distribution map, and calculate the sub-pixel level offset vector of the deep feature layer relative to the shallow feature layer based on the sampling density to construct the hidden displacement field; S4. Perform spatial resampling on the deep feature layer based on the implicit displacement field to align the deep feature layer with the shallow feature layer in the pixel coordinate system, and output the aligned feature layer. S5. Calculate the feature norm distribution of the aligned feature layer, and construct a weight mask matrix in combination with the entropy distribution map. Use the weight mask matrix to perform pixel-wise weighted fusion of the shallow feature layer and the aligned feature layer to generate a fused feature map. S6. Calculate the local topological correlation matrix of the fused feature map and the original topological correlation matrix of the shallow feature layer respectively, and compare them to obtain the topological deviation value; S7. When the topology deviation value is within the preset threshold range, perform classification and regression operations on the fused feature map and output the target's category probability and bounding box coordinates; when the topology deviation value exceeds the preset threshold, adjust the weight parameters of the displacement vector prediction branch according to the topology deviation value and return to step S3.
2. The image target detection method based on multi-scale feature fusion according to claim 1, characterized in that, The process of generating the entropy distribution map includes: performing channel averaging on the shallow feature layer to convert the three-dimensional feature tensor into a two-dimensional single-channel feature map; calculating the maximum and minimum feature intensity values in the two-dimensional single-channel feature map and performing linear normalization; performing discretization mapping on the normalized data according to a preset quantization level to generate a quantized feature map; setting a sliding window of size k×k on the quantized feature map, counting the frequency of each quantization level pixel within the window, calculating the probability of occurrence based on the ratio of the frequency to the total number of pixels in the window, and then calculating the local information entropy using a logarithmic operation with the natural constant as the base based on the probability of occurrence, and traversing the entire quantized feature map to obtain the entropy distribution map.
3. The image target detection method based on multi-scale feature fusion according to claim 1, characterized in that, The method for determining the sampling density based on the entropy value in the entropy distribution map is as follows: the entropy distribution map is converted into a sampling density mapping matrix using a sampling density mapping function. The sampling density mapping function is in the form of an exponential S-shaped growth curve with the natural constant e as the base, wherein the sampling density factor is positively correlated with the entropy value.
4. The image target detection method based on multi-scale feature fusion according to claim 3, characterized in that, The construction of the hidden displacement field includes: performing concatenation on the shallow feature layer, the upsampled deep feature layer, and the sampling density mapping matrix along the channel axis to generate a multidimensional associated feature tensor; performing linear transformation through two sets of parallel 1×1 convolutional layers to calculate the offset component in the horizontal direction and the offset component in the vertical direction of each pixel coordinate point, thereby constructing the hidden displacement field.
5. The image target detection method based on multi-scale feature fusion according to claim 1, characterized in that, The spatial resampling includes: calculating the sub-pixel coordinates to be sampled in the deep feature layer based on the implicit displacement field, determining four adjacent integer pixel points surrounding the sub-pixel coordinates to be sampled, and performing bilinear interpolation operation based on the horizontal interpolation coefficients and the vertical interpolation coefficients to obtain the aligned feature layer.
6. The image target detection method based on multi-scale feature fusion according to claim 1, characterized in that, The weight mask matrix is constructed as follows: the entropy distribution map is multiplied by a preset structural weight parameter, the eigenvector L2 norm of the aligned feature layer is multiplied by a preset semantic weight parameter, the sum of the two products is input into the Sigmoid activation function to obtain pixel-wise weight values, thus forming the weight mask matrix; wherein the structural weight parameter and the semantic weight parameter are trainable parameters.
7. The image target detection method based on multi-scale feature fusion according to claim 1, characterized in that, The local topological correlation matrix is calculated as follows: for each pixel position, a local search window is defined, and the negative of the square of the second norm of the feature vector of the center pixel and the adjacent pixels in the window is calculated by the Gaussian kernel function and divided by the exponent value of twice the preset topological smoothing coefficient. This exponent is used as the topological correlation weight between the two pixel positions. The local topological correlation matrix is obtained by traversing all pixels.
8. The image target detection method based on multi-scale feature fusion according to claim 1, characterized in that, The adjustment of the weight parameters of the displacement vector prediction branch includes: constructing a joint optimization objective function, which is a weighted sum of the detection task loss function and the topology loss function, wherein the topology loss function is calculated based on the difference between the local topological correlation matrix of the fused feature map and the shallow feature layer; and using the backpropagation algorithm to backpropagate the gradient of the joint optimization objective function to the displacement vector prediction branch to perform weight parameter updates.
9. An image target detection system based on multi-scale feature fusion, characterized in that, It includes a processor and a memory storing computer-executed instructions, the processor executing by invoking the computer-executed instructions: The feature extraction backbone network is used to output shallow and deep feature maps; The local information entropy analysis module is used to perform statistical analysis on the shallow feature map and generate an entropy distribution map. The implicit semantic displacement compensation module is used to determine the sampling density based on the entropy distribution map, calculate the sub-pixel level offset vector of the deep feature map relative to the shallow feature map and construct the implicit displacement field, and perform spatial resampling on the deep feature map to output an aligned feature map; The dynamic mask fusion module is used to construct a weighted mask matrix based on the feature norm of the entropy distribution map and the aligned feature map, and to perform pixel-level weighted fusion of the shallow feature map and the aligned feature map to generate a fused feature map. The topology consistency monitoring module is used to calculate the local topology correlation matrix between the fused feature map and the shallow feature map and compare them to obtain the topology deviation value. When the topology deviation value exceeds a preset threshold, a correction signal is sent to the implicit semantic displacement compensation module. The detection head module is used to receive the fused feature map and output the target category probability and bounding box coordinates.
10. The system according to claim 9, characterized in that, The implicit semantic displacement compensation module includes: The feature splicing unit is used to splice the shallow feature map, the upsampled deep feature map, and the sampling density mapping matrix generated according to the entropy distribution map along the channel axis to generate a multidimensional associated feature tensor. The offset vector output unit is used to calculate the offset components of each pixel coordinate point in the horizontal and vertical directions through two sets of parallel 1×1 convolutional layers, thereby forming the implicit displacement field. The sampling execution unit is used to calculate the sub-pixel coordinates to be sampled in the deep feature map based on the implicit displacement field, and to obtain the aligned feature map through bilinear interpolation.