An image target detection method based on multi-scale attention feature fusion

Abstract

This invention discloses an image target detection method based on multi-scale attention feature fusion, relating to the fields of image recognition and deep learning. The method receives an input image and processes it into a standard image. After extracting image features, a scale-adjusting module is used to obtain scale-adjusted features, generating edge response maps, structural response maps, and semantic response maps. A structural constraint transfer cost matrix is ​​constructed, and a structural constraint transfer plan is obtained through entropy regularization and alternating scaling, forming a structural constraint response transfer map. Based on the scale-adjusted features, baseline scale fusion features and transfer-guided scale fusion features are obtained, the fusion route result is determined, and target consistency fusion features are obtained through positional attention analysis. After target output mapping of the target consistency fusion features, candidate retention values ​​are calculated, and the detected target is determined by grouping and sorting overlapping bounding boxes. This method allows edge information, structural information, and semantic information to participate in target detection within the same feature coordinate system.

Description

Technical Field

[0001] This invention relates to the fields of image recognition and deep learning technology, specifically to an image target detection method based on multi-scale attention feature fusion. Background Technology

[0002] Image recognition and deep learning technologies belong to the field of computer vision. Image object detection is an important branch of computer vision, mainly used to identify object categories and determine object locations in input images. With the increasing complexity of image content in scenarios such as industrial inspection, traffic monitoring, security identification, and remote sensing image analysis, object detection tasks have gradually shifted from single-scale object recognition to object localization and classification in scenarios where multiple scales of objects coexist. Current image object detection technologies typically require resizing and pixel normalization of the input image, extracting shallow texture features, mid-level structural features, and deep semantic features, and then merging features from different scales into the same feature coordinate system. Finally, based on the fused features, the system outputs category response, location distance, and confidence response to obtain candidate and detected objects.

[0003] In multi-scale target detection, shallow texture features contain more target edge information, mid-level structural features contain more target subject information, and deep semantic features contain more contextual semantic information. Existing technologies often use direct stitching or weighted fusion to process features at different scales, failing to fully analyze the distribution differences of edge response maps, structural response maps, and semantic response maps in the same feature coordinate system, and lacking constraint analysis on coordinate displacement costs, structural path costs, and cross-scale response migration states. Consequently, when small targets, occluded targets, weakly textured targets, and complex backgrounds coexist, mismatches between edge and semantic information easily occur between features at different scales, target subject responses are mixed with background responses, and the target region in the fused features is unclear. When the detection head generates candidate categories, candidate boxes, and candidate confidence scores based on such fused features, problems such as missed detection of small targets, false detection of the background, and target box offset are prone to occur. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an image target detection method based on multi-scale attention feature fusion, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention provides an image target detection method based on multi-scale attention feature fusion, comprising the following steps: S1. Receive the input image and perform image processing to obtain a standard image. Extract the image features of the standard image and organize them into a unified feature coordinate system to obtain scale-organized features. Then, generate edge response maps, structural response maps, and semantic response maps based on the scale-organized features to form cross-scale response samples. S2. Calculate the coordinate displacement cost and structural path cost based on the cross-scale response samples and add them together to obtain the structural constraint transmission cost. After arranging them, output the structural constraint transmission cost matrix and combine it with the edge response map and semantic response map to perform entropy regularization transformation and alternating scaling to obtain the structural constraint transmission plan, calculate the response value and construct the structural constraint response migration map. S3. Based on the scale-organization features, establish the baseline scale fusion features, read the response values ​​of each spatial location in the structural constraint response migration map, multiply them with the feature values ​​of each channel of the scale-organization features and fuse them to obtain the migration-guided scale fusion features, and then perform response consistency analysis to obtain the baseline response consistency value and the guided response consistency value and determine the fusion route result. S4. Perform positional attention analysis on the fusion route results to obtain the first positional filtering features and the second positional filtering features, and determine the target-consistent fusion features. S5. Map the target output to the target consistent fusion features and determine the candidate confidence, region consistent response value and region response value. After multiplication, obtain the candidate retained value. Group the candidate retained values ​​into overlapping boxes and sort them. Determine the candidate target with the first position in the sort as the detection target.

[0006] Preferably, S1 includes S11 and S12; S11. Receive the input image, perform size adjustment and pixel normalization on the input image to obtain a standard image, and input the standard image into the backbone feature extraction network. Output shallow texture features, mid-level structural features and deep semantic features according to the processing order from shallow to deep. S12. Establish a scaling model based on a convolutional neural network and train the model. Input shallow texture features, mid-level structural features and deep semantic features into the trained scaling model, and output scaling features based on the scaling model. The scale-adjusted features include a first scale-adjusted feature, a second scale-adjusted feature, and a third scale-adjusted feature. The first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature are located in the same feature coordinate system. The same feature coordinate system means that the first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature have the same coordinate position, and the same coordinate position of the three is mapped to the same image region in the standard image.

[0007] Preferably, S1 also includes S13; S13. The first-scale features are compressed using the channel-wise arithmetic mean algorithm to obtain the first single-channel feature map. Then, the gradient of the first single-channel feature map is calculated using the Sobel edge detection algorithm to obtain the edge amplitude map. Finally, the edge amplitude map is normalized using the Softmax normalization algorithm to obtain the edge response map. The channel norm of the second-scale features is calculated along the channel direction using the Euclidean norm algorithm to obtain the structural amplitude map. Then, the structural amplitude map is normalized using the Softmax normalization algorithm to obtain the structural response map. The channel norm of the third-scale features is calculated along the channel direction using the Euclidean norm algorithm to obtain the semantic amplitude map. Then, the semantic amplitude map is normalized using the Softmax normalization algorithm to obtain the semantic response map. Wherein, the channel response represents the feature values ​​of the first-scale organized features, the second-scale organized features, and the third-scale organized features at any position in the width and height directions; The edge response map, structural response map, and semantic response map are combined to form a cross-scale response sample. The edge response map, structural response map, and semantic response map inherit the feature coordinate system. The feature coordinate positions with the same row coordinates and column coordinates in the edge response map, structural response map, and semantic response map all point to the same image region in the standard image. The feature coordinate system includes the row direction and column direction of each response map. The row direction coordinates and column direction coordinates constitute the feature coordinate positions.

[0008] Preferably, S2 includes S21; S21. Take each feature coordinate position in the edge response map as the transmission start point and each feature coordinate position in the semantic response map as the transmission end point, and calculate the structural constraint transmission cost from the transmission start point to the transmission end point. Structural constraint transmission costs include coordinate displacement costs and structural path costs; Coordinate displacement cost: Read the row and column coordinates of the transmission start point and transmission end point in the same characteristic coordinate system, calculate the absolute value of the difference between the row coordinates and the absolute value of the difference between the column coordinates, and add the two together to obtain the coordinate displacement cost; Structural path cost: A linear grid traversal method is used, with the row and column coordinates of the transmission start point as the starting coordinates and the row and column coordinates of the transmission end point as the ending coordinates. A grid path from the starting coordinates to the ending coordinates is generated in the same feature coordinate system. The feature coordinate positions contained in the grid path are recorded as the feature coordinate position set. The structural response map is sampled according to the feature coordinate position set to obtain the structural path sequence. The difference is processed by taking the value 1 as the minuend and each item in the structural path sequence as the subtrahend. All the difference results are summed to obtain the structural path cost. The coordinate displacement cost and the structural path cost are added together to obtain the structural constraint transmission cost from the transmission start point to the transmission end point; the structural constraint transmission costs are arranged according to all transmission start points and all transmission end points to obtain the structural constraint transmission cost matrix.

[0009] Preferably, S2 also includes S22 and S23; S22. Sort the edge response map and semantic response map according to the coordinate order in the same feature coordinate system to generate the edge response sequence and semantic response sequence respectively, and perform entropy regularization transformation according to the structural constraint transmission cost matrix to obtain the transmission kernel matrix. The Sinkhorn iterative method is used to alternately scale the transfer kernel matrix to obtain a structurally constrained transfer plan. This plan represents the proportion of response allocated from an edge position in the edge response sequence to a semantic position in the semantic response sequence under the constraints of the structural response map, as follows: Alternating scaling is used to repeatedly perform row scaling and column scaling based on a fixed number of iterations during the training phase; Row scaling processing: The transmission kernel matrix is ​​summed row by row to obtain a row sum value sequence. Each item in the edge response sequence is divided by the item with the same index in the row sum value sequence to obtain a row scaling coefficient sequence. Each item in the row scaling coefficient sequence is then multiplied by all items in the same row of the transmission kernel matrix to obtain a row-scaled transmission matrix. Column scaling processing: Summing column by column of the row scaling transmission matrix to obtain the column sum value sequence, and dividing each item in the semantic response sequence by the item with the same index in the column sum value sequence to obtain the column scaling coefficient sequence. Then, multiplying each item in the column scaling coefficient sequence with all items in the same column of the row scaling transmission matrix to obtain the column scaling transmission matrix. S23. Read the response proportion allocation values ​​of each feature coordinate position in the semantic response map from the structural constraint transfer plan, and for each feature coordinate position, accumulate all the response proportion allocation values ​​of the current feature coordinate position to obtain the response value of the current feature coordinate position. Then, perform response value calculation on all feature coordinate positions in the semantic response map to obtain the transfer response map. Perform point-by-point multiplication of the transfer response map and the structural response map to obtain the structural constraint transfer map. Perform point-by-point difference absolute value calculation on the structural constraint transfer map and the semantic response map to obtain the structural constraint response transfer map.

[0010] Preferably, S3 includes S31; S31. Establish a scale sequence fusion model based on a convolutional neural network and train the model. Input the first scale organized features, the second scale organized features and the third scale organized features into the scale sequence fusion model to obtain the baseline scale fusion features. Based on the same feature coordinate system, at any spatial position of the first-scale organized feature, the response value of the structural constraint response migration map at the same spatial position is read, and the current response value is multiplied with the feature values ​​of each channel of the first-scale organized feature at the same spatial position to obtain the first spatial guidance feature; according to the same processing method, the second-scale organized feature and the third-scale organized feature are processed to obtain the second spatial guidance feature and the third spatial guidance feature. The first spatial guiding feature, the second spatial guiding feature, and the third spatial guiding feature are input into the scale sequence fusion model, and the transfer-guided scale fusion feature is output.

[0011] Preferably, S3 also includes S32 and S33; S32. Based on the structural constraint response migration diagram, perform response consistency analysis on the baseline scale fusion characteristics and the migration-guided scale fusion characteristics to obtain the baseline response consistency value and the guided response consistency value. Response consistency analysis: The baseline single-channel fusion response is obtained by calculating the Euclidean norm of the baseline scale fusion feature along the channel direction. The baseline single-channel fusion response is then multiplied point-by-point with the response values ​​at the same spatial position in the structural constraint response migration diagram, and all product results are summed to obtain the baseline response consistency value. The guide scale fusion feature is also calculated by calculating the Euclidean norm along the channel direction to obtain the guide single-channel fusion response. The guide single-channel fusion response is then multiplied point-by-point with the response values ​​at the same spatial position in the structural constraint response migration diagram, and all product results are summed to obtain the guide response consistency value. S33. Determine the fusion route result based on the baseline response consistency value and the guide response consistency value; If the consistency value of the guiding response is greater than or equal to the consistency value of the baseline response, the fusion route result is the migration-guided fusion route; If the consistency value of the guiding response is less than the consistency value of the baseline response, then the fusion route result is the baseline fusion route.

[0012] Preferably, S4 includes S41, S42 and S43; S41. Determine the input features based on the fusion route results; When the fusion route result is the baseline fusion route, the baseline scale fusion features are taken as features to be noted; When the fusion path result is a transfer-guided fusion path, the transfer-guided scale fusion feature is used as the feature to be paid attention to, and the structural constraint response transfer map is used as the position guidance input; wherein, the structural constraint response transfer map and the transfer-guided scale fusion feature are located in the same feature coordinate system; S42. Perform global average pooling on the features to be paid attention to obtain the channel description results; perform one-dimensional convolution and sigmoid activation on the channel description results to obtain the channel attention results; perform channel-wise multiplication of the channel attention results with the features to be paid attention to obtain the channel filtering features. S43. Perform location attention analysis based on the fusion route results; When the fusion route result is the baseline fusion route, the channel filtering features are subjected to width-direction average pooling and height-direction average pooling to obtain the position attention result. The position attention result is then multiplied with the channel filtering features in a spatial position manner to obtain the first position filtering feature. When the fusion route result is a migration-guided fusion route, the structural constraint response migration map and the channel filtering feature are first multiplied by spatial position to obtain the migration-guided feature. Then, the migration-guided feature is subjected to average pooling in the width direction and average pooling in the height direction to obtain the position attention result. The position attention result is then multiplied by the migration-guided feature by spatial position to obtain the second position filtering feature. When the fusion route result is the baseline fusion route, the first position filter feature is used as the target consistent fusion feature; When the fusion route result is a migration-guided fusion route, the second position filtering feature is used as the target-consistent fusion feature.

[0013] Preferably, S5 includes S51; S51. Based on the target consistency fusion features, perform target output mapping to obtain category response features, location distance features, and confidence response features; wherein, target output mapping includes category mapping processing, location distance mapping processing, and confidence mapping processing; Category mapping processing: Category mapping convolution is used to perform convolutional mapping on the target consistent fusion features to obtain category response features; Location distance mapping processing: Location distance mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain location distance features; among them, the location distance features output four boundary distances at each feature coordinate position, and the four boundary distances represent the distances from the current feature coordinate position to the left boundary, top boundary, right boundary and bottom boundary of the target box, respectively; Confidence mapping processing: Confidence mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain confidence response features; whereby the confidence response features are used to represent the degree of response of the target at each feature coordinate position in the target consistent fusion features; For any feature coordinate position in the target consistency fusion features, read the response value of the category response feature at the current feature coordinate position, and take the category with the largest response value as the candidate category; read the distance of the location distance feature to the four boundaries at the current feature coordinate position, and generate candidate boxes based on the current feature coordinate position and the four boundary distances; read the response value of the confidence response feature at the current feature coordinate position as the candidate confidence; combine the candidate category, candidate box, and candidate confidence to form a candidate target. Perform target output mapping on all feature coordinate positions in the target consistency fusion feature to obtain a set of candidate targets.

[0014] Preferably, S5 also includes S52; S52. Calculate the Euclidean norm of the target consistency fusion feature along the channel direction to obtain the target consistency response map; for any candidate target in the candidate target set, read the candidate box coverage area of ​​the current candidate target, and read the response value within the candidate box coverage area in the target consistency response map, calculate the arithmetic mean of all response values ​​to obtain the region consistency response value; and read all response values ​​within the same candidate box coverage area in the structural constraint response transition map and calculate the arithmetic mean to obtain the region response value. The candidate confidence score, regional consensus response value, and regional response value of the current candidate target are multiplied to obtain the candidate retention value of the current candidate target; If the candidate bounding boxes of two candidate targets cover at least one of the same feature coordinate positions, then the two candidate targets are divided into the same overlapping box group, and the candidate targets in each overlapping box group are arranged in descending order of candidate retention value. The candidate target at the top of the sort is retained and marked as the detection target of the current overlapping group.

[0015] This invention provides an image target detection method based on multi-scale attention feature fusion. It has the following beneficial effects: (1) This method receives the input image and performs size adjustment and pixel normalization to obtain a standard image; then, it obtains shallow texture features, mid-level structural features, and deep semantic features through a backbone feature extraction network, and obtains first-scale adjusted features, second-scale adjusted features, and third-scale adjusted features through a scale adjustment model. The first-scale adjusted features, second-scale adjusted features, and third-scale adjusted features are located in the same feature coordinate system, so that the same feature coordinate positions can point to the same image region in the standard image. Based on this, an edge response map is generated according to the first-scale adjusted features, a structural response map is generated according to the second-scale adjusted features, and a semantic response map is generated according to the third-scale adjusted features. The edge response map, structural response map, and semantic response map are combined to form a cross-scale response sample. Through the above processing, the current method can retain the target edge information, target main structure information, and contextual semantic information respectively, and establish a clear data foundation for subsequent cross-scale response transfer and multi-scale feature fusion.

[0016] (2) This method performs structural constraint transmission analysis based on edge response maps, structural response maps, and semantic response maps. Each feature coordinate position in the edge response map is taken as the transmission starting point, and each feature coordinate position in the semantic response map is taken as the transmission ending point. The structural constraint transmission cost matrix is ​​obtained based on the coordinate displacement cost and the structural path cost. Then, based on the edge response sequence, semantic response sequence, and structural constraint transmission cost matrix, entropy regularization transformation and alternating scaling processing are performed to obtain the structural constraint transmission plan, and further, the migration response map, structural constraint migration map, and structural constraint response migration map are obtained. Through the above processing, when the target edge response in the shallow texture features migrates to the semantic response in the deep semantic features, it can be constrained by the target subject structure formed by the second-scale sorting features. This makes the edge response, structural response, and semantic response form a clearer spatial analysis basis in the same feature coordinate system. Compared with direct splicing or fixed weighted fusion, it is more suitable for processing multi-scale targets, occluded targets, and weak texture targets.

[0017] (3) This method establishes baseline scale fusion features based on the first scale features, second scale features, and third scale features, and obtains first spatial guidance features, second spatial guidance features, and third spatial guidance features based on the structural constraint response migration map. Then, the migration guidance scale fusion features are obtained by the scale sequence fusion model. After determining the fusion route result through the baseline response consistency value and the guidance response consistency value, the first position screening feature or the second position screening feature is obtained based on the fusion route result, and the target consistency fusion feature is determined. Finally, the category response feature, position distance feature, and confidence response feature are obtained based on the target consistency fusion feature, and the candidate retention value is obtained based on the candidate confidence, region consistency response value, and region response value. The candidate targets in the overlapping box group are sorted and the detection target is determined. Through the above processing, the current method can make the determination of the detection target refer to the target category response, target position distance, confidence response, target consistency response map, and region response status in the structural constraint response migration map, which improves the problems of candidate box offset, background false detection, small target missed detection, and multi-scale feature mismatch in complex backgrounds. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the steps of an image target detection method based on multi-scale attention feature fusion according to the present invention; Figure 2 This is a logic block diagram for judging an image target detection method based on multi-scale attention feature fusion according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1 Please see Figure 1 This invention provides an image target detection method based on multi-scale attention feature fusion. To achieve the above objectives, this invention employs the following technical solution, comprising the following steps: S1. Receive the input image and perform image processing to obtain a standard image. Extract the image features of the standard image and organize them into a unified feature coordinate system to obtain scale-organized features. Then, generate edge response maps, structural response maps, and semantic response maps based on the scale-organized features to form cross-scale response samples. S2. Calculate the coordinate displacement cost and structural path cost based on the cross-scale response samples and add them together to obtain the structural constraint transmission cost. After arranging them, output the structural constraint transmission cost matrix and combine it with the edge response map and semantic response map to perform entropy regularization transformation and alternating scaling to obtain the structural constraint transmission plan, calculate the response value and construct the structural constraint response migration map. S3. Based on the scale-organization features, establish the baseline scale fusion features, read the response values ​​of each spatial location in the structural constraint response migration map, multiply them with the feature values ​​of each channel of the scale-organization features and fuse them to obtain the migration-guided scale fusion features, and then perform response consistency analysis to obtain the baseline response consistency value and the guided response consistency value and determine the fusion route result. S4. Perform positional attention analysis on the fusion route results to obtain the first positional filtering features and the second positional filtering features, and determine the target-consistent fusion features. S5. Map the target output to the target consistent fusion features and determine the candidate confidence, region consistent response value and region response value. After multiplication, obtain the candidate retained value. Group the candidate retained values ​​into overlapping boxes and sort them. Determine the candidate target with the first position in the sort as the detection target.

[0021] In this embodiment, in the traffic monitoring image application scenario, the input image is first resized and pixel normalized by S1 to obtain a standard image. Then, the backbone feature extraction network extracts the image features of the standard image, and the shallow texture features, mid-level structural features, and deep semantic features are organized into a unified feature coordinate system to obtain scale-organized features. Based on this, an edge response map is generated according to the first scale-organized features, a structural response map is generated according to the second scale-organized features, and a semantic response map is generated according to the third scale-organized features. The edge response map, structural response map, and semantic response map together form a cross-scale response sample.

[0022] Through the above processing, vehicle edges, pedestrian contours, non-motorized vehicle body structures, and road environment semantic information in traffic monitoring images can participate in subsequent analysis at the same spatial location. Taking a road intersection image as an example, when a small electric vehicle in the distance, a bus in the foreground, and a pedestrian partially obscured by vehicles are simultaneously present in the image, shallow texture features can reflect edge information such as the electric vehicle wheels and pedestrian outlines, mid-level structural features can reflect the vehicle body and pedestrian torso areas, and deep semantic features can reflect the semantic distribution of roads, vehicles, and pedestrians in the traffic scene. This invention does not directly stitch together or weight features of different scales, but first analyzes the spatial location and response distribution of features of different scales, and then obtains cross-scale response samples, so that subsequent fusion processing has a unified spatial reference basis. This helps to improve the problems of feature location confusion, target edge deviation from semantic region, and background texture entering the target region when multiple scale targets coexist.

[0023] This invention further performs structural constraint transmission analysis on cross-scale response samples through S2. Specifically, each feature coordinate position in the edge response map is taken as the transmission start point, and each feature coordinate position in the semantic response map is taken as the transmission end point. First, the row and column coordinates of the transmission start and end points in a unified feature coordinate system are read, and the coordinate displacement cost is calculated. Then, a linear grid traversal method is used to sample the set of feature coordinate positions between the transmission start and end points in the structural response map to obtain a structural path sequence, and the structural path cost is calculated based on the structural path sequence. Then, the coordinate displacement cost and the structural path cost are added to obtain the structural constraint transmission cost matrix. Subsequently, the edge response map generates an edge response sequence, the semantic response map generates a semantic response sequence, the structural constraint transmission cost matrix is ​​transformed by entropy regularization to obtain a transmission kernel matrix, and then alternating scaling is used to obtain a structural constraint transmission plan. The response value is calculated based on the structural constraint transmission plan, and a structural constraint response migration map is obtained.

[0024] Taking a pedestrian partially obscured by a bus in a traffic monitoring image as an example, if an edge response location needs to be migrated to a pedestrian semantic response location, the current migration path will pass through the pedestrian's main structural area. The path response in the structural response map is relatively concentrated, and the structural path cost is low. However, if the migration path passes through road markings, tree shadows, or vehicle reflections, the path response in the structural response map is dispersed, and the structural path cost is high. Therefore, the structurally constrained response migration map can reflect the spatial consistency between edge responses, structural responses, and semantic responses. This allows the cross-scale response migration process to not only depend on distance or semantic strength but also consider the target's main structure, helping to improve erroneous response interference caused by background texture, occlusion boundaries, and weakly textured targets in traffic monitoring images.

[0025] This invention utilizes the structural constraint response migration map in steps S3 to S5 for feature fusion, attention analysis, and candidate target selection. First, a baseline scale fusion feature is established based on the scale-organized feature. The response values ​​at each spatial location of the structural constraint response migration map are then read, multiplied, and fused with the feature values ​​of each channel of the scale-organized feature to obtain the migration-guided scale fusion feature. Next, response consistency analysis is performed on the baseline scale fusion feature and the migration-guided scale fusion feature based on the structural constraint response migration map to obtain baseline and guided response consistency values, and the fusion path result is determined. Finally, channel attention processing and position attention analysis are performed based on the fusion path result to obtain either a first position selection feature or a second position selection feature, and the target consistency fusion feature is determined. Finally, target output mapping is performed on the target consistency fusion features to obtain category response features, location distance features, and confidence response features. Candidate categories are determined based on the category response features, candidate boxes are generated based on the location distance features, and candidate confidence is determined based on the confidence response features. Then, region consistency response values ​​are obtained based on the target consistency response map, and region response values ​​are obtained based on the structural constraint response transition map. The candidate confidence, region consistency response values, and region response values ​​are multiplied to obtain candidate retention values. The candidate retention values ​​are grouped into overlapping boxes and sorted, and the candidate target with the highest ranking is determined as the detection target.

[0026] Taking a distant small electric vehicle in a traffic monitoring image as an example, if a candidate box has a high candidate confidence, but the target consistency response map within the candidate box's coverage area is scattered and the regional response value in the structural constraint response transition map is low, then the candidate retention value of the current candidate target will be limited. If another candidate box covers the main area of ​​the electric vehicle, and both the target consistency response map and the structural constraint response transition map have concentrated responses within the coverage area of ​​the current candidate box, then the current candidate target is more likely to be retained in the overlapping box grouping and sorting. Compared with the existing technology that mainly relies on category response features and confidence response features to select candidate targets, this invention uses the structural constraint response transition map, target consistency fusion features, regional consistency response value, and regional response value together for candidate target judgment, making the detected target closer to the real target main area in the traffic monitoring image. This helps to improve the problems of missed detection, false detection, and candidate box offset of small electric vehicles, occluded pedestrians, and vehicle targets in complex road backgrounds.

[0027] Example 2 Please refer to Figure 2 Based on Example 1, specifically: S1 includes S11 and S12; S11. Receive the input image, perform size adjustment and pixel normalization on the input image to obtain a standard image, and input the standard image into the backbone feature extraction network. Output shallow texture features, mid-level structural features and deep semantic features according to the processing order from shallow to deep. S12. Establish a scaling model based on the convolutional neural network and train the model. Model Training: Historical standard image samples are collected as the training set and input into the convolutional neural network for forward computation to obtain shallow texture features, mid-level structural features, and deep semantic features. These features are then input into the scaling model, which outputs training scaling features. The training scaling features and standard scaling features are compared using a loss function to calculate the training loss. The first, second, third, fourth, fifth, and sixth convolutional units in the scaling model are then updated in reverse. When the training loss meets the error range condition, the trained scaling model is obtained. The first, second, third, fourth, fifth, and sixth convolutional units each include a convolutional layer, a normalization layer, and an activation layer. The shallow texture features, mid-level structural features, and deep semantic features are input into the trained scale-scaling model, and the scale-scaling model outputs scale-scaling features, as follows: The shallow texture features are processed by a first-scale processing branch using a first convolution. A parallel downsampling unit then performs max pooling and average pooling on the features processed by the first convolution, outputting a downsampled and concatenated feature. A second convolution unit performs a second convolution on the downsampled and concatenated feature to obtain the first-scale processed feature. The mid-level structural features are then processed by a second-scale processing branch using a second convolution, yielding the second-scale processed feature. The deep semantic features are then processed by a third-scale processing branch using a fourth convolution. A parallel downsampling unit then performs max pooling and average pooling on the features processed by the fourth convolution, outputting a downsampled and concatenated feature. A fifth convolution unit performs a fifth convolution on the downsampled and concatenated feature to obtain the third-scale processed feature. A channel concatenation unit concatenates the first-scale processed feature, the second-scale processed feature, and the third-scale processed feature along the channel direction. The scale-adjusted features include a first scale-adjusted feature, a second scale-adjusted feature, and a third scale-adjusted feature. The first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature are located in the same feature coordinate system. The channel direction represents the feature channel dimension direction in the first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature. The same feature coordinate system means that the first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature have the same coordinate position, and the same coordinate position of the three is mapped to the same image region in the standard image.

[0028] S1 also includes S13; S13. The first-scale features are compressed using the channel-wise arithmetic mean algorithm to obtain the first single-channel feature map. Then, the gradient of the first single-channel feature map is calculated using the Sobel edge detection algorithm to obtain the edge amplitude map. Finally, the edge amplitude map is normalized using the Softmax normalization algorithm to obtain the edge response map. Channel compression is performed as follows: For each spatial location of the first-scale organized features, all channel responses of each spatial location of the first-scale organized features are read and the arithmetic mean is calculated according to the channel-wise arithmetic mean algorithm. The arithmetic mean is used as the pixel value of the current spatial location. The Sobel edge detection algorithm includes using a horizontal Sobel convolution kernel to obtain the horizontal gradient and a vertical Sobel convolution kernel to obtain the vertical gradient. The absolute values ​​of the horizontal and vertical gradients are added together to obtain the edge amplitude map. The normalization process is performed as follows: The Softmax normalization algorithm is used to perform exponential operations on all pixel responses in the edge amplitude map, and the sum of all exponential operation results is used as the normalization benchmark. The channel norm of the second-scale features is calculated along the channel direction using the Euclidean norm algorithm to obtain the structural amplitude map. Then, the structural amplitude map is normalized using the Softmax normalization algorithm to obtain the structural response map. The channel norm is calculated as follows: read all channel responses at each spatial location of the second-scale organized features, calculate the square value of each channel response according to the Euclidean norm algorithm, sum the square values ​​and perform a square root operation, and use the square root operation result as the structural amplitude of the current spatial location; the normalization process is as follows: perform exponential operations on all structural amplitudes in the structural amplitude map according to the Softmax normalization algorithm, and use the sum of all exponential operation results as the normalization benchmark to obtain the structural response map; The channel norm of the third-scale features is calculated along the channel direction using the Euclidean norm algorithm to obtain the semantic amplitude map. Then, the semantic amplitude map is normalized using the Softmax normalization algorithm to obtain the semantic response map. The channel norm is calculated as follows: at each spatial location of the features organized in the third scale, all channel responses at the current spatial location are read, and the square value of each channel response is calculated according to the Euclidean norm algorithm. The square values ​​are summed and then the square root is calculated. The result of the square root operation is used as the semantic amplitude of the current spatial location. The normalization process is as follows: according to the Softmax normalization algorithm, all semantic amplitudes in the semantic amplitude map are exponentially operated, and the sum of all exponential operation results is used as the normalization benchmark to obtain the semantic response map. Wherein, the channel response represents the feature values ​​of the first-scale organized features, the second-scale organized features, and the third-scale organized features at any position in the width and height directions; The edge response map, structural response map, and semantic response map are combined to form a cross-scale response sample. The edge response map, structural response map, and semantic response map inherit the feature coordinate system. The feature coordinate positions with the same row coordinates and column coordinates in the edge response map, structural response map, and semantic response map all point to the same image region in the standard image. The feature coordinate system includes the row direction and column direction of each response map. The row direction coordinates and column direction coordinates form the feature coordinate positions. Among them, the edge response map is used to represent the target edge response in the first-scale organized features; the structural response map is used to represent the target subject response in the second-scale organized features; and the semantic response map is used to represent the contextual semantic response in the third-scale organized features.

[0029] In the process of generating edge response maps based on the first scale, structural response maps based on the second scale, and semantic response maps based on the third scale, channel compression, Sobel gradient operation, Euclidean norm operation, and Softmax normalization operation do not change the row and column coordinates of the feature maps. Therefore, the edge response maps, structural response maps, and semantic response maps inherit the feature coordinate system.

[0030] In this embodiment, the input image is first received and its size is adjusted and pixels are normalized to obtain a standard image. Then, the standard image is input into the backbone feature extraction network, and shallow texture features, mid-level structural features and deep semantic features are output according to the processing order from shallow to deep. Subsequently, the scale differences, channel differences, and spatial coordinate differences of shallow texture features, mid-level structural features, and deep semantic features are analyzed. Then, the trained scale-sorting model is used to process the first, second, and third scale-sorting branches to obtain the first, second, and third scale-sorting features, and these features are located in the same feature coordinate system. Specifically, the first scale-sorting branch obtains the first scale-sorting features through the first convolution, max pooling, average pooling, downsampling and concatenation, and second convolution. The second scale-sorting branch obtains the second scale-sorting features through convolution. The third scale-sorting branch obtains the third scale-sorting features through the fourth convolution, max pooling, average pooling, downsampling and concatenation, and fifth convolution. Subsequently, based on the first-scale feature processing, a channel-by-channel arithmetic mean is performed on all channel responses to obtain the first single-channel feature map. Then, the Sobel edge detection algorithm is used to obtain the edge magnitude map, and the Softmax normalization algorithm is used to obtain the edge response map. Based on the second-scale feature processing, Euclidean norm calculation is performed along the channel direction to obtain the structure magnitude map, and the Softmax normalization algorithm is used to obtain the structure response map. Based on the third-scale feature processing, Euclidean norm calculation is performed along the channel direction to obtain the semantic magnitude map, and the Softmax normalization algorithm is used to obtain the semantic response map. Since channel compression, Sobel gradient operation, Euclidean norm operation, and Softmax normalization operation do not change the row and column coordinates of the feature map, the edge response map, structure response map, and semantic response map inherit the feature coordinate system. Feature coordinates with the same row and column coordinates all point to the same image region in the standard image.

[0031] Through the above implementation process, the current method can organize the target edge response, target subject response, and contextual semantic response into a unified spatial reference before entering cross-scale transmission and feature fusion. This solves problems such as spatial location confusion, deviation between edge details and semantic regions, and insufficient participation of subject structural information that are prone to occur when features at different scales are directly stitched together or simply weighted. For example, in an image containing vehicles, pedestrians, and road backgrounds, the first-scale organized features can reflect details such as vehicle edges and pedestrian outlines, the second-scale organized features can reflect the vehicle subject and pedestrian torso regions, and the third-scale organized features can reflect the target category semantics and the semantics of the surrounding environment. After the edge response map, structural response map, and semantic response map form a cross-scale response sample under the same feature coordinate system, the subsequent structural constraint transmission cost matrix, structural constraint transmission plan, structural constraint response migration map, baseline scale fusion features, migration-guided scale fusion features, and target consistency fusion features can all be processed based on the same spatial foundation. This also improves the problem that small targets, occluded targets, and weakly textured targets are easily interfered with by background responses during multi-scale feature fusion.

[0032] Example 3 Please refer to Figure 2 Based on Example 2, specifically: S2 includes S21; S21. Take each feature coordinate position in the edge response map as the transmission start point and each feature coordinate position in the semantic response map as the transmission end point, and calculate the structural constraint transmission cost from the transmission start point to the transmission end point. Structural constraint transmission costs include coordinate displacement costs and structural path costs; Coordinate displacement cost: Read the row and column coordinates of the transmission start point and transmission end point in the same characteristic coordinate system, calculate the absolute value of the difference between the row coordinates and the absolute value of the difference between the column coordinates, and add the two together to obtain the coordinate displacement cost; Structural path cost: A linear grid traversal method is used, with the row and column coordinates of the transmission start point as the starting coordinates and the row and column coordinates of the transmission end point as the ending coordinates. A grid path from the starting coordinates to the ending coordinates is generated in the same feature coordinate system. The feature coordinate positions contained in the grid path are recorded as the feature coordinate position set. The structural response map is sampled according to the feature coordinate position set to obtain the structural path sequence. The difference is processed by taking the value 1 as the minuend and each item in the structural path sequence as the subtrahend. All the difference results are summed to obtain the structural path cost. The coordinate displacement cost and the structural path cost are added together to obtain the structural constraint transmission cost from the transmission start point to the transmission end point; the structural constraint transmission costs are arranged according to all transmission start points and all transmission end points to obtain the structural constraint transmission cost matrix.

[0033] S2 also includes S22 and S23; S22. Sort the edge response map and semantic response map according to the coordinate order in the same feature coordinate system to generate the edge response sequence and semantic response sequence respectively, and perform entropy regularization transformation according to the structural constraint transmission cost matrix to obtain the transmission kernel matrix. In this process, the rows of the transmission kernel matrix are matched one-to-one with each item in the edge response sequence, and the columns of the transmission kernel matrix are matched one-to-one with each item in the semantic response sequence. The Sinkhorn iterative method is used to alternately scale the transfer kernel matrix to obtain a structurally constrained transfer plan. This plan represents the proportion of response allocated from an edge position in the edge response sequence to a semantic position in the semantic response sequence under the constraints of the structural response map, as follows: Alternating scaling is used to repeatedly perform row scaling and column scaling based on a fixed number of iterations during the training phase; Row scaling processing: The transmission kernel matrix is ​​summed row by row to obtain a row sum value sequence. Each item in the edge response sequence is divided by the item with the same index in the row sum value sequence to obtain a row scaling coefficient sequence. Each item in the row scaling coefficient sequence is then multiplied by all items in the same row of the transmission kernel matrix to obtain a row-scaled transmission matrix. Column scaling processing: Summing column by column of the row scaling transmission matrix to obtain the column sum value sequence, and dividing each item in the semantic response sequence by the item with the same index in the column sum value sequence to obtain the column scaling coefficient sequence. Then, multiplying each item in the column scaling coefficient sequence with all items in the same column of the row scaling transmission matrix to obtain the column scaling transmission matrix. S23. Read the response proportion allocation values ​​of each feature coordinate position in the semantic response map from the structural constraint transfer plan, and for each feature coordinate position, accumulate all the response proportion allocation values ​​of the current feature coordinate position to obtain the response value of the current feature coordinate position. Then, perform response value calculation on all feature coordinate positions in the semantic response map to obtain the transfer response map. Perform point-by-point multiplication of the transfer response map and the structural response map to obtain the structural constraint transfer map. Perform point-by-point difference absolute value calculation on the structural constraint transfer map and the semantic response map to obtain the structural constraint response transfer map.

[0034] In this embodiment, after obtaining the edge response map, structural response map, and semantic response map, the spatial distribution of each feature coordinate position in the edge response map and each feature coordinate position in the semantic response map under the same feature coordinate system is analyzed first. Then, each feature coordinate position in the edge response map is taken as the transmission start point, and each feature coordinate position in the semantic response map is taken as the transmission end point. The row coordinates and column coordinates of the transmission start point and the transmission end point are read respectively, the absolute value of the difference between the row coordinates and the absolute value of the difference between the column coordinates are calculated, and the two are added together to obtain the coordinate displacement cost. At the same time, a linear grid traversal method is adopted, with the row coordinates and column coordinates of the transmission start point as the starting coordinates and the transmission end point as the starting coordinates. The row and column coordinates are used as the termination coordinates. A grid path is generated in the same feature coordinate system, and the feature coordinate positions contained in the grid path are recorded as the feature coordinate position set. The structural response map is sampled according to the feature coordinate position set to obtain the structural path sequence. Then, the difference is processed by taking 1 as the minuend and each item in the structural path sequence as the subtrahend. All difference results are summed to obtain the structural path cost. Then, the coordinate displacement cost and the structural path cost are added to obtain the structural constraint transmission cost from the transmission start point to the transmission end point. The structural constraint transmission costs are arranged according to all transmission start points and all transmission end points to obtain the structural constraint transmission cost matrix. Subsequently, the edge response map and semantic response map are sorted according to the coordinate order in the same feature coordinate system to generate edge response sequences and semantic response sequences, respectively. An entropy regularization transformation is performed based on the structural constraint transfer cost matrix to obtain the transfer kernel matrix. Then, the Sinkhorn iterative method is used to repeatedly perform row scaling and column scaling on the transfer kernel matrix for a fixed number of iterations during the training phase to obtain the structural constraint transfer plan. The structural constraint transfer plan represents the response proportion allocation value of the edge positions in the edge response sequence to the semantic positions in the semantic response sequence under the constraints of the structural response map. Finally, the response proportion allocation values ​​of each feature coordinate position in the semantic response map are read from the structural constraint transfer plan. For each feature coordinate position, all response proportion allocation values ​​of the current feature coordinate position are accumulated to obtain the response value of the current feature coordinate position. Response value calculation is performed on all feature coordinate positions in the semantic response map to obtain the transfer response map. Then, the transfer response map and the structural response map are multiplied point-by-point to obtain the structural constraint transfer map. Finally, the structural constraint transfer map and the semantic response map are calculated using point-by-point absolute difference calculation to obtain the structural constraint response transfer map.

[0035] Through the above implementation process, the current method can simultaneously refer to spatial distance and target subject structure when migrating from edge response to semantic response. This makes the transmission path that is close but passes through the background texture area subject to structural path cost constraints, so that the transmission path that passes through the target subject area can obtain a more reasonable response proportion allocation value in the structural constraint transmission plan. For example, when a target edge position is spatially close to multiple semantic positions, if the grid path between one of the semantic positions and the current target edge position passes through the target subject area in the structural response map, the structural path cost of the current path is small. Conversely, if the grid path passes through background lines, shadows, or non-target texture areas, the structural path cost of the current path is large.

[0036] Example 4 Please refer to Figure 2 Based on Example 3, specifically: S3 includes S31; S31. Establish a scale sequence fusion model based on a convolutional neural network and train the model. Model Training: Collect historical standard image samples as training set and input them into the convolutional neural network for forward computation to obtain first-scale, second-scale, and third-scale features. Input the first-scale, second-scale, and third-scale features into the scale sequence fusion model. Based on the scale sequence fusion model, output the training scale fusion features. Then, calculate the training loss using the loss function based on the training scale fusion features and the standard image annotation data to update the scale sequence fusion model in reverse. When the training loss meets the error range, the trained scale sequence fusion model is obtained. The first-scale, second-scale, and third-scale processed features are input into the scale sequence fusion model to obtain the baseline scale fusion features, as follows: Based on a convolutional neural network, a channel-organizing convolutional layer, a scale-stacking layer, a 3D convolutional layer, a normalization layer, and an activation layer are established. The channel-organizing convolutional layer organizes the first-scale organized features, the second-scale organized features, and the third-scale organized features into the same number of channels. The scale-stacking layer stacks the channel-organized first-scale organized features, the second-scale organized features, and the third-scale organized features along the scale dimension to obtain scale-stacking features. The 3D convolutional layer performs 3D convolution processing on the scale-stacking features in the scale dimension, height direction, and width direction to obtain scale convolutional features. The normalization layer normalizes the scale convolutional features to obtain the baseline scale fusion features. Based on the same feature coordinate system, at any spatial position of the first-scale organized feature, the response value of the structural constraint response migration map at the same spatial position is read, and the current response value is multiplied with the feature values ​​of each channel of the first-scale organized feature at the same spatial position to obtain the first spatial guidance feature; according to the same processing method, the second-scale organized feature and the third-scale organized feature are processed to obtain the second spatial guidance feature and the third spatial guidance feature. The first spatial guiding feature, the second spatial guiding feature, and the third spatial guiding feature are input into the scale sequence fusion model, and the transfer-guided scale fusion feature is output.

[0037] S3 also includes S32 and S33; S32. Based on the structural constraint response migration diagram, perform response consistency analysis on the baseline scale fusion characteristics and the migration-guided scale fusion characteristics to obtain the baseline response consistency value and the guided response consistency value. Response consistency analysis: The baseline single-channel fusion response is obtained by calculating the Euclidean norm of the baseline scale fusion feature along the channel direction. The baseline single-channel fusion response is then multiplied point-by-point with the response values ​​at the same spatial position in the structural constraint response migration diagram, and all product results are summed to obtain the baseline response consistency value. The guide scale fusion feature is also calculated by calculating the Euclidean norm along the channel direction to obtain the guide single-channel fusion response. The guide single-channel fusion response is then multiplied point-by-point with the response values ​​at the same spatial position in the structural constraint response migration diagram, and all product results are summed to obtain the guide response consistency value. S33. Determine the fusion route result based on the baseline response consistency value and the guide response consistency value; If the consistency value of the guiding response is greater than or equal to the consistency value of the baseline response, the fusion route result is the migration-guided fusion route; If the consistency value of the guiding response is less than the consistency value of the baseline response, then the fusion route result is the baseline fusion route.

[0038] In this embodiment, the channel differences, scale differences, and spatial response distributions of the first-scale, second-scale, and third-scale features are first analyzed. Then, a scale sequence fusion model is established based on a convolutional neural network, and the model is trained using historical standard image samples. During training, historical standard image samples are input into the convolutional neural network for forward computation to obtain the first-scale, second-scale, and third-scale features. These features are then input into the scale sequence fusion model, which outputs the trained scale fusion features. The training loss is calculated based on the training scale fusion features and standard image annotation data using a loss function. This training loss is then used to back-update the scale sequence fusion model, resulting in the trained model. In actual detection, the first, second, and third scale-sorted features are input into the trained model. A channel-sorting convolutional layer first sorts these features to the same number of channels. A scale-stacking layer then stacks these sorted features along the scale dimension to obtain scale-stacking features. The 3D convolutional layer then processes the scale-stacking features... The system performs 3D convolution processing on the scale dimension, height direction, and width direction to obtain scale convolution features. A normalization layer normalizes these scale convolution features to obtain baseline scale fusion features. Subsequently, based on the same feature coordinate system, the response values ​​of the structural constraint response migration map at the same spatial position in the first, second, and third scale features are read. The current response value is then multiplied by the channel feature values ​​at the same spatial position in each of the first, second, and third scale features to obtain the first, second, and third spatial guided features. Finally, the first spatial guided feature... The second and third spatial guiding features are input into the scale sequence fusion model to obtain the migration guiding scale fusion features. Then, the Euclidean norm of the baseline scale fusion features is calculated along the channel direction to obtain the baseline single-channel fusion response. The baseline single-channel fusion response and the response values ​​at the same spatial position in the structural constraint response migration diagram are then multiplied point by point and summed to obtain the baseline response consistency value. At the same time, the Euclidean norm of the migration guiding scale fusion features is calculated along the channel direction to obtain the guiding single-channel fusion response. The guiding single-channel fusion response and the response values ​​at the same spatial position in the structural constraint response migration diagram are then multiplied point by point and summed to obtain the guiding response consistency value.Finally, the fusion route result is determined based on the baseline response consistency value and the bootstrap response consistency value. When the bootstrap response consistency value is greater than or equal to the baseline response consistency value, the fusion route result is the migration bootstrap fusion route; when the bootstrap response consistency value is less than the baseline response consistency value, the fusion route result is the baseline fusion route.

[0039] Through the above implementation, the current method does not fixate on using a single route to fuse multi-scale features. Instead, it first analyzes the response consistency between the baseline scale fusion features, the migration-guided scale fusion features, and the structural constraint response migration map, and then obtains the fusion route result. This allows the scale sequence fusion process to select a route based on the spatial consistency of the target edge response, the target subject response, and the contextual semantic response. For example, in an image containing a large nearby target, a small distant target, and a partially occluded target, if the structural constraint response migration map shows a concentrated response in the distant small target subject region, and the migration-guided scale fusion features are more consistent with the current concentrated response region, then the fusion route result selects the migration-guided fusion route, allowing subsequent positional attention analysis to continue processing around the current target subject region. If the baseline scale fusion features have better spatial consistency with the structural constraint response migration map, then the fusion route result selects the baseline fusion route, preserving the multi-scale integrated information in the baseline scale fusion features. Therefore, the current method can improve the problem of easy mixing between shallow details, mid-level structure, and deep semantics under a fixed fusion method, making the multi-scale feature fusion result closer to the target subject region.

[0040] Example 5 Please refer to Figure 2 Based on Example 4, specifically: S4 includes S41, S42 and S43; S41. Determine the input features based on the fusion route results; When the fusion route result is the baseline fusion route, the baseline scale fusion features are taken as features to be noted; When the fusion path result is a transfer-guided fusion path, the transfer-guided scale fusion feature is used as the feature to be paid attention to, and the structural constraint response transfer map is used as the position guidance input; wherein, the structural constraint response transfer map and the transfer-guided scale fusion feature are located in the same feature coordinate system; S42. Perform global average pooling on the features to be paid attention to obtain channel description results; perform one-dimensional convolution and sigmoid activation on the channel description results to obtain channel attention results; perform channel attention results and channel-wise multiplication on the features to be paid attention to obtain channel selection features; wherein, channel-wise multiplication means that the weight of each channel in the channel attention results is multiplied with all spatial position feature values ​​of the same channel in the features to be paid attention. S43. Perform location attention analysis based on the fusion route results; When the fusion route result is the baseline fusion route, the channel filtering features are subjected to width-direction average pooling and height-direction average pooling to obtain the position attention result. The position attention result is then multiplied with the channel filtering features in a spatial position manner to obtain the first position filtering feature. When the fusion route result is a migration-guided fusion route, the structural constraint response migration map and the channel filtering features are first multiplied by spatial location to obtain the migration guidance features. Then, the migration guidance features are subjected to average pooling in the width direction and average pooling in the height direction to obtain the position attention result. The position attention result is then multiplied by spatial location with the migration guidance features to obtain the second position filtering feature. The spatial location multiplication process refers to multiplying the response value at any spatial location in the position attention result or the structural constraint response migration map with all channel feature values ​​at the same spatial location in the channel filtering feature or the migration guidance feature. When the fusion route result is the baseline fusion route, the first position filter feature is used as the target consistent fusion feature; When the fusion route result is a migration-guided fusion route, the second position filtering feature is used as the target-consistent fusion feature.

[0041] In this embodiment, the input features are first analyzed based on the fusion path results, and then the features to be noted are obtained. When the fusion path result is a baseline fusion path, the baseline scale fusion features are used as the features to be noted; when the fusion path result is a transfer-guided fusion path, the transfer-guided scale fusion features are used as the features to be noted, and the structural constraint response transfer map is used as the position-guided input. Subsequently, global average pooling is performed on the features to be noted to obtain channel description results. Then, one-dimensional convolution and sigmoid activation are performed on the channel description results to obtain channel attention results. Finally, each channel weight in the channel attention results is multiplied channel-by-channel with all spatial position feature values ​​of the same channel in the features to be noted to obtain channel selection features. Next, positional attention analysis is performed based on the fusion route results. When the fusion route result is the baseline fusion route, the channel filtering features are subjected to width-direction average pooling and height-direction average pooling to obtain positional attention results. The positional attention results are then multiplied with the channel filtering features in a spatially positional manner to obtain the first positional filtering feature. When the fusion route result is a migration-guided fusion route, the structural constraint response migration map is first multiplied with the channel filtering features in a spatially positional manner to obtain the migration-guided feature. Then, the migration-guided feature is subjected to width-direction average pooling and height-direction average pooling to obtain the positional attention result. The positional attention result is then multiplied with the migration-guided feature in a spatially positional manner to obtain the second positional filtering feature. Finally, when the fusion route result is the baseline fusion route, the first positional filtering feature is used as the target consistency fusion feature. When the fusion route result is a migration-guided fusion route, the second positional filtering feature is used as the target consistency fusion feature.

[0042] Through the above implementation methods, the current method can employ different positional attention analysis methods between the baseline fusion route and the migration-guided fusion route, so that the target consistency fusion features can simultaneously retain target category discrimination information at the channel level and target region pointing information at the spatial level. Compared with the existing technology that only performs single-channel attention or single-space attention processing on the fusion features, the current method uses the fusion route results and structural constraint response migration map to select the attention processing path, so that the first position filtering feature, the second position filtering feature and the target consistency fusion feature are closer to the target body region, improving the problems of background texture participating in target judgment, weak response targets being weakened by channel filtering, and position judgment deviation caused by occlusion boundaries.

[0043] Example 6 Please refer to Figure 2 Based on Example 5, specifically: S5 includes S51; S51. Based on the target consistency fusion features, perform target output mapping to obtain category response features, location distance features, and confidence response features; wherein, target output mapping includes category mapping processing, location distance mapping processing, and confidence mapping processing; Category mapping processing: Category mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain category response features; whereby the category response features are used to represent the target category response at the coordinate positions of each feature in the target consistent fusion features; Location distance mapping processing: Location distance mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain location distance features; among them, the location distance features output four boundary distances at each feature coordinate position, and the four boundary distances represent the distances from the current feature coordinate position to the left boundary, top boundary, right boundary and bottom boundary of the target box, respectively; Confidence mapping processing: Confidence mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain confidence response features; whereby the confidence response features are used to represent the degree of response of the target at each feature coordinate position in the target consistent fusion features; For any feature coordinate position in the target consistency fusion features, read the response value of the category response feature at the current feature coordinate position, and select the category with the largest response value as the candidate category; read the four boundary distances of the position distance feature at the current feature coordinate position, and generate candidate boxes based on the current feature coordinate position and the four boundary distances, specifically: subtract the left boundary distance from the column coordinate of the current feature coordinate position to obtain the left boundary coordinate of the candidate box; subtract the top boundary distance from the row coordinate of the current feature coordinate position to obtain the top boundary coordinate of the candidate box; add the right boundary distance to the column coordinate of the current feature coordinate position to obtain the right boundary coordinate of the candidate box; add the bottom boundary distance to the row coordinate of the current feature coordinate position to obtain the bottom boundary coordinate of the candidate box; read the response value of the confidence response feature at the current feature coordinate position as the candidate confidence; combine the candidate category, candidate box, and candidate confidence to form a candidate target. Perform target output mapping on all feature coordinate positions in the target consistency fusion feature to obtain a set of candidate targets.

[0044] S5 also includes S52; S52. Calculate the Euclidean norm of the target consistency fusion feature along the channel direction to obtain the target consistency response map; for any candidate target in the candidate target set, read the candidate box coverage area of ​​the current candidate target, and read the response value within the candidate box coverage area in the target consistency response map, calculate the arithmetic mean of all response values ​​to obtain the region consistency response value; and read all response values ​​within the same candidate box coverage area in the structural constraint response transition map and calculate the arithmetic mean to obtain the region response value. The candidate confidence score, regional consensus response value, and regional response value of the current candidate target are multiplied to obtain the candidate retention value of the current candidate target; If the candidate bounding boxes of two candidate targets cover at least one of the same feature coordinate positions, then the two candidate targets are divided into the same overlapping box group, and the candidate targets in each overlapping box group are arranged in descending order of candidate retention value. The candidate target at the top of the sort is retained and marked as the detection target of the current overlapping group.

[0045] In this embodiment, target output mapping is first performed based on the target consistency fusion features. Category response features, location distance features, and confidence response features are obtained through category mapping processing, location distance mapping processing, and confidence mapping processing, respectively. Among them, the category response features are used to represent the target category response at each feature coordinate position in the target consistency fusion features, the location distance features are used to output the left boundary distance, upper boundary distance, right boundary distance, and lower boundary distance at each feature coordinate position, and the confidence response features are used to represent the degree of response of the presence of a target at each feature coordinate position in the target consistency fusion features. Subsequently, for any feature coordinate position in the target consistency fusion feature, the response value of the category response feature at the current feature coordinate position is read, and the category with the largest response value is selected as the candidate category; the distance of the position distance feature at the current feature coordinate position is read, and candidate boxes are generated based on the row coordinates, column coordinates, and left, top, right, and bottom boundary distances of the current feature coordinate position; the response value of the confidence response feature at the current feature coordinate position is read as the candidate confidence, and the candidate category, candidate box, and candidate confidence are combined into a candidate target. After performing the above processing on all feature coordinate positions in the target consistency fusion feature, a set of candidate targets is obtained. Next, the Euclidean norm of the target consistency fusion features is calculated along the channel direction to obtain the target consistency response map. For any candidate target in the candidate target set, the candidate box coverage area of ​​the current candidate target is read, and the response value within the candidate box coverage area is read in the target consistency response map. The arithmetic mean of all response values ​​is calculated to obtain the region consistency response value. At the same time, all response values ​​within the same candidate box coverage area are read in the structural constraint response transition map and the arithmetic mean is calculated to obtain the region response value. Then, the candidate confidence, region consistency response value, and region response value of the current candidate target are multiplied to obtain the candidate retention value of the current candidate target. If the candidate boxes of two candidate targets cover at least one identical feature coordinate position, the two candidate targets are assigned to the same overlapping box group. The candidate targets within each overlapping box group are arranged in descending order of candidate retention value, and the candidate target at the top of the sort is retained and labeled as the detection target of the current overlapping group.Taking traffic monitoring images as an example, when there are small electric vehicles, cars, and partially occluded pedestrians in the same image, the target consistency fusion feature can serve as the input basis for the target output mapping. The category response feature can be used to distinguish candidate categories, the location distance feature can be used to determine the candidate box boundary, and the confidence response feature can be used to determine the response degree of the target at the current feature coordinate position. If a candidate box only covers part of the pedestrian's outer contour, but the category response feature gives a high pedestrian response, the current candidate target still needs to be analyzed again through the region consistency response value in the target consistency response map and the region response value in the structural constraint response migration map. If the target body response within the area covered by the current candidate box is not concentrated, the candidate retention value will not be ranked first in the overlapping box group. If another candidate box covers the pedestrian body area, and both the target consistency response map and the structural constraint response migration map show concentrated responses within the area covered by the current candidate box, then the current candidate target is more likely to be labeled as a detection target in the overlapping box grouping.

[0046] Through the above implementation method, the current step undertakes the tasks of candidate category judgment, candidate box generation, candidate confidence acquisition, candidate target set construction, candidate retention value calculation, and overlapping box grouping and screening. This makes the detection target no longer rely solely on category response features or confidence response features for retention judgment, but simultaneously combines the region consistency response value formed by the target consistency fusion feature and the region response value formed by the structural constraint response migration map for screening. This helps to improve problems such as candidate boxes deviating from the target subject area, background areas being mistakenly used as target areas, duplicate candidate target retention caused by occlusion boundaries, and unstable candidate confidence of weak texture targets, so that the finally retained detection targets are closer to the real target subject area.

[0047] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended technical solutions and their equivalents.

Claims

1. An image target detection method based on multi-scale attention feature fusion, characterized in that, Includes the following steps: S1. Receive the input image and perform image processing to obtain a standard image. Extract the image features of the standard image and organize them into a unified feature coordinate system to obtain scale-organized features. Then, generate edge response maps, structural response maps, and semantic response maps based on the scale-organized features to form cross-scale response samples. S2. Calculate the coordinate displacement cost and structural path cost based on the cross-scale response samples and add them together to obtain the structural constraint transmission cost. After arranging them, output the structural constraint transmission cost matrix and combine it with the edge response map and semantic response map to perform entropy regularization transformation and alternating scaling to obtain the structural constraint transmission plan, calculate the response value and construct the structural constraint response migration map. S3. Based on the scale-organization features, establish the baseline scale fusion features, read the response values ​​of each spatial location in the structural constraint response migration map, multiply them with the feature values ​​of each channel of the scale-organization features and fuse them to obtain the migration-guided scale fusion features, and then perform response consistency analysis to obtain the baseline response consistency value and the guided response consistency value and determine the fusion route result. S4. Perform positional attention analysis on the fusion route results to obtain the first positional filtering features and the second positional filtering features, and determine the target-consistent fusion features. S5. Map the target output to the target consistent fusion features and determine the candidate confidence, region consistent response value and region response value. After multiplication, obtain the candidate retained value. Group the candidate retained values ​​into overlapping boxes and sort them. Determine the candidate target with the first position in the sort as the detection target.

2. The image target detection method based on multi-scale attention feature fusion according to claim 1, characterized in that, S1 includes S11 and S12; S11. Receive the input image, perform size adjustment and pixel normalization on the input image to obtain a standard image, and input the standard image into the backbone feature extraction network. Output shallow texture features, mid-level structural features and deep semantic features according to the processing order from shallow to deep. S12. Establish a scaling model based on a convolutional neural network and train the model. Input shallow texture features, mid-level structural features and deep semantic features into the trained scaling model, and output scaling features based on the scaling model. The scale-adjusted features include a first scale-adjusted feature, a second scale-adjusted feature, and a third scale-adjusted feature. The first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature are located in the same feature coordinate system. The same feature coordinate system means that the first scale-adjusted feature, the second scale-adjusted feature, and the third scale-adjusted feature have the same coordinate position, and the same coordinate position of the three is mapped to the same image region in the standard image.

3. The image target detection method based on multi-scale attention feature fusion according to claim 2, characterized in that, S1 also includes S13; S13. The first-scale features are compressed using the channel-wise arithmetic mean algorithm to obtain the first single-channel feature map. Then, the gradient of the first single-channel feature map is calculated using the Sobel edge detection algorithm to obtain the edge amplitude map. Finally, the edge amplitude map is normalized using the Softmax normalization algorithm to obtain the edge response map. The channel norm of the second-scale features is calculated along the channel direction using the Euclidean norm algorithm to obtain the structural amplitude map. Then, the structural amplitude map is normalized using the Softmax normalization algorithm to obtain the structural response map. The channel norm of the third-scale features is calculated along the channel direction using the Euclidean norm algorithm to obtain the semantic amplitude map. Then, the semantic amplitude map is normalized using the Softmax normalization algorithm to obtain the semantic response map. The edge response map, structural response map, and semantic response map are combined to form a cross-scale response sample. The edge response map, structural response map, and semantic response map inherit the feature coordinate system. The feature coordinate positions with the same row coordinates and column coordinates in the edge response map, structural response map, and semantic response map all point to the same image region in the standard image. The feature coordinate system includes the row direction and column direction of each response map. The row direction coordinates and column direction coordinates constitute the feature coordinate positions.

4. The image target detection method based on multi-scale attention feature fusion according to claim 3, characterized in that, S2 includes S21; S21. Take each feature coordinate position in the edge response map as the transmission start point and each feature coordinate position in the semantic response map as the transmission end point, and calculate the structural constraint transmission cost from the transmission start point to the transmission end point. Structural constraint transmission costs include coordinate displacement costs and structural path costs; Coordinate displacement cost: Read the row and column coordinates of the transmission start point and transmission end point in the same characteristic coordinate system, calculate the absolute value of the difference between the row coordinates and the absolute value of the difference between the column coordinates, and add the two together to obtain the coordinate displacement cost; Structural path cost: A linear grid traversal method is used, with the row and column coordinates of the transmission start point as the starting coordinates and the row and column coordinates of the transmission end point as the ending coordinates. A grid path from the starting coordinates to the ending coordinates is generated in the same feature coordinate system. The feature coordinate positions contained in the grid path are recorded as the feature coordinate position set. The structural response map is sampled according to the feature coordinate position set to obtain the structural path sequence. The difference is processed by taking the value 1 as the minuend and each item in the structural path sequence as the subtrahend. All the difference results are summed to obtain the structural path cost. Add the coordinate displacement cost and the structural path cost to obtain the structural constraint transmission cost from the transmission start point to the transmission end point; The structural constraint transmission cost matrix is ​​obtained by arranging the structural constraint transmission costs according to all transmission start points and all transmission end points.

5. The image target detection method based on multi-scale attention feature fusion according to claim 4, characterized in that, S2 also includes S22 and S23; S22. Sort the edge response map and semantic response map according to the coordinate order in the same feature coordinate system to generate the edge response sequence and semantic response sequence respectively, and perform entropy regularization transformation according to the structural constraint transmission cost matrix to obtain the transmission kernel matrix. The Sinkhorn iterative method is used to alternately scale the transfer kernel matrix to obtain a structurally constrained transfer plan. This plan represents the proportion of response allocated from an edge position in the edge response sequence to a semantic position in the semantic response sequence under the constraints of the structural response map, as follows: Alternating scaling is used to repeatedly perform row scaling and column scaling based on a fixed number of iterations during the training phase; Row scaling processing: The transmission kernel matrix is ​​summed row by row to obtain a row sum value sequence. Each item in the edge response sequence is divided by the item with the same index in the row sum value sequence to obtain a row scaling coefficient sequence. Each item in the row scaling coefficient sequence is then multiplied by all items in the same row of the transmission kernel matrix to obtain a row-scaled transmission matrix. Column scaling processing: Summing column by column of the row scaling transmission matrix to obtain the column sum value sequence, and dividing each item in the semantic response sequence by the item with the same index in the column sum value sequence to obtain the column scaling coefficient sequence. Then, multiplying each item in the column scaling coefficient sequence with all items in the same column of the row scaling transmission matrix to obtain the column scaling transmission matrix. S23. Read the response proportion allocation values ​​of each feature coordinate position in the semantic response map from the structural constraint transfer plan, and for each feature coordinate position, accumulate all the response proportion allocation values ​​of the current feature coordinate position to obtain the response value of the current feature coordinate position. Then, perform response value calculation on all feature coordinate positions in the semantic response map to obtain the transfer response map. Perform point-by-point multiplication of the transfer response map and the structural response map to obtain the structural constraint transfer map. Perform point-by-point difference absolute value calculation on the structural constraint transfer map and the semantic response map to obtain the structural constraint response transfer map.

6. The image target detection method based on multi-scale attention feature fusion according to claim 5, characterized in that, S3 includes S31; S31. Establish a scale sequence fusion model based on a convolutional neural network and train the model. Input the first scale organized features, the second scale organized features and the third scale organized features into the scale sequence fusion model to obtain the baseline scale fusion features. Based on the same feature coordinate system, at any spatial position of the first-scale organized feature, the response value of the structural constraint response migration map at the same spatial position is read, and the current response value is multiplied with the feature values ​​of each channel of the first-scale organized feature at the same spatial position to obtain the first spatial guidance feature. Following the same processing method, the second-scale and third-scale processing features are processed respectively to obtain the second-space guided features and the third-space guided features; The first spatial guiding feature, the second spatial guiding feature, and the third spatial guiding feature are input into the scale sequence fusion model, and the transfer-guided scale fusion feature is output.

7. The image target detection method based on multi-scale attention feature fusion according to claim 6, characterized in that, S3 also includes S32 and S33; S32. Based on the structural constraint response migration diagram, perform response consistency analysis on the baseline scale fusion characteristics and the migration-guided scale fusion characteristics to obtain the baseline response consistency value and the guided response consistency value. Response consistency analysis: The Euclidean norm of the fusion feature at the baseline scale is calculated along the channel direction to obtain the baseline single-channel fusion response. The baseline single-channel fusion response and the response values ​​at the same spatial position in the structural constraint response migration diagram are multiplied point by point, and all product results are summed to obtain the baseline response consistency value. The Euclidean norm of the migration guidance scale fusion feature is calculated along the channel direction to obtain the guidance single-channel fusion response. The guidance single-channel fusion response and the response values ​​at the same spatial position in the structural constraint response migration map are multiplied point by point, and all product results are summed to obtain the guidance response consistency value. S33. Determine the fusion route result based on the baseline response consistency value and the guide response consistency value; If the consistency value of the guiding response is greater than or equal to the consistency value of the baseline response, the fusion route result is the migration-guided fusion route; If the consistency value of the guiding response is less than the consistency value of the baseline response, then the fusion route result is the baseline fusion route.

8. The image target detection method based on multi-scale attention feature fusion according to claim 7, characterized in that, S4 includes S41, S42 and S43; S41. Determine the input features based on the fusion route results; When the fusion route result is the baseline fusion route, the baseline scale fusion features are taken as features to be noted; When the fusion path result is a transfer-guided fusion path, the transfer-guided scale fusion feature is used as the feature to be paid attention to, and the structural constraint response transfer map is used as the position guidance input; wherein, the structural constraint response transfer map and the transfer-guided scale fusion feature are located in the same feature coordinate system; S42. Perform global average pooling on the features to be paid attention to obtain the channel description results; perform one-dimensional convolution and sigmoid activation on the channel description results to obtain the channel attention results; perform channel-wise multiplication of the channel attention results with the features to be paid attention to obtain the channel filtering features. S43. Perform location attention analysis based on the fusion route results; When the fusion route result is the baseline fusion route, the channel filtering features are subjected to width-direction average pooling and height-direction average pooling to obtain the position attention result. The position attention result is then multiplied with the channel filtering features in a spatial position manner to obtain the first position filtering feature. When the fusion route result is a migration-guided fusion route, the structural constraint response migration map and the channel filtering feature are first multiplied by spatial position to obtain the migration-guided feature. Then, the migration-guided feature is subjected to average pooling in the width direction and average pooling in the height direction to obtain the position attention result. The position attention result is then multiplied by the migration-guided feature by spatial position to obtain the second position filtering feature. When the fusion route result is the baseline fusion route, the first position filter feature is used as the target consistent fusion feature; When the fusion route result is a migration-guided fusion route, the second position filtering feature is used as the target-consistent fusion feature.

9. The image target detection method based on multi-scale attention feature fusion according to claim 8, characterized in that, S5 includes S51; S51. Based on the target consistency fusion features, perform target output mapping to obtain category response features, location distance features, and confidence response features; wherein, target output mapping includes category mapping processing, location distance mapping processing, and confidence mapping processing; Category mapping processing: Category mapping convolution is used to perform convolutional mapping on the target consistent fusion features to obtain category response features; Location distance mapping processing: Location distance mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain location distance features; among them, the location distance features output four boundary distances at each feature coordinate position, and the four boundary distances represent the distances from the current feature coordinate position to the left boundary, top boundary, right boundary and bottom boundary of the target box, respectively; Confidence mapping processing: Confidence mapping convolution is used to perform convolution mapping on the target consistent fusion features to obtain confidence response features; whereby the confidence response features are used to represent the degree of response of the target at each feature coordinate position in the target consistent fusion features; For any feature coordinate position in the target consistency fusion features, read the response value of the category response feature at the current feature coordinate position, and take the category with the largest response value as the candidate category; read the distance of the location distance feature to the four boundaries at the current feature coordinate position, and generate candidate boxes based on the current feature coordinate position and the four boundary distances; read the response value of the confidence response feature at the current feature coordinate position as the candidate confidence; combine the candidate category, candidate box, and candidate confidence to form a candidate target. Perform target output mapping on all feature coordinate positions in the target consistency fusion feature to obtain a set of candidate targets.

10. The image target detection method based on multi-scale attention feature fusion according to claim 9, characterized in that, S5 also includes S52; S52. Calculate the Euclidean norm of the target consistency fusion feature along the channel direction to obtain the target consistency response map; for any candidate target in the candidate target set, read the candidate box coverage area of ​​the current candidate target, and read the response value within the candidate box coverage area in the target consistency response map, calculate the arithmetic mean of all response values, and obtain the region consistency response value. Then, in the structural constraint response transition map, all response values ​​within the same candidate box coverage area are read and the arithmetic mean is calculated to obtain the region response value; The candidate confidence score, regional consensus response value, and regional response value of the current candidate target are multiplied to obtain the candidate retention value of the current candidate target; If the candidate bounding boxes of two candidate targets cover at least one of the same feature coordinate positions, then the two candidate targets are divided into the same overlapping box group, and the candidate targets in each overlapping box group are arranged in descending order of candidate retention value. The candidate target at the top of the sort is retained and marked as the detection target of the current overlapping group.

Images