Target detection method and device based on deep learning and electronic equipment

Abstract

The invention discloses a target detection method and device based on deep learning and electronic equipment, and relates to the field of data processing. The method comprises the following steps: processing an input image to obtain a plurality of initial feature layers with different resolutions; performing global context modeling on each initial feature layer to obtain a plurality of feature layers, and calculating scale sensing foreground scores for a plurality of tokens in each feature layer; generating a confidence score of each feature layer, and screening out an optimal feature layer and a suboptimal feature layer; fusing a plurality of target feature layers, which are not screened out, in the feature layers to the optimal feature layer and the suboptimal feature layer respectively to obtain an enhanced optimal feature layer and an enhanced suboptimal feature layer; and performing sparse processing based on the scale perception foreground score to obtain a sparse token set, and inputting the sparse token set into a detector of a preset target detection model to obtain a target detection result of the input image. By implementing the technical scheme provided by the invention, the calculation redundancy and the memory overhead are reduced.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a deep learning-based object detection method, apparatus, and electronic device. Background Technology

[0002] Object detection, as one of the fundamental core tasks of computer vision, aims to identify object instances in images or videos and determine their locations. It has wide application value in many fields such as autonomous driving, security monitoring, industrial quality inspection, and medical image analysis.

[0003] A core challenge in object detection lies in effectively handling objects of vastly different sizes within an image. To address this issue, existing techniques widely employ feature pyramid networks and their various variants. These methods use deep convolutional neural networks as the backbone, extracting feature layers with varying spatial resolutions and semantic intensities at different stages. By constructing a top-down pathway and incorporating lateral connections, this approach progressively transmits and fuses strong semantic information from deeper layers into shallower features, thereby providing corresponding, information-rich feature representations for objects of different scales.

[0004] However, the problem with this traditional multi-scale fusion strategy is that it employs a static processing paradigm. Regardless of the specific content of the input image, the model indiscriminately generates and processes a complete set of feature pyramids, and then feeds all the fused feature layers into the subsequent detection head for prediction. This fixed processing flow leads to a large amount of computational redundancy and unnecessary memory overhead. Summary of the Invention

[0005] This application provides a deep learning-based object detection method, apparatus, and electronic device that reduces computational redundancy and memory overhead.

[0006] A first aspect of this application provides a deep learning-based object detection method, comprising: processing an input image using the backbone network of a preset object detection model to obtain multiple initial feature layers with different resolutions; performing global context modeling on each of the initial feature layers to obtain multiple feature layers, and calculating scale-aware foreground scores for multiple tokens in each feature layer; generating a confidence score for each feature layer based on the scale-aware foreground scores, and selecting an optimal feature layer and a suboptimal feature layer from the multiple feature layers according to the confidence scores; fusing multiple target feature layers that were not selected from the feature layers into the optimal feature layer and the suboptimal feature layer respectively to obtain an enhanced optimal feature layer and an enhanced suboptimal feature layer; performing sparsification processing on the tokens of the enhanced optimal feature layer and the tokens of the enhanced suboptimal feature layer based on the scale-aware foreground scores to obtain a sparse token set, and inputting the sparse token set into the detector of the preset object detection model to obtain the object detection result of the input image.

[0007] By employing the aforementioned technical solution, multi-scale feature extraction is performed on the input image to obtain initial feature layers of different resolutions. Global context modeling is then performed on each initial feature layer to capture global dependencies and contextual information within the feature layer, enhancing feature representation capabilities. Simultaneously, a scale-aware foreground score is introduced as an importance measure for feature layers and tokens, and a feature layer confidence score is generated based on this score to filter out the optimal and suboptimal feature layers, ensuring that subsequent processing focuses on the feature layers with the most information. Furthermore, unfiltered target feature layers are fused into the optimal and suboptimal feature layers respectively, further enhancing feature representation capabilities. Finally, the tokens of the fused optimal and suboptimal feature layers are sparsified to obtain a sparse token set, reducing computational redundancy and noise interference, and improving detection efficiency and accuracy. This method, through a series of operations including multi-scale feature extraction, global context modeling, scale-aware foreground score calculation, feature layer filtering and fusion, and token sparsification, fully utilizes the multi-scale information and global contextual information of the image, while highlighting important feature layers and tokens, suppressing redundancy and noise, and ultimately obtaining efficient and accurate target detection results.

[0008] Optionally, the step of fusing multiple unselected target feature layers from the feature layers into the optimal feature layer and the suboptimal feature layer to obtain an enhanced optimal feature layer and an enhanced suboptimal feature layer specifically includes: calculating a first spatial resolution ratio between the first target feature layer and the optimal feature layer, and calculating a second spatial resolution ratio between the first target feature layer and the suboptimal feature layer, wherein the first target feature layer is any one of the multiple target feature layers; based on the first spatial resolution ratio and the second spatial resolution ratio, using a preset rule, determining the parameters corresponding to the first target feature layer. The first target feature layer is selected as either the optimal or suboptimal feature layer. If the reference feature layer is the optimal feature layer, a dimension alignment operation is performed on the first target feature layer to obtain a second target feature layer. All second target feature layers whose reference feature layer is the optimal feature layer are aggregated to generate a fused feature block. The fused feature block is then superimposed on the optimal feature layer using residual connections to obtain the enhanced optimal feature layer. If the reference feature layer is the suboptimal feature layer, the first target feature layer is fused with the suboptimal feature layer to obtain the enhanced suboptimal feature layer.

[0009] By employing the above technical solution, the spatial resolution ratio of the target feature layer to the optimal and suboptimal feature layers is calculated, and a reference feature layer corresponding to each target feature layer is determined based on preset rules, thus realizing an adaptive feature layer fusion strategy. For target feature layers whose reference feature layer is the optimal feature layer, a dimension alignment operation is performed to make its spatial resolution consistent with that of the optimal feature layer. The aligned target feature layers are then aggregated to generate fused feature blocks, which are then superimposed on the optimal feature layer using residual connections. This achieves the fusion of semantic and detail information at different scales into the optimal feature layer, enhancing its feature representation capability. For target feature layers whose reference feature layer is the suboptimal feature layer, they are directly fused into the suboptimal feature layer, avoiding unnecessary computational overhead. This fusion strategy fully considers the differences in spatial resolution between different target feature layers and the optimal and suboptimal feature layers. By adaptively determining the reference feature layer and employing operations such as dimension alignment and residual connections, it flexibly and efficiently fuses feature information at different scales into the optimal and suboptimal feature layers, improving the richness and robustness of feature representation, which is beneficial for subsequent object detection tasks.

[0010] Optionally, the step of performing a dimension alignment operation on the first target feature layer to obtain a second target feature layer specifically includes: determining the size relationship between the first spatial resolution of the first target feature layer and the second spatial resolution of the optimal feature layer; if the first spatial resolution is less than the second spatial resolution, then upsampling the first target feature layer to obtain the second target feature layer; if the first spatial resolution is greater than the second spatial resolution, then downsampling the first target feature layer to obtain the second target feature layer.

[0011] By employing the above technical solution, the spatial resolution relationship between the target feature layer and the optimal feature layer is determined, and upsampling or downsampling operations are adaptively selected, achieving flexible and efficient dimension alignment. When the spatial resolution of the target feature layer is smaller than that of the optimal feature layer, the target feature layer is upsampled to increase its spatial resolution, making it consistent with the resolution of the optimal feature layer. When the spatial resolution of the target feature layer is larger than that of the optimal feature layer, the target feature layer is downsampled to decrease its spatial resolution, making it consistent with the resolution of the optimal feature layer. By adaptively selecting appropriate upsampling or downsampling algorithms, the spatial resolution of the target feature layer is efficiently and flexibly adjusted to match that of the optimal feature layer, facilitating subsequent feature fusion and improving the effectiveness and efficiency of feature fusion.

[0012] Optionally, the step of performing global context modeling on each initial feature layer to obtain multiple feature layers specifically includes: expanding the initial feature layer into multiple one-dimensional feature sequences along a preset scanning direction, wherein the preset scanning direction includes top to bottom, bottom to top, left to right, and right to left; applying a preset state space model operator to each one-dimensional feature sequence to perform sequence modeling, thereby obtaining multiple processed one-dimensional sequences; merging all processed one-dimensional sequences according to their corresponding original spatial positions and reshaping them into a two-dimensional structure to obtain the feature layer.

[0013] By adopting the above technical solution, the spatial dependencies within the feature layer are fully utilized. By unfolding the two-dimensional feature layer into a one-dimensional sequence and applying powerful sequence modeling operators, the global contextual information within the feature layer is efficiently captured and fused, enhancing the semantics and contextual awareness of the feature representation, and providing richer and more robust feature representations for subsequent object detection tasks.

[0014] Optionally, calculating the scale-aware foreground score for each of the multiple tokens in each feature layer specifically includes: for each deformable reference point corresponding to a token, calculating the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box of the deformable reference point to generate an IU score for each token; enhancing the channel weights of each feature layer and performing convolution and normalization operations on the channel weights of each feature layer to obtain a foreground-background confidence score for each token; performing a weighted summation of the IU score and the foreground-background confidence score to obtain a weighted summation result; and performing maximum / minimum normalization on the weighted summation result to obtain the scale-aware foreground score corresponding to each token, wherein the formula for calculating the scale-aware foreground score is: ; Among them, F ij Let P represent the scale-aware foreground score of the j-th token in the i-th feature layer, where Norm() is the extremum normalization function. box D represents the predicted bounding box. box Represents the true bounding box. Let α be the intersection-union ratio of the reference point. fg Let W be the weighting factor, W be the channel weight matrix, X be the eigenvector representation of the token, and b be the bias term. The foreground-background confidence score is given.

[0015] By adopting the above technical solution, the importance of feature layers and tokens is effectively quantified, tokens with high matching degree with the target and belonging to the foreground region are highlighted, and background and redundant tokens are suppressed. This provides effective guidance for subsequent feature layer selection and token sparsification, and improves the accuracy and efficiency of target detection.

[0016] Optionally, the step of generating a confidence score for each feature layer based on the scale-aware foreground score, and selecting the optimal and suboptimal feature layers from multiple feature layers based on the confidence scores, specifically includes: processing each feature layer using a multilayer perceptron trained with the scale-aware foreground score as a supervision signal to obtain a confidence score corresponding to each feature layer; performing a top-down score modulation process, adjusting a second confidence score based on a first confidence score, where the first confidence score is the confidence score corresponding to the highest-level feature layer, and the second confidence score is the confidence score below the first confidence score, generating multiple modulated confidence scores; calculating the score percentage of each feature layer relative to the total score of all feature layers based on the modulated confidence scores; and determining the feature layer with the highest score percentage as the optimal feature layer and the feature layer with the second highest score percentage as the suboptimal feature layer based on the score percentage.

[0017] By adopting the above technical solution, the supervision information provided by the scale-aware foreground score is fully utilized. Through multilayer perceptron learning and top-down score modulation, the importance of each feature layer is adaptively evaluated. Based on the score ratio, the optimal and second-best feature layers are flexibly selected, so that the subsequent feature fusion and target detection processes are more focused on the feature layers with large information content and high contribution, thereby improving detection accuracy and efficiency.

[0018] Optionally, the step of sparsifying the tokens of the enhanced optimal feature layer and the enhanced suboptimal feature layer based on the scale-aware foreground score to obtain a sparsified token set specifically includes: determining a first-level focus ratio corresponding to the enhanced optimal feature layer and a second-level focus ratio corresponding to the enhanced suboptimal feature layer, wherein the first-level focus ratio is greater than the second-level focus ratio; determining a first preset quantity based on the first-level focus ratio and a preset ratio, and determining a second preset quantity based on the second-level focus ratio and the preset ratio; selecting the first preset quantity of tokens from the tokens of the enhanced optimal feature layer based on the scale-aware foreground score to obtain a first token set; selecting the second preset quantity of tokens from the tokens of the enhanced suboptimal feature layer based on the scale-aware foreground score to obtain a second token set; merging the first token set and the second token set to obtain a candidate sparsified token set; and performing nonmaximum suppression processing on the candidate sparsified token set to obtain the sparsified token set.

[0019] By adopting the above technical solution, the importance differences of different feature layers are fully considered. Through the adaptive setting of the layer focus ratio and preset number, the number of tokens to be screened in different feature layers can be flexibly controlled, highlighting the key tokens in the optimal and suboptimal feature layers. At the same time, by removing redundancy through nonmaximum suppression, a concise and effective sparse token set is obtained, which improves the accuracy and efficiency of target detection.

[0020] A second aspect of this application provides a deep learning-based object detection device, comprising an initial feature layer acquisition module, a global context modeling module, a confidence scoring module, a feature fusion module, and an output result module, wherein: the initial feature layer acquisition module is used to process the input image using the backbone network of a preset object detection model to obtain multiple initial feature layers of different resolutions; the global context modeling module is used to perform global context modeling on each of the initial feature layers to obtain multiple feature layers, and calculate scale-aware foreground scores for multiple tokens in each of the feature layers; the confidence scoring module is used to generate a result for each of the initial feature layers based on the scale-aware foreground scores. The feature layer is scored with confidence, and the optimal and suboptimal feature layers are selected from multiple feature layers based on the confidence scores. The feature fusion module is used to fuse multiple target feature layers that were not selected from the feature layers into the optimal and suboptimal feature layers respectively, to obtain enhanced optimal and enhanced suboptimal feature layers. The output result module is used to perform sparsification processing on the tokens of the enhanced optimal feature layer and the tokens of the enhanced suboptimal feature layer based on the scale-aware foreground score, to obtain a sparse token set, and input the sparse token set into the detector of the preset target detection model to obtain the target detection result of the input image.

[0021] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0022] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0023] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. By using global context modeling and scale-aware foreground scoring, the optimal and suboptimal feature layers are dynamically selected, and information from the secondary feature layers is fused to enhance representation capabilities. Combined with sparsification of the feature layers, this method effectively reduces computational redundancy and memory overhead, while improving adaptability to different target scales and detection accuracy. This scheme significantly enhances detection performance while improving efficiency, making it suitable for real-time and resource-constrained scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a deep learning-based object detection method disclosed in an embodiment of this application; Figure 2 This is another flowchart illustrating a deep learning-based object detection method disclosed in an embodiment of this application; Figure 3 This is a schematic diagram of a deep learning-based target detection device disclosed in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0025] Figure labeling: 301, Initial feature layer acquisition module; 302, Global context modeling module; 303, Confidence scoring module; 304, Feature fusion module; 305, Output result module; 400, Electronic device; 401, Processor; 402, Communication bus; 403, User interface; 404, Network interface; 405, Memory. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0027] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0028] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0029] This application provides a deep learning-based object detection method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating a deep learning-based object detection method provided in an embodiment of this application. The method is applied to a server, which executes the deep learning-based object detection program. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The method includes steps S101 to S105, as follows: Step S101: Process the input image using the backbone network of the preset target detection model to obtain multiple initial feature layers with different resolutions.

[0030] In step S101, the backbone network of the preset object detection model refers to a deep neural network, typically a convolutional neural network, whose core function is to extract hierarchical visual features from the original input image. The input image represents the digital image to be detected. Multiple initial feature layers with different resolutions refer to the set of feature maps output by the backbone network at different depth stages. These feature maps have different spatial dimensions and number of channels, capturing semantic and detail information of the image at different scales. The feature layers with lower resolution contain richer global semantic information, while the feature layers with higher resolution retain more spatial details.

[0031] Specifically, the server first receives an input image to be detected. Then, the server feeds this image as input into a pre-defined, weighted backbone network of an object detection model, such as a ResNet-50 network. This backbone network performs forward propagation computation on the input image through a series of convolutions, pooling, and non-linear activation operations. During this process, the server extracts feature maps from multiple predetermined stages of the backbone network, such as the outputs of the third, fourth, and fifth residual blocks. These extracted feature maps constitute multiple initial feature layers. For example, for a standard input, four initial feature layers might be obtained, with the spatial resolution halved layer by layer while the channel dimension increases accordingly, forming a feature pyramid structure that provides a rich multi-scale visual foundation for subsequent analysis.

[0032] Step S102: Perform global context modeling for each initial feature layer to obtain multiple feature layers, and calculate the scale-aware foreground score for each of the multiple tokens in each feature layer.

[0033] In step S102, global context modeling refers to an operation used to enhance feature representation capabilities. It aims to enable every location in the feature map to perceive information across the entire world, thereby capturing long-distance dependencies between features. The feature layer represents the initial feature layer after global context modeling, containing richer contextual information. A token is a basic unit in the feature layer, which can be understood as a feature vector representing a specific spatial region in the image. Scale-aware foreground score is a key metric used to quantify the probability that each token belongs to a foreground object, and this score adaptively reflects the importance of the object at different scales.

[0034] Specifically, the server independently performs global context modeling for each initial feature layer from step S101. This process is implemented through a two-dimensional selective scanning mechanism. The server first unfolds the two-dimensional initial feature layer into one-dimensional feature sequences along four scanning directions: top to bottom, bottom to top, left to right, and right to left. Next, a state-space model operator is applied to each one-dimensional sequence for efficient sequence modeling, capturing long-range dependencies within the sequence. Finally, the four processed one-dimensional sequences are merged according to their original spatial positions and reconstructed back into a two-dimensional structure, thus obtaining a feature layer with a global receptive field. After modeling is completed, the server calculates the scale-aware foreground score for each token in each newly generated feature layer. This score calculation includes two parts: first, calculating the intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box for each token's deformable reference point to obtain the reference point IU score; second, enhancing channel weights through an attention mechanism and combining convolution operations to calculate a foreground-background confidence score. The server normalizes and weights these two scores to generate a scale-aware foreground score that comprehensively reflects both location accuracy and foreground probability.

[0035] In one possible implementation, global context modeling is performed on each initial feature layer to obtain multiple feature layers, specifically including steps S1021-S2023, as follows: Step S1021: Expand the initial feature layer into multiple one-dimensional feature sequences along a preset scanning direction. The preset scanning direction includes top to bottom, bottom to top, left to right, and right to left.

[0036] In step S1021, the feature layer refers to the image feature representation with a two-dimensional spatial structure extracted by the backbone network, such as a data block with height H, width W, and number of channels C. The preset scan direction refers to four predefined paths used to convert two-dimensional data into a one-dimensional sequence. A one-dimensional feature sequence is a linear sequence formed by arranging all feature vectors in the two-dimensional feature layer according to a specific scan order. The purpose of this step is to convert spatial data into sequence data to adapt to subsequent sequence modeling operators.

[0037] Specifically, the server receives an initial feature layer as input. To capture its spatial dependencies from different perspectives, the server generates four independent one-dimensional feature sequences based on all four preset scanning directions. The first sequence is generated from left to right; the server scans the feature layer row by row, concatenating the feature vectors of each row from left to right to form a long sequence. The second sequence is generated from right to left; the server also scans row by row, but concatenates the feature vectors within each row from right to left. The third sequence is generated from top to bottom; the server changes to scanning column by column, concatenating the feature vectors of each column from top to bottom. The fourth sequence is generated from bottom to top; the server scans column by column, concatenating the feature vectors within each column from bottom to top. After this step, a two-dimensional feature layer is transformed into four long one-dimensional feature sequences containing the same data but arranged in different orders.

[0038] Step S1022: Apply a preset state-space model operator to each one-dimensional feature sequence to perform sequence modeling, and obtain multiple processed one-dimensional sequences.

[0039] In step S1022, the preset state-space model operator refers to a specific computational model used to process sequence data, such as the S6 operator used in the Mamba model, which can efficiently capture long-distance dependencies in a sequence with linear computational complexity. Sequence modeling is a process that processes the input sequence through model operators, so that each element of the output sequence contains its contextual information. The processed one-dimensional sequence is a new sequence obtained after computation by the state-space model operator, and the feature vector at each position incorporates global contextual information in its scan direction.

[0040] Specifically, the server takes the four one-dimensional feature sequences generated in step S1021 as input. The server independently applies the same preset state-space model operator to each of these four sequences. For sequences from left to right, the operator processes the sequence starting from the first element and progressively passes information backward, ensuring that the output at any position in the sequence includes information about all elements to its left. Similarly, for sequences from right to left, the output includes information about all elements to its right. For sequences from top to bottom and bottom to top, the information flow proceeds along the vertical direction, respectively. In this way, the server generates a corresponding processed one-dimensional sequence with global context modeling for each scanning direction. Finally, the server obtains four processed one-dimensional sequences that correspond one-to-one with the input sequences.

[0041] Step S1023: Merge all processed one-dimensional sequences according to their corresponding original spatial positions and reshape them into a two-dimensional structure to obtain the feature layer.

[0042] In step S1023, the original spatial location refers to the row and column coordinates of each feature vector in the one-dimensional sequence before transformation in the original two-dimensional feature layer. Merging refers to the process of combining multiple feature vectors from different processed sequences that correspond to the same original spatial location into a single vector, for example, by summing or averaging. Reshaping into a two-dimensional structure is the reverse process of merging, that is, restoring the merged feature vectors to their original spatial coordinates, thereby reconstructing a two-dimensional feature layer. The final feature layer is the output of this global context modeling module; it is an information-enhanced feature layer of the same size as the input.

[0043] Specifically, the server receives four processed one-dimensional sequences from step S1022. The server first creates an empty two-dimensional data structure with the same size as the original input feature layer as the output container. Then, the server iterates through each spatial location in this two-dimensional structure, for example, the location at coordinates (i, j). For this location, the server finds the four feature vectors that originally originated from this location from the four processed one-dimensional sequences. Next, the server performs a merging operation on these four feature vectors, such as calculating their element-wise average, to obtain a final feature vector that integrates contextual information from all four directions. Finally, the server fills this final feature vector into the (i, j) position of the output container. Once all spatial locations are filled, this output container forms the final feature layer, where each pixel efficiently integrates global contextual information.

[0044] In one possible implementation, scale-aware foreground scores are calculated for multiple tokens in each feature layer, specifically including steps S1024-S1027, as follows: Step S1024: For each deformable reference point corresponding to a token, calculate the intersection-union ratio (IU) between the predicted bounding box and the true bounding box of the deformable reference point, and generate the IU score of the reference point for each token. In step S1024, deformable reference points are a sparsely sampled set of points associated with each token in the feature space, used to replace the dense computation of the entire feature map in traditional attention mechanisms. The predicted bounding box refers to the location box that the model predicts might contain the target based on the information from the deformable reference points. The ground truth bounding box is the actual location box of the target, pre-labeled in the training data. The intersection-union ratio (IU) is a metric that measures the degree of overlap between two bounding boxes; its value is the ratio of their intersection area to their union area. The reference point IU score is the final calculated IU value, used to represent the accuracy of the predicted geometric location.

[0045] Specifically, when processing a feature layer, the server iterates through each token. For each token, the server first determines its corresponding deformable reference point. Then, using a feedforward neural network, the server generates a predicted bounding box using the information from these reference points. Simultaneously, the server retrieves the ground truth bounding boxes associated with the current token from the training dataset. Next, the server calculates the overlapping area of ​​these two bounding boxes, i.e., the intersection area, and the total area they cover, i.e., the union area. Finally, the server divides the intersection area by the union area, obtaining a value between 0 and 1. This value is designated as the reference point intersection-union ratio (IURR) for that token and is stored for subsequent steps.

[0046] Step S1025: Enhance the channel weights of each feature layer, and perform convolution and normalization operations on the channel weights of each feature layer to obtain the foreground-background confidence score of each token; In step S1025, channel weights refer to the importance or activation level of each channel in the feature layer. Enhancement means dynamically adjusting these weights through attention mechanisms and other means, making the model focus more on channels containing important information. Convolution is a computational process that uses convolution kernels to extract local features; here, it specifically refers to 1x1 convolution, used to fuse and transform the channel information of each token without changing the spatial dimension. Normalization function processing refers to applying a function such as Softmax to convert a set of values ​​into a probability distribution that sums to 1. The foreground-background confidence score is a probability value representing that a token belongs to the foreground object rather than the background.

[0047] Specifically, the server first applies an attention mechanism to the input feature layer to recalibrate and enhance its channel weights, giving higher weights to channels that respond more strongly to foreground targets. After processing, the server performs a 1x1 convolution operation on this channel-weight-enhanced feature layer. This operation iterates through each token in the feature layer, mapping its multi-channel feature vector to a single scalar value, which can be seen as the raw score indicating that the token belongs to the foreground. To convert the raw scores of all tokens into standardized probabilities, the server then applies a Softmax normalization function to all scores across the entire feature layer. After processing, each token receives a value between 0 and 1, which is the foreground-background confidence score for that token.

[0048] Step S1026: The reference point intersection and union score and the foreground-background confidence score are weighted and summed to obtain the weighted sum result.

[0049] In step S1026, weighted summation is an information fusion method that assigns different weight factors to different information sources and then adds them together to obtain a comprehensive evaluation value. The weighted summation result is an intermediate value generated by this fusion operation, which simultaneously includes information on the geometrical position accuracy of the token and information on its semantic category importance.

[0050] Specifically, for each token, the server retrieves two key scores from previous steps: the reference point intersection-union score generated in step S1024 and the foreground-background confidence score generated in step S1025. The server also reads a preset weighting factor, which adjusts the relative importance of the foreground-background confidence score in the total score. During calculation, the server first multiplies the foreground-background confidence score by the weighting factor, and then directly adds the product to the reference point intersection-union score. This sum is the weighted sum for that token.

[0051] Step S1027: Perform maximum / minimum normalization on the weighted summation result to obtain the scale-aware foreground score corresponding to each token. The formula for calculating the scale-aware foreground score is as follows:

[0052] Among them, F ij Let P represent the scale-aware foreground score of the j-th token in the i-th feature layer, where Norm() is the extremum normalization function. box D represents the predicted bounding box. box Represents the true bounding box. Let α be the intersection, union, and ratio of fractions with reference point. fg Let W be the weighting factor, W be the channel weight matrix, X be the eigenvector representation of the token, and b be the bias term. Foreground-background confidence score.

[0053] In step S1027, the extreme value normalization process is a technique that linearly maps data to a specific interval, typically between 0 and 1. It is calculated by subtracting the minimum value from the current value and then dividing by the difference between the maximum and minimum values. The scale-aware foreground score is the final output of this series of steps. It is a comprehensive, normalized score that accurately reflects the importance of a token as a foreground target at a specific scale. This importance considers both geometric accuracy and semantic saliency.

[0054] Specifically, the server first collects all weighted summation results calculated for all tokens in all relevant feature layers of the current image in step S1026. The server iterates through this score set, finding the maximum and minimum values. Then, the server iterates through each token again, applying an extremum normalization formula to its weighted summation result. That is, the weighted summation result of the token is subtracted from the previously found global minimum value, and the difference is divided by the difference between the global maximum and the global minimum value. This calculation process ensures that regardless of the range of the original scores, the final score will be uniformly scaled to the range of 0 to 1. This final score after normalization is the scale-aware foreground score of the token. The principle of the scale-aware foreground score calculation formula is as follows: The purpose of this formula is to calculate the scale-aware foreground score for each token, comprehensively considering the token's matching degree with the real target at different scales and the confidence level that the token belongs to the foreground or background. The formula can be broken down into the following parts: 1. Reference points, intersection, union, and comparison of fractions:

[0055] This item calculates the predicted bounding box (P) of the deformable reference point corresponding to the token. box ) and true bounding box (D box The Intersection over Union (IoU) of the predicted bounding box and the ground truth bounding box. The higher the IoU value, the better the token matches the ground truth target at the current scale.

[0056] 2. Foreground-Background Confidence Score:

[0057] This section enhances the channel weights of the feature layer and applies convolution operations and normalization functions to obtain the confidence score of each token belonging to the foreground or background. W is the channel weight matrix, X is the feature vector representation of the token, and b is the bias term. The convolution result is normalized to the range [0, 1] using the Softmax function to obtain the foreground-background confidence score. The larger the value, the more likely the token belongs to the foreground region.

[0058] 3. Weighted summation:

[0059] This term is a weighted sum of the reference point intersection-union score and the foreground-background confidence score. α fg This is a weighting factor used to balance the weights of the two terms. The weighted sum takes into account both the target matching degree of the token at different scales and the confidence of the foreground region; the larger the value, the more important the token.

[0060] 4. Maximum / minimum value normalization:

[0061] The result of the weighted summation is normalized to its maximum value, mapping it to the range [0, 1].

[0062] The normalized result is the scale-perceived foreground score F for each token. ij The larger the value, the more important the token is when considering factors such as scale and prospect confidence.

[0063] In summary, this formula, by fusing the target matching degree and foreground region confidence of tokens at different scales and performing weighted summation and normalization, yields a scale-aware foreground score for each token. This score fully considers scale and foreground information, effectively quantifying the importance of tokens and providing guidance for subsequent feature layer selection and sparsification.

[0064] Step S103: Based on the scale-aware foreground score, generate a confidence score for each feature layer, and select the best and second-best feature layers from multiple feature layers based on the confidence score.

[0065] In step S103, the confidence score is a numerical value obtained after evaluating the importance of the entire feature layer, representing the contribution or relevance of that feature layer to the target detection task in the current image. The optimal feature layer is the feature layer with the highest confidence score among all feature layers, and it is considered the dominant scale containing the most critical target information. The second-best feature layer is the feature layer with the second-highest confidence score, serving as an important supplement to the optimal feature layer.

[0066] Specifically, the server first uses the scale-aware foreground scores of all tokens within each feature layer calculated in step S102 as supervision signals. It then processes each feature layer using a pre-trained multilayer perceptron to generate an initial confidence score for each layer. Subsequently, the server performs a top-down score modulation process to optimize these scores. This process starts with the feature layer with the lowest resolution and highest semantic level, using its confidence score as a baseline. Then, it proceeds layer by layer downwards, multiplying the modulated confidence score of the previous layer by the original confidence score of the current layer to generate the modulated confidence score for the current layer. This process allows the judgment of higher-level semantics to influence and calibrate the scores of lower-level feature layers. After all feature layers have obtained modulated confidence scores, the server calculates the proportion of each feature layer's score to the total score of all layers, i.e., the score percentage. Finally, the server sorts the feature layers according to their score percentages, determining the feature layer with the highest percentage as the optimal feature layer and the feature layer with the second highest percentage as the suboptimal feature layer.

[0067] In one possible implementation, a confidence score for each feature layer is generated based on the scale-aware foreground score, and the optimal and second-best feature layers are selected from multiple feature layers based on the confidence scores. Specifically, this includes steps S1031-S1034, as follows: Step S1031: Using a multilayer perceptron trained with scale-aware foreground scores as supervision signals, process each feature layer to obtain the confidence score corresponding to each feature layer.

[0068] In step S1031, the scale-aware foreground score refers to the normalized score calculated for each token in the previous step, which integrates location and category information. The supervision signal represents the ground truth data used as guidance or target during model training. In this scenario, the overall foreground importance of a feature layer is determined by the scale-aware foreground scores of its multiple tokens. A multilayer perceptron is a feedforward neural network used to learn complex mappings from input to output. The confidence score is a single numerical value output by the multilayer perceptron after processing a complete feature layer, representing the predicted importance or contribution of that layer to the object detection task in the current image.

[0069] Specifically, the server configures an independent multilayer perceptron for each feature layer, and these multilayer perceptrons have been pre-trained. During training, these networks learn to map the overall representation of a feature layer to its importance score in the current image. The ground truth value of this importance score, i.e., the supervision signal, is obtained by aggregating the scale-aware foreground scores of all tokens on that feature layer. In the current implementation step, the server inputs each feature layer into its corresponding pre-trained multilayer perceptron. The multilayer perceptron performs information compression and nonlinear transformation on the input feature layers, ultimately outputting a single scalar value for each feature layer. This output value is the initial confidence score corresponding to that feature layer.

[0070] Step S1032: Perform a top-down score modulation process, adjust the second confidence score based on the first confidence score, the first confidence score is the confidence score corresponding to the highest level feature layer, and the second confidence score is the confidence score below the first confidence score, and generate multiple modulated confidence scores.

[0071] In step S1032, the top-down score modulation process is an information transfer mechanism that allows higher-level feature layers with rich semantic information to influence and adjust the scores of lower-level feature layers containing more detailed information. The highest-level feature layer refers to the feature layer with the lowest spatial resolution but the highest semantic level in the feature pyramid. The first confidence score is the initial confidence score of this highest-level feature layer. The second confidence score refers to the confidence score of any feature layer in the feature pyramid that is lower than the highest level. The modulated confidence score is the new score obtained after the original confidence scores of the lower-level feature layers are influenced by the higher-level scores.

[0072] Specifically, the server first identifies the highest-level feature layer from all feature layers and obtains its confidence score calculated in step S1031, which is the first confidence score. Then, the server processes the remaining feature layers layer by layer in a top-down order. For the layer immediately below the highest-level layer, the server multiplies its own confidence score with the confidence score of the layer above it (the highest-level layer), and the result is used as the new post-modulation confidence score for that layer. Next, for the layer below that, the server multiplies its original confidence score with the post-modulation confidence score just calculated by the layer immediately above it. This process is passed down sequentially, and the score of each layer is adjusted by the cumulative effect of the scores of all its upper layers. Finally, all feature layers except the highest-level feature layer will obtain a post-modulation confidence score.

[0073] Step S1033: Based on the post-modulation confidence score, calculate the score ratio of each feature layer to the total score of all feature layers.

[0074] In step S1033, the post-modulation confidence score refers to the set of scores for each level generated in step S1032, which has incorporated high-level semantic information. The total score is the sum of the post-modulation confidence scores of all feature layers. The score percentage is a relative value, calculated by dividing the score of a single feature layer by the total score, and is used to represent the relative importance of that layer among all feature layers. The score percentages of all layers are summed to one.

[0075] Specifically, the server first obtains the confidence scores for all feature layers, using the original confidence score for the highest layer and the modulated confidence scores generated in step S1032 for the remaining layers. The server adds all these scores together to obtain a total score. Then, the server iterates through each feature layer, dividing its confidence score by the calculated total score. The result of this division is the score percentage for that feature layer. The server performs this calculation for each feature layer, ultimately generating a list of score percentages equal to the number of feature layers.

[0076] Step S1034: Based on the score ratio, the feature layer with the highest score ratio is determined as the optimal feature layer, and the feature layer with the second highest score ratio is determined as the suboptimal feature layer.

[0077] In step S1034, the score percentage refers to the quantitative representation of the relative importance of each feature layer. The optimal feature layer is the one rated as the most critical and primary feature layer in the current image detection task. The second-best feature layer is the second most critical feature layer, second only to the optimal feature layer in importance. The purpose of this step is to select the two most valuable layers from all candidate layers based on the quantitative score for subsequent refinement processing.

[0078] Specifically, the server receives the list of score proportions for all feature layers calculated in step S1033. The server sorts or searches this list to determine the feature layer with the largest score proportion. The feature layer corresponding to this highest score proportion is marked as the optimal feature layer. Next, the server searches the list for the second-largest score proportion and marks the corresponding feature layer as the suboptimal feature layer. At this point, the selection process is complete, and the server has successfully and dynamically selected two of the most suitable feature layers from multiple scales.

[0079] For example, suppose the server has four feature layers F2, F3, F4, and F5, with F5 being the highest level. Due to the large target size, higher-level semantic information is more crucial. In step S1031, the trained multilayer perceptron generates initial confidence scores for each layer, possibly resulting in: F2 0.3, F3 0.6, F4 0.9, and F5 0.8. Then, step S1032 is executed for top-down modulation. The score of F5, 0.8, is the first confidence score and remains unchanged. The new score for F4 is its original score of 0.9 multiplied by the score of F5, 0.8, resulting in 0.72. The new score for F3 is its original score of 0.6 multiplied by the new score of F4, 0.72, resulting in 0.432. The new score for F2 is its original score of 0.3 multiplied by the new score of F3, 0.432, resulting in approximately 0.130. The modulated scores become: F2 is 0.130, F3 is 0.432, F4 is 0.72, and F5 is 0.8. Then, step S1033 is executed. The server calculates the sum of all modulated scores: 0.130 + 0.432 + 0.72 + 0.8 equals 2.082. Subsequently, the score percentages for each layer are calculated: F2 accounts for approximately 6.2%, F3 for approximately 20.7%, F4 for approximately 34.6%, and F5 for approximately 38.4%. Finally, step S1034 is executed. The server compares all score percentages and finds that F5 has the highest percentage at 38.4%, therefore F5 is determined as the optimal feature layer. F4 has the second highest percentage at 34.6%, therefore F4 is determined as the second-best feature layer. Ultimately, the server successfully selected the most important high-level feature layer F5 and the second most important high-level feature layer F4 for this image containing a large target.

[0080] Step S104: Merge multiple target feature layers that were not selected in the feature layer into the optimal feature layer and the suboptimal feature layer respectively to obtain the enhanced optimal feature layer and the enhanced suboptimal feature layer.

[0081] In step S104, the multiple target feature layers that were not filtered out refer to all feature layers other than the optimal and suboptimal feature layers in step S103. Fusion refers to the process of selectively merging the information contained in these unfiltered feature layers into the filtered feature layers, with the aim of compensating for useful information that may be lost due to filtering. Enhancing the optimal feature layer and enhancing the suboptimal feature layer refer to the optimal and suboptimal feature layers whose content representation is strengthened after receiving fused information from the unfiltered layers, respectively.

[0082] Specifically, the server first processes all target feature layers that were not selected in step S103. For any unselected target feature layer, the server calculates the ratio of its spatial resolution to the optimal feature layer resolution and the ratio to the second-best feature layer resolution. Based on these two ratios, the server uses a preset rule to select layers with closer resolutions as fusion targets, determining whether the unselected layer should be fused to the optimal or second-best feature layer. If it is determined to be fused to the optimal feature layer, the server performs a dimension alignment operation on the unselected layer, i.e., upsampling (such as bilinear interpolation) or downsampling (such as max pooling) to make its spatial resolution completely consistent with the optimal feature layer. All dimension-aligned feature layers designated to be fused to the optimal feature layer are aggregated to form a fused feature block. Finally, the server superimposes this fused feature block onto the original optimal feature layer using residual joins, thereby obtaining an enhanced optimal feature layer with supplemented information. For unselected layers that need to be fused to the second-best feature layer, the server performs the exact same process, ultimately obtaining an enhanced second-best feature layer.

[0083] In one possible implementation, please refer to Figure 2 The multiple target feature layers that were not selected in the feature layer are fused into the optimal feature layer and the suboptimal feature layer respectively to obtain the enhanced optimal feature layer and the enhanced suboptimal feature layer. Specifically, this includes steps S201-S208, as follows: Step S201: Calculate the first spatial resolution ratio of the first target feature layer to the optimal feature layer, and calculate the second spatial resolution ratio of the first target feature layer to the second-best feature layer. The first target feature layer is any one of the multiple target feature layers.

[0084] In step S201, the first target feature layer represents a specific layer currently selected for processing from the set of all unselected feature layers. The optimal and second-best feature layers refer to the two most important feature layers selected based on confidence scores in previous steps. The first spatial resolution ratio and the second spatial resolution ratio are both dimensionless values, used to quantify the size difference between the spatial size of the first target feature layer and the optimal and second-best feature layers, respectively. For example, if the resolution of one feature layer is twice that of the other, the ratio is 2 or 0.5.

[0085] Specifically, the server first selects one from all unselected target feature layers and designates it as the first target feature layer. Next, the server reads the spatial dimension information of this first target feature layer, namely its height and width. Simultaneously, the server also obtains the spatial dimension information of the previously determined optimal and suboptimal feature layers. Then, the server performs two ratio calculations: First, it divides the spatial resolution of the first target feature layer (e.g., the total number of pixels in the height multiplied by the width, or, if the aspect ratio is fixed, only the height or width) by the spatial resolution of the optimal feature layer; the result is the first spatial resolution ratio. Second, in the same manner, it divides the spatial resolution of the first target feature layer by the spatial resolution of the suboptimal feature layer to obtain the second spatial resolution ratio. These two calculated ratios together constitute the quantitative input for subsequent decision-making steps, providing a crucial basis for determining which selected feature layer the first target feature layer should compensate for.

[0086] Step S202: Based on the ratio of the first spatial resolution to the ratio of the second spatial resolution, a reference feature layer corresponding to the first target feature layer is determined using a preset rule. The reference feature layer is either the optimal feature layer or the suboptimal feature layer.

[0087] In step S202, the preset rule refers to a predefined algorithm or logical judgment standard used for decision-making, the purpose of which is to select the most suitable fusion target based on the input ratio. The reference feature layer refers to the one selected from the optimal feature layer and the second-best feature layer according to the preset rule, which will serve as the destination for receiving information from the first target feature layer. The process of determining the reference feature layer is described below.

[0088] Specifically, after obtaining the ratios of the first and second spatial resolutions, the server applies a pre-defined decision rule to determine the reference feature layer. The core idea of ​​this rule is to minimize information distortion that may be introduced in subsequent dimensional alignment operations. To this end, the server needs to determine which of the optimal and suboptimal feature layers the first target feature layer is closer to in terms of spatial resolution. The server achieves this determination by calculating the absolute difference between each ratio and the value 1. That is, the server calculates the first distance value, which is equal to the absolute value of the first spatial resolution ratio minus 1; simultaneously, it calculates the second distance value, which is equal to the absolute value of the second spatial resolution ratio minus 1. The server then compares these two distance values. If the first distance value is less than or equal to the second distance value, it indicates that the resolution difference between the first target feature layer and the optimal feature layer is smaller, and the server therefore determines the optimal feature layer as the reference feature layer for this fusion operation. Conversely, if the second distance value is smaller, the server determines the suboptimal feature layer as the reference feature layer. Through this explicit step, the server finds the most suitable fusion object for the currently processed first target feature layer, thus laying the foundation for subsequent feature fusion and information compensation.

[0089] For example, suppose the server generates four feature layers F2, F3, F4, and F5, with spatial resolutions of 128x128, 64x64, 32x32, and 16x16, respectively. After filtering, F2 is determined to be the optimal feature layer, and F3 is determined to be the second-best feature layer. Therefore, the unfiltered target feature layers are F4 and F5.

[0090] Now, the server determines its reference feature layer for F4. First, the server executes step S201 with F4 as the first target feature layer. It calculates the first spatial resolution ratio of F4 to the optimal feature layer F2, which is 32 divided by 128, resulting in 0.25. Next, it calculates the second spatial resolution ratio of F4 to the second-best feature layer F3, which is 32 divided by 64, resulting in 0.5.

[0091] Then, the server executes step S202. It applies a preset rule to compare which ratio is closer to 1. The server calculates the absolute difference between the first ratio and 1, i.e., |0.25-1|, resulting in 0.75. It then calculates the absolute difference between the second ratio and 1, i.e., |0.5-1|, resulting in 0.5. Since 0.5 is less than 0.75, this indicates that the spatial resolution of F4 differs less from that of the suboptimal feature layer F3. Therefore, according to the preset rule, the server determines that the reference feature layer corresponding to F4 is the suboptimal feature layer F3. Subsequent fusion operations will then fuse the feature information of F4 into F3 to compensate for any information that may have been lost during the selection of F4.

[0092] Step S203: If the reference feature layer is the optimal feature layer, then determine the relationship between the first spatial resolution of the first target feature layer and the second spatial resolution of the optimal feature layer.

[0093] In step S203, the reference feature layer refers to the target layer for information fusion determined in the previous step for the first target feature layer. The optimal feature layer is the layer with the highest confidence score among all feature layers. The first target feature layer represents the feature layer currently being processed that has not been filtered out. The first spatial resolution refers to the spatial size of the first target feature layer, usually expressed in pixels for its height and width. The second spatial resolution refers to the spatial size of the optimal feature layer. The core of this step is a conditional judgment used to determine whether the size of the first target feature layer needs to be enlarged or reduced in the subsequent steps to align with the optimal feature layer.

[0094] Specifically, the server first performs a precondition check to confirm whether the reference feature layer determined for the current first target feature layer in step S202 is indeed the optimal feature layer. Only if this condition is met will the server continue with this step. If the condition is met, the server then obtains the spatial height and width of the first target feature layer, as well as the spatial height and width of the optimal feature layer. Subsequently, the server determines the spatial resolution relationship between the two feature layers by comparing their total pixel count or, if the aspect ratios are the same, directly comparing a single dimension such as the height value. This determination will produce three possible results: the first spatial resolution is less than, greater than, or equal to the second spatial resolution. This result will serve as the basis for executing subsequent steps S204 and S205.

[0095] Step S204: If the first spatial resolution is less than the second spatial resolution, then the first target feature layer is upsampled to obtain the second target feature layer.

[0096] In step S204, upsampling refers to a technique that increases the spatial resolution of a feature layer, with the aim of enlarging a smaller feature map. The second target feature layer is a new feature layer generated after upsampling, and its spatial resolution is completely consistent with the second spatial resolution of the optimal feature layer. This step describes how to unify their dimensions through an enlargement operation when the size of the first target feature layer is smaller than that of the optimal feature layer.

[0097] Specifically, the server initiates this step based on the judgment result of step S203. If the judgment result is that the first spatial resolution is less than the second spatial resolution, the server will perform an upsampling operation on the first target feature layer. This operation is implemented using a bilinear interpolation algorithm. The server first determines the target size for upsampling, i.e., the second spatial resolution of the optimal feature layer. Then, for each position in the newly generated second target feature layer, the bilinear interpolation algorithm finds its four nearest neighbor pixels on the original first target feature layer. The feature value of the new position is obtained by interpolating the feature values ​​of these four points using a distance-based weighted average. The server repeats this process for all positions in the second target feature layer, ultimately generating a second target feature layer with an increased size that is perfectly aligned with the dimension of the optimal feature layer. This layer smoothly preserves the semantic information of the original feature layer.

[0098] Step S205: If the first spatial resolution is greater than the second spatial resolution, then downsample the first target feature layer to obtain the second target feature layer.

[0099] In step S205, downsampling is a technique for reducing the spatial resolution of a feature layer, used to shrink a large feature map. The second target feature layer is a new feature layer generated after downsampling, and its spatial resolution is exactly the same as the second spatial resolution of the optimal feature layer. This step details how to align the dimensions of the first target feature layer with the optimal feature layer through a shrinking operation when the size of the first target feature layer is larger than that of the optimal feature layer.

[0100] Specifically, the execution of this step also depends on the judgment result of step S203. If the result is that the first spatial resolution is greater than the second spatial resolution, the server performs a downsampling operation on the first target feature layer. This operation is achieved through max pooling. The server first calculates the required pooling kernel size and stride based on the ratio of the first spatial resolution to the second spatial resolution. For example, to halve the resolution, a 2x2 pooling kernel with a stride of 2 is used. Then, the server divides the first target feature layer into multiple non-overlapping rectangular regions, each region being the same size as the pooling kernel. For each region, the server takes only the maximum value of all feature values ​​within that region as the new feature value at the corresponding position on the output second target feature layer. By sliding this operation across the entire first target feature layer, the server ultimately generates a second target feature layer with a reduced size that perfectly matches the dimension of the optimal feature layer, effectively preserving the most significant feature information from the original feature layer.

[0101] For example, suppose the server obtains four feature layers from the backbone network, with spatial resolutions of 256x256 pixels for layer F2, 128x128 pixels for layer F3, 64x64 pixels for layer F4, and 32x32 pixels for layer F5. After prior confidence scoring and selection, layer F4, with its 64x64 resolution, is determined to be the optimal feature layer. The other layers that were not selected are layers F2 and F5. Now, the server needs to fuse the information from layers F2 and F5 into the selected layer.

[0102] First, the F2 layer is processed. Assume that in step S202, the server determines that the reference feature layer for F2 is the optimal feature layer F4. Proceeding to step S203, the server confirms that the reference feature layer for F2 is indeed the optimal feature layer, and then compares the first spatial resolution of F2 (256x256) with the second spatial resolution of F4 (64x64). Because 256x256 is greater than 64x64, the result is "greater than".

[0103] Based on this result, the server skips step S204 and directly executes step S205. The server needs to downsample the F2 layer. Since it needs to be reduced from a side length of 256 pixels to a side length of 64 pixels, the scaling factor is 4. Therefore, the server uses a 4x4 pooling kernel with a stride of 4 to perform max pooling on the F2 layer, finally obtaining a second target feature layer with a size of 64x64.

[0104] Next, we process layer F5. Again, we assume its reference feature layer is determined to be the optimal feature layer F4. We proceed to step S203 again, where the server compares the first spatial resolution of layer F5 (32x32) with the second spatial resolution of layer F4 (64x64). Because 32x32 is less than 64x64, the result is "less than".

[0105] Based on this result, the server executes step S204. The server needs to upsample the F5 layer. Using the 64x64 resolution of the F4 layer as the target size, the server employs a bilinear interpolation algorithm to double the size of the 32x32 F5 layer, generating a second target feature layer with a size of 64x64. After this, step S205 is skipped.

[0106] Through the above process, layers F2 and F5 were successfully converted into feature layers with the same size as the optimal feature layer F4, thus preparing for subsequent feature fusion.

[0107] Step S206: Aggregate all second target feature layers with all reference feature layers as the optimal feature layers to generate fused feature blocks.

[0108] In step S206, the second target feature layer refers to the set of all feature layers that have been size-aligned and referenced the optimal feature layer in previous steps. Aggregation refers to an operation that combines information from multiple independent feature layers into a single feature layer, for example, by adding them pixel by pixel. The fused feature block is the final product of the aggregation operation; it is a single data block that integrates all related, unfiltered feature layer information, and its dimensions are exactly the same as the optimal feature layer.

[0109] Specifically, the server first filters out all second target feature layers generated in the preceding step S204 or S205 from its processing cache. The sole criterion for filtering is that the original reference feature layer of these layers is determined to be the optimal feature layer. Then, the server creates a zero-initialized tensor with the same height, width, and number of channels as the optimal feature layer, serving as an accumulator. Next, the server iterates through all the filtered second target feature layers and adds them element-wise to this tensor. Once all the eligible second target feature layers have been accumulated, this accumulated tensor, containing rich supplementary information, is finally determined as the fused feature block.

[0110] Step S207: The fused feature blocks are superimposed onto the optimal feature layer in a residual connection manner to obtain the enhanced optimal feature layer.

[0111] In step S207, the fused feature block refers to the feature set generated in step S206 that summarizes information from multiple other scales. Residual connections are a common structure in deep learning networks. They help the network learn identity mappings and alleviate the vanishing gradient problem by directly adding the input of one or more layers to the output of subsequent layers. Superposition here specifically refers to element-wise addition. The enhanced optimal feature layer is the output of this step; it is a strengthened version of the original optimal feature layer, supplemented with information from other scales.

[0112] Specifically, the server executes this step immediately after generating the fused feature block. The server obtains this fused feature block and the original optimal feature layer. Since the design dimensions of the fused feature block are exactly the same as those of the optimal feature layer, the server can directly superimpose the two. This superposition operation is performed in the form of residual connections, that is, by adding all corresponding elements at the same position in the two feature layers. In this way, fine-grained or macroscopic information from other scales is effectively injected into the optimal feature layer, rather than simply replacing it. The new feature layer obtained after this addition operation is the enhanced optimal feature layer. It retains the core information of the optimal feature layer itself and absorbs useful features from other discarded scales, thus achieving information compensation and enhancement.

[0113] Assume the server has four feature layers F2, F3, F4, and F5, with resolutions of 256x256, 128x128, 64x64, and 32x32, respectively. The optimal feature layer is determined to be F4 (64x64), and the second-best feature layer is determined to be F3 (128x128). The unselected target feature layers are F2 and F5. After step S202, the server determines that the resolution of layer F5 (32x32) is closer to that of layer F4 (64x64), therefore its reference feature layer is the optimal feature layer F4. The resolution of layer F2 (256x256) is closer to that of layer F3 (128x128), therefore its reference feature layer is the second-best feature layer F3.

[0114] In the previous steps, layer F5 has been upsampled to generate a 64x64 second target feature layer. Now, step S206 is executed. The server searches for the second target feature layer that is the optimal feature layer F4 among all reference feature layers. In this example, only the one from layer F5 is found. Therefore, the server directly uses this 64x64 layer as the fused feature block, as it does not need to be aggregated with other layers.

[0115] Next, step S207 is executed. The server adds this fused feature block, i.e., the upsampled F5 layer, onto the original optimal feature layer F4 using an element-wise additive approach. After the operation is completed, a 64x64 enhanced optimal feature layer is obtained. At this point, the enhancement path for the optimal feature layer is complete.

[0116] Subsequently, the server processes layer F2. Upon inspection, the server finds that the reference feature layer for layer F2 is the suboptimal feature layer F3, thus triggering step S208. The server first performs dimension alignment on layer F2, downsampling its 256x256 resolution to the same 128x128 resolution as layer F3 using max pooling. Then, the server element-wise superimposes this downsampled F2 layer onto the original suboptimal feature layer F3. After this operation, an enhanced 128x128 suboptimal feature layer is obtained.

[0117] Ultimately, through this entire process, the original optimal feature layer F4 and suboptimal feature layer F3 absorbed information from the discarded F5 and F2 layers, respectively, successfully generating enhanced optimal and suboptimal feature layers with richer information.

[0118] Step S208: If the reference feature layer is a suboptimal feature layer, then the first target feature layer is fused with the suboptimal feature layer to obtain an enhanced suboptimal feature layer.

[0119] In step S208, referring to the suboptimal feature layer as the reference feature layer refers to a preliminary conditional judgment. This conditional branch specifically handles those unfiltered feature layers that are closer to the suboptimal feature layer in spatial resolution. Fusion here is a general term encompassing the two core operations of dimensional alignment and feature overlay. The enhanced suboptimal feature layer is the final output of this fusion process; it represents a strengthened version obtained by supplementing the suboptimal feature layer with additional scale information.

[0120] Specifically, for each unfiltered first target feature layer, the server checks its reference feature layer determined in step S202. If the reference feature layer of a first target feature layer is a suboptimal feature layer, the server initiates the processing flow for that layer. This flow first includes a dimension alignment operation, similar to steps S204 and S205. The server determines the relationship between the spatial resolution of the first target feature layer and the spatial resolution of the suboptimal feature layer, and then generates an intermediate feature layer with the exact same size as the suboptimal feature layer through upsampling or downsampling operations. Next, the server adds this dimension-aligned intermediate feature layer onto the current suboptimal feature layer using a residual join approach, i.e., element-wise addition. This process is repeated for each feature layer with the suboptimal feature layer as a reference, and the final feature layer is the enhanced suboptimal feature layer.

[0121] Step S105: Based on the scale-aware foreground score, the tokens of the enhanced optimal feature layer and the enhanced suboptimal feature layer are sparsified to obtain a sparsified token set. The sparsified token set is then input into the detector of the preset target detection model to obtain the target detection result of the input image.

[0122] In step S105, sparsification is a filtering mechanism aimed at removing a large amount of redundant background information while retaining key foreground information, thereby reducing the complexity of subsequent calculations. The sparsified token set refers to the union of high-value tokens selected from the enhanced optimal feature layer and the enhanced suboptimal feature layer. The detector in the pre-defined object detection model typically refers to the final part of the model, such as the encoder and decoder of a Transformer, which is responsible for receiving feature representations and ultimately predicting the location and category of the target. The object detection result is the final output of this method, consisting of the bounding box coordinates and category labels for each target in the input image.

[0123] Specifically, the server obtains the enhanced optimal feature layer and enhanced suboptimal feature layer generated in step S104. For each token in these two layers, the server reuses the scale-aware foreground score calculated in step S102. Based on these scores, the server filters the tokens. For example, the server can set a threshold or a retention ratio (e.g., retaining only the top 30% of tokens in each layer), treating tokens with scores below the threshold or not ranked highly as background and discarding them. After this processing, the tokens retained from the two layers together constitute a sparse token set. Subsequently, the server inputs this concise but information-rich sparse token set, along with its positional encoding information, into the detector part of the object detection model. The Transformer encoder and decoder in the detector perform self-attention and cross-attention calculations on these tokens, decode and extract the target information, and finally output a series of prediction results, including the bounding box position, size, and corresponding class confidence of each detected target. This is the final object detection result for the input image.

[0124] In one possible implementation, the tokens for enhancing the optimal feature layer and the suboptimal feature layer are sparsified based on the scale-aware foreground score to obtain a sparse token set. Specifically, this includes steps S1051-S1056, as follows: Step S1051: Determine the first-level focus ratio corresponding to the enhanced optimal feature layer, and determine the second-level focus ratio corresponding to the enhanced suboptimal feature layer, wherein the first-level focus ratio is greater than the second-level focus ratio.

[0125] In step S1051, the enhanced optimal feature layer and the enhanced suboptimal feature layer refer to the two feature layers that were selected and enhanced through information fusion in the previous steps. The first-level focus ratio is a preset value used to represent the proportion of tokens retained from the optimal feature layer, for example, retaining 70%. The second-level focus ratio is another preset value used to represent the proportion of tokens retained from the suboptimal feature layer, for example, retaining 50%. The core idea of ​​this step is to assign a higher retention rate to the more important feature layers to ensure that key information is retained to the greatest extent.

[0126] Specifically, the server first receives the enhanced optimal feature layer and enhanced suboptimal feature layer determined in the previous steps. The server accesses its internal preset parameter configuration, which defines different level focus ratios for feature layers of different importance levels. The server allocates a higher ratio value, i.e., the first-level focus ratio, to the enhanced optimal feature layer. Simultaneously, it allocates a lower ratio value, i.e., the second-level focus ratio, to the enhanced suboptimal feature layer. This differentiated allocation strategy ensures that when the model performs information sparsification, it prioritizes retaining information from the most critical feature layers while appropriately reducing the amount of information from secondary feature layers.

[0127] Step S1052: Determine the first preset quantity based on the first level focus ratio and the preset ratio, and determine the second preset quantity based on the second level focus ratio and the preset ratio.

[0128] In step S1052, the preset ratio in this context refers to the total number of tokens contained in each feature layer. The first preset number refers to the specific number of tokens that need to be selected from the optimal feature layer, calculated based on the retention ratio. The second preset number refers to the specific number of tokens that need to be selected from the suboptimal feature layer accordingly. This step aims to convert the relative retention ratio into an absolute selection number.

[0129] Specifically, the server first processes the enhanced optimal feature layer, calculating the total number of tokens contained in that layer. This total number is the preset ratio used for calculation. Then, the server multiplies this total number by the first-level focus ratio determined in step S1051. After rounding the product, the specific number of tokens to be selected from that layer is obtained, i.e., the first preset number. The server processes the enhanced suboptimal feature layer in the same way, multiplying its total number of tokens by the second-level focus ratio and rounding it to determine the second preset number.

[0130] Step S1053: In the tokens of the enhanced optimal feature layer, select a first preset number of tokens based on the scale-aware foreground score to obtain the first token set.

[0131] In step S1053, the scale-aware foreground score is a comprehensive score previously calculated for each token, measuring its importance as foreground. The selection operation refers to the process of sorting and filtering according to the scores. The first token set is the set of tokens with the highest scores retained from the optimal feature layer after filtering.

[0132] Specifically, the server first obtains all tokens in the enhanced optimal feature layer, along with the scale-aware foreground score corresponding to each token. The server then sorts all tokens in descending order of their scale-aware foreground scores, placing the token with the highest score at the front of the list. Next, the server selects a first predetermined number of tokens from this sorted list, starting from the beginning. These selected tokens together constitute the first token set.

[0133] Step S1054: In the tokens of the enhanced suboptimal feature layer, select a second preset number of tokens based on the scale-aware foreground score to obtain the second token set.

[0134] In step S1054, the entity words involved are similar to those in step S1053, but the operation target is the suboptimal feature layer. The second token set is the set of tokens with the highest scores retained from the suboptimal feature layer after filtering.

[0135] Specifically, the server performs the same processing procedure as step S1053 on the enhanced suboptimal feature layer. The server obtains all tokens of this layer and their corresponding scale-aware foreground scores, and sorts the tokens in descending order according to the scores. Then, the server selects a second preset number of tokens from the sorted list. These selected tokens constitute the second token set.

[0136] Step S1055: Merge the first token set and the second token set to obtain a candidate sparse token set.

[0137] In step S1055, the first token set and the second token set are the high-scoring token sets selected in the first two steps. The merging operation refers to combining two independent sets into a larger set. The candidate sparse token set is the preliminary set obtained after merging, containing all the selected tokens. It is called a candidate because it still needs to undergo final redundancy removal processing.

[0138] Specifically, the server takes the first token set generated in step S1053 and the second token set generated in step S1054 as input. The server performs a union operation on the sets, combining all tokens from the two sets into a new set. This newly generated set, which contains all high-scoring foreground tokens from both feature layers, is defined as the candidate sparse token set.

[0139] Step S1056: Perform nonmaximum suppression processing on the candidate sparse token set to obtain the sparse token set.

[0140] In step S1056, non-maximum suppression (NMS) is a post-processing algorithm used to eliminate redundant detection results or features that are spatially adjacent and overlap in information. In this scenario, it is applied at the token level to remove spatially overly dense low-scoring foreground tokens. The sparse token set is the final token set after NMS processing, which more efficiently represents foreground information in the image and significantly reduces the burden of subsequent computation.

[0141] Specifically, the server performs non-maximum suppression on the candidate sparse token set. First, the server sorts all tokens in the set in descending order of their scale-aware foreground scores. Then, the server iterates starting with the token with the highest score. For the current token, the server checks all other tokens in the set. If a token's spatial distance from the current token is less than a preset threshold, and its score is lower than the current token's, then that token is suppressed or removed. This process is repeated until all tokens in the set have been processed. The final set of tokens retained is the sparse token set, which contains the most representative and spatially distributed foreground tokens.

[0142] To illustrate the method of this application more clearly, the following example is used to illustrate a scenario of detecting a small aircraft in a remote sensing image.

[0143] Suppose the input is a high-resolution (e.g., 2048x2048 pixels) remote sensing image.

[0144] First, in step S101, the server processes the image using the ResNet-50 backbone network, generating four initial feature layers with different resolutions, which we call P2, P3, P4, and P5, with the resolution decreasing sequentially and the semantic information increasing sequentially.

[0145] Then, in step S102, the server performs global context modeling on the four initial feature layers P2, P3, P4, and P5 respectively, resulting in new feature layers F2, F3, F4, and F5 containing long-range dependency information. Since the aircraft is a small target, when calculating the scale-aware foreground score, the tokens corresponding to the aircraft region in layers F2 and F3 will receive higher scores due to their higher level of detail and better localization accuracy (higher intersection-union score).

[0146] Next, in step S103, the server calculates the confidence score for each feature layer based on these scores. Because the aircraft is a small target, the model determines that preserving high-resolution features is crucial for detection. Therefore, after top-down modulation, layer F2 is likely to obtain the highest score percentage and is identified as the optimal feature layer; layer F3 scores second and is identified as the suboptimal feature layer. Layers F4 and F5 become the unselected target feature layers.

[0147] Next, in step S104, the server needs to fuse the unfiltered F4 and F5 layers. Since the resolution of layer F4 is closer to that of layer F3 (the suboptimal feature layer), the server will fuse it into layer F3. Specifically, layer F4 is upsampled to align its resolution with that of layer F3, and then superimposed on layer F3 using residual connections to generate an enhanced suboptimal feature layer. Similarly, layer F5 is also upsampled and fused into either layer F3 or F2, which has a closer resolution. Assuming that both F4 and F5 are ultimately fused into F3, we obtain the original F2 (enhanced optimal feature layer) and the fused F3 (enhanced suboptimal feature layer).

[0148] Finally, in step S105, the server uses the original foreground scores calculated in step S102 to perform sparsification on the enhanced F2 and F3 layers, retaining only the tokens with the highest scores. These tokens are highly likely to correspond to the aircraft itself or its key components. This sparsified token set, composed of tokens selected from the two layers, is fed into the Transformer detector. The detector processes this selected feature information and ultimately accurately outputs the bounding box locations and "aircraft" category labels for all small aircraft in the image, completing the detection task efficiently and accurately.

[0149] Reference Figure 3This application also provides a deep learning-based object detection device, which is a server. The server includes an initial feature layer acquisition module 301, a global context modeling module 302, a confidence scoring module 303, a feature fusion module 304, and an output result module 305. Specifically: the initial feature layer acquisition module 301 processes the input image using the backbone network of a preset object detection model to obtain multiple initial feature layers of different resolutions; the global context modeling module 302 performs global context modeling on each initial feature layer to obtain multiple feature layers, and calculates scale-aware foreground scores for multiple tokens in each feature layer; the confidence scoring module 303... The module 304 is used to generate a confidence score for each feature layer based on the scale-aware foreground score, and to select the best and second-best feature layers from multiple feature layers based on the confidence score; the feature fusion module 304 is used to fuse multiple target feature layers that were not selected from the feature layers into the best and second-best feature layers respectively, to obtain the enhanced best feature layer and the enhanced second-best feature layer; the output result module 305 is used to perform sparsification processing on the tokens of the enhanced best feature layer and the enhanced second-best feature layer based on the scale-aware foreground score, to obtain a sparse token set, and input the sparse token set into the detector of the preset target detection model to obtain the target detection result of the input image.

[0150] In one possible implementation, the confidence scoring module 303 fuses multiple unselected target feature layers in the feature layer into the optimal feature layer and the suboptimal feature layer, respectively, to obtain an enhanced optimal feature layer and an enhanced suboptimal feature layer. Specifically, the confidence scoring module 303 calculates a first spatial resolution ratio between the first target feature layer and the optimal feature layer, and calculates a second spatial resolution ratio between the first target feature layer and the suboptimal feature layer, wherein the first target feature layer is any one of the multiple target feature layers; based on the first spatial resolution ratio and the second spatial resolution ratio, the confidence scoring module 303 determines the first target feature layer using a preset rule. The corresponding reference feature layer can be either the optimal or suboptimal feature layer. If the reference feature layer is the optimal feature layer, the confidence scoring module 303 performs a dimension alignment operation on the first target feature layer to obtain the second target feature layer. The confidence scoring module 303 aggregates all second target feature layers whose reference feature layers are optimal to generate a fused feature block. The confidence scoring module 303 then superimposes the fused feature block onto the optimal feature layer using residual connections to obtain an enhanced optimal feature layer. If the reference feature layer is a suboptimal feature layer, the confidence scoring module 303 fuses the first target feature layer with the suboptimal feature layer to obtain an enhanced suboptimal feature layer.

[0151] In one possible implementation, the confidence scoring module 303 performs a dimension alignment operation on the first target feature layer to obtain the second target feature layer. Specifically, the confidence scoring module 303 determines the relationship between the first spatial resolution of the first target feature layer and the second spatial resolution of the optimal feature layer. If the first spatial resolution is less than the second spatial resolution, the confidence scoring module 303 upsamples the first target feature layer to obtain the second target feature layer. If the first spatial resolution is greater than the second spatial resolution, the confidence scoring module 303 downsamples the first target feature layer to obtain the second target feature layer.

[0152] In one possible implementation, the global context modeling module 302 performs global context modeling on each initial feature layer to obtain multiple feature layers. Specifically, the global context modeling module 302 expands the initial feature layer into multiple one-dimensional feature sequences along a preset scanning direction, where the preset scanning direction includes top to bottom, bottom to top, left to right, and right to left. The global context modeling module 302 applies a preset state space model operator to each one-dimensional feature sequence to perform sequence modeling, resulting in multiple processed one-dimensional sequences. The global context modeling module 302 merges all processed one-dimensional sequences according to their corresponding original spatial positions and reshapes them into a two-dimensional structure to obtain the feature layers.

[0153] In one possible implementation, the global context modeling module 302 calculates scale-aware foreground scores for multiple tokens in each feature layer. Specifically, this includes: for each deformable reference point corresponding to a token, the global context modeling module 302 calculates the intersection-union ratio (IU) between the predicted bounding box and the ground truth bounding box of the deformable reference point, generating an IU score for each token; the global context modeling module 302 enhances the channel weights of each feature layer and performs convolution and normalization operations on the channel weights of each feature layer to obtain a foreground-background confidence score for each token; the global context modeling module 302 performs a weighted summation of the IU score and the foreground-background confidence score to obtain a weighted summation result; the global context modeling module 302 performs maximum / minimum normalization on the weighted summation result to obtain a scale-aware foreground score for each token. The formula for calculating the scale-aware foreground score is as follows:

[0154] Among them, F ij Let P represent the scale-aware foreground score of the j-th token in the i-th feature layer, where Norm() is the extremum normalization function. box D represents the predicted bounding box. box Represents the true bounding box. Let α be the intersection, union, and ratio of fractions with reference point. fgLet W be the weighting factor, W be the channel weight matrix, X be the eigenvector representation of the token, and b be the bias term. Foreground-background confidence score.

[0155] In one possible implementation, the confidence scoring module 303 generates a confidence score for each feature layer based on the scale-aware foreground score, and selects the optimal and suboptimal feature layers from multiple feature layers based on the confidence scores. Specifically, the confidence scoring module 303 uses a multilayer perceptron trained with the scale-aware foreground score as the supervision signal to process each feature layer and obtain a confidence score corresponding to each feature layer; the confidence scoring module 303 performs a top-down score modulation process, adjusting a second confidence score based on a first confidence score, where the first confidence score is the confidence score corresponding to the highest-level feature layer, and the second confidence score is the confidence score below the first confidence score, generating multiple modulated confidence scores; the confidence scoring module 303 calculates the score percentage of each feature layer relative to the total score of all feature layers based on the modulated confidence scores; and the confidence scoring module 303 determines the feature layer with the highest score percentage as the optimal feature layer and the feature layer with the second highest score percentage as the suboptimal feature layer based on the score percentage.

[0156] In one possible implementation, the output result module 305 performs sparsification processing on the tokens of the enhanced optimal feature layer and the enhanced suboptimal feature layer based on the scale-aware foreground score to obtain a sparse token set. Specifically, the output result module 305 determines the first-level focus ratio corresponding to the enhanced optimal feature layer and the second-level focus ratio corresponding to the enhanced suboptimal feature layer, wherein the first-level focus ratio is greater than the second-level focus ratio; the output result module 305 determines a first preset quantity based on the first-level focus ratio and a preset ratio, and determines a second preset quantity based on the second-level focus ratio and the preset ratio. The output result module 305 selects a first preset number of tokens from the tokens in the enhanced optimal feature layer based on the scale-aware foreground score, to obtain a first token set; the output result module 305 selects a second preset number of tokens from the tokens in the enhanced suboptimal feature layer based on the scale-aware foreground score, to obtain a second token set; the first token set and the second token set are merged to obtain a candidate sparsified token set; the output result module 305 performs nonmaximum suppression processing on the candidate sparsified token set to obtain a sparsified token set.

[0157] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0158] This application also provides an electronic device. (See reference...) Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 400 may include: at least one processor 401, at least one network interface 404, a user interface 403, a memory 405, and at least one communication bus 402. The communication bus 402 is used to enable communication between these components. The user interface 403 may include a display screen or a camera; optionally, the user interface 403 may also include a standard wired interface or a wireless interface. The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The processor 401 may include one or more processing cores. The processor 401 connects to various parts of the server using various interfaces and lines, and performs various functions of the server and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 405, and by calling data stored in the memory 405.

[0159] The memory 405 may include random access memory (RAM) or read-only memory. (See reference...) Figure 4 The memory 405, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program based on a deep learning-based object detection method.

[0160] exist Figure 4In the illustrated electronic device 400, the user interface 403 is mainly used to provide an input interface for the user and acquire user input data; while the processor 401 can be used to call an application program of a deep learning-based object detection method stored in the memory 405. When executed by one or more processors 401, the electronic device 400 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0161] This application also provides a computer-readable storage medium storing instructions. When executed by one or more processors 401, these instructions cause the electronic device 400 to perform one or more of the methods described in the above embodiments.

[0162] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0163] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A deep learning-based object detection method, characterized in that, The method includes: The input image is processed using the backbone network of a pre-defined target detection model to obtain multiple initial feature layers with different resolutions; Global context modeling is performed on each of the initial feature layers to obtain multiple feature layers, and scale-aware foreground scores are calculated for each of the multiple tokens in each feature layer. Based on the scale-aware foreground score, a confidence score is generated for each feature layer, and the optimal and second-best feature layers are selected from multiple feature layers according to the confidence score. Multiple target feature layers that were not selected in the feature layer are respectively fused into the optimal feature layer and the suboptimal feature layer to obtain the enhanced optimal feature layer and the enhanced suboptimal feature layer; Based on the scale-aware foreground score, the tokens of the enhanced optimal feature layer and the enhanced suboptimal feature layer are sparsified to obtain a sparse token set. The sparse token set is then input into the detector of the preset target detection model to obtain the target detection result of the input image.

2. The method according to claim 1, characterized in that, The step of fusing multiple unselected target feature layers from the feature layers into the optimal feature layer and the suboptimal feature layer, respectively, to obtain an enhanced optimal feature layer and an enhanced suboptimal feature layer, specifically includes: Calculate the first spatial resolution ratio between the first target feature layer and the optimal feature layer, and calculate the second spatial resolution ratio between the first target feature layer and the suboptimal feature layer, wherein the first target feature layer is any one of the plurality of target feature layers; Based on the first spatial resolution ratio and the second spatial resolution ratio, a reference feature layer corresponding to the first target feature layer is determined using a preset rule. The reference feature layer is the optimal feature layer or the suboptimal feature layer. If the reference feature layer is the optimal feature layer, then the first target feature layer is subjected to a dimension alignment operation to obtain the second target feature layer; Aggregate all reference feature layers that are the optimal feature layers of the second target feature layer to generate a fused feature block; The fused feature blocks are superimposed onto the optimal feature layer using residual connections to obtain the enhanced optimal feature layer; If the reference feature layer is the suboptimal feature layer, then the first target feature layer is fused with the suboptimal feature layer to obtain the enhanced suboptimal feature layer.

3. The method according to claim 2, characterized in that, The step of performing a dimension alignment operation on the first target feature layer to obtain the second target feature layer specifically includes: Determine the relationship between the first spatial resolution of the first target feature layer and the second spatial resolution of the optimal feature layer; If the first spatial resolution is less than the second spatial resolution, then the first target feature layer is upsampled to obtain the second target feature layer; If the first spatial resolution is greater than the second spatial resolution, then the first target feature layer is downsampled to obtain the second target feature layer.

4. The method according to claim 1, characterized in that, The step of performing global context modeling on each of the initial feature layers to obtain multiple feature layers specifically includes: The initial feature layer is expanded into multiple one-dimensional feature sequences along a preset scanning direction, which includes top to bottom, bottom to top, left to right, and right to left. A preset state-space model operator is applied to each of the one-dimensional feature sequences to perform sequence modeling, resulting in multiple processed one-dimensional sequences. All processed one-dimensional sequences are merged according to their corresponding original spatial positions and reshaped into a two-dimensional structure to obtain the feature layer.

5. The method according to claim 1, characterized in that, The calculation of scale-aware foreground scores for each of the multiple tokens in each feature layer specifically includes: For each deformable reference point corresponding to the token, calculate the intersection-union ratio (IU) between the predicted bounding box and the true bounding box of the deformable reference point, and generate the IU score for the reference point of each token; The channel weights of each feature layer are enhanced, and the channel weights of each feature layer are processed by convolution and normalization functions to obtain the foreground-background confidence score of each token. The reference point intersection-union score and the foreground-background confidence score are weighted and summed to obtain a weighted sum result; The weighted summation result is subjected to extremum normalization to obtain the scale-aware foreground score corresponding to each token. The formula for calculating the scale-aware foreground score is as follows: ； Among them, F ij Let P represent the scale-aware foreground score of the j-th token in the i-th feature layer, where Norm() is the extremum normalization function. box D represents the predicted bounding box. box Represents the true bounding box. Let α be the intersection-union ratio of the reference point. fg Let W be the weighting factor, W be the channel weight matrix, X be the eigenvector representation of the token, and b be the bias term. The foreground-background confidence score is given.

6. The method according to claim 1, characterized in that, The process of generating a confidence score for each feature layer based on the scale-aware foreground score, and selecting the optimal and second-best feature layers from multiple feature layers based on the confidence scores, specifically includes: A multilayer perceptron trained with the scale-aware foreground score as a supervision signal is used to process each feature layer to obtain a confidence score corresponding to each feature layer. A top-down score modulation process is performed, and a second confidence score is adjusted based on a first confidence score. The first confidence score is the confidence score corresponding to the highest level feature layer, and the second confidence score is the confidence score below the first confidence score, generating multiple modulated confidence scores. Based on the post-modulation confidence score, calculate the score percentage of each feature layer in the total score of all feature layers; Based on the score percentage, the feature layer with the highest score percentage is determined as the optimal feature layer, and the feature layer with the second highest score percentage is determined as the suboptimal feature layer.

7. The method according to claim 1, characterized in that, The process of sparsifying the tokens of the enhanced optimal feature layer and the enhanced suboptimal feature layer based on the scale-aware foreground score to obtain a sparse token set specifically includes: Determine the first-level focus ratio corresponding to the enhanced optimal feature layer, and determine the second-level focus ratio corresponding to the enhanced suboptimal feature layer, wherein the first-level focus ratio is greater than the second-level focus ratio; Based on the first level focusing ratio and the preset ratio, a first preset quantity is determined, and based on the second level focusing ratio and the preset ratio, a second preset quantity is determined; In the tokens of the enhanced optimal feature layer, based on the scale-aware foreground score, the first preset number of tokens are selected to obtain the first token set; In the tokens of the enhanced suboptimal feature layer, based on the scale-aware foreground score, a second preset number of tokens are selected to obtain a second token set; The first token set and the second token set are merged to obtain a candidate sparse token set; The candidate sparse token set is subjected to nonmaximum suppression processing to obtain the sparse token set.

8. A target detection device based on deep learning, characterized in that, The device includes an initial feature layer acquisition module (301), a global context modeling module (302), a confidence scoring module (303), a feature fusion module (304), and an output result module (305), wherein: The initial feature layer acquisition module (301) is used to process the input image using the backbone network of a preset target detection model to obtain multiple initial feature layers with different resolutions. The global context modeling module (302) is used to perform global context modeling on each of the initial feature layers to obtain multiple feature layers, and to calculate the scale-aware foreground score for each of the multiple tokens in each feature layer. The confidence scoring module (303) is used to generate a confidence score for each feature layer based on the scale-aware foreground score, and to select the best and second-best feature layers from multiple feature layers based on the confidence scores. The feature fusion module (304) is used to fuse multiple target feature layers that have not been screened in the feature layer into the optimal feature layer and the suboptimal feature layer respectively, to obtain the enhanced optimal feature layer and the enhanced suboptimal feature layer; The output result module (305) is used to perform sparsification processing on the tokens of the enhanced optimal feature layer and the tokens of the enhanced suboptimal feature layer based on the scale-aware foreground score, to obtain a sparsified token set, and input the sparsified token set into the detector of the preset target detection model to obtain the target detection result of the input image.

9. An electronic device, characterized in that, The device includes a processor (401), a memory (405), a user interface (403), and a network interface (404). The memory (405) is used to store instructions. The user interface (403) and the network interface (404) are used to communicate with other devices. The processor (401) is used to execute the instructions stored in the memory (405) to cause the electronic device (400) to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images