A target detection method based on multi-scale attention and network architecture search

Abstract

This application discloses a target detection method based on multi-scale attention and network architecture search. By introducing deep-guided adaptive feature extraction and a multi-scale contextual fusion attention mechanism, this invention can dynamically adjust the feature extraction strategy and optimize the network architecture design, thereby significantly improving the accuracy and robustness of target detection in complex scenes. Through the multi-scale contextual fusion attention mechanism, the spatial relationships between image regions can be modeled at both global and local levels, enhancing the model's understanding of complex scenes and further improving the accuracy of target detection. Utilizing neural network architecture search technology, the optimal network architecture can be automatically explored, avoiding the limitations of traditional manual network architecture design. The NAS framework dynamically optimizes the feature extraction strategy, improving detection performance while minimizing computational costs, making it more efficient and practical in real-world applications and improving the model's adaptability and performance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and relates to a method for object detection based on multi-scale attention and network architecture search. Background Technology

[0002] Network Architecture Search (NAS)-based object detection methods aim to improve the accuracy and robustness of organ detection in medical images by automating the design of optimal neural network architectures. With the widespread application of deep learning in medical image analysis, while traditional manual network architecture design has achieved some success, it still faces challenges in complex scenarios (such as blurred organ boundaries, lesion interference, or partial occlusion). Especially in CT or MRI images, organs may be partially occluded by surrounding tissues, lesion areas, or other organs, leading to incomplete target feature extraction and affecting detection performance. Therefore, utilizing network architecture search technology to automatically explore highly adaptable network architectures with excellent feature extraction capabilities is crucial for solving occlusion problems and improving the accuracy of object detection. By introducing a depth-guided attention mechanism and a multi-scale feature fusion strategy, NAS can dynamically adjust the network structure, enhancing the model's ability to learn features from occluded areas, thereby achieving more accurate target localization and recognition in complex medical images.

[0003] This approach offers a flexible and cost-effective solution for practice due to its efficient architecture, capable of accurately detecting and locating targets from single CT or MRI images. It rapidly identifies target regions in complex images through automated feature extraction and optimized network design, reducing manual intervention and adapting to resource-constrained medical environments, achieving high-precision detection without expensive equipment. However, target detection still faces two major challenges: first, targets in a single image may be blurred due to image quality, noise, or occlusion, making it difficult for traditional methods to accurately extract features; second, target feature requirements differ at different depth ranges, and traditional methods struggle to account for the differences between nearby organs and distant lesions, affecting detection results.

[0004] To address the aforementioned issues, we explore advanced deep learning and multi-scale feature fusion techniques to enhance the model's ability to identify fuzzy targets and optimize the network architecture, thereby improving detection accuracy and robustness and providing more reliable support for practical applications. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a method based on multi-scale contextual fusion attention and neural architecture search (NAS) to solve the problem of decreased detection performance caused by target blurring in a single image and different feature requirements at different depth ranges in existing technologies. By introducing a depth-guided module and a multi-scale contextual fusion attention mechanism, the feature extraction strategy can be dynamically adjusted, the network structure optimized, and the target accurately located.

[0006] In a first aspect, embodiments of this application provide a target detection method based on multi-scale attention and network architecture search, comprising the following steps:

[0007] S1: Acquire the input human body image, extract features from the human body image through the backbone network, and obtain multi-scale feature maps and the visual and depth features of the multi-scale feature maps.

[0008] S2: Input the visual features and the depth features into a multi-scale context fusion encoder to perform multi-scale context fusion and obtain the multi-scale context fusion result;

[0009] S3: The multi-scale context fusion results are processed by the depth prediction module to generate a panoramic depth map;

[0010] S4: Divide the generated panoramic depth map into multiple depth ranges, and perform feature extraction and search on the multiple depth ranges through multiple network search architectures to obtain multiple target search results;

[0011] S5: Calculate the prediction loss corresponding to the multiple target search results based on the MLP detection method;

[0012] S6: Select the target search results obtained by the network search architecture corresponding to the minimum prediction loss, and output them as the final target.

[0013] Preferably, the human images are derived from publicly available datasets used for visual analysis of endoscopic surgery; the backbone network includes a ResNet-50 model and a lightweight depth predictor.

[0014] Preferably, step S1 includes:

[0015] S11: The human body image is processed by the ResNet-50 model to extract features and obtain multi-scale feature maps;

[0016] S12: Select the feature map with the highest-level semantics from the multi-scale feature maps as the visual feature;

[0017] S13: The downsampling rate of the multi-scale feature map is unified by bilinear pooling, and then the multi-scale feature map after unification is fused by element-wise addition. Finally, the fused multi-scale feature map is processed by a convolutional layer to obtain the depth feature.

[0018] Preferably, step S2 includes:

[0019] S21: Divide and mark the human body image into blocks;

[0020] S22: A multi-layer self-attention module based on Transformer models the global dependency relationship of the partitioned and labeled patches to obtain the global dependency relationship of the human body image;

[0021] S23: Based on a multi-scale context fusion encoder, feature aggregation is performed on the segmented and labeled patches and their surrounding patches to obtain local context fusion results;

[0022] The global dependency relationship and the local context fusion result constitute the multi-scale context fusion result.

[0023] Preferably, the loss function used to calculate the loss in step S5 is:

[0024] (1);

[0025] in, For loss function, The focus loss is calculated from the panoramic depth map corresponding to the target in the search results. The number of target search results. This refers to the attribute information of the panoramic depth map corresponding to the target in the search results. For web search architecture.

[0026] Preferably, This can be expressed by the following formula:

[0027] (2);

[0028] in, For web search architecture, , For balance coefficient, For web search architecture The mean accuracy of the corresponding target. For web search architecture The target is to maximize the performance metrics. For web search architecture The intersection-union ratio of the corresponding targets.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] By introducing Deep Guided Adaptive Feature Extraction (DGAFE) and a multi-scale contextual fusion attention mechanism, this invention can dynamically adjust the feature extraction strategy and optimize the network architecture design, thereby significantly improving the accuracy and robustness of object detection in complex scenes (such as blurred or occluded objects). Through the multi-scale contextual fusion attention mechanism, this invention can model the spatial relationships between image regions at both the global and local levels, enhancing the model's understanding of complex scenes and further improving the accuracy of object detection. Utilizing Neural Architecture Search (NAS) technology, this invention can automatically explore the optimal network architecture, avoiding the limitations of traditional manual network architecture design. The NAS framework dynamically optimizes the feature extraction strategy, improving detection performance while minimizing computational costs, making the method more efficient and practical in real-world applications and improving the model's adaptability and performance. Attached Figure Description

[0031] Exemplary embodiments of the present invention can be more fully understood by referring to the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the present invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0032] Figure 1 A flowchart illustrating a target detection method based on multi-scale attention and network architecture search, provided for embodiments of this application. Detailed Implementation

[0033] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0034] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0035] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0036] Reference Figure 1 This embodiment discloses a target detection method based on multi-scale attention and network architecture search, including the following steps:

[0037] S1: Acquire the input human body image, extract features from the human body image through the backbone network, and obtain multi-scale feature maps and the visual and depth features of the multi-scale feature maps.

[0038] Specifically, this example uses the Endovis2018 dataset, a public dataset for visual analysis in endoscopic surgery, primarily used to advance research in the field of computer-assisted surgery. It contains eight surgical sequences, each consisting of multiple frames of images, each with a resolution of 1920x1080 or 1280x1024. Taking the images as input, our framework utilizes feature backbones, such as ResNet50, and a lightweight depth predictor to generate their visual and depth features, respectively. Given an image... H and W represent its height and width, respectively, and its multi-scale feature map is obtained from the last three stages of ResNet-50. The reduction sampling ratio from the original size is... The highest level with sufficient semantics. Visual features of the input image , This represents the number of channels in the input layer of the image.

[0039] Depth features are obtained from images using a lightweight depth predictor. First, the sizes of the three-level features are uniformly downsampled to the same 1 / 16 downsampling rate using bilinear pooling, and then they are fused together using element-wise addition. This approach integrates multi-scale visual appearances and preserves fine-grained patterns for small objects. Then, two 3×3 convolutional layers are applied to obtain the depth features of the input image. , This represents the number of channels in the input image.

[0040] S2: Input the visual features and the depth features into the multi-scale context fusion encoder to perform multi-scale context fusion and obtain the multi-scale context fusion result;

[0041] Step S2 includes:

[0042] S21: Divide and label human body images into blocks;

[0043] S22: A multi-layer self-attention module based on Transformer models the global dependency relationship of the partitioned and labeled patches to obtain the global dependency relationship of the human body image;

[0044] S23: Based on a multi-scale context fusion encoder, feature aggregation is performed on the segmented and labeled patches and their surrounding patches to obtain local context fusion results;

[0045] The global dependency relationship and the local context fusion result constitute the multi-scale context fusion result.

[0046] The highest-level semantics refers to the most abstract and global semantic information captured by the last or deepest layer during feature extraction. Multi-scale feature maps were obtained from the last three stages of ResNet-50. Input visual and deep features were then used by a multi-scale contextual fusion encoder to model global dependencies and perform local optimization.

[0047] While depth-guided feature extraction focuses on depth-specific adaptation, the multi-scale contextual fusion attention mechanism enhances the spatial relationships between image regions. This mechanism operates in two phases: global dependency modeling and local optimization.

[0048] (1) Global dependency modeling

[0049] First, divide and label the image patches: Divide the input image I into P × P patches, with each patch serving as an independent label. Let... This is represented by the label for block i. The sequence of blocks is represented as follows: These blocks model global dependencies through a multi-layered self-attention module based on Transformer, as shown below:

[0050] (3);

[0051] Here, A() represents a self-attention operation based on an autotransformer, adaptively capturing long-distance dependencies across token sequences. By leveraging the adaptive attention mechanism of the autotransformer, each block... While maintaining computational efficiency, it effectively integrates global contextual information. Adaptive capture of long-distance dependencies = global attention mechanism + data-driven weight learning. The model can dynamically learn and model complex relationships between any two elements (such as pixels, objects, words, etc.) in the input sequence, regardless of their distance in the sequence. This capability is one of the core advantages of Transformer, especially suitable for tasks requiring global contextual understanding.

[0052] (2) Local context integration

[0053] For local refinement, multi-scale context fusion attention is applied to a central patch of an image and its surrounding patches. Aggregate features, where the features represent the area surrounding the central block. The set of adjacent blocks is represented as:

[0054] (4);

[0055] in, This represents a set of P×P patches of image I, ordered sequentially. The weight is represented by the following formula:

[0056] (5);

[0057] Variance-based weight allocation: To ensure the robustness of feature fusion, variance-based weight allocation is introduced. (Computation block) The characteristic variance is:

[0058] (6);

[0059] Among them, variance Smaller blocks are assigned higher weights during the fusion process, reflecting consistency within the local context. This method integrates a deep-guided module for adaptive feature extraction and a multi-scale contextual fusion attention mechanism for spatial relationship modeling. By combining depth perception with sophisticated attention-based feature fusion, the framework achieves robust performance in object detection.

[0060] S3: The depth prediction module processes the multi-scale context fusion results to generate a panoramic depth map;

[0061] Specifically, the depth distribution of foreground objects is predicted, rather than the complete dense depth map, thereby reducing computational overhead and improving detection accuracy. Based on the visual and depth features generated in step S1, the two types of features are then encoded globally and locally in S2 to obtain new visual and depth features. These features are then input into the depth-guided decoder in S3-S4 to perform network architecture search and output the best result.

[0062] S4: Divide the generated panoramic depth map into multiple depth ranges, and perform feature extraction and search on the multiple depth ranges through multiple network search architectures to obtain multiple target search results.

[0063] Specifically, accurate object detection in space requires precise depth information to distinguish objects at different distances. The depth guidance module addresses this challenge by dividing the input depth map (depth features output from the encoder, identical to the original input image) into different ranges and applying customized feature extraction strategies to each range, which are dynamically optimized using a NAS framework.

[0064] Depth Prediction: The depth guidance module first uses the depth prediction module (a lightweight CNN module that predicts discrete depth ranges of objects, provides geometric priors for the Transformer, and outputs depth features (for the encoder) and a foreground depth map) to generate a panoramic depth map D. Formally, given an input image... Panoramic depth map Calculated as:

[0065] ;

[0066] in This represents a deep prediction model.

[0067] Depth range will Divided into K predefined depth ranges The threshold is The segmented depth region Defined as:

[0068] ;

[0069] in, This is a pixel index for the image. Each range corresponds to an object at a different distance (e.g., near, middle, far).

[0070] Personalized feature extraction based on NAS: for each segment depth range The NAS framework is used to search for the optimal feature extraction strategy. The NAS search space S contains configurations of transformer layers and attention mechanisms at different depths:

[0071] ;

[0072] in, Representative candidate architectures (e.g., shallow or deep transformers with multi-head self-attention).

[0073] The search objective is to maximize performance metrics. Mean accuracy and intersection At the same time, minimize computational costs. The optimization problem is formulated as follows:

[0074] (2);

[0075] in, Indicates the architecture selected for the NAS. It is the balance coefficient.

[0076] Specifically, maximizing performance metrics This represents the overall performance of the optimized model through Neural Architecture Search (NAS) in the object detection task. Mean accuracy. This represents the mean precision (AP) of all categories at different recall rates, measuring the overall accuracy of the detector under varying difficulty levels. Crossover Union (CLU) This represents the ratio of the intersection to the union of the predicted bounding box and the ground truth bounding box, measuring localization accuracy. It is directly optimized by dynamically adjusting the depth-related architecture parameter θ and the fusion strategy. and At the same time, constrain the computational cost (i.e. The key innovations are depth guidance and multi-scale attention mechanisms, which respectively model depth ambiguity and spatial relationships to jointly improve detection performance.

[0077] S5: Calculate the prediction loss corresponding to the multiple target search results based on the MLP detection method;

[0078] S6: Select the target search results obtained by the network search architecture corresponding to the minimum prediction loss, and output them as the final target.

[0079] To correctly match each query with the ground-truth object, the loss of each query-label (which represents the matching relationship between learnable object queries and ground truth labels, its core function is to solve the correspondence problem between "unordered prediction" and "ordered labeling," achieving true end-to-end detection without the need for manually designed anchors or NMS post-processing) is calculated, and the global best match is found using the Hungarian algorithm. The Hungarian algorithm is used to perform optimal one-to-one matching between learnable object queries and ground truth labels, ensuring that each real object is predicted by only one query, avoiding redundancy. After matching, from... The query yielded Valid pairs, among which This represents the number of real objects. Then, the overall loss for the training images is formulated as:

[0080] (1);

[0081] in, For loss function, The focus loss is calculated from the panoramic depth map corresponding to the target in the search results. The number of target search results. This refers to the attribute information of the panoramic depth map corresponding to the target in the search results. For web search architecture.

[0082] Specifically, The focus loss for the panoramic depth map corresponding to the target in the target search results is actually obtained based on the set of all panoramic depth images generated in the target search results. Specifically, it focuses on training a sparse set of difficult samples (composed of all panoramic depth maps generated in the target search). By directly introducing two penalty factors into the standard cross-entropy loss model (which is based on existing technology and will not be elaborated here), the weight of easily classified samples is reduced, making the model focus more on difficult samples during training. Therefore, the faster the model converges, the more focus loss of the image set can be obtained when the model converges. This can be expressed by the following formula:

[0083] ;

[0084] in, Let be the probability of a positive sample in the sample set. Let be the probability of a negative sample in the sample set; where, For control parameters, and 1- These are used to control the ratio of positive to negative samples, and their values ​​range from [0, 1]. The value of can generally be selected through cross-validation; The focusing parameter, with a value range of [0, +∞), reduces the weight of easily classified samples, thus allowing the model to focus more on difficult samples during training. When it is 0, The cross-entropy loss is used. The sample set consists of panoramic depth maps corresponding to all targets in the target search results.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] By introducing Deep Guided Adaptive Feature Extraction (DGAFE) and a multi-scale contextual fusion attention mechanism, this invention can dynamically adjust the feature extraction strategy and optimize the network architecture design, thereby significantly improving the accuracy and robustness of object detection in complex scenes (such as blurred or occluded objects). Through the multi-scale contextual fusion attention mechanism, this invention can model the spatial relationships between image regions at both the global and local levels, enhancing the model's understanding of complex scenes and further improving the accuracy of object detection. Utilizing Neural Architecture Search (NAS) technology, this invention can automatically explore the optimal network architecture, avoiding the limitations of traditional manual network architecture design. The NAS framework dynamically optimizes the feature extraction strategy, improving detection performance while minimizing computational costs, making the method more efficient and practical in real-world applications and improving the model's adaptability and performance.

[0087] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0090] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0092] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application.

Claims

1. A target detection method based on multi-scale attention and network architecture search, characterized in that, Includes the following steps: S1: Acquire the input human body image, extract features from the human body image through the backbone network, and obtain multi-scale feature maps and the visual and depth features of the multi-scale feature maps. S2: Input the visual features and the depth features into a multi-scale context fusion encoder to perform multi-scale context fusion and obtain the multi-scale context fusion result; S21: Divide and mark the human body image into blocks; S22: A multi-layer self-attention module based on Transformer models the global dependency relationship of the partitioned and labeled patches to obtain the global dependency relationship of the human body image; The global dependency modeling includes: First, divide and label the image patches: Divide the input image I into P×P patches, and let each patch be an independent label. The label representing block i; The block sequence is represented as These blocks model global dependencies through a multi-layered self-attention module based on Transformer, as shown below: ； Where A() represents the self-attention operation based on the autotransformer, and H and W represent the height and width of the input image I; S23: Based on a multi-scale context fusion encoder, feature aggregation is performed on the segmented and labeled patches and their surrounding patches to obtain local context fusion results; The global dependency relationship and the local context fusion result constitute the multi-scale context fusion result; Local context fusion includes: For local refinement, multi-scale context fusion attention is applied to a central patch of an image and its surrounding patches. Aggregate features, where the features represent the area surrounding the central block. The set of adjacent blocks is represented as: ； in, This represents a set of P×P patches that are ordered sequentially from image I. The weight is represented by the following formula: ； Variance-based weight allocation: To ensure the robustness of feature fusion, variance-based weight allocation is introduced, and the computation block... The characteristic variance is: ； S3: The multi-scale context fusion results are processed by the depth prediction module to generate a panoramic depth map; S4: Divide the generated panoramic depth map into multiple depth ranges, and perform feature extraction and search on the multiple depth ranges through multiple network search architectures to obtain multiple target search results; S5: Calculate the prediction loss corresponding to the multiple target search results based on the MLP detection method; S6: Select the target search results obtained by the network search architecture corresponding to the minimum prediction loss, and output them as the final target.

2. The method according to claim 1, characterized in that, The human images are derived from a publicly available dataset used for visual analysis of endoscopic surgery; the backbone network includes a ResNet-50 model and a lightweight depth predictor.

3. The method according to claim 2, characterized in that, Step S1 includes: S11: The human body image is processed by the ResNet-50 model to extract features and obtain multi-scale feature maps; S12: Select the feature map with the highest-level semantics from the multi-scale feature maps as the visual feature; S13: The downsampling rate of the multi-scale feature map is unified by bilinear pooling, and then the multi-scale feature map after unification is fused by element-wise addition. Finally, the fused multi-scale feature map is processed by a convolutional layer to obtain the depth feature.

4. The method according to claim 1, characterized in that, The loss function used to calculate the loss in step S5 is: （1）； in, For loss function, The focus loss is calculated from the panoramic depth map corresponding to the target in the search results. The number of target search results. This refers to the attribute information of the panoramic depth map corresponding to the target in the search results. For web search architecture.

5. The method according to claim 4, characterized in that, This can be expressed by the following formula: （2）； in, For web search architecture, , For balance coefficient, For web search architecture The mean accuracy of the corresponding target. For web search architecture The target is to maximize the performance metrics. For web search architecture The intersection-union ratio of the corresponding targets.

Images