A nasopharyngeal lesion detection method and system based on cross-layer adaptive guidance

Abstract

The application discloses a nasopharyngeal lesion detection method and system based on cross-layer adaptive guidance, and belongs to the technical field of medical image processing and intelligent auxiliary diagnosis. Multi-scale basic features are extracted through a backbone network, and after semantic encoding of deep layer features, the features are input into a stable residual gate calibration module to perform bidirectional multi-scale fusion and residual gate calibration, and then the cross-layer adaptive guidance module is used to perform channel projection, spatial alignment and guidance weight enhancement on different scale features, and finally the multi-scale features after cross-layer guidance are input into a Transformer decoder to output lesion categories, positions and confidence. The application realizes bidirectional fusion of high-level semantics and shallow details, can adjust sampling weights according to local content, and selectively enhances multi-scale lesion responses, which is helpful to improve the feature expression and positioning of nasopharyngeal endoscopic lesions. In the experimental embodiment, the method is used for nasopharyngeal lesion detection in a nasopharyngeal region of a white light endoscopic image.

Description

Technical Field

[0001] This application relates to the fields of medical image processing, computer vision and intelligent assisted diagnosis technology, specifically to a method and system for detecting nasopharyngeal lesions based on cross-layer adaptive guidance. Background Technology

[0002] Nasopharyngeal endoscopy is an important tool for screening and diagnosing nasopharyngeal carcinoma, laryngeal cancer, precancerous lesions, and upper respiratory tract-related diseases. However, the anatomical structure of the nasopharynx and larynx is complex, and endoscopic imaging is easily affected by factors such as changes in viewing angle, instrument obstruction, mucus adhesion, motion blur, and local strong reflections. As a result, lesions may only appear as subtle signs in the image, such as slight color changes, local bulging, abnormal mucosal texture, or weakened boundaries. Therefore, high requirements are placed on the fine-grained characterization ability and localization accuracy of the detection algorithm.

[0003] Existing technologies, such as [1] Tao Xiaoyao. Application research on real-time identification and localization of concealed targets under otolaryngology endoscopy based on YOLO deep learning model [D]. Anhui Medical University, 2025. Based on the YOLO single detection algorithm, a dual-modal intelligent system covering static images and dynamic videos was constructed, realizing functions such as fishbone foreign body identification and recurrent laryngeal nerve monitoring under laryngoscopy. However, this method and existing technologies still have the following shortcomings.

[0004] (1) Although shallow features have high spatial resolution, they are easily affected by mucosal texture, reflective points and background structure; although deep features have strong semantic discrimination ability, spatial details have been compressed. Existing methods often use element-wise addition, channel splicing or fixed interpolation when fusing across scales, which can easily lead to spatial mismatch between high-level semantics and shallow boundaries, affecting the detection accuracy of small lesions and weak boundary lesions.

[0005] (2) The sampling weights of fixed nearest neighbor interpolation or bilinear interpolation are determined only by the relative position of pixels, and cannot adaptively adjust the feature reconstruction region according to the lesion outline, mucosal texture and local context. When the lesion size is small, the boundary is blurred or the contrast is low, the upsampling process will further dilute the lesion response; when there is strong reflection or repetitive texture in the image, conventional fusion will also amplify the non-lesion response synchronously.

[0006] (3) Although the end-to-end Transformer detector can reduce the impact of anchor box design and complex post-processing, its performance still depends on the quality of multi-scale features before entering the decoder. If the local lesion clues in the multi-scale features are not stably calibrated, the decoding query is difficult to form an accurate category and location representation within the limited decoding layer. Problems such as missed detection of small lesions, bounding box offset and false detection of reflective areas are still prominent.

[0007] In summary, existing methods still need improvement in areas such as multi-scale semantic and spatial detail fusion, lesion boundary information recovery, suppression of shallow background interference, and stability in detecting small and weakly boundaryed lesions. Therefore, it is necessary to propose a technical solution that simultaneously performs high-level semantic downlinking, shallow detail reflow, content-aware upsampling, cross-layer adaptive guidance, and local spatial refinement before detection and decoding, to improve the multi-scale feature representation and localization of nasopharyngeal endoscopic lesions. Summary of the Invention

[0008] The purpose of this application is to provide a method and system for detecting nasopharyngeal lesions based on cross-layer adaptive guidance, in order to solve the problems in existing technologies such as insufficient fusion of multi-scale semantic and spatial details, insufficient recovery of lesion boundary information due to fixed upsampling, easy amplification of shallow background interference, and low stability in detecting small lesions and lesions with weak boundaries. The technical solution to achieve the purpose of this application is as follows: A method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance includes the following steps: S101. Obtain the nasopharyngeal endoscope image to be detected to obtain the model input image; S102. Input the model input image into the backbone network and extract basic features at multiple different spatial resolution levels. The basic features include at least shallow features, mid-level features and deep features. S103. Semantically encode the deep features to obtain semantically enhanced deep features; S104. Perform bidirectional multi-scale fusion on the shallow features, the mid-level features, and the semantically enhanced deep features: After content-aware upsampling of the semantically enhanced deep features, they are fused with the mid-level features. Then, after content-aware upsampling of the fused mid-level features, they are fused with the shallow features to form a top-down semantic propagation path. The fused shallow features are downsampled and then aggregated with the fused mid-level features. The aggregated mid-level features are then downsampled and then aggregated with the semantically enhanced deep features, forming a bottom-up detail aggregation path. S105. For at least one scale feature after the bidirectional multi-scale fusion, generate residual features and gating weights using the calibration branch, and update the main branch features with residuals with scaling factors to complete residual gating calibration. S106. For each target scale of the multi-scale features after residual gating calibration, the features of other scales are channel-projected and spatially aligned to the target scale. Guiding weights are generated based on the aligned context features and the features of the target scale. The guiding weights are used to selectively enhance the features of the target scale to obtain the multi-scale features after cross-layer guidance. S107. Input the multi-scale features after cross-layer guidance into the Transformer decoder, and output the detection results after decoding. The detection results include lesion type, lesion location and corresponding confidence level.

[0009] Furthermore, the spatial resolutions of the shallow features, the mid-level features, and the deep features are H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32 of the model input image, respectively, where H and W are the height and width of the model input image, respectively; the semantic encoding uses a Transformer encoder to perform at least one layer of self-attention encoding on the deep features.

[0010] Furthermore, the content-aware upsampling includes: predicting a location-related reconstruction kernel based on the content of the input features, using the reconstruction kernel to perform weighted reconstruction of the local neighborhood of the input features, and the weights of the reconstruction kernel are normalized.

[0011] Furthermore, the residual gating calibration specifically includes:

[0012]

[0013] in, i Indicates the feature scale index. i ∈{3, 4}, Indicates the first i The features obtained after residual gating calibration. Indicates the first i Features obtained after bidirectional multi-scale fusion. Indicates the first i Learnable scaling factor for scale residual gating calibration Indicates the first i Scale-based gating weights, The first term used to generate the gating weights i Scale intermediate features Indicates the first i The residual features generated by the scale calibration branch express and splicing along the channel dimension, This indicates that a convolution transformation is performed on the spliced ​​result. This indicates that the gating weights are obtained by applying Sigmoid activation to the convolution result. , This indicates element-wise multiplication.

[0014] Furthermore, the cross-layer guidance specifically includes: features targeting the target scale. Features at other scales Channel projection is performed using scale-specific projection units, and then the spatial dimensions are aligned to the target scale using upsampling or downsampling to obtain the aligned context features. ; Features at the target scale Aligned context features Aggregate to generate guiding weights , ,in, Indicates the first j Scale features are transferred to the first channel after being projected and aligned with spatial dimensions. i Contextual features at the target scale, where Indicates the target scale features Target features obtained by performing scale-specific channel projection. This indicates that the target feature and each aligned context feature are concatenated along the channel dimension. This indicates the aggregation transformation and weight generation transformation performed on the spliced ​​features.

[0015] The guided weights are used to selectively enhance the features at the target scale, and cross-layer guided features are obtained through residual connections. , ,in, For the target scale features And candidate augmentation features generated across layer contexts, This is the learnable scaling factor for cross-layer guidance.

[0016] Furthermore, the learnable scaling factor of the residual gating calibration and the learnable scaling factor of the cross-layer guidance Initialize with zero or an absolute value less than 0.01.

[0017] Further, S105 specifically includes: performing local spatial refinement on the fusion features of the shallow and / or mid-scale after the bidirectional multi-scale fusion: channel compression is performed on the fusion features of the scale, a spatial attention response map is generated through local spatial modeling branches, the fusion features of the scale are modulated using the spatial attention response map, and the refinement result is obtained by adding the residuals; a direct path is adopted for the fusion features of the deep scale after the bidirectional multi-scale fusion; the refinement result and the output of the direct path together constitute the output features of the residual gating calibration.

[0018] Furthermore, the downsampling employs a learnable downsampling unit, which uses an SCDown structure.

[0019] Furthermore, the Transformer decoder is a query-based decoding structure. It sequentially passes through self-attention, deformable cross-attention, and feedforward networks through learnable object queries, and outputs candidate detection results after multiple iterations. The classification head of the Transformer decoder outputs the lesion category probability corresponding to each candidate detection result, and the bounding box regression head of the Transformer decoder outputs normalized bounding box parameters.

[0020] This application also provides a nasopharyngeal lesion detection system based on cross-layer adaptive guidance, including: The image acquisition and preprocessing module is used to acquire the nasopharyngeal endoscope image to be detected, and to preprocess the endoscope image to obtain the model input image.

[0021] The multi-scale feature extraction module is used to input the model input image into the backbone network and extract basic features at multiple different spatial resolution levels. The basic features include at least shallow features, mid-level features and deep features.

[0022] The high-level semantic encoding module is used to perform semantic encoding on the deep features to obtain semantically enhanced deep features.

[0023] A bidirectional multi-scale fusion module is used to perform bidirectional multi-scale fusion on the shallow features, the mid-level features, and the semantically enhanced deep features: the semantically enhanced deep features are content-aware upsampled and then fused with the mid-level features; the fused mid-level features are then content-aware upsampled and then fused with the shallow features, forming a top-down semantic propagation path; the fused shallow features are downsampled and then aggregated with the fused mid-level features; the aggregated mid-level features are then downsampled and then aggregated with the semantically enhanced deep features, forming a bottom-up detail aggregation path.

[0024] The residual gating calibration module is used to generate residual features and gating weights for at least one scale feature after the bidirectional multi-scale fusion, and update the main branch features in a residual manner with scaling factor to complete the residual gating calibration.

[0025] The cross-layer adaptive guidance module is used to perform channel projection and spatial alignment of features from other scales to the target scale for each target scale of the multi-scale features after residual gating calibration. Guiding weights are generated based on the aligned context features and the features of the target scale. The guiding weights are then used to selectively enhance the features of the target scale, and the multi-scale features after cross-layer guidance are output.

[0026] The detection output module is used to input the multi-scale features after cross-layer guidance into the Transformer decoder, and output the detection results after decoding. The detection results include lesion type, lesion location and corresponding confidence level.

[0027] Compared with the prior art, this application has the following beneficial effects: (1) By constructing a bidirectional fusion path through top-down semantic propagation and bottom-up detail aggregation, information exchange is made between high-level lesion semantics and shallow boundary texture, which helps to improve spatial mismatch in the cross-scale fusion process and enhance the feature expression of small lesions and weak boundary lesions.

[0028] (2) Content-aware feature reorganization is used to dynamically adjust the sampling weights according to the lesion outline and local texture, which avoids further dilution of the lesion response when the contrast is low or the boundary is blurred, and improves the recovery quality of the lesion boundary.

[0029] (3) The network is accessed through residual gating calibration with learnable scaling coefficients in an approximate identity mapping manner, resulting in high training stability; the target scale features are selectively enhanced by cross-layer guided weights, making the lesion response consistent across multiple scales stand out, while suppressing reflection and background texture interference.

[0030] (4) Perform local spatial refinement on high-resolution features to enhance the response continuity of small lesions, weak boundaries and low contrast regions; connect the calibrated multi-scale features to the query-based Transformer decoder to output lesion categories and bounding box positions in a unified manner. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A schematic flowchart of the nasopharyngeal lesion detection method based on cross-layer adaptive guidance provided in the embodiments of this application; Figure 2 This is a schematic diagram of the overall structure of the SCAG-Net detection network provided in the embodiments of this application; Figure 3 A schematic diagram of the Stable Residual Gated Calibration (SRGC) module and the bidirectional multi-scale fusion structure provided in the embodiments of this application; Figure 4 This is a schematic diagram of the cross-layer adaptive guidance and LSA local spatial refinement module structure provided in an embodiment of this application; Figure 5This is a comparison chart of lesion detection results provided in the embodiments of this application; Figure 6 A comparison chart of attention responses provided for embodiments of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] This application provides a method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance, such as... Figure 1 The diagram shown is a flowchart of the method described in this application.

[0035] Step S101: Obtain the nasopharyngeal endoscope image to be detected, preprocess the endoscope image to obtain the model input image.

[0036] The nasopharyngeal endoscope image I to be tested is acquired. This image can originate from a real-time video stream output by the endoscope or from historical image data stored on a storage medium. For video data, frames can be extracted at preset time intervals, or keyframes can be selected based on resolution, inter-frame differences, and effective field of view ratio to reduce redundant computation. Preferably, at least one of the following processes is performed on the image: effective field of view extraction, resizing, pixel normalization, and tensor format conversion, to obtain the model input image I′.

[0037] Effective field of view extraction is performed on the image. Nasopharyngeal endoscopy images typically have a circular or elliptical field of view, with black backgrounds or invalid pixels in the edge areas. Preferably, threshold segmentation or a preset mask is used to extract the effective field of view area to reduce the impact of black border areas on feature learning.

[0038] The image is resized and normalized. The image after extracting the effective field of view is resized to the model input size. Preferably, the resizing is done by scaling while maintaining the aspect ratio, filling in insufficient areas (e.g., zero-padding), and saving the scaling factor and padding offset so that the predicted bounding boxes can be restored to the original image coordinate system in subsequent steps. Pixel normalization can use training data statistics or normalization parameters corresponding to the pre-trained backbone network. In one embodiment, the normalization parameters are the mean and standard deviation of the ImageNet dataset; in another embodiment, the statistics of the training set itself are used to better adapt to the pixel distribution characteristics of the endoscopic image.

[0039] Furthermore, image preprocessing also includes converting image data from HWC (height, width, number of channels) format to CHW (number of channels, height, width) format to meet the tensor input requirements of deep learning frameworks.

[0040] Step S102: Input the model input image I′ into the backbone network and obtain the basic features of three resolution levels from different network layers or intermediate feature nodes, including shallow feature P3, medium feature P4 and deep feature P5.

[0041] like Figure 2 As shown, the backbone network adopts a Transformer architecture capable of outputting multi-scale visual features. In a preferred embodiment, DINOv3 is used as the backbone network. Utilizing its pre-trained visual representation capabilities, basic features P3, P4, and P5 at three resolution levels are extracted from different network layers. P3, P4, and P5 are taken from layers 5, 8, and 11 of the DINOv3 backbone network, respectively, and their spatial dimensions relative to the input image are H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32, respectively. Here, H and W represent the height and width of the model's input image, respectively. The values ​​of H and W depend on the resolution of the model's input image after size adjustment, and those skilled in the art can determine the specific values ​​based on the actual input image size.

[0042] The DINOv3 backbone network can be initialized using pre-trained parameters. During the training phase, it can be fully fine-tuned, unfrozen in stages, or only trained on preset layers. In a preferred embodiment, the backbone network adopts the DINOv3STAs architecture, with the model name dinov3_vits16plus, and loads the pre-trained weight file dinov3_vits16plus_pretrain_lvd1689m-4057cbaa.pth. To adapt to the nasopharyngeal lesion detection task, the network interaction layer indexes are set to [5,8,11], fine-tuning is set to True, and conv_inplane is set to 64. Intermediate features are extracted from the 5th, 8th, and 11th interaction layers and output as P3 / P4 / P5 through the adapter. The number of channels in each layer is 256, and the strides are 8, 16, and 32, respectively.

[0043] P3 corresponds to a high spatial resolution feature, preserving relatively rich edge, texture, and local detail information. It is more sensitive to small lesions and lesions with weak boundaries, but it is also easily affected by mucosal texture, reflective points, and background structures. P5 corresponds to a low spatial resolution feature, with a large receptive field and strong semantic discriminative ability. It can represent the overall morphology and contextual relationships of lesions, but spatial details are significantly compressed. P4 corresponds to a medium spatial resolution feature, used to represent local structures and mid-level semantics. The feature flow in the figure can be summarized as: I′→{P3, P4, P5}.

[0044] Step S103: Semantically encode the deep feature P5 to obtain semantically enhanced P5 features.

[0045] In the hybrid encoder, P3, P4, and P5 are first mapped to a unified channel dimension C using scale-specific channel projection units. For consistency with the accompanying figure labels, the channel-unified features are still denoted as P3, P4, and P5. The channel projection unit can consist of a 1×1 convolution, a normalized layer, and a ReLU activation function.

[0046]

[0047] in, Indicates the first i The scale of the channel projection unit will be the basic feature The features obtained after mapping to a unified channel dimension C. At least one layer of Transformer self-attention encoding is performed on the channel-unified deep features P5 to establish the contextual relationship between distant anatomical structures and lesion regions using the self-attention mechanism, resulting in semantically enhanced P5 features.

[0048]

[0049] in, This represents the semantically enhanced features obtained after the Transformer encoder performs self-attention encoding on the channel-unified deep features P5. This represents the enhanced high-level semantic features. In this embodiment, Transformer encoding is performed only on the deepest P5 features to control computational load while enhancing high-level semantics.

[0050] In step S104, the P3, P4 and semantically enhanced P5 features are input into the stable residual gated calibration module to perform bidirectional multi-scale fusion.

[0051] In one specific embodiment, the relevant parameters are configured as follows: the number of input channels of the hybrid encoder is [256, 256, 256], hidden_dim is 256, dim_feedforward is 1024, expansion is 1.25, depth_mult is 1.37, fuse_op is sum, csp_type is csp2, use_hfp, use_carafe and use_lsa are all set to True, and hfp_positions is set to [fpn, pan].

[0052] The Stable Residual Gated Calibration (SRGC) module comprises four stages: top-down fusion, content-aware upsampling, bottom-up path aggregation, and selective local spatial attention (LSA) refinement.

[0053] The key to this adaptation lies in the joint configuration of the three-layer feature extraction levels, feature scale transformation positions, local refinement levels, and cross-layer guidance order: High-level semantics are first propagated to the mid- and shallow layers through content-aware upsampling. Shallow details are then downsampled through spatial-channel decoupled downsampling (SCDown) and flow back to the mid- and high-level layers step by step. LSA only performs local calibration on P3 and P4 level high-resolution features. CAGL then uses the alignment context between each scale to generate guiding weights. Thus, the edge information of local lesions is constrained by high-level semantics, while high-level semantics simultaneously obtains low-level texture supplementation, forming a collaborative feature processing path adapted to small lesions, weak boundaries, and reflective interference scenarios in nasopharyngeal endoscopy.

[0054] like Figure 3 As shown, firstly, the semantically enhanced P5 feature is subjected to a 2x upsampling of Content-Aware ReAssembly of Features (CARAFE) to make its spatial size consistent with P4, and then added element-wise with P4 at the addition node in the figure to obtain the enhanced P4 feature; then, the enhanced P4 feature is subjected to a 2x upsampling of CARAFE again, and then added element-wise with P3 at the next addition node to obtain the enhanced P3 feature. Both of the above top-down fusions use element-wise addition.

[0055] The content-aware upsampling unit predicts the location-related reconstruction kernel based on the input features, and uses the reconstruction kernel to weight and reconstruct local neighborhood features, so that different sampling weights can be used near the lesion outline and in the uniform background area. This represents the output feature map obtained after content-aware upsampling. This indicates the spatial location of the output feature map. The feature vector at the output position. The corresponding input local neighborhood is This neighborhood contains | | position, and the first one r Each position is denoted as q r ,in r =1, 2, ..., | |, then It is represented as follows.

[0056]

[0057] in, Indicates the location of the input feature map in the neighborhood. q r The eigenvector at that location; | | indicates the total number of positions contained in the local neighborhood; Indicates the output position and neighboring locations q r The predicted recombination weights are obtained by Softmax normalization. The lower bound for summation is... r =1, upper limit is | |, that is, to The weighted summation is performed across all positions in the dataset. In this embodiment, CARAFE is set in two top-down upsampling paths, P5 to P4 and P4 to P3, with an upsampling factor of 2. The number of input and output channels is 256 each, the encoding convolution kernel k_enc is 3, the reconstruction kernel k_up is 5, the number of intermediate channels c_mid is 64, and the reconstruction weights are normalized using Softmax. Compared to bilinear interpolation or nearest-neighbor interpolation, content-aware upsampling can adaptively adjust the sampling weights based on lesion contours and local textures, thereby improving the spatial mismatch problem in the cross-scale fusion process.

[0058] In the bottom-up path aggregation stage, the enhanced P3 features are downsampled to the P4 scale and added element-wise with the enhanced P4 features to obtain the P4-level aggregated features; then, the P4-level aggregated features are downsampled to the P5 scale and added element-wise with the semantically enhanced P5 features to obtain the P5-level aggregated features. For example... Figure 3 As shown, the features at levels P3, P4, and P5, which are respectively sent to subsequent processing, are denoted as P3′, P4′, and P5′.

[0059] In one specific embodiment, the learnable downsampling unit adopts an SCDown(256,256,k=3,s=2) structure: the first layer is a 1×1 convolution and BatchNorm, with 256 input and output channels, a stride of 1, padding of 0, groups of 1, and no bias; the second layer is a 3×3 depthwise convolution and BatchNorm, with 256 input and output channels, a stride of 2, padding of 1, groups of 256, and no bias. Therefore, the downsampling process can be represented as: `downsample_feat` is set to `BN2(DWConv3×3_s2(BN1(Conv1×1_s1(feat_low))))`, and then fused with the adjacent high-level feature `feat_height` in the bottom-up path using an additive approach, i.e., `fused_feat = downsample_feat + feat_height`. To prevent shallow noise from propagating indiscriminately to higher levels, residual gating is set in the aggregation transform; the learnable downsampling unit uses a 3×3 depthwise convolution with a stride of 2 to complete the scaling transformation.

[0060] Step S105 generates residual features using a calibration branch for at least one scale feature among P3′, P4′, and P5′ after bidirectional multi-scale fusion. and gating weights The main branch features are updated using residuals with scaling factors to complete residual gating calibration.

[0061] like Figure 3 and Figure 4 As shown, in this embodiment, Local Thinning (LSA) is performed on P3′ and P4′, and an identity mapping path is used for P5′; thus, the correspondences of P3′→Z3, P4′→Z4, and P5′→Z5 are formed, where Z5 can be equal to P5′. The number of input channels is 256, the compression ratio is 4, and the number of intermediate channels is 64. 1×1 channel compression, 3×3 depthwise convolution, 1×1 channel restoration, and sigmoid activation are performed sequentially, where γ is the LSA local thinning scaling factor with an initial value of 0.001. Subsequently, attention weights are used to modulate the input features, and the thinned P3′ and P4′ are obtained through residual thinning connections.

[0062]

[0063] in, Indicates the first i Local spatial attention response map at scale, where Compress represents channel compression transformation and Convlocal represents local spatial modeling convolution. This represents the Sigmoid activation function. Indicates the first i The scale is the local refinement scaling factor of LSA, Restore represents the transformation that restores the compressed features to the original number of channels, and ⊙ represents element-wise multiplication. The local spatial modeling branch in this embodiment uses a 3×3 depthwise convolution. LSA is performed on P3′ and P4′ to enhance the spatial continuity of small lesions, weak boundaries, and low-contrast regions; P5′ uses a direct pass path to preserve global representation and control computational cost.

[0064] In one optional structure, the calibration branch includes channel compression, depthwise separable convolution, parallel convolution with different receptive fields, normalization, activation, and channel recovery operations. In this embodiment, the stable residual gated calibration module employs a lightweight residual calibration branch, first performing global average pooling on the base output, then generating the calibration increment delta through 1×1 convolution, SiLU activation, 1×1 convolution, and Sigmoid gating. The module output is base_out + ·delta, where This is the SRGC residual scaling factor, initialized to 0.

[0065]

[0066]

[0067] in, i Indicates the feature scale index. i ∈{3, 4}, Indicates the first i The features obtained after residual gating calibration. Indicates the first i Features obtained after bidirectional multi-scale fusion. Indicates the first i Learnable scaling factor for scale residual gating calibration Indicates the first i Scale-based gating weights, The first term used to generate the gating weights i Scale intermediate features Indicates the first i The residual features generated by the scale calibration branch express and splicing along the channel dimension, This indicates that a convolution transformation is performed on the spliced ​​result. This indicates that the gating weights are obtained by applying Sigmoid activation to the convolution result. , This indicates element-wise multiplication.

[0068] Preferably, In this embodiment, the initialization is performed using a value of zero or an absolute value less than 0.01. Initialize to 0 to make the module approximately an identity mapping at the start of training, and then gradually learn the calibration strength based on the training data, thereby reducing gradient oscillations and abrupt changes in feature distribution caused by new branches.

[0069] Step S106: Input Z3, Z4, and Z5 output from the Stable Residual Gated Calibration Module (SRGC) into the cross-layer adaptive guidance module, and for any target scale, first adjust the other scales... The features are channel-projected and spatially aligned respectively. Guiding weights are generated based on the aligned context features and target scale features. The guiding weights are used to selectively enhance the target scale features to obtain multi-scale features after cross-layer guidance.

[0070] The Z3, Z4, and Z5 values ​​output from the Stable Residual Gated Calibration Module (SRGC) are input into the cross-layer adaptive guidance module CAGL. This applies to any target scale. i First, features at other scales are transformed to a preset number of channels using scale-specific projection. Then, upsampling or downsampling is used to align their spatial dimensions to the target scale, thus obtaining the alignment context. .like Figure 2 As shown, the final outputs of the cross-layer adaptive bootstrapping module (CAGL) are uniformly denoted as Q3, Q4, and Q5, which are the final bootstrapping features sent to the DEIM (DETR with Improved Matching for Fast Convergence) converter decoder. The overall flow is: {Z3, Z4, Z5} → CAGL → {Q3, Q4, Q5}.

[0071]

[0072] in, j Scale index representing contextual features and j ≠ i ; Indicates the first j The characteristics output by the scale through the stable residual gated calibration module; Indicates to Contextual feature representation obtained by performing scale-specific channel projection; Indicates the first j Spatial dimensions of scale context features aligned to the first i Upsampling or downsampling operations at the target scale.

[0073] Target scale features Aggregate with alignment contexts from other scales to complete feature-query alignment and generate target-scale guiding weights. This embodiment concatenates the target-scale projection features with the alignment context of other scales along the channel dimension, and then performs convolutional aggregation and sigmoid activation to obtain spatial-channel joint guiding weights. Unlike methods that generate attention weights solely from the target scale itself, the guiding weights in this embodiment simultaneously use the target-scale features and the alignment context of other scales, ensuring that isolated responses generated by reflections, mucosal folds, or instrument edges at one scale are constrained by the consistency of features from other scales.

[0074]

[0075] in, Indicates the target scale features Target features obtained by performing scale-specific channel projection. The target features and each aligned context feature are concatenated along the channel dimension. This indicates the aggregation transformation and weight generation transformation performed on the spliced ​​features.

[0076] Using guiding weights Target scale features Alternatively, selective enhancements can be performed on its transformed features, and the final cross-layer guided features can be obtained through residual connections. Areas with higher gating values ​​are enhanced, while areas with lower gating values ​​retain their original main branch information, thereby reducing the risk of over-amplification of reflections, instrument edges, and non-lesion textures.

[0077] in, To be based on target scale features And candidate augmentation features generated across layer contexts, In this embodiment, the scaling factor is a learnable factor. Initialized to 0.001. Independent projection and weighting parameters are used at each scale to accommodate the distribution differences in shallow textures, mid-level structures, and high-level semantics. This embodiment employs a combined structure of SRGC bidirectional multi-scale fusion, LSA local spatial refinement, and CAGL cross-layer adaptive guidance.

[0078] Step S107: Input the multi-scale features after cross-layer guidance into the Transformer decoder, and output the detection results after decoding.

[0079] The Q3, Q4, and Q5 outputs from the Cross-Layer Adaptive Guidance Module (CAGL) are input into the DEIM transformer decoder. The decoder has the following parameters: num_layers = 6, eval_idx = -1, num_queries = 300, hidden_dim = 256, feat_channels = [256, 256, 256], feat_strides = [8, 16, 32], num_levels = 3, nhead = 8, dim_feedforward = 2048, num_denoising = 100, num_points = [3, 6, 3], reg_max = 32, reg_scale = 4, activation and mlp_act are both silu, and cross_attn_method and query_select_method are both default. The target query is iteratively optimized through self-attention, deformable cross-attention, feedforward neural network, and bounding box, and outputs candidate detection results after 6 iterations.

[0080] The DEIM transform decoder's classification head outputs the lesion category probability or background probability corresponding to each candidate, while the bounding box regression head outputs normalized bounding box parameters, including the normalized center point x-coordinate cx, center point y-coordinate cy, bounding box width w, and height h. The normalized bounding boxes are restored to the original image coordinate system according to the scaling factor and padding offset recorded in the preprocessing stage, and the detection results are retained based on a confidence threshold; in one evaluation embodiment, the confidence threshold is 0.5.

[0081] In a preferred embodiment, model training includes the following steps: First, training data and data preprocessing were performed. This embodiment uses the YC-ENTDATA dataset, primarily sourced from the Department of Otolaryngology at Yinchuan Regional Hospital. A retrospective collection of 707 examination videos from January 16, 2024 to May 26, 2025 was conducted. The data was strictly anonymized before model development and approved by the hospital's ethics committee (ethics approval number KY-2023-079). This dataset contains 4880 images and 2815 lesion bounding boxes from 198 patients. The training, validation, and test frames were 3462, 762, and 656 respectively, and the data was categorized by patient level. An independent external test was conducted using the XA-ENTDATA dataset from Xi'an Regional Hospital, collected from April 2023 to March 2024. This dataset was not used for training, hyperparameter tuning, or model selection. Annotators drew rectangular bounding boxes for the lesion regions and assigned category labels. Labels can be set to binary classification of lesions and non-lesions, or further subdivided into tumors, precancerous lesions, inflammation, polyps, or other lesion types. Preferably, the training, validation, and test sets are divided according to patient identification, ensuring that different frames from the same patient do not appear across sets, thus reducing the risk of data leakage. For consecutive video frames, further deduplication of similar frames or time-interval sampling can be performed to avoid highly repetitive images biasing the training distribution.

[0082] Augmentation processing of training data can include at least one of the following: random horizontal flipping, slight rotation, scaling, random cropping, color perturbation, brightness or contrast adjustment, blurring, noise injection, and local occlusion. The augmentation should be performed in a manner that does not compromise the medical semantics of the lesions or the validity of the bounding boxes. For randomly cropped images, the lesion box coordinates need to be recalculated, and invalid annotations with a retention rate below a threshold need to be removed. For data with imbalanced class distribution, class-aware sampling, hard sample resampling, loss weighting, or data augmentation can be used to increase the probability of minority classes. For circular or elliptical endoscopic fields of view containing obvious black borders, threshold segmentation or a preset mask can be used to extract the effective field of view to reduce the impact of black border regions on feature learning.

[0083] Next, the trained images are input into SCAG-Net to obtain a set of predicted categories and a set of predicted bounding boxes. The predicted results are then matched with the ground truth annotations using bipartite graph matching or other set matching methods. The matching cost can be a weighted sum of classification cost, bounding box L1 distance cost, and generalized intersection-union cost.

[0084]

[0085] in, Lmatch This represents the total cost of the Hungarian match; Lcls This represents the cost of classification matching, corresponding to cost_class in the matcher, and corresponding to the classification loss term during training. L1 Represents bounding boxL1 Regression cost, corresponding to loss_bbox; Lgiou This represents the cost of the generalized intersection-union comparison, corresponding to loss_giou; , and These represent the weights of the classification term, L1 bounding box regression term, and GIoU term, respectively. During the matching phase, these weights are set to 2, 5, and 2 respectively; during the training loss phase, they are set to 1, 5, and 2 respectively. After matching is complete, the total detection loss is calculated. The total detection loss includes at least the classification loss and the bounding box regression loss; depending on the detector structure, it may also include intermediate decoding layer auxiliary loss, encoder candidate loss, denoising training loss, or a combination thereof. The classification loss can be cross-entropy loss, focus loss, or quality-aware classification loss; the bounding box loss can be L1 loss, IoU class loss, or a combination thereof.

[0086] In this embodiment, the Adaptive Moment Estimation Weight Decay Optimization (AdamW) algorithm is used during training. The training input size is 640×640, the training duration is 110 epochs, the main learning rate is 0.0001, the weight decay is 0.000125, the batch size is 4, and EMA (Exponential Moving Average), AMP (Automatic Mixed Precision), linear warm-up, and gradient pruning strategies are employed. The backbone network and newly added modules use different learning rates; the learning rate of the backbone network is set to 0.0000125. The detector uses 300 queries, and the HungarianMatcher is used for training. The weights for class cost, bounding box cost, and GIoU cost are 2, 5, and 2, respectively. In the early stages of training, a portion of the backbone network layers can be frozen, and joint fine-tuning can be performed after the newly added modules have stabilized.

[0087] During the validation phase, models are selected based on metrics such as mean precision, mean precision at different lesion scales, recall, sensitivity, false positives per image, and IoU. Model selection should not rely solely on overall mean precision; recall for small lesions should also be a key consideration.

[0088] In one specific embodiment, the internal dataset YC-ENTDATA contains a total of 4880 nasopharyngeal endoscopy images, including 3462 training frames, 762 validation frames, and 656 test frames, which include images of tumors, cysts, vocal cord leukoplakia, adenoid hypertrophy, tonsil enlargement, and negative images; the external validation set XA-ENTDATA contains a total of 363 images, which are used to verify the model's generalization ability on data from different sources.

[0089] In one internal validation embodiment, the model achieved precision 0.547769, recall 0.806408, mAP50 0.762779, mAP75 0.612504, and mAP50-95 0.547769. These results illustrate the impact of structure on lesion detection recall and localization quality.

[0090] In an external validation example, compared to the DEIMv2-DINOv3x baseline, the model in this example improved recall on XA-ENTDATA from 0.736393 to 0.756172, sensitivity from 0.425105 to 0.455066, and one-vs-rest specificity from 0.953924 to 0.966163, illustrating the model's trade-off between detection and class-level specificity across cross-source data.

[0091] Ablation experiments showed that, compared to DINOv3XBase, adding SRGC alone increased Recall from 0.7540 to 0.7990, mAP50 from 0.6270 to 0.7550, and mAP50-95 from 0.4560 to 0.5410; adding CAGL alone increased mAP75 from 0.5090 to 0.5818; and adding both SRGC and CAGL simultaneously resulted in Recall of 0.8060, mAP50 of 0.7630, mAP75 of 0.6130, and mAP50-95 of 0.5480.

[0092] like Figure 5 As shown, the comparison chart displays the outputs of SCAG-Net, multiple contrastive detection models, and ground truth annotations, categorized by lesion type. The red numbers on each subplot represent the detection confidence level for the predicted category corresponding to that bounding box. For example, "cyst 0.97" indicates that the model predicts the target within the bounding box as a cyst, with a detection confidence level of 0.97. Detection confidence is obtained by: the model outputting the coordinates of the predicted bounding box and the prediction scores for each category; applying a Sigmoid activation function to each category's prediction score to obtain the confidence value for each bounding box in different categories, mapping it to the corresponding category and bounding box, and finally annotating the confidence value next to the bounding box with two decimal places. In the shown samples of tumors, cysts, vocal cord leukoplakia, tonsillar enlargement, and adenoid hypertrophy, SCAG-Net provides detection boxes corresponding to the ground truth annotation areas; some contrastive models show no detection, low confidence, or additional bounding boxes in the shown samples. This chart is used to visually illustrate the detection differences in the listed samples and does not represent a statistical conclusion for all samples. Figure 6As shown, the attention response comparison map displays different lesion categories by row and shows the response distribution by column for the original image and ground truth annotations, the base model, the addition of SRGC, the addition of CAGL, and the simultaneous addition of SRGC and CAGL. Compared with the base model, after adding SRGC, the response area is more concentrated near the annotation box in several examples; after adding CAGL, the cross-region response is adjusted; when SRGC and CAGL are added simultaneously, the high response areas in the examples of tumors, vocal cord leukoplakia, tonsillar enlargement, adenoid hypertrophy, and cysts show a more obvious spatial correspondence with the lesion annotation areas. Figure 5 and Figure 6 All of these are exemplary visual evidence used to assist in understanding the technical effects of the technical solutions in this application, and should not be construed as an absolute limitation on the performance of the detection method or system in this application on any specific dataset, nor as a substitute for quantitative statistics or clinical validation of the entire test set.

[0093] Based on the same concept as the above method, this application also provides a nasopharyngeal lesion detection system based on cross-layer adaptive guidance. The system is applied to the aforementioned method and includes: The image acquisition and preprocessing module is configured to acquire the nasopharyngeal endoscope image to be detected, preprocess the endoscope image, and obtain the model input image. The multi-scale feature extraction module is configured to input the model input image into the backbone network and extract basic features at multiple different spatial resolution levels. The basic features include at least shallow features, mid-level features, and deep features. The high-level semantic coding module is configured to perform semantic coding on deep features to obtain semantically enhanced deep features; The bidirectional multi-scale fusion module is configured to perform bidirectional multi-scale fusion of shallow features, mid-level features, and semantically enhanced deep features. The bidirectional multi-scale fusion includes: content-aware upsampling of semantically enhanced deep features and fusing them with mid-level features, then content-aware upsampling of the fused mid-level features and fusing them with shallow features to form a top-down semantic propagation path; and downsampling of the fused shallow features and aggregating them with mid-level features, then downsampling of the aggregated mid-level features and aggregating them with semantically enhanced deep features to form a bottom-up detail aggregation path. The residual gating calibration module is configured to generate residual features and gating weights using calibration branches for at least one scale feature after bidirectional multi-scale fusion, and update the main branch features with residuals with scaling factors to complete residual gating calibration. The cross-layer adaptive guidance module is configured to take the multi-scale features after residual gating calibration, and for each target scale, perform channel projection and spatial alignment of the features of other scales to the target scale. Based on the aligned context features and target scale features, guidance weights are generated. The guidance weights are used to selectively enhance the target scale features and output the multi-scale features after cross-layer guidance. The detection output module is configured to input the multi-scale features after cross-layer guidance into the Transformer decoder, and output the detection results after decoding. The detection results include lesion type, lesion location and corresponding confidence level.

[0094] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0095] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance, characterized in that, Includes the following steps: S101. Obtain the nasopharyngeal endoscope image to be detected to obtain the model input image; S102. Input the model input image into the backbone network and extract basic features at multiple different spatial resolution levels. The basic features include at least shallow features, mid-level features and deep features. S103. Semantically encode the deep features to obtain semantically enhanced deep features; S104. Perform bidirectional multi-scale fusion on the shallow features, the mid-level features, and the semantically enhanced deep features: After content-aware upsampling of the semantically enhanced deep features, they are fused with the mid-level features. Then, after content-aware upsampling of the fused mid-level features, they are fused with the shallow features to form a top-down semantic propagation path. The fused shallow features are downsampled and then aggregated with the fused mid-level features. The aggregated mid-level features are then downsampled and then aggregated with the semantically enhanced deep features, forming a bottom-up detail aggregation path. S105. For at least one scale feature after the bidirectional multi-scale fusion, generate residual features and gating weights using the calibration branch, and update the main branch features with residuals with scaling factors to complete residual gating calibration. S106. For each target scale of the multi-scale features after residual gating calibration, the features of other scales are channel-projected and spatially aligned to the target scale. Guiding weights are generated based on the aligned context features and the features of the target scale. The guiding weights are used to selectively enhance the features of the target scale to obtain the multi-scale features after cross-layer guidance. S107. Input the multi-scale features after cross-layer guidance into the Transformer decoder, and output the detection results after decoding. The detection results include lesion type, lesion location and corresponding confidence level.

2. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, The spatial resolutions of the shallow features, the middle features, and the deep features are H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32 of the model input image, respectively, where H and W are the height and width of the model input image, respectively. The semantic encoding employs a Transformer encoder to perform at least one layer of self-attention encoding on the deep features.

3. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, The content-aware upsampling includes: predicting a location-related reconstruction kernel based on the content of the input features, using the reconstruction kernel to perform weighted reconstruction of the local neighborhood of the input features, and normalizing the weights of the reconstruction kernel.

4. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, The residual gating calibration described in S105 specifically refers to: in, i Indicates the feature scale index. i ∈{3, 4}, Indicates the first i The features obtained after residual gating calibration. Indicates the first i Features obtained after bidirectional multi-scale fusion. Indicates the first i Learnable scaling factor for scale residual gating calibration Indicates the first i Scale-based gating weights, The first term used to generate the gating weights i Scale intermediate features Indicates the first i The residual features generated by the scale calibration branch express and splicing along the channel dimension, This indicates that a convolution transformation is performed on the spliced ​​result. This indicates that the gating weights are obtained by applying Sigmoid activation to the convolution result. , This indicates element-wise multiplication.

5. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, The cross-layer guidance specifically includes: Features for the target scale Features at other scales Channel projection is performed using scale-specific projection units, and then the spatial dimensions are aligned to the target scale using upsampling or downsampling to obtain the aligned context features. ; Features at the target scale Aligned context features Aggregate to generate guiding weights , ,in, Indicates the first j Scale features are transferred to the first channel after being projected and aligned with spatial dimensions. i Contextual features at the target scale, where , Indicates the target scale features Target features obtained by performing scale-specific channel projection. This indicates that the target feature and each aligned context feature are concatenated along the channel dimension. This indicates the aggregation transformation and weight generation transformation performed on the spliced ​​features. The guided weights are used to selectively enhance the features at the target scale, and cross-layer guided features are obtained through residual connections. , ,in, For the target scale features And candidate augmentation features generated across layer contexts, This is the learnable scaling factor for cross-layer guidance.

6. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 4 or 5, characterized in that, The learnable scaling factor of the residual gate calibration and the learnable scaling factor of the cross-layer guidance Initialize with zero or an absolute value less than 0.

01.

7. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, Specifically, S105 includes: Local spatial refinement is performed on the shallow-scale and / or mid-scale fusion features after the bidirectional multi-scale fusion: Channel compression is performed on the fusion features at the scale, a spatial attention response map is generated through a local spatial modeling branch, the fusion features at the scale are modulated using the spatial attention response map, and a refinement result is obtained by summing the residuals; A direct path is used for the deep-scale fusion features after the bidirectional multi-scale fusion. The refinement result and the output of the direct path together constitute the output characteristics of the residual gating calibration.

8. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, The downsampling employs a learnable downsampling unit, which uses an SCDown structure.

9. The method for detecting nasopharyngeal lesions based on cross-layer adaptive guidance according to claim 1, characterized in that, The Transformer decoder described in S107 is a query-based decoding structure. It sequentially passes through self-attention, deformable cross-attention, and feedforward networks by querying learnable objects, and outputs candidate detection results after multiple iterations. The classification head of the Transformer decoder outputs the lesion category probability corresponding to each candidate detection result, and the bounding box regression head of the Transformer decoder outputs the normalized bounding box parameters.

10. A nasopharyngeal lesion detection system based on cross-layer adaptive guidance, characterized in that, include: The image acquisition and preprocessing module is configured to acquire the nasopharyngeal endoscope image to be detected, and preprocess the endoscope image to obtain the model input image; The multi-scale feature extraction module is configured to input the model input image into the backbone network and extract basic features at multiple different spatial resolution levels, wherein the basic features include at least shallow features, mid-level features and deep features; A high-level semantic coding module is configured to perform semantic coding on the deep features to obtain semantically enhanced deep features; The bidirectional multi-scale fusion module is configured to perform bidirectional multi-scale fusion on the shallow features, the mid-level features, and the semantically enhanced deep features: after content-aware upsampling of the semantically enhanced deep features, it is fused with the mid-level features; then, after content-aware upsampling of the fused mid-level features, it is fused with the shallow features to form a top-down semantic propagation path; after downsampling of the fused shallow features, it is aggregated with the fused mid-level features; then, after downsampling of the aggregated mid-level features, it is aggregated with the semantically enhanced deep features to form a bottom-up detail aggregation path. The residual gating calibration module is configured to generate residual features and gating weights using calibration branches for at least one scale feature after the bidirectional multi-scale fusion, and update the main branch features with residuals with scaling factors to complete the residual gating calibration. The cross-layer adaptive guidance module is configured to perform channel projection and spatial alignment of features from other scales to the target scale for each target scale of the multi-scale features after residual gating calibration. It generates guidance weights based on the aligned context features and the features of the target scale, and uses the guidance weights to selectively enhance the features of the target scale, and outputs the multi-scale features after cross-layer guidance. The detection output module is configured to input the multi-scale features after cross-layer guidance into the Transformer decoder, and output the detection results after decoding. The detection results include lesion type, lesion location and corresponding confidence level.

Images