A method and system for target detection of an esophagus endoscope image

Abstract

The present application relates to the technical field of computer image processing, and provide a kind of esophagus endoscope image target detection method and system, method includes: S1, obtaining the esophagus endoscope image to be detected;S2, after pre-processing esophagus endoscope image is respectively carried out main feature extraction, color density constraint pilot and mucosa texture boundary perception;S3, the same scale under main semantic feature, color pilot feature and texture boundary feature are carried out heterogenous priori adaptive convergence, obtain multi-scale fusion enhanced feature;S4, multi-scale fusion enhanced feature radial class category prototype bias suppression, obtain class enhanced feature;S5, class enhanced feature is detected decoding, obtains the class, confidence and boundary box position of lesion target in esophagus endoscope image;S6, output lesion detection result.The present application improves the detection accuracy, classification stability and few-sample class recognition ability of lesion target in esophagus endoscope image.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and in particular to a method and system for target detection in esophageal endoscopy images. Background Technology

[0002] Lesion detection and classification in esophageal endoscopy images are crucial for intelligent assisted diagnosis of the digestive tract. Existing research indicates that automated analysis of endoscopic images can assist physicians in identifying lesion location and type. However, practical applications still face challenges such as glare, mucosal folds, localized blurring, unclear lesion boundaries, subtle color differences, and uneven category distribution. Especially in esophageal lesion scenarios, the differences between lesions and the background mucosa often manifest simultaneously as color differences, edge structure differences, and localized texture differences. Simply relying on general detection networks often struggles to reliably balance detection and localization accuracy with the ability to identify categories in small sample sizes.

[0003] Most existing deep learning detection methods employ a single backbone network to extract uniform features from RGB images, and then complete classification and localization through feature pyramids and detection heads. While these methods have achieved good results in general object detection, they lack specific design for addressing fine-grained color differences, local texture anomalies, and class imbalances in esophageal endoscopy images. This results in insufficient ability to identify lesions with few samples, weak boundaries, and significant staining differences but inconspicuous morphology.

[0004] Existing research provides a dedicated target detection technique for esophageal endoscopy image scenarios. By constructing multiple prior auxiliary branches and adaptive fusion structures that match the lesion discrimination mechanism, the model's ability to identify color anomalies, texture anomalies, and few-sample categories is enhanced without significantly increasing the inference burden.

[0005] For example, one existing technology is a general object detection scheme based on a single-backbone convolutional neural network or a Transformer detection framework. This type of scheme typically uses esophageal endoscopy images as three-channel RGB input, extracts multi-scale semantic features through the backbone network, aggregates information at different scales through a feature pyramid structure, and finally inputs it into a classification and regression detection head to obtain the lesion category and bounding box location.

[0006] Taking the RT-DETR method as an example, its typical implementation process includes: inputting the original RGB image; extracting multi-layer feature maps through the backbone network; constructing multi-scale features through the encoder, upsampling, and downsampling paths; feeding the multi-scale features into the decoder for target query and bounding box prediction; and outputting the target category and location. Specific methods are as follows: Figure 1 As shown in the figure. The core of this technical solution lies in using a unified backbone network to perform end-to-end learning on images, which has good performance in detecting targets with obvious shape boundaries and category differences in natural images.

[0007] The shortcomings of existing technology 1 are analyzed as follows:

[0008] (1) The esophageal endoscopy images were processed as ordinary RGB images without specific modeling of the differences in color density and staining reaction related to the lesions;

[0009] (2) The global feature extraction region usually covers the entire image and is easily affected by black borders, specular reflection and invalid background areas, resulting in the dispersion of effective computing resources;

[0010] (3) Insufficient extraction of weakly significant features such as mucosal texture, fine edges and local roughness, making it difficult to accurately characterize lesions with indistinct color differences but obvious texture;

[0011] (4) When the class distribution is unbalanced, the network is more likely to favor high-frequency classes, resulting in a decrease in the recall rate of classes with fewer samples.

[0012] For example, existing technology two involves adding a single auxiliary branch to the backbone network for detection, such as adding an edge enhancement module, an attention module, or a channel recalibration module. This type of method typically constructs an additional feature stream from the RGB image, such as extracting edge information, color space transformation features, or channel attention weights, and then weights or fuses the auxiliary branch output with the backbone features to enhance the network's sensitivity to lesion color differences.

[0013] In this type of scheme, the auxiliary branch generally serves as a single piece of prior information, providing only edge grayscale enhancement or salient region emphasis information. The fusion methods are mostly simple weighting, element-wise addition, or channel concatenation followed by convolutional compression. Specific methods include... Figure 2 As shown.

[0014] However, the single auxiliary branch solution provided by the existing technology 2 still has the following shortcomings:

[0015] (1) The information type of auxiliary branches is singular, usually only emphasizing one feature prior, and cannot simultaneously depict changes such as edge structure, local texture and fine-grained morphology;

[0016] (2) The branch fusion method is relatively simple and cannot dynamically select the contribution ratio of main features, color features and texture features according to different scales, different spatial regions and different samples;

[0017] (3) There is a lack of structural mitigation mechanisms for the discrimination bias caused by class imbalance;

[0018] (4) When the lesion is not significantly different from the background in color, it is difficult for a single color branch to continuously and stably improve the detection performance. Summary of the Invention

[0019] The purpose of this invention is to provide a target detection method and system for esophageal endoscopy images. This target detection method uses a master detection network as its core, combined with a color density constraint leading branch, a mucosal texture boundary perception branch, and a heterogeneous prior adaptive convergence module, to collaboratively model color, texture, and global semantics at multi-scale feature levels. Furthermore, a class prototype bias suppression module is introduced to enhance the discrimination ability for few-sample classes.

[0020] The technical problem solved by this invention is as follows:

[0021] (1) Solve the problem that existing esophageal endoscopy target detection methods cannot specifically model color density differences;

[0022] (2) To address the problem that existing methods are insufficient in characterizing mucosal texture, edge structure, and local contrast anomalies;

[0023] (3) Solve the problem of excessive bias of the model towards high-frequency classes and decreased accuracy of class recognition with few samples under class imbalance conditions;

[0024] (4) A multi-prior collaborative detection framework suitable for esophageal endoscopy image scenarios is provided, which maintains good engineering feasibility while ensuring improved detection accuracy.

[0025] This invention provides the following technical solution:

[0026] A method for target detection in esophageal endoscopy images, comprising the following steps:

[0027] S1. Acquire the esophageal endoscope image to be detected, and preprocess the esophageal endoscope image;

[0028] S2. Perform main trunk feature extraction, color density constraint guidance, and mucosal texture boundary perception on the preprocessed esophageal endoscope images, specifically including:

[0029] S2.1 Extract multi-scale semantic features through the backbone feature extraction, including shallow scale features, medium scale features and deep scale features;

[0030] S2.2 Extract multi-scale color leader features using the color density constraint leader and generate a color effective region constraint map;

[0031] S2.3 Extract multi-scale texture boundary features through the mucosal texture boundary perception to compensate for the structural information that the color density constraint leader cannot stably describe, and enhance the roughness of mucosal texture, edges and local anomalies;

[0032] S3. Perform heterogeneous prior adaptive convergence on the backbone semantic features, the color leading features and the texture boundary features at the same scale to obtain multi-scale fusion enhanced features;

[0033] S4. Suppress the radial category prototype bias of the multi-scale fusion enhancement feature to obtain the category enhancement feature;

[0034] S5. Detect and decode the category enhancement features to obtain the category, confidence level, and bounding box position of the lesion target in the esophageal endoscopy image;

[0035] S6. Calculate the confidence level of each candidate target based on the category probability and bounding box position of the lesion target, filter the candidate targets, and output the lesion detection results in the esophageal endoscopy image.

[0036] Preferably, the color density constraint leading step in step S2.2 includes color input construction, effective channel filtering, effective color region constraint, and color consistency redundancy suppression;

[0037] The color input construction includes extracting the red, green, and blue channels from the esophagoscopic image and constructing a color difference representation that includes the green channel, blue channel, green and blue differences, red-green differences, and red-blue differences.

[0038] The effective channel screening includes screening effective color channels based on one or more of the following indicators: effective response ratio, coefficient of variation, local response entropy, or response concentration.

[0039] The effective color region constraint includes generating a color effective region constraint map based on the effective color channels, and using the color effective region constraint map to impose region constraints on the backbone semantic features or color leading features;

[0040] The color consistency redundancy suppression includes measuring the difference between the green channel features and the blue channel features. When the difference between the two is less than a preset threshold, one of the color features is retained. When the difference between the two is greater than or equal to the preset threshold, the two color features are fused.

[0041] Preferably, the color effective region restriction map is used to apply weighted constraints to the backbone semantic features at the s-th scale, and the constrained backbone semantic features are represented as follows:

[0042] (1)

[0043] Where Fm(s) represents the backbone semantic features at the s-th scale, Mc(s) represents the color effective region constraint map downsampled to the s-th scale, and β represents the enhancement coefficient. This represents the core semantic features after being constrained by the effective color region.

[0044] The difference between the green channel features and the blue channel features is measured using normalized mean square error, expressed as:

[0045] (2)

[0046] Where Fg represents the green channel feature, Fb represents the blue channel feature, MSE(Fg, Fb) represents the mean square error between the green channel feature and the blue channel feature, Var(Fg) and Var(Fb) represent the variances of the green channel feature and the blue channel feature, respectively, and ε is a minimal positive stable term.

[0047] Preferably, the mucosal texture boundary perception in step S2.3 includes texture input construction, texture boundary salient region generation, and multi-scale texture encoding;

[0048] The texture input construction includes converting the esophageal endoscopy image into a grayscale image, and obtaining a gradient magnitude map, a Laplacian response map, and a local contrast map based on the grayscale image;

[0049] The generation of salient regions of texture boundaries includes jointly encoding gradient magnitude maps and local contrast maps to generate salient maps of mucosal structures.

[0050] The multi-scale texture encoding includes obtaining multi-scale texture boundary features corresponding to the backbone semantic features through convolutional encoding or downsampling operations;

[0051] The grayscale image, gradient magnitude image, and local contrast image are respectively represented as follows:

[0052] (3)

[0053] (4)

[0054] The local contrast image Cl is obtained by the difference between the grayscale image and its local mean:

[0055] (5)

[0056] Where R, G, and B represent the red, green, and blue channels, respectively; Y represents the grayscale image; Gx and Gy represent the horizontal and vertical gradients, respectively; Gm represents the gradient magnitude image; Cl represents the local contrast image; and ε is the minimum positive stable term.

[0057] Preferably, the heterogeneous prior adaptive convergence in step S3 is the fusion of backbone semantic features, color leading features, and texture boundary features at the same detection scale;

[0058] For the s-th detection scale, let the backbone semantic feature be Fm(s), the color leading feature be Fc(s), and the texture boundary feature be Ft(s). After aligning the color leading feature and the texture boundary feature in terms of spatial size and channel dimension, we obtain F'c(s) and F't(s).

[0059] The joint features are obtained by concatenating Fm(s), F'c(s), and F't(s):

[0060] (6)

[0061] Based on the joint features, three-way fusion weights are generated, and modal-level fusion features are obtained according to the three-way fusion weights;

[0062] The three-way fusion weights include backbone semantic feature weights, color-leading feature weights, and texture boundary feature weights, denoted as αm(s), αc(s), and αt(s), respectively, and satisfy the following:

[0063] (7)

[0064] The modality-level fusion feature is represented as follows:

[0065] (8)

[0066] Preferably, the heterogeneous prior adaptive convergence further includes local detail compensation, wherein the local detail compensation extracts local detail compensation features Fdetail(s) from the joint features, and obtains the fusion enhancement features of the s-th detection scale through a residual approach:

[0067] (9)

[0068] Where Fout(s) represents the fusion enhancement feature of the s-th detection scale.

[0069] Preferably, the category prototype bias suppression in step S4 includes category prototype establishment, sample-level prototype association, and category-aware feature recalibration;

[0070] The establishment of the category prototype includes setting a learnable category prototype vector for each lesion category;

[0071] The sample-level prototype association includes obtaining a sample-level query vector based on the fusion enhancement features, and calculating the correlation between the sample-level query vector and the prototype vectors of each category.

[0072] The category-aware feature recalibration includes generating a category context vector based on the correlation, and using the category context vector to recalibrate the fused enhanced features;

[0073] The category prototype bias suppression includes a category frequency adjustment factor, which adjusts the response of each category prototype. The category response weights are expressed as follows:

[0074] (10)

[0075] Where q(s) represents the sample-level query vector corresponding to the s-th scale, Pk represents the category prototype vector corresponding to the k-th lesion, rk represents the category frequency adjustment factor corresponding to the k-th lesion, λ represents the adjustment coefficient, and Ak represents the category response weight of the k-th lesion.

[0076] Preferably, the detection decoding in step S5 is based on the interaction between the target query vector and the multi-scale category enhancement features to obtain the category probability and bounding box coordinates of the candidate lesion target;

[0077] For the i-th candidate lesion target, its category prediction result and bounding box prediction result are expressed as follows:

[0078] (11)

[0079] Where D represents detection decoding, Qi represents the i-th target query vector, Fenh(s) represents the category augmentation feature at the s-th scale, Pi represents the category probability distribution, and Bi represents the predicted bounding box.

[0080] The bounding box Bi is represented as:

[0081] (12)

[0082] Preferably, the detection decoding is performed by obtaining detection confidence based on the category probability of candidate lesion targets and then filtering the candidate lesion targets.

[0083] The detection confidence score of the i-th candidate lesion is expressed as:

[0084] (13)

[0085] When the detection confidence level is lower than a preset threshold, the corresponding candidate lesion target is filtered out; when multiple candidate lesion targets correspond to the same lesion region, the repeated box suppression strategy is used to retain the candidate lesion targets with higher detection confidence.

[0086] On the other hand, the present invention provides an esophageal endoscopy lesion target detection system, comprising:

[0087] An image input unit is used to acquire an esophageal endoscope image to be detected and to preprocess the esophageal endoscope image.

[0088] The processing module is used to perform main feature extraction, color density constraint guidance, and mucosal texture boundary perception on the preprocessed esophageal endoscope images, specifically including:

[0089] The backbone feature extraction unit is used to extract multi-scale backbone semantic features, including shallow-scale features, mid-scale features and deep-scale features, through the backbone feature extraction.

[0090] Color density constrained leader branch is used to extract multi-scale color leader features through the color density constrained leader and generate a color effective region constraint map;

[0091] The mucosal texture boundary perception branch is used to extract multi-scale texture boundary features through the mucosal texture boundary perception, compensate for the structural information that the color density constraint leading branch cannot stably describe, and enhance the roughness of mucosal texture, edges and local anomalies.

[0092] The heterogeneous prior adaptive convergence module is used to perform heterogeneous prior adaptive convergence of the backbone semantic features, the color leading features and the texture boundary features at the same scale to obtain multi-scale fusion enhanced features.

[0093] The category prototype bias suppression module is used to suppress the radial category prototype bias of the multi-scale fused enhancement features to obtain category enhancement features;

[0094] The detection and decoding unit is used to detect and decode the category enhancement features to obtain the category, confidence level, and bounding box position of the lesion target in the esophageal endoscopy image.

[0095] The results output unit calculates the confidence level of each candidate target based on the category probability and bounding box position of the candidate lesion targets, filters the candidate targets, and outputs the lesion detection results in the esophageal endoscopy image.

[0096] Compared with the prior art, the present invention has the following beneficial effects:

[0097] (1) The effective color responses such as G and B are quantified and screened by the color density constraint leading branch, and a color effective area restriction map is constructed to make the backbone network and auxiliary feature extraction process more focused on the lesion-related area and reduce the invalid interference caused by highlights, background and borders.

[0098] (2) By explicitly modeling gradient, Laplacian response and local contrast through the mucosal texture boundary perception branch, the characterization of abnormal surface texture, irregular boundary and local roughness changes of esophageal mucosa can be enhanced, making up for the shortcomings of relying solely on color prior.

[0099] (3) By using heterogeneous prior adaptive convergence, dynamic weight allocation and spatial selection are performed on the three features of the main semantics, color lead and texture boundary, so that different types of lesions can obtain a suitable fusion method at different scales and in different regions, avoiding information conflicts caused by simple splicing or addition.

[0100] (4) By suppressing the class prototype bias, the class imbalance problem can be alleviated at the structural level, the separability and recall of classes with few samples can be improved, and the bias of the model on high-frequency classes can be reduced.

[0101] Under the same training data and detection framework, this invention can improve the model's feature response strength to color abnormal regions, texture boundary abnormal regions, and lesions of few sample categories, thereby improving the recall rate, classification stability, and boundary localization consistency of lesion detection. Attached Figure Description

[0102] Figure 1 An example diagram of RT-DETR target detection is provided for the background art of this application.

[0103] Figure 2 This is an example diagram of multi-branch target detection provided for the background technology of this invention application.

[0104] Figure 3 This is a flowchart illustrating a target detection method for esophageal endoscopy images provided in an embodiment of the present invention. Detailed Implementation

[0105] Under the existing detection framework, this invention adopts the following technical approach to address the technical deficiencies of the prior art:

[0106] (1) Based on color density response and channel validity screening, a color density-constrained leading branch is constructed, and the range of global feature extraction is narrowed by color effective region restriction map;

[0107] (2) Construct a mucosal texture boundary perception branch based on gradient, Laplacian and local contrast to supplement structural discrimination cues in addition to color information;

[0108] (3) Based on the three-way information of backbone semantics, color guidance and texture boundary, a heterogeneous prior adaptive convergence module is constructed to realize multi-scale dynamic fusion;

[0109] (4) Construct a class prototype bias suppression module based on class prototype response to alleviate the high-frequency class bias problem caused by class imbalance.

[0110] The present invention will now be described in detail with reference to embodiments and accompanying drawings. However, it should be understood that the embodiments and drawings are for illustrative purposes only and do not constitute any limitation on the scope of protection of the present invention. All reasonable modifications and combinations included within the inventive spirit of the present invention fall within the scope of protection of the present invention.

[0111] Example 1

[0112] like Figure 3 This embodiment provides a target detection method for esophageal endoscopy images. The overall process is as follows: Based on the esophageal endoscopy image, on the one hand, the backbone detection network extracts global semantic features; on the other hand, the color density constraint leading branch and the mucosal texture boundary perception branch are input in parallel to extract color priors and texture boundary priors, respectively; then, at the multi-scale level, features are dynamically fused through heterogeneous prior adaptive convergence; the fused multi-scale features are then input into class prototype bias suppression for class discrimination enhancement; finally, the detection decoding outputs the lesion target box and the corresponding class to obtain the detection result. Specifically:

[0113] S1. Obtain the image

[0114] In this embodiment, the laboratory collaborates with surrounding hospitals to collect data to obtain esophageal endoscopy images. Currently, the dataset of esophageal cancer endoscopy images is only 1518 images. The imbalance in the number of lesion categories seriously affects the performance of the lesion detection model (due to the design of the category prototype bias suppression module).

[0115] S2, Multiscale Modeling

[0116] Multi-scale features include three pathways: core semantics, color-leading features, and texture boundary features. Specifically, they include:

[0117] S2.1 Main Semantic Feature Extraction

[0118] S2.1.1 Shallow scale features are used to characterize local edges, fine textures, superficial lesion outlines, and local color changes in esophageal endoscopy images. At this scale, the backbone semantic features provide basic spatial structure information, the color leading features provide lesion-related color density responses, and the texture boundary features provide information on fine textures and boundary abrupt changes on the mucosal surface.

[0119] S2.1.2 Mid-scale features are used to characterize the local structure of the lesion area, the differences between the abnormal color area and the surrounding normal mucosa, and the joint features of lesion boundary and texture changes. At this scale, the backbone semantic features provide local regional semantics, the color leading features enhance the staining reaction and color density differences, and the texture boundary features supplement the information on local roughness, wrinkle changes and structural disorder.

[0120] S2.1.3 Deep-scale features are used to characterize the high-level semantic information, category discrimination information and global context information of lesion targets. At this scale, the backbone semantic features provide the overall lesion category related expression, the color leading features help distinguish the color performance differences of different lesion categories, and the texture boundary features help identify the differences of different lesion categories in tissue structure and boundary morphology.

[0121] S2.2 Color Density Constrained Leading Branch (CDCP Branch)

[0122] This branch is used to extract lesion-related priors from the perspectives of color density and channel response, and to constrain the scope of interest for the backbone features using effective color response regions. Specifically, it includes:

[0123] S2.2.1 Color Input Structure

[0124] The input endoscopic image is an RGB three-channel image, denoted as R, G, and B respectively. First, the original image is standardized to reduce the impact of equipment differences, brightness fluctuations, and changes in imaging conditions. Standardization methods can include Z-score normalization, linear normalization, or mean-variance based normalization.

[0125] Subsequently, a color difference representation tensor is constructed, including but not limited to the following components: green channel G; blue channel B; absolute value of the difference between green and blue |G−B|; red-green difference R−G; red-blue difference R−B. Preferably, the above five components are combined into a color leader input vector.

[0126] S2.2.2 Effective Channel Filtering Mechanism

[0127] To avoid subjective judgments about "which channel is useful," this embodiment introduces a quantitative screening method for channel effectiveness. One or more of the following metrics are calculated for candidate color channels:

[0128] a. Effective Response Ratio (ER): The percentage of pixels whose pixel values ​​deviate from the background range;

[0129] b. Coefficient of Variation (CV): Measures the degree of dispersion in the distribution of pixels in a channel;

[0130] c. Local response entropy: measures the amount of information in a local region;

[0131] d. Response Concentration Index: Measures whether the response region is focused on a local area rather than generalizing to the whole map.

[0132] In a preferred embodiment, ER and CV are calculated for the G channel and the B channel, and a channel that meets one of the following conditions is considered a valid channel:

[0133] a. ER is within the preset range, and CV is higher than the threshold;

[0134] b. Or the local response entropy is higher than the threshold;

[0135] c. Or the channel selection gating output learned during the training phase is higher than the threshold.

[0136] This design enables branch lines to automatically identify color information that is more distinctive to lesion areas.

[0137] S2.2.3 Constraints on the Effective Color Area

[0138] This embodiment further utilizes the effective response regions extracted from the G and B channels to generate a color density constraint map.

[0139] This constraint map is used to indicate the spatial extent of color reactions that may be associated with lesions in an image. It can be generated as follows:

[0140] a. Take the union of the effective G and B response regions;

[0141] b. Combine |G−B|, R−G, and R−B to form a color saliency map;

[0142] c. Then, a soft mask between 0 and 1 is obtained through convolution and the Sigmoid function.

[0143] Then, the soft mask is applied to the global feature extraction path of the backbone network or the feature extraction process inside the auxiliary branch, so that subsequent convolution calculations and feature aggregation focus on the effective color region, thereby reducing the global extraction range, reducing invalid background interference and improving feature utilization efficiency.

[0144] In a preferred embodiment, the color density constraint map is denoted as Mc. For the feature Fm(s) output by the backbone network at the s-th scale, Mc(s) is obtained by downsampling Mc to the corresponding scale, and the backbone features are constrained in the following way:

[0145] (1)

[0146] in, These are learnable or preset enhancement coefficients. This method does not directly delete the background area, but rather enhances the main feature response within the effective color response area, while reducing auxiliary enhancement in the ineffective area, thereby avoiding interference from black borders, highlight areas, and dark areas in the lumen on lesion detection.

[0147] S2.2.4 Color Consistency Redundancy Suppression

[0148] Calculate the normalized difference measure (NMSE) for the green channel feature Fg and the blue channel feature Fb:

[0149] (2)

[0150] If NMSE is less than the preset threshold τ, it is considered that there is high redundancy in the two channel features, and the feature with the higher channel variation coefficient is retained as the color consistency feature; if NMSE is greater than or equal to τ, it is considered that the two channels contain complementary lesion responses, and Fg and Fb are concatenated and compressed by 1×1 convolution to obtain the fused color feature. It should be a very small positive number to prevent the denominator from being 0 or too small, thus avoiding unstable calculations.

[0151] This method reduces redundant calculations when color responses are consistent and preserves information about potential lesions when color responses differ significantly.

[0152] S2.2.5 Color Vector Output

[0153] After color convolutional encoding, gating enhancement, and masking constraint, multi-scale color leading feature vectors are output, corresponding to the shallow, medium, and deep scale features in the main detection network, respectively, which are used for subsequent heterogeneous convergence with the backbone features.

[0154] S2.3 Texture Boundary Aware Branch (MTBP Branch)

[0155] This branch is used to compensate for structural information that color branches cannot stably describe, specifically enhancing mucosal texture, edges, and localized roughness. It includes the following steps:

[0156] S2.3.1 Texture Input Construction

[0157] The input RGB image is converted into a grayscale basic representation, and the following texture boundary priors are further computed: grayscale map; gradient magnitude map; Laplacian response absolute value map; local contrast map; optional directional gradient coding map, local binary pattern map or multi-scale structure tensor map.

[0158] In a preferred embodiment, four components are used as branch inputs: grayscale image, gradient magnitude image, Laplace absolute response image, and local contrast image.

[0159] In a preferred embodiment, the grayscale image Y is obtained by weighting the R, G, and B channels of the input image:

[0160] (3)

[0161] The gradient magnitude map Gm is calculated from the horizontal gradient Gx and the vertical gradient Gy:

[0162] (4)

[0163] In the above formula, It is a very small positive stable term, which is mainly used to prevent numerical instability during the calculation process.

[0164] The local contrast image Cl is obtained by the difference between the grayscale image and its local mean:

[0165] (5)

[0166] The Y, Gm, absolute values ​​of the Laplacian response, and Cl are concatenated and input into the texture boundary perception branch to obtain multi-scale texture boundary features.

[0167] S2.3.2 Generation of significant texture boundary regions

[0168] Gradient magnitude and local contrast are jointly convolved to generate a saliency map of the mucosal structure. This saliency map is used to identify regions with abrupt changes in edge size, rough texture, and abnormal tissue structures. Subsequently, a texture boundary mask is obtained through two-layer convolutional encoding and a spatial gating network, and this mask is applied to the texture feature encoding result to highlight lesion edges and abnormal texture regions.

[0169] S2.3.3 Multi-scale Texture Encoding

[0170] Within the branch, stride convolution or downsampling convolution is used to form texture feature maps of three scales, which are then connected to the multi-scale feature layers P3, P4, and P5 in the main network, respectively.

[0171] Compared to using a single texture enhancement module, this embodiment employs an independent texture boundary perception branch, allowing texture priors to enter the fusion stage as parallel heterogeneous information sources, thereby improving the detection capability for lesions with indistinct colors but abnormally obvious textures.

[0172] S3, Heterogeneous Prior Adaptive Convergence (HPAA)

[0173] Heterogeneous prior adaptive convergence is used to jointly model and dynamically fuse the global semantic features output by the backbone network, the color prior features output by the color density constraint leading branch, and the texture boundary prior features output by the mucosal texture boundary perception branch on the same scale. This avoids the problems of information conflict, feature submersion, and loss of weak lesion response caused by simple addition or direct splicing in the existing technology.

[0174] In this embodiment, for any detection scale s, let the output feature of the backbone network be... The color branch output characteristics are The texture branch output features are .

[0175] First, the color branch features and texture branch features are aligned in terms of spatial size and channel dimension to ensure that all three features have a consistent feature representation dimension. The aligned color and texture features are denoted as follows: and .

[0176] Subsequently, the three features are concatenated along the channel dimension to form a joint feature:

[0177] (6)

[0178] Based on this, global pooling and a weight generation unit are used to calculate the fusion weights of the three features. The three weights are denoted as follows: , , And satisfy:

[0179] (7)

[0180] This yields the modality-level fusion features:

[0181] (8)

[0182] The core features provide overall semantic information, color features highlight lesion color density and areas of abnormal staining reaction, and texture features supplement information such as mucosal boundaries, local roughness, and structural disorder. Through this weighting method, the contribution ratios of the three branches can be dynamically adjusted according to the performance characteristics of different lesion samples.

[0183] To further prevent the features of small lesions or weakly defined regions from being weakened during the fusion process, this embodiment introduces a local detail compensation mechanism after fusion. This mechanism performs convolutional refinement on the fused features and outputs the final result as a residual. The final output features can be represented as:

[0184] (9)

[0185] in, This represents the local detail compensation feature extracted from the joint features, used to recover edge details and weak response information that may be lost during the fusion process.

[0186] Through the above design, heterogeneous prior adaptive convergence can achieve adaptive fusion of backbone global semantic features, color prior features and texture boundary features on the same detection scale, so that color anomalies, texture anomalies and high-level semantic information can be effectively and collaboratively represented in a unified detection framework, thereby improving the detection accuracy and classification stability of lesion targets in esophageal endoscopy images.

[0187] S4, Class Prototype Bias Suppression (CPBS)

[0188] To address the issue of uneven sample numbers for different lesion categories in esophageal endoscopy images, this embodiment introduces class prototype bias suppression before the fused multi-scale features enter the detection and decoding unit. This is used to enhance the discrimination response of fewer sample categories and suppress the excessive occupation of the feature space by high-frequency categories.

[0189] Let the fused features be Fout(s). First, global pooling is performed on them to obtain the sample-level query vector q(s). Simultaneously, a learnable category prototype vector Pk is assigned to each lesion category, where k represents the category number. By calculating the correlation between q(s) and each category prototype Pk, the response distribution between the current sample and each category is obtained.

[0190] In a preferred embodiment, a category frequency adjustment factor rk can be further introduced to reduce the response advantage of high-frequency categories and compensate for the insufficient response of low-frequency categories. The category response weight can be expressed as:

[0191] (10)

[0192] Where λ is the adjustment coefficient, and rk is determined based on the number of samples in the k-th class in the training set. For classes with fewer samples, rk takes a larger value; for classes with more samples, rk takes a smaller value.

[0193] The class prototypes are weighted and summed according to the class response weights Ak to obtain the class context vector Pc. Pc is then jointly encoded with the original fused features to generate class-aware gating coefficients. Finally, the fused features are recalibrated using these gating coefficients to obtain class-enhanced features.

[0194] In this way, class prototype bias suppression can compensate for class imbalance at the model structure level, enabling the few-sample lesion class to obtain stronger discriminative feature expression before detection and decoding, thereby improving the detection stability of the few-sample class.

[0195] S5, Detection and Decoding

[0196] The multi-scale enhancement features output by the category prototype bias suppression module are input into the detection decoding unit. These multi-scale enhancement features include shallow-scale enhancement features, mid-scale enhancement features, and deep-scale enhancement features. Based on the interaction between the target query vector and the multi-scale enhancement features, the detection decoding unit generates decoding features for multiple candidate lesion targets and performs category prediction and bounding box regression on each.

[0197] For the i-th candidate lesion target, its category prediction result and bounding box prediction result can be expressed as:

[0198] (11)

[0199] Where D represents the detection decoding unit, Qi represents the i-th target query vector, Fenh(s) represents the augmentation feature at the s-th scale, Pi represents the class probability distribution, and Bi represents the predicted bounding box.

[0200] The bounding box Bi can be represented as:

[0201] (12)

[0202] Where cxi and cyi represent the coordinates of the center point of the predicted bounding box, and wi and hi represent the width and height of the predicted bounding box, respectively.

[0203] S6. Obtain the test results

[0204] Based on the class probability and bounding box location of the candidate lesion targets, the confidence score of each candidate target is calculated, and the candidate targets are then filtered. Preferably, the candidate target with the highest class probability is taken as its detection confidence score.

[0205] (13)

[0206] When the score is below a preset confidence threshold, the candidate target is filtered out. When multiple candidate boxes correspond to the same lesion region and have a high degree of overlap, the detection boxes with higher confidence are retained through a duplicate box suppression strategy. Finally, the retained detection results are mapped back to the original esophageal endoscopy image coordinate system to output the lesion target's category label, confidence score, and bounding box position.

[0207] This embodiment also provides some other alternative solutions:

[0208] 1. Alternatives to color-coded branches

[0209] In the color density-constrained leading branch, the color input construction in step S2.2.1 is not limited to G, B, |G−B|, R−G, R−B, and can also be replaced by any one or more of the following combinations: S and V components in HSV space; a and b components in Lab space; Cb and Cr components in YCbCr space; pseudo-color response map after color enhancement; and color response map obtained through training.

[0210] 2. Alternatives to effective channel screening

[0211] In step S2.2.2, the ER and CV threshold filtering can be replaced by: a learnable channel selection gating network; automatic filtering based on entropy; filtering based on mutual information; and dynamic filtering based on sample statistical priors.

[0212] 3. Alternatives for texture boundary branches

[0213] In step S2.3.1 of the texture boundary perception branch, the texture prior is not limited to gradient, Laplacian and local contrast, but can also be: Local Binary Pattern (LBP); Gabor filter response; Structural Tensor Features; Histogram of Oriented Gradients (HOG); Wavelet Multiscale Texture Decomposition Results.

[0214] 4. Alternative solutions for the fusion module

[0215] Step S3, heterogeneous prior adaptive convergence, can be replaced by: a three-way fusion structure based on cross-attention; a fusion structure based on gated weighting and dynamic convolution; a heterogeneous fusion structure based on Transformer token recombination; or a multi-branch relationship modeling structure based on graph neural networks.

[0216] 5. Alternatives to class imbalance suppression

[0217] Step S4, category prototype bias suppression, can also be replaced by: adaptive scaling of category frequency; few-sample enhancement based on prototype memory; category balancing based on query reweighting; and discriminative enhancement based on category center separation constraints.

[0218] 6. Alternatives to the main detection framework for backbone feature extraction

[0219] This embodiment is not limited to the RT-DETR framework, and can also be adapted to: Faster R-CNN; RetinaNet; YOLO series; Deformable DETR; DINO detection framework; and other target detection networks with multi-scale detection structures.

[0220] Example 2

[0221] This embodiment provides an esophageal endoscopy lesion target detection system, which includes:

[0222] An image input unit is used to acquire an esophageal endoscope image to be detected and to preprocess the esophageal endoscope image.

[0223] The processing module is used to perform main feature extraction, color density constraint guidance, and mucosal texture boundary perception on the preprocessed esophageal endoscope images, specifically including:

[0224] The backbone feature extraction unit is used to extract multi-scale backbone semantic features, including shallow-scale features, mid-scale features and deep-scale features, through the backbone feature extraction.

[0225] Color density constrained leader branch is used to extract multi-scale color leader features through the color density constrained leader and generate a color effective region constraint map;

[0226] The mucosal texture boundary perception branch is used to extract multi-scale texture boundary features through the mucosal texture boundary perception, compensate for the structural information that the color branch cannot stably describe, and enhance the roughness of mucosal texture, edges and local anomalies.

[0227] The heterogeneous prior adaptive convergence module is used to perform heterogeneous prior adaptive convergence of the backbone semantic features, the color leading features and the texture boundary features at the same scale to obtain multi-scale fusion enhanced features.

[0228] The category prototype bias suppression module is used to suppress the radial category prototype bias of the multi-scale fused enhancement features to obtain category enhancement features;

[0229] The detection and decoding unit is used to detect and decode the category enhancement features to obtain the category, confidence level, and bounding box position of the lesion target in the esophageal endoscopy image.

[0230] The results output unit calculates the confidence level of each candidate target based on the category probability and bounding box position of the candidate lesion targets, filters the candidate targets, and outputs the lesion detection results in the esophageal endoscopy image.

[0231] The esophageal endoscopy lesion detection system receives the esophageal endoscopy image to be detected and performs color prior extraction, texture boundary prior extraction, multi-scale feature fusion, class bias suppression, and detection decoding. The final output includes lesion category, detection confidence score, and bounding box coordinates.

[0232] If the input image has been scaled, padded, or cropped before detection, the system maps the predicted bounding box coordinates back to the original image coordinate system and annotates the lesion detection results on the original image. These detection results can be used for lesion region indication, image visualization, detection report generation, or subsequent medical auxiliary analysis.

[0233] The above embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A target detection method for esophageal endoscopy images, characterized in that, Includes the following steps: S1. Acquire the esophageal endoscope image to be detected, and preprocess the esophageal endoscope image; S2. Perform main trunk feature extraction, color density constraint guidance, and mucosal texture boundary perception on the preprocessed esophageal endoscope images, specifically including: S2.1 Extract multi-scale semantic features through the backbone feature extraction, including shallow scale features, medium scale features and deep scale features; S2.2 Extract multi-scale color leader features using the color density constraint leader and generate a color effective region constraint map; S2.3 Extract multi-scale texture boundary features through the mucosal texture boundary perception to compensate for the structural information that the color density constraint leader cannot stably describe, and enhance the roughness of mucosal texture, edges and local anomalies; S3. Perform heterogeneous prior adaptive convergence on the backbone semantic features, the color leading features and the texture boundary features at the same scale to obtain multi-scale fusion enhanced features; S4. Suppress the radial category prototype bias of the multi-scale fusion enhancement feature to obtain the category enhancement feature; S5. Detect and decode the category enhancement features to obtain the category, confidence level, and bounding box position of the lesion target in the esophageal endoscopy image; S6. Calculate the confidence level of each candidate target based on the category probability and bounding box position of the lesion target, filter the candidate targets, and output the lesion detection results in the esophageal endoscopy image.

2. The method for detecting lesions using esophageal endoscopy according to claim 1, characterized in that, The steps leading to the color density constraint in step S2.2 include color input construction, effective channel filtering, effective color region constraint, and color consistency redundancy suppression. The color input construction includes extracting the red, green, and blue channels from the esophagoscopic image and constructing a color difference representation that includes the green channel, blue channel, green and blue differences, red-green differences, and red-blue differences. The effective channel screening includes screening effective color channels based on one or more of the following: effective response ratio, coefficient of variation, local response entropy, or response concentration index. The effective color region constraint includes generating a color effective region constraint map based on the effective color channels, and using the color effective region constraint map to impose region constraints on the backbone semantic features or color leading features; The color consistency redundancy suppression includes measuring the difference between the green channel features and the blue channel features. When the difference between the two is less than a preset threshold, one of the color features is retained. When the difference between the two is greater than or equal to the preset threshold, the two color features are fused.

3. The method for detecting lesions using esophageal endoscopy according to claim 2, characterized in that, The color effective region restriction map is used to weight and constrain the backbone semantic features at the s-th scale. The constrained backbone semantic features are represented as follows: （1） Where Fm(s) represents the backbone semantic features at the s-th scale, Mc(s) represents the color effective region constraint map downsampled to the s-th scale, and β represents the enhancement coefficient. This represents the core semantic features after being constrained by the effective color region. The difference between the green channel features and the blue channel features is measured using normalized mean square error, expressed as: （2） Where Fg represents the green channel feature, Fb represents the blue channel feature, MSE(Fg, Fb) represents the mean square error between the green channel feature and the blue channel feature, Var(Fg) and Var(Fb) represent the variances of the green channel feature and the blue channel feature, respectively, and ε is a minimal positive stable term.

4. The method for detecting lesions using esophageal endoscopy according to claim 3, characterized in that, Step S2.3, mucosal texture boundary perception, includes texture input construction, generation of salient texture boundary regions, and multi-scale texture encoding. The texture input construction includes converting the esophageal endoscopy image into a grayscale image, and obtaining a gradient magnitude map, a Laplacian response map, and a local contrast map based on the grayscale image; The generation of salient regions of texture boundaries includes jointly encoding gradient magnitude maps and local contrast maps to generate salient maps of mucosal structures. The multi-scale texture encoding includes obtaining multi-scale texture boundary features corresponding to the backbone semantic features through convolutional encoding or downsampling operations; The grayscale image, gradient magnitude image, and local contrast image are respectively represented as follows: （3） （4） The local contrast image Cl is obtained by the difference between the grayscale image and its local mean: （5） Where R, G, and B represent the red, green, and blue channels, respectively; Y represents the grayscale image; Gx and Gy represent the horizontal and vertical gradients, respectively; Gm represents the gradient magnitude image; Cl represents the local contrast image; and ε is the minimum positive stable term.

5. The method for detecting lesions using esophageal endoscopy according to claim 4, characterized in that, The heterogeneous prior adaptive convergence described in step S3 is the fusion of backbone semantic features, color leading features, and texture boundary features at the same detection scale; For the s-th detection scale, let the backbone semantic feature be Fm(s), the color leading feature be Fc(s), and the texture boundary feature be Ft(s). After aligning the color leading feature and the texture boundary feature in terms of spatial size and channel dimension, we obtain F'c(s) and F't(s). The joint features are obtained by concatenating Fm(s), F'c(s), and F't(s): （6） Based on the joint features, three-way fusion weights are generated, and modal-level fusion features are obtained according to the three-way fusion weights; The three-way fusion weights include backbone semantic feature weights, color-leading feature weights, and texture boundary feature weights, denoted as αm(s), αc(s), and αt(s), respectively, and satisfy the following: （7） The modality-level fusion feature is represented as follows: （8）。 6. The method for detecting lesions in esophageal endoscopy according to claim 5, characterized in that, The heterogeneous prior adaptive convergence also includes local detail compensation, which extracts local detail compensation features Fdetail(s) from the joint features and obtains the fusion enhancement features at the s-th detection scale through residual methods: （9） Where Fout(s) represents the fusion enhancement feature of the s-th detection scale.

7. The method for detecting lesions using esophageal endoscopy according to claim 1, characterized in that, The category prototype bias suppression mentioned in step S4 includes category prototype establishment, sample-level prototype association, and category-aware feature recalibration; The establishment of the category prototype includes setting a learnable category prototype vector for each lesion category; The sample-level prototype association includes obtaining a sample-level query vector based on the fusion enhancement features, and calculating the correlation between the sample-level query vector and the prototype vectors of each category. The category-aware feature recalibration includes generating a category context vector based on the correlation, and using the category context vector to recalibrate the fused enhanced features; The category prototype bias suppression includes a category frequency adjustment factor, which adjusts the response of each category prototype. The category response weights are expressed as follows: （10） Where q(s) represents the sample-level query vector corresponding to the s-th scale, Pk represents the category prototype vector corresponding to the k-th lesion, rk represents the category frequency adjustment factor corresponding to the k-th lesion, λ represents the adjustment coefficient, and Ak represents the category response weight of the k-th lesion.

8. The method for detecting lesions using esophageal endoscopy according to claim 1, characterized in that, The detection and decoding described in step S5 is based on the interaction between the target query vector and the multi-scale category enhancement features to obtain the category probability and bounding box coordinates of the candidate lesion target; For the i-th candidate lesion target, its category prediction result and bounding box prediction result are expressed as follows: （11） Where D represents detection decoding, Qi represents the i-th target query vector, Fenh(s) represents the category augmentation feature at the s-th scale, Pi represents the category probability distribution, and Bi represents the predicted bounding box. The bounding box Bi is represented as: （12）。 9. The method for detecting lesions using esophageal endoscopy according to claim 8, characterized in that, The detection decoding is based on the category probability of candidate lesion targets to obtain detection confidence and to filter candidate lesion targets. The detection confidence score of the i-th candidate lesion is expressed as: （13） When the detection confidence level is lower than a preset threshold, the corresponding candidate lesion target is filtered out; when multiple candidate lesion targets correspond to the same lesion region, the repeated box suppression strategy is used to retain the candidate lesion targets with higher detection confidence.

10. An esophageal endoscopic lesion target detection system, characterized in that, include: An image input unit is used to acquire an esophageal endoscope image to be detected and to preprocess the esophageal endoscope image. The processing module is used to perform main feature extraction, color density constraint guidance, and mucosal texture boundary perception on the preprocessed esophageal endoscope images, specifically including: The backbone feature extraction unit is used to extract multi-scale backbone semantic features, including shallow-scale features, mid-scale features and deep-scale features, through the backbone feature extraction. Color density constrained leader branch is used to extract multi-scale color leader features through the color density constrained leader and generate a color effective region constraint map; The mucosal texture boundary perception branch is used to extract multi-scale texture boundary features through the mucosal texture boundary perception, compensate for the structural information that the color density constraint leading branch cannot stably describe, and enhance the roughness of mucosal texture, edges and local anomalies. The heterogeneous prior adaptive convergence module is used to perform heterogeneous prior adaptive convergence of the backbone semantic features, the color leading features and the texture boundary features at the same scale to obtain multi-scale fusion enhanced features. The category prototype bias suppression module is used to suppress the radial category prototype bias of the multi-scale fused enhancement features to obtain category enhancement features; The detection and decoding unit is used to detect and decode the category enhancement features to obtain the category, confidence level, and bounding box position of the lesion target in the esophageal endoscopy image. The results output unit calculates the confidence level of each candidate target based on the category probability and bounding box position of the candidate lesion targets, filters the candidate targets, and outputs the lesion detection results in the esophageal endoscopy image.

Images