Feature classification method and system based on alimentary canal internal image

Abstract

The invention discloses a feature classification method and system based on an alimentary canal internal image, and relates to the technical field of image recognition, and the method comprises the steps: collecting an alimentary canal image in real time through an endoscope, and carrying out the preprocessing; based on a position estimation network formed by a convolutional neural network and a time sequence encoder, the image sequence is mapped to a standardized alimentary canal template, and endoscope position evaluation is achieved; the attention weight of each sample is calculated according to the time sequence similarity of the similar positions, and individual adaptive optimization of the image recognition model is completed; outputting an identification result, severity and confidence distribution of a focus type through an image identification model, constructing a graph structure model taking the focus type as a node, performing information propagation in a graph structure, and deducing focus probability distribution of an undetected area; and an inference result is fed back to the attention weight of the classification output layer, so that dynamic self-learning and global optimization of the model are realized. According to the method, the accuracy and stability of alimentary canal image recognition are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a feature classification method and system based on internal images of a digestive tract. BACKGROUND

[0002] With the popularization of clinical digestive endoscopy, the amount of image data collected by doctors at the stomach, intestine and other digestive tract sites is growing exponentially. Traditional image recognition methods rely on manual observation or detection algorithms based on shallow features, which cannot fully utilize the temporal continuous frame information and the semantic association of complex spatial structures. The digestive tract has strong non-rigid, curved and repetitive texture characteristics. The same lesion presents different morphologies in different postures, which leads to recognition drift in positioning and classification of existing convolutional neural networks. At the same time, existing models generally use static training parameters, which cannot be updated adaptively according to the physiological differences of different patients and environmental factors, and lack dynamic learning ability for individual features.

[0003] In addition, internal lesions of the digestive tract often have pathological symbiotic relationships, such as physiological coupling between pathological progression of types such as erosion, ulcer and polyp. However, existing algorithms only identify single frames or local areas, do not establish global structural relationships between lesion types, and cannot infer potential risk distribution in unexplored areas based on partially explored areas. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a feature classification method based on internal images of a digestive tract, which solves the problems of unstable positioning, poor individual adaptability and lack of global correlation reasoning between lesions in existing endoscopic image recognition.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a feature classification method based on internal images of a digestive tract, which includes real-time acquisition of internal images of the digestive tract by an endoscope, and completion of noise removal and image enhancement preprocessing;

[0008] According to the acquired real-time images and the depth of the endoscope, the position of the endoscope is evaluated, and the training process of the image recognition model is invaded according to the evaluation results: by similarity calculation at similar positions, the attention of each training sample is analyzed, and the individual features are adapted in the image recognition model;

[0009] The image recognition model is used to recognize the internal images of the digestive tract collected at the next moment;

[0010] According to the result of the image recognition, the spatial correlation scattering influence is carried out, the environment inference of the lesion is realized, and the environment inference is fed back to the image recognition model, so that the image recognition model is supplemented in the adaptive process.

[0011] By using the result of the image recognition, the classification of the internal image of the digestive tract is carried out, and the feature classification result of all internal images of the digestive tract is obtained.

[0012] As a preferred scheme of the feature classification method based on the internal image of the digestive tract, wherein: the position evaluation includes constructing a feature sequence with the continuous frame images in the internal digestive tract and the depth corresponding to each frame image; the feature sequence is input into a position estimation network composed of a convolutional neural network and a time encoding encoder; the spatial texture and structure gradient features are extracted through the convolutional layer, and the dynamic changes between adjacent frames are modeled through the time encoding encoder.

[0013] In the training stage of the position estimation network, the output feature is mapped to the standardized digestive tract three-dimensional template coordinate system, and the matching error with the template position label is calculated.

[0014] In the inference stage of the position estimation network, the corresponding region A of the current endoscope field of view in the standard digestive tract template is determined according to the pose probability distribution output by the model.

[0015] In the training set and the verification set of the image recognition model, any image in each sample uniquely corresponds to a coordinate in the standard digestive tract template; the image of the sample corresponding to region A in the training set and the verification set is captured as the similar position of sequence a, recorded as sequence b; wherein, sequence a is a continuous frame image sequence of fixed length extracted from the current time as the starting point in the continuous frame image.

[0016] The dynamic time warping algorithm is used to time-align sequence a and sequence b to obtain a set of aligned frame pairs; the similarity of all aligned frame pairs is weighted and averaged to obtain the average similarity of sequence a and similar position sequence b; wherein, the similarity of the i th sample and sequence a is recorded as: , as the attention between the current patient and the sample.

[0017] As a preferred scheme of the feature classification method based on the internal image of the digestive tract, wherein: the adaptation of the image recognition model includes accumulating the attention of each sample in each period according to the analysis period of the attention, obtaining the attention base of each sample; after normalizing the attention base of all samples, the attention between the current patient and each sample is obtained.

[0018] Adjust the training process through attention between the current patient and each sample, so as to obtain image recognition model parameters for the current patient;

[0019] The image recognition model is a deep neural network structure combining convolution feature extraction and attention fusion, including a residual convolutional encoder, a cross-scale self-attention feature fusion module and a classification output head.

[0020] As a preferred scheme of the feature classification method based on the internal image of the digestive tract, the image recognition result includes the recognition result of different types of lesions in the digestive tract, the severity of each lesion recognition result and the confidence distribution of each lesion recognition result.

[0021] The confidence distribution is the probability of each type of lesion in each frame of image.

[0022] As a preferred scheme of the feature classification method based on the internal image of the digestive tract, the spatial correlation scattering effect includes taking each type of lesion as a node, taking the corresponding region obtained by the position evaluation and the severity of each lesion recognition result as node features, and constructing a graph structure.

[0023] The weighted coupling relationship between different lesion types is constructed as the edge of the graph, and the coupling weight is obtained by clinical priori and data-driven learning.

[0024] The image recognition result collected according to the attention analysis period is used for node generation to obtain a feature node set of the patient.

[0025] The feature node set of the patient is used for information propagation and aggregation in the graph structure to obtain the lesion judgment result of each sub-region in the undetected region and the probability distribution of each lesion type. The sub-region is the smallest region obtained by finite element division of the undetected region.

[0026] The lesion judgment result of each sub-region and the probability distribution of each lesion type are updated with the image collection of the attention analysis period.

[0027] The lesion judgment result of each sub-region and the probability distribution of each lesion type are fed back to the adaptive process of the image recognition model: in the process of real-time image collection, the corresponding n sub-regions of the corresponding region A are obtained through the corresponding region A obtained by the position evaluation; the lesion judgment result in the n sub-regions is counted, and the average probability of each type of lesion is obtained by using the probability distribution of each type of lesion in the n sub-regions, denoted as .

[0028] Where m represents the number of lesion types, including the recognition result of health. represents the mth lesion type; represents the probability of the mth lesion; G represents a set of attention-supplemented probabilities;

[0029] In the adaptive process, each lesion type is provided with a standard attention coefficient , in G is taken as an attention additional item of , and is calculated as an adaptive attention coefficient of each lesion type; wherein u represents an index of a lesion type, represents a standard attention coefficient of the uth lesion;

[0030] is taken as an attention weight of the classification output head, and dynamic attention adaptive adjustment of the model to different lesion types is realized.

[0031] As a preferred scheme of the feature classification method based on the internal image of the digestive tract according to the application, wherein: the classification result includes that each image can be classified into multiple classification results; a classification label is preset for each lesion;

[0032] The confidence of each lesion recognition result is compared with a preset judgment threshold; if the confidence of any lesion recognition result is higher than the judgment threshold, the image is classified into the classification label of the corresponding lesion, and the confidence of the lesion recognition result in the current classification label is labeled to obtain the feature classification result of the current image;

[0033] If the confidence of all lesion recognition results is not higher than the judgment threshold, the classification and the confidence under each classification label are labeled through the correlation analysis of the context to obtain the feature classification result of the current image;

[0034] The feature classification results of each image are integrated to obtain the feature classification results of all internal images of the digestive tract.

[0035] As a preferred scheme of the feature classification method based on the internal image of the digestive tract according to the application, wherein: the correlation analysis of the context includes that the current image is set as pending, and image acquisition and recognition are continued;

[0036] The confidence of each lesion recognition result is updated by taking the mean of the confidence of each lesion recognition result of the context image of the pending image;

[0037] ​The confidence of the to-be-determined image is updated for each lesion recognition result, and the comparison of the judgment threshold is performed, if the to-be-determined image obtains a feature classification result after the confidence is updated, the correlation analysis is completed, if the to-be-determined image cannot obtain a feature classification result after the confidence is updated, the to-be-determined image is specially marked.

[0038] In a second aspect, the present application provides a feature classification system based on internal images of a digestive tract, comprising: a collection unit, which collects internal images of the digestive tract in real time through an endoscope, and completes noise removal and image enhancement preprocessing;

[0039] An analysis module, which evaluates the position of the endoscope according to the collected real-time images and the depth of the endoscope, and invades the training process of the image recognition model according to the evaluation result: through similarity calculation in similar positions, the attention of each training sample is analyzed, and the individual features are adapted in the image recognition model;

[0040] An identification module, which performs image recognition on the internal images of the digestive tract collected at the next moment through the image recognition model;

[0041] A supplement module, which performs scattering influence of spatial correlation according to the result of the image recognition, realizes environmental inference of the lesion, and feeds back the environmental inference to the image recognition model, realizes the supplement of the image recognition model in the adaptive process;

[0042] A classification module, which performs classification of the internal images of the digestive tract using the result of the image recognition, and obtains the feature classification result of all internal images of the digestive tract.

[0043] In a third aspect, the present application provides a computer device, comprising a memory and a processor, and the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the feature classification method based on internal images of a digestive tract according to the first aspect of the present application is realized.

[0044] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein: when the computer program is executed by the processor, any step of the feature classification method based on internal images of a digestive tract according to the first aspect of the present application is realized.

[0045] The present application has the beneficial effects that: the present application realizes accurate positioning of the endoscope field of view in the standardized digestive tract template by introducing a position estimation network based on a convolutional neural network and a time sequence encoder; through the combination of time sequence similarity and attention mechanism, the image recognition model can be adaptively optimized according to different patient characteristics, and the robustness of the model to individual differences is significantly improved. Further, the present application constructs a weighted graph structure taking the lesion type as a node and the physiological correlation as an edge, and performs information propagation and aggregation on the graph to realize the lesion type inference of the undetected area. By feeding back the graph propagation result to the image recognition model, the attention weights of each lesion type are dynamically adjusted, so that the continuous evolution and global optimization of the recognition model are realized. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 Flowchart of the feature classification method based on the internal image of the digestive tract. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0049] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0050] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0051] REFERENCE Figure 1 For one embodiment of the present application, the embodiment provides a feature classification method based on the internal image of the digestive tract, comprising the following steps:

[0052] S1: Real-time acquisition of the internal image of the digestive tract by endoscope, and completion of noise removal and image enhancement preprocessing.

[0053] A continuous image acquisition of the inside of the digestive tract is performed by an endoscope system. The endoscope system is equipped with a high-brightness light source, a high-definition image sensor, and a posture detection module, and can record time, position, and depth information synchronously during the acquisition process. The acquired raw image signals are transmitted in real time to a data processing terminal through a high-speed link to ensure the continuity of the time sequence and the integrity of the image.

[0054] In order to improve the input quality of the subsequent recognition model, the image is preprocessed by noise removal and image enhancement. First, for the random noise introduced by liquid reflection, light flicker and sensor jitter, adaptive median filtering and bilateral filtering method is used for denoising processing, and time sequence noise suppression algorithm is used to maintain the texture consistency between frames. Second, according to the light source output intensity and the local brightness distribution, the light correction is carried out, the color deviation is eliminated by Retinex algorithm and white balance compensation, and the visibility of the lesion area is improved by using contrast limited adaptive histogram equalization. Subsequently, the image edge and texture are enhanced, the mucosa surface texture information is enhanced by using multi-scale Laplacian pyramid, and the frames with slight blur are sharpened by using frequency domain deconvolution. Finally, the size of all images is standardized and the frame is synchronized, each frame image is timestamped with the endoscope depth information, and the depth-image aligned feature sequence is constructed, which provides high signal-to-noise ratio input data for subsequent position evaluation and template mapping.

[0055] S2: According to the real-time image and the endoscope depth acquired, the position of the endoscope is evaluated, and the training process of the image recognition model is invaded according to the evaluation result: through the similarity calculation of similar positions, the attention of each training sample is analyzed, and the individual feature is adapted in the image recognition model. (Make the current digestive tract environment more effectively identified, and when each kind of digestive tract environment is identified, the attention of different lesions is enhanced)

[0056] The position evaluation includes constructing a feature sequence with the continuous frame images inside the digestive tract and the depth corresponding to each frame image; inputting the feature sequence into a position estimation network based on convolutional neural network and time sequence encoder; extracting spatial texture and structure gradient features through convolutional layer, and modeling the dynamic changes between adjacent frames through time sequence encoder. In the training stage of the position estimation network, the output features are mapped to the standardized digestive tract three-dimensional template coordinate system, and the matching error with the template position label is calculated. In the inference stage of the position estimation network, the corresponding area A of the current endoscope field of view in the standard digestive tract template is determined according to the pose probability distribution output by the model; realizing the nonlinear mapping from image sequence and depth features to digestive tract template space.

[0057] To say, the position evaluation realizes high-precision positioning and field of view tracking of the endoscope in a complex digestive tract environment by combining a neural network mapping of an image sequence and depth features. Traditional positioning methods based on geometric reconstruction or optical flow matching are limited by the non-rigid structure of the digestive tract and the influence of liquid reflection, and are prone to registration error accumulation and position drift. The present application can capture the mucosal surface morphology change and blood vessel distribution law by extracting multi-level spatial texture and structure gradient features through a convolutional neural network; at the same time, a temporal encoder is introduced to model the dynamic relationship between adjacent frames, so that the model can understand the image sequence change law corresponding to the endoscope advancing, rotating or retracting, etc., thereby having continuity and stability in the time dimension. In the training stage, the model output features are mapped to the standardized digestive tract three-dimensional template coordinate system, and the matching error with the template position label is calculated, so that the network can learn the nonlinear correspondence from local visual features to global spatial coordinates; in the inference stage, the region position of the current frame in the template is determined through the pose probability distribution, realizing unified positioning of the digestive tract space. Without relying on additional positioning sensors, self-calibration and continuous tracking can be realized only with image and depth data, which can effectively solve the problem of insufficient accuracy of traditional methods in non-rigid scenes.

[0058] It is also known that the convolutional neural network has strong spatial feature expression capability, and the temporal encoder can effectively capture the dynamic change law, and the combination of the two can build an end-to-end trainable nonlinear mapping model; under the constraint of the standardized template, stable convergence can be achieved through large-scale labeled sample training, so as to ensure high-precision position estimation in different individuals and different postures.

[0059] In the training set and the validation set of the image recognition model, any image in each sample (each sample is a digestive tract image sequence of a patient) uniquely corresponds to a coordinate in the standard digestive tract template; the images of the sample in region A in the training set and the validation set are captured as the similar position of sequence a, denoted as sequence b; wherein sequence a is a continuous frame image sequence of a fixed length taken forward from the current time as the starting point in the continuous frame image.

[0060] The dynamic time warping algorithm is used to time-align sequence a and sequence b to obtain a set of aligned frame pairs; the similarity of all aligned frame pairs is weighted averaged to obtain the average similarity of sequence a and similar position sequence b; wherein the similarity of the i th sample and sequence a is denoted as: , as the attention between the current patient and the sample.

[0061] It is to be known that, due to the significant structural differences and morphological variations of the digestive tract among individuals, the traditional recognition model trained based on single patient data is difficult to maintain stable performance on new patients. The present application establishes a position-consistent inter-individual alignment relationship by spatially corresponding the image samples in the training set and the validation set to the image sequence of the current patient under the coordinate system of the standardized digestive tract template, so that each frame of image can be uniquely mapped to a coordinate position in the template. Further, the dynamic time warping (DTW) algorithm is used to time-align the image sequences at the same position of different patients, which can eliminate the time offset caused by the differences in endoscope pushing speed and frame sampling frequency, and ensure the time synchronization of the comparison sequences. By calculating the feature similarity between the aligned frame pairs and taking the weighted average, the average similarity between the current patient and the historical samples is obtained The similarity reflects the feature consistency between patients in similar anatomical regions and can be used as attention weight to participate in the model adaptation process. The generalization ability of the model between different individuals is significantly enhanced, so that the recognition network pays more attention to the samples most similar to the anatomical features of the current patient during the learning process, improving the model migration efficiency and stability.

[0062] It is also to be understood that the dynamic time warping algorithm can effectively solve the alignment problem between sequences of different lengths, and in the image feature space, the inter-frame measurement can be realized through convolution feature cosine similarity or structural similarity index. At the same time, the standardized template provides a unified spatial reference, making the position correspondence and time alignment have mathematical constraints and trainability, thereby realizing the automation of cross-patient feature alignment and attention weight calculation.

[0063] Further, the attention of each sample in each period is accumulated according to the analysis period of the attention (preset period), and the attention base of each sample is obtained. After normalizing the attention base of all samples, the attention between the current patient and each sample is obtained.

[0064] Through the attention between the current patient and each sample, the training process is adjusted to obtain the image recognition model parameters for the current patient.

[0065] The image recognition model is a deep neural network structure combining convolution feature extraction and attention fusion, including a residual convolutional encoder, a cross-scale self-attention feature fusion module, and a classification output head.

[0066] The residual convolutional encoder is composed of multiple layers of convolution, batch normalization and ReLU activation, contains a residual connection unit, and is used to extract multi-layer spatial texture and structural gradient features of the colonoscopy image. It can suppress feature drift caused by illumination and posture changes. The cross-scale self-attention feature fusion module establishes a spatial and channel direction self-attention connection between feature maps of different scales; it uses a multi-head attention mechanism to fuse context information and realize long-range dependency modeling between the lesion area and the surrounding tissue. Its output is a unified feature map after fusion. The classification output head performs global average pooling (GAP) on the fused features, and outputs the lesion category and confidence through the fully connected layer and the Softmax layer. It can be extended to multi-task output for classification, positioning or segmentation at the same time.

[0067] The three modules are sequentially connected in structure: input image → residual convolutional encoder → cross-scale self-attention feature fusion module → classification output head.

[0068] The data is transmitted between modules through feature tensors (Feature Map), and an end-to-end trainable network is formed with a unified gradient backpropagation path.

[0069] Residual convolutional encoder → cross-scale self-attention feature fusion module:

[0070] The residual convolutional encoder (ResNet-Encoder) first extracts multi-layer features of the colonoscopy image to generate a set of feature maps of different scales, each containing different levels of semantic information and spatial details. These feature maps are input to the cross-scale self-attention feature fusion module.

[0071] The cross-scale module takes a multi-scale pyramid structure + multi-head self-attention mechanism (MHSA) as the core, dynamically adjusts feature weights by calculating the correlation between channels and spaces, thereby strengthening the prominent areas of the lesion and weakening the background interference. Its output is a unified feature map after fusion, which has global context awareness.

[0072] Cross-scale self-attention feature fusion module → classification output head:

[0073] The classification output head receives the fused feature map and aggregates the global semantics. It extracts a global semantic vector through Global Average Pooling (Global Average Pooling), and then outputs the lesion category and confidence through the fully connected layer + Softmax layer. This output head also supports additional multi-task branches (such as positioning or segmentation) to output the lesion position bounding box or saliency mask when needed.

[0074] Residual convolutional encoder: responsible for basic feature extraction (local texture, morphology, structure). Cross-scale self-attention module: responsible for global feature fusion (context enhancement and multi-scale alignment). Classification output head: responsible for decision output (class determination and confidence calculation). The three form a hierarchical progressive relationship of "feature extraction-feature fusion-result output", which constitutes a complete image recognition neural network.

[0075] It is worth mentioning that the above self-adaptation is realized through a periodic attention accumulation mechanism, which enables the image recognition model to adaptively update between different patients, allowing the network parameters to dynamically adjust according to individual feature differences, thereby improving the model's generalization ability and recognition stability in complex anatomical environments. Due to the uneven lighting, similar texture, and drastic posture changes in the internal environment of the digestive tract, traditional static parameter models cannot adjust according to the physiological differences of different patients, resulting in a decrease in recognition accuracy on new individuals. The present application accumulates the attention between each sample and the current patient within a preset attention analysis period, forms an attention base, and obtains the attention distribution of the current patient to each sample after normalization, realizing personalized weighted update of model parameters.

[0076] During parameter adjustment, attention weights are introduced into the model's training loss function to redistribute sample gradient contributions: samples with high similarity to the current patient's features are given higher gradient weights, thereby enhancing their influence on parameter updates; while samples with large feature differences are automatically given lower weights, causing the model to gradually move closer to the current patient's feature distribution during training. In this way, the model maintains its overall learning ability while obtaining adaptive convergence characteristics for specific patients.

[0077] This mechanism relies on the synergistic action of the residual convolutional encoder, cross-scale self-attention feature fusion module, and classification output head. The residual convolutional encoder is responsible for multi-layer spatial feature extraction, providing fine-grained texture information for subsequent fusion; the cross-scale self-attention module establishes dependencies between features of different scales through a multi-head attention mechanism, allowing the network to focus on key features in the lesion area within the global context; the classification output head uses the attention-adjusted fusion features for decision output, generating a confidence distribution for lesion categories through the Softmax function. In adaptive updating, the adjustment process of model parameters is achieved through backpropagation: after each analysis period, the normalized attention coefficients are used to recalculate the weighted gradients of the loss function, updating the residual convolutional layer weights, attention fusion module channel weights, and fully connected parameters of the classification output layer, allowing the model to prioritize features in the next period for image classification.

[0078] An individualized dynamic self-learning mechanism is realized: the model can correct the weight distribution according to the attention feedback on different patient data, thereby continuously optimizing the recognition performance. Among them, the attention accumulation and gradient update are based on the differentiable calculation process, which can be realized end-to-end in the standard deep learning framework, and the periodic update strategy can prevent overfitting and realize stable convergence in the cross-patient environment.

[0079] S3: performing image recognition on the next time digestive tract image collected by the image recognition model.

[0080] Further, the image recognition result includes the recognition result of different types of lesions in the digestive tract, the severity of each lesion recognition result, and the confidence distribution of each lesion recognition result. The confidence distribution is the probability of each type of lesion in each frame of image.

[0081] It should be noted that the traditional endoscopic image recognition relies on artificial experience for lesion identification, which is not only time-consuming and labor-intensive, but also subjective in results, making it difficult to quantitatively analyze the severity of lesions. The present application can capture the subtle texture and abnormal features of the mucosal surface of the digestive tract in multi-scale space by combining residual convolutional encoder and cross-scale self-attention feature fusion module, and accurately distinguish different lesion types (such as erosion, ulcer, polyp, etc.). At the same time, the Softmax probability calculation mechanism is introduced in the classification output head, so that the recognition result not only outputs a single category, but also generates the confidence distribution corresponding to each type of lesion, thereby reflecting the uncertainty and classification reliability of the model in multi-class determination.

[0082] The confidence distribution can be used as input for subsequent spatial correlation analysis and graph structure propagation, allowing the system to associate lesions and perform global optimization based on probability relationships; at the same time, the severity index can provide objective lesion grading reference for clinical practice, which is helpful for discovering early micro lesions or potential risk areas.

[0083] The image recognition model has been individualized and optimized through pre-position evaluation and attention adaptive training, and the model has strong multi-scale semantic feature extraction capability and cross-frame temporal stability. In the implementation process, the input frame quality is guaranteed to be consistent through standardization preprocessing; after the feature tensor output by the fusion module is input into the classification output head, the model calculates the probability distribution of each category using the Softmax function, and realizes automatic judgment combined with the confidence threshold, thereby realizing high-precision lesion recognition and dynamic result update under the condition that hardware can be processed in real time.

[0084] S4: According to the results of the image recognition, the scattering influence of the spatial correlation is realized, and the environmental inference of the lesion is realized. The environmental inference is fed back to the image recognition model, and the image recognition model is supplemented in the adaptive process.

[0085] Further, the spatial correlation scattering effect includes taking each lesion type as a node, taking the corresponding area of the lesion through the position evaluation and the severity of the identification result of each lesion as a node feature, and constructing a graph structure. By constructing a graph structure model with the lesion type as the core, spatial correlation modeling and global information propagation between lesions are realized, thereby breaking through the limitation of traditional recognition methods which only make independent judgments based on single frames or local features, and improving the lesion inference ability and overall cognitive level of the system for undetected areas.

[0086] The physiological correlation between different lesion types is constructed as a weighted coupling relationship as the edge of the graph, and the coupling weight is obtained by clinical priori and data-driven learning.

[0087] The image recognition result collected according to the attention analysis period is used for node generation to obtain a feature node set of the patient.

[0088] The feature node set of the patient is used for information propagation and aggregation in the graph structure to obtain a lesion judgment result of each sub-region in the undetected area and a probability distribution of each lesion type; wherein the sub-region is the smallest region obtained by finite element division of the undetected area.

[0089] It is to be noted that there is usually a physiological correlation and pathological evolution relationship between different types of lesions in the digestive tract, for example, polyps and erosions may coexist in adjacent areas, and ulcers may interact with local blood vessel dilation or inflammatory response. The traditional image classification model cannot capture such cross-regional semantic relationship, but by abstracting the lesion type as a node and taking the corresponding area in the standardized digestive tract template and the severity of the identification result as a node feature, a structured representation reflecting the pathological spatial distribution can be established. On this basis, the physiological correlation between lesion types is used to construct a weighted coupling edge, and the coupling weight is determined by clinical statistical priori and data-driven learning, so that the model can automatically learn the interaction strength between different lesions. By continuously collecting the recognition result according to the attention analysis period, a feature node set of the patient is generated, and information propagation and aggregation are performed in the graph structure to realize the diffusion reasoning of known lesion features to unknown areas. The sub-region design of finite element division ensures the refinement of spatial resolution, so that the lesion judgment result of the undetected area can be accurately positioned to a specific anatomical location.

[0090] The model has global lesion correlation understanding and spatial propagation capability, can make high-confidence inference on unobserved areas under the condition that part of the area has been detected, and realizes the cross-space reasoning of "from local recognition to global inference". The information propagation of the graph structure can be realized by a graph convolution network (GCN) or a graph attention network (GAT), the node features can be composed of high-dimensional feature vectors output by the convolutional encoder, and the edge weight can be adaptively optimized by a trainable parameter matrix; in the template space of finite element division, the lesion probability distribution of each sub-region can be stably updated by a standard numerical calculation method, thereby ensuring the calculation efficiency of the algorithm and the actual deployability of the medical scene.

[0091] The lesion judgment result of each sub-region and the probability distribution of each lesion type are updated with the image acquisition of the attention analysis period.

[0092] The lesion judgment result of each sub-region and the probability distribution of each lesion type are fed back to the adaptive process of the image recognition model: in the process of real-time image acquisition, the n sub-regions corresponding to the corresponding region A are obtained through the position evaluation of the corresponding region A; the lesion judgment results in the n sub-regions are counted, and the average probability of each lesion type is calculated using the probability distribution of each lesion type in the n sub-regions; denoted as .

[0093] Wherein, m represents the number of lesion types, including the identification result of health; represents the mth lesion type; represents the probability of the mth lesion; G represents the probability set supplemented by attention.

[0094] In the adaptive process, each lesion type is provided with a standard attention coefficient , and the in G is taken as the attention additional term of , and is calculated as the attention coefficient after adaptation, and the attention coefficient after adaptation of each lesion type is ; wherein, u represents the index of the lesion type, represents the standard attention coefficient of the uth lesion.

[0095] The is taken as the attention weight of the classification output head, and the dynamic attention adaptive adjustment of the model to different lesion types is realized.

[0096] To say, the traditional deep learning model is fixed after training, and cannot be optimized in real time according to the newly collected images and individual differences of patients, resulting in insufficient recognition effect on rare lesions or atypical manifestations. In the present application, in each attention analysis period, the average probability of each lesion type in the n sub-regions under the current corresponding region A is calculated according to the inferred sub-region lesion judgment result and probability distribution, and an attention supplement set is formed. This set reflects the spatial confidence response of the model to each lesion type in the current sampling period, so as to quantitatively describe the attention distribution of the model on different types of lesions. On this basis, a standard attention coefficient is set for each lesion type, and the probability of the corresponding lesion type in the set is taken as an additional item to calculate a new attention coefficient, which is used as the dynamic attention weight of the classification output head. This design enables the model to adjust the focus after each period according to the statistical results of the newly sampled, for example, to increase the weight of the relevant feature channel when the probability of ulcerous lesions rises, and to automatically reduce the attention in healthy or low-risk areas, thereby realizing the optimal allocation of attention resources.

[0097] Among them, the attention weight adjustment process can be embedded in the back propagation mechanism of the neural network, so that the fixed weight is replaced by the attention weight to participate in the gradient update, realizing the differentiable dynamic optimization; at the same time, the update and statistics of the sub-region lesion probability can be efficiently completed through the numerical calculation of the standard finite element division, ensuring the real-time performance on the premise of controllable calculation complexity. This mechanism forms a closed-loop process of "graph structure inference-attention update-model relearning", which makes the system maintain adaptability and stability in long-term operation, and has the intelligent characteristics of continuous optimization.

[0098] S5: using the result of image recognition, classifying the internal image of the digestive tract to obtain the feature classification result of all internal images of the digestive tract.

[0099] The classification result includes that each image can be classified into multiple classification results; a classification label is preset for each lesion. The confidence of each lesion recognition result is compared with the preset judgment threshold; if the confidence of any lesion recognition result is higher than the judgment threshold, the image is classified into the classification label of the corresponding lesion, and the confidence of the lesion recognition result in the current classification label is labeled to obtain the feature classification result of the current image.

[0100] ​In view of the situation that multiple lesion types or complex pathological characteristics may exist in the digestive tract image at the same time, a multi-label classification and confidence threshold determination mechanism is introduced to realize fine and multi-dimensional recognition of the image content. The traditional single-label classification model can only output one dominant class and cannot cope with the situation that multiple lesions coexist in the same image, the lesion boundary is blurred or the pathological morphology is overlapped, which is easy to cause missed detection or misjudgment. The present application presets independent classification labels for each lesion type and allows the image to belong to multiple categories at the same time, so that the recognition process is more in line with the actual distribution characteristics of digestive tract lesions. By comparing each lesion recognition confidence output by the model with the preset threshold one by one, the system can filter out the lesion types with high confidence and label the confidence value under the corresponding label, thereby generating a quantifiable feature classification result. The confidence labeling not only improves the interpretability of the model output, but also provides a reliable quantitative basis for subsequent spatial correlation analysis and lesion severity assessment. The multi-label classification framework can be directly extended based on the Softmax or Sigmoid output layer, and an independent probability distribution is output for each class. The confidence threshold can be automatically set through the validation set or statistical distribution to balance the sensitivity and specificity. The confidence labeling process can be directly generated from the model output result without additional computational overhead. This mechanism is compatible with existing deep neural networks in engineering and can be quickly executed in real-time inference, ensuring that the system has multi-label expression capability while maintaining high-precision recognition.

[0101] If the confidence of all lesion recognition results is not higher than the determination threshold, the context association analysis is performed to classify and label the confidence under each classification label to obtain the feature classification result of the current image.

[0102] The feature classification results of each image are integrated to obtain the feature classification results of all digestive tract internal images.

[0103] Further, the context association analysis includes setting the current image as pending, and continuing image acquisition and recognition.

[0104] The confidence of each lesion recognition result is calculated using the context image of the pending image, and the confidence of each lesion recognition result of the pending image is updated.

[0105] The confidence update result of each lesion recognition result of the pending image is compared with the determination threshold. If the pending image obtains a feature classification result after confidence update, the association analysis is completed. If the pending image does not obtain a feature classification result after confidence update, the pending image is specially marked. The special marking can be used as a key analysis point for lesion evolution.

[0106] In order to solve the problem of low confidence sample determination uncertainty caused by light change, posture deviation or local blur in the process of digestive tract image recognition, by introducing the context correlation analysis mechanism, the classification confidence of the image to be determined is dynamically corrected by using the temporal consistency and spatial continuity between continuous images, so as to improve the continuity, stability and clinical reliability of the whole recognition. The digestive tract endoscopic video is a typical time sequence continuous image data, and single frame image may cause recognition confidence to decrease due to liquid shielding, motion blur or light reflection, but its adjacent frame often contains identifiable information of the same area. When it is detected that the confidence of all lesions is lower than the threshold, the current image is temporarily set as "to be determined", and the recognition results of the front and rear frame images are called to update the mean value or weighted fusion of the confidence of each lesion type, so as to improve the stability of confidence judgment by using the redundant information of context. It can effectively reduce misjudgment and omission, ensure the continuity of image classification in time dimension, and make the system have anti-interference and self-correction ability in actual endoscopic video stream. Through the re-comparison after the mean value update of the confidence, if the to-be-determined image re-obtains stable lesion type judgment, the classification can be completed and included in the result set; if it still cannot be determined, it will be specially marked as a key reference point for subsequent lesion evolution analysis and manual review. This marking mechanism enables the system to automatically capture potential new lesions or abnormal change areas, providing early risk prompt for clinical use.

[0107] It is to be noted that the context correlation analysis can be realized by sliding window or circular buffer, and the recognition results of adjacent frames are called in time sequence; the confidence mean value calculation can adopt simple weighted average or attention-based time weighting model, which is light in calculation and can be executed in real time; the generation and recording of special marking can be embedded in the recognition result output module, without additional hardware support. The whole process can run in parallel with the existing neural network inference process, without affecting the real-time performance, so it can be efficiently realized in endoscopic real-time detection and post-processing analysis.

[0108] In other optional embodiments, the continuity of the "to-be-determined image" can be analyzed, if a plurality of continuous images are "to-be-determined images", the recognition is performed by introducing other algorithms; if not continuous, the similarity between the confidence of the current "to-be-determined image" and the confidence of the context is analyzed, and the current "to-be-determined image" is classified into the classification with higher similarity in the context. It is to be said that the endoscopic video belongs to strong time sequence data stream, if a plurality of continuous images are recognized as "to-be-determined images", it is indicated that the visual features in the whole time period are unstable, which can be affected by motion blur, liquid obstruction or sudden illumination change. At this time, the recognition is re-performed by introducing other algorithms (such as optical flow estimation, frame interpolation recovery, feature backtracking network or cross-frame fusion algorithm), the comprehensive judgment is performed on a longer time window, so as to make up for the recognition blind area of the deep model in the local uncertain scene. When the "to-be-determined image" is discontinuous, the system analyzes the confidence similarity of the current frame and the context frame, infers by using the semantic consistency of adjacent frames, and classifies the current image into the classification result with the closest confidence distribution. In this way, the stability of the judgment is ensured, and the interference of irrelevant frames is avoided.

[0109] The embodiment also provides a feature classification system based on internal images of a digestive tract, comprising:

[0110] The acquisition unit acquires the internal images of the digestive tract in real time by the endoscope, and completes the preprocessing of noise removal and image enhancement.

[0111] The analysis module evaluates the position of the endoscope according to the acquired real-time images and the endoscope depth, and invades the training process of the image recognition model according to the evaluation result: the attention of each training sample is analyzed through the similarity calculation of similar positions, and the individual features are adapted in the image recognition model.

[0112] The recognition module performs image recognition on the internal images of the digestive tract acquired at the next moment by the image recognition model.

[0113] The supplement module performs the scattering influence of the spatial correlation according to the result of the image recognition, realizes the environment inference of the lesion, and feeds back the environment inference to the image recognition model, realizes the supplement of the image recognition model in the adaptive process.

[0114] The classification module classifies the internal images of the digestive tract by using the result of the image recognition, and obtains the feature classification result of all internal images of the digestive tract.

[0115] The embodiment also provides a computer device suitable for the feature classification method based on internal images of a digestive tract, comprising a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, and realizes the feature classification method based on internal images of a digestive tract as proposed in the above embodiment.

[0116] The computer device can be a terminal, which includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0117] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the feature classification method based on the internal image of the digestive tract as described in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0118] In summary, the present application realizes the full-process intelligent analysis of the internal image of the digestive tract by constructing a deep intelligent recognition system integrating position evaluation-individual adaptation-graph structure reasoning-dynamic feedback-multi-label classification. The position estimation network established by the convolutional neural network and the time sequence encoder realizes the accurate positioning of the endoscope field in the standardized three-dimensional digestive tract template, laying the foundation for cross-individual image registration and time sequence alignment. Through the similarity analysis based on dynamic time warping and the attention accumulation mechanism, the adaptive adjustment of the model parameters between different patients is realized, so that the system has individualized learning ability. Through the graph structure space reasoning model, different lesion types are regarded as nodes, and the physiological correlation is combined to realize the information transmission between lesions and the inference of undetected areas, which significantly improves the global consistency of recognition. Through the attention feedback mechanism, the inference results are fed back to the recognition model, and the weights of the classification output head are dynamically corrected to realize the continuous self-evolution of the model. Finally, through the multi-label classification strategy of confidence threshold determination and context association analysis, the stability, continuity and explainability of the recognition results in complex scenes are ensured. The method constructs a closed-loop learning framework from local image recognition to global lesion inference, has high precision, strong robustness and scalability, and can be widely applied to the fields of intelligent endoscopic diagnosis and clinical auxiliary analysis.

[0119] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A feature classification method based on images of the digestive tract interior, characterized in that: This includes real-time acquisition of images of the digestive tract via endoscopy, and preprocessing for noise removal and image enhancement; Based on the acquired real-time images and endoscope depth, the position of the endoscope is evaluated, and the training process of the image recognition model is initiated based on the evaluation results: by calculating the similarity of similar positions, the attention of each training sample is analyzed to achieve the adaptation of individual features in the image recognition model. The image recognition model is used to perform image recognition on the digestive tract images acquired at the next moment; Based on the image recognition results, the spatial correlation scattering effect is analyzed to achieve environmental inference of the lesion; and the environmental inference is fed back to the image recognition model to supplement the image recognition model in the adaptive process; Using the results of the image recognition, the images of the digestive tract are classified to obtain the feature classification results of all images of the digestive tract.

2. The feature classification method based on images of the digestive tract interior as described in claim 1, characterized in that: The location assessment includes constructing a feature sequence using consecutive frame images of the digestive tract and the depth corresponding to each frame; inputting the feature sequence into a location estimation network based on a convolutional neural network and a temporal encoder; extracting spatial texture and structural gradient features through convolutional layers; and modeling dynamic changes between adjacent frames through a temporal encoder. During the training phase of the location estimation network, the output features are mapped to a standardized three-dimensional template coordinate system of the digestive tract, and the matching error with the template location label is calculated. During the inference phase of the position estimation network, the corresponding region A of the current endoscopic field of view in the standard digestive tract template is determined based on the pose probability distribution output by the model. In the training and validation sets of the image recognition model, any image in each sample uniquely corresponds to a coordinate in the standard digestive tract template; the images of the samples in the training and validation sets corresponding to region A are captured and used as the similar position to sequence a, denoted as sequence b; wherein, sequence a is a fixed-length sequence of consecutive frame images that are cropped forward from the current time in the consecutive frame images. The dynamic time warping algorithm is used to temporally align sequence a and sequence b, resulting in a set of aligned frame pairs. A weighted average of the similarities of all aligned frame pairs is then taken to obtain the average similarity between sequence a and sequence b at similar positions. The similarity between the i-th sample and sequence a is denoted as: This serves as the current attention between the patient and the sample.

3. The feature classification method based on images of the digestive tract interior as described in claim 2, characterized in that: The adaptive aspect of the image recognition model includes accumulating the attention of each sample in each cycle according to the attention analysis cycle to obtain the attention base number of each sample. After normalizing the attention base of all samples, the attention between the current patient and each sample is obtained; The training process is adjusted by the attention between the current patient and each sample, thereby obtaining the parameters of the image recognition model for the current patient; The image recognition model is a deep neural network structure that combines convolutional feature extraction and attention fusion, including: a residual convolutional encoder, a cross-scale self-attention feature fusion module, and a classification output head.

4. The feature classification method based on images of the digestive tract interior as described in claim 3, characterized in that: The image recognition results include the recognition results for different types of lesions in the digestive tract, the severity of each lesion recognition result, and the confidence distribution of each lesion recognition result; The confidence distribution is the probability of each type of lesion in each frame of the image.

5. The feature classification method based on images of the digestive tract interior as described in claim 4, characterized in that: The scattering effect of spatial correlation includes constructing a graph structure by taking each lesion type as a node, and taking the corresponding region obtained by the location assessment of the lesion and the severity of each lesion identification result as node features. We construct weighted coupling relationships based on the physiological correlations between different lesion types as the edges of a graph, with the coupling weights obtained from clinical priors and data-driven learning. The image recognition results collected according to the attention analysis cycle are used to generate nodes, resulting in a set of feature nodes for the patient. The patient's feature node set is used to propagate and aggregate information in the graph structure to obtain the lesion judgment result of each sub-region in the undetected region and the probability distribution of each lesion type; wherein, the sub-region is: the smallest region of the undetected region divided by finite element method; The lesion judgment results and probability distribution of each lesion type in each sub-region are updated with the image acquisition of the attention analysis cycle; The lesion judgment results and probability distributions of each lesion type in each sub-region are fed back into the adaptive process of the image recognition model: During real-time image acquisition, based on the corresponding region A obtained through the location evaluation, n sub-regions corresponding to region A are acquired; the lesion judgment results in the n sub-regions are statistically analyzed, and the average probability of each lesion type is calculated using the probability distributions of each lesion type in the n sub-regions; denoted as... ; Where m represents the number of lesion types, including the results of healthy identification; Indicates the type of the m-th lesion; Let G represent the probability of the m-th lesion; G represents the set of probabilities for attention supplementation. In the adaptive process, each lesion type is assigned a standard attention coefficient. , in G As Attention addendum, calculation As the adapted attention coefficient, the adapted attention coefficient for each lesion type Where u represents the index of the lesion type, The standard attention coefficient represents the u-th lesion; Will As attention weights in the classification output head, the model dynamically and adaptively adjusts its attention to different lesion types.

6. The feature classification method based on images of the digestive tract interior as described in claim 5, characterized in that: The classification results include: each image can be classified into multiple classification results; and a classification label is preset for each type of lesion. The confidence score of each lesion recognition result is compared with a preset judgment threshold. If the confidence score of any lesion recognition result is higher than the judgment threshold, the image is classified into the classification label of the corresponding lesion, and the confidence score of the lesion recognition result in the current classification label is marked to obtain the feature classification result of the current image. If the confidence level of all lesion identification results is not higher than the judgment threshold, then the feature classification result of the current image is obtained by performing context association analysis and labeling the confidence level under each classification label. The feature classification results of each image are integrated to obtain the feature classification results of all images of the digestive tract interior.

7. The feature classification method based on images of the digestive tract interior as described in claim 6, characterized in that: The contextual analysis includes setting the current image as pending and continuing image acquisition and recognition; Using the context image of the image to be determined, the average confidence score of each lesion identification result is calculated, and the confidence score of each lesion identification result of the image to be determined is updated. The judgment threshold is compared with the confidence update results of each lesion identification result using the pending image. If the pending image obtains a feature classification result after the confidence update, the association analysis is completed; if the pending image does not obtain a feature classification result after the confidence update, the pending image is specially marked.

8. A feature classification system based on images of the digestive tract interior, based on the feature classification method based on images of the digestive tract interior as described in any one of claims 1 to 7, characterized in that: It includes an acquisition unit that acquires images of the digestive tract in real time through an endoscope and performs preprocessing such as noise removal and image enhancement. The analysis module assesses the position of the endoscope based on the acquired real-time images and endoscope depth, and then integrates the image recognition model training process based on the assessment results: by calculating the similarity of similar positions, it analyzes the attention of each training sample to achieve the adaptation of individual features in the image recognition model. The recognition module performs image recognition on the digestive tract image acquired at the next moment using the image recognition model; The supplementary module, based on the image recognition results, analyzes the spatial correlation scattering effects to infer the environment of the lesion; and feeds the environmental inference back to the image recognition model, thus supplementing the image recognition model during the adaptive process. The classification module uses the image recognition results to classify the images inside the digestive tract, obtaining feature classification results for all images inside the digestive tract.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the feature classification method based on images of the digestive tract as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the feature classification method based on images of the digestive tract interior as described in any one of claims 1 to 7.

Images