A digestive endoscopy auxiliary system and method based on AI technology
Through the digestive endoscopy auxiliary system based on AI technology, multi-scale feature perception and deep learning models are used to solve the problem of insufficient ability of traditional systems to detect lesions in different sizes and shapes, and intelligent analysis of digestive tract endoscopy images and Diagnosis improves detection capabilities and system intelligence.
Patent Information
- Application Number
- CN202411269319.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-09-11
AI Technical Summary
Traditional digestive endoscopy systems are inadequate in detecting digestive lesions of different sizes and shapes, and cannot effectively identify tiny or undetectable lesions. The feature extraction method that relies on artificial design cannot fully capture the complex characteristics of the lesions, resulting in diagnostic accuracy. and reduce reliability.
Adopting an AI-based digestive endoscopy auxiliary system, by acquiring the endoscopy images of the digestive tract, using multi-scale feature perception and deep learning models to automatically extract image features, generate multi-scale perceived feature maps, and improve the detection of lesions of different sizes and shapes. and reduce dependence on artificial design features.
It realizes the analysis of images on multiple scales, improves the detection ability of small and large-scale abnormalities, improves the intelligence of the digestive endoscopy auxiliary system, and enhances the accuracy and reliability of diagnosis.
Smart Images

Figure CN118781118B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of digestive endoscopy, and more specifically, to a digestive endoscopy auxiliary system and method based on AI technology. Background Art
[0002] Digestive tract diseases are one of the common health problems worldwide, including but not limited to gastritis, gastric ulcer, duodenal ulcer, polyps, etc. Early detection and treatment of these diseases can significantly improve the survival rate and quality of life of patients. Digestive tract endoscopy is one of the important tools for diagnosing these diseases. It can provide images of the inside of the digestive tract, help doctors observe the state of the digestive tract mucosa and collect biopsy samples.
[0003] The internal environment of the digestive tract is complex and diverse, with a wide variety of lesions, including polyps, tumors, inflammation, etc. These lesions are often similar in appearance and easy to confuse. However, traditional digestive endoscopy image analysis systems usually perform feature extraction at a single scale, resulting in their inability to detect lesions of different sizes and shapes. This limitation makes it difficult for the system to adapt to the diversity of digestive tract lesions and cannot effectively identify tiny or imperceptible lesions. In addition, traditional systems rely on manually designed feature extraction methods, which often fail to fully capture the complex characteristics and nuances of lesions. This experience-based feature selection method may ignore important information in the data, resulting in reduced accuracy and reliability of diagnosis.
[0004] Therefore, an optimized digestive endoscopy auxiliary system is desired. Summary of the invention
[0005] The present application provides a digestive endoscopy auxiliary system and method based on AI technology, which can analyze images at multiple scales at the same time, improving the detection ability of lesions of different sizes and shapes, including microlesions and large-scale abnormalities. At the same time, the deep learning model is used to automatically extract image features, reducing the dependence on manually designed features and improving the intelligence of the digestive endoscopy auxiliary system.
[0006] In a first aspect, the present invention provides a digestive endoscopy auxiliary system based on AI technology, including: a patient digestive tract endoscopy image acquisition module, used to acquire a digestive tract endoscopy image of the patient being examined; a digestive tract state feature processing module, used to pass the digestive tract endoscopy image through a digestive tract state feature extractor to obtain a digestive tract state feature map; a digestive tract state multi-scale feature perception module, used to pass the digestive tract state feature map through a digestive tract state enhancement expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception feature map; a digestive tract state multi-scale perception feature enhancement module, used to pass the digestive tract state multi-scale perception feature map through a cross-channel feature space self-attention enhancement adaptive feature module to obtain an attention-enhanced digestive tract state multi-scale perception feature map; a recognition result generation module, used to obtain a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issue a warning prompt signal based on the recognition result;
[0007] The digestive tract state multi-scale feature perception module includes: a digestive tract state multi-layer convolution unit, which is used to perform convolution processing on the digestive tract state feature map based on a multi-layer convolution layer to obtain a deep digestive tract state feature map, a middle digestive tract state feature map and a shallow digestive tract state feature map; a deep digestive tract state convolution encoding unit, which is used to perform convolution encoding on the deep digestive tract state feature map to obtain a deep digestive tract state convolution feature map; a deep digestive tract state upsampling processing unit, which is used to perform upsampling processing on the deep digestive tract state convolution feature map to obtain an upsampled deep digestive tract state feature map; a middle digestive tract state convolution encoding unit, which is used to perform convolution encoding on the deep digestive tract state feature map to obtain a deep digestive tract state convolution feature map; a deep digestive tract state upsampling processing unit, which is used to perform upsampling processing on the deep digestive tract state convolution feature map to obtain an upsampled deep digestive tract state feature map; a deep ... A digestive tract state perception extraction unit is used to perform multi-scale feature perception processing on the middle-layer digestive tract state feature map to obtain a first multi-scale perception digestive tract state feature map; a shallow-layer digestive tract state extraction unit is used to perform multi-scale feature perception processing on the shallow-layer digestive tract state feature map to obtain a second multi-scale perception digestive tract state feature map; and a digestive tract state multi-scale feature fusion unit is used to fuse the deep-layer digestive tract state convolution feature map, the first multi-scale perception digestive tract state feature map and the second multi-scale perception digestive tract state feature map to obtain the digestive tract state multi-scale perception feature map.
[0008] In a possible implementation, the digestive tract state feature processing module is used to: pass the digestive tract endoscopic image through a digestive tract state feature extractor based on a void convolutional neural network to obtain the digestive tract state feature map.
[0009] In a possible implementation, the middle-layer perception digestive tract state extraction unit includes: a middle-layer multi-scale digestive tract state perception subunit, used to input the middle-layer digestive tract state feature map into a multi-scale perception network to obtain a middle-layer multi-scale perception digestive tract state feature map; a digestive tract state feature fusion subunit, used to perform feature fusion on the middle-layer multi-scale perception digestive tract state feature map and the upsampled deep-layer digestive tract state feature map to obtain a first digestive tract state fusion feature map, and then perform convolution encoding on the first digestive tract state fusion feature map to obtain the first multi-scale perception digestive tract state feature map.
[0010] In a possible implementation, the middle-level multi-scale digestive tract state perception subunit is used to: copy the middle-level digestive tract state feature map to obtain a middle-level digestive tract state copy feature map; first pass the middle-level digestive tract state feature map through a convolution layer with a convolution kernel of 1×1, and then successively perform activation processing and standardized batch processing based on the ReLU function on the obtained feature map to obtain a first-scale middle-level standardized digestive tract state feature map; first pass the first-scale middle-level standardized digestive tract state feature map through a convolution layer with a convolution kernel of 3×3, and then successively perform activation processing and standardized batch processing based on the ReLU function on the obtained feature map to obtain to a second-scale middle-level standardized digestive tract state feature map; the second-scale middle-level standardized digestive tract state feature map is first passed through a convolution layer with a convolution kernel of 5×5, and then the obtained feature map is sequentially subjected to the activation processing based on the ReLU function and the standardized batch processing to obtain a third-scale middle-level standardized digestive tract state feature map; the middle-level digestive tract state replication feature map, the first-scale middle-level standardized digestive tract state feature map, the second-scale middle-level standardized digestive tract state feature map and the third-scale middle-level standardized digestive tract state feature map are cascaded through a feature fusion layer to obtain the middle-level multi-scale perception digestive tract state feature map.
[0011] In a possible implementation, the shallow perception digestive tract state extraction unit is used to: input the shallow digestive tract state feature map into the multi-scale perception network to obtain a shallow multi-scale perception digestive tract state feature map; upsample the first multi-scale perception digestive tract state feature map to obtain an upsampled middle-layer multi-scale perception digestive tract state feature map; perform feature fusion on the shallow multi-scale perception digestive tract state feature map and the upsampled middle-layer multi-scale perception digestive tract state feature map to obtain a second digestive tract state fusion feature map, and then perform convolution encoding on the second digestive tract state fusion feature map to obtain the second multi-scale perception digestive tract state feature map.
[0012] In a possible implementation, the digestive tract state multi-scale perception feature enhancement module includes: a digestive tract state multi-scale perception normalization unit, which is used to perform layer normalization on the digestive tract state multi-scale perception feature map to obtain a normalized digestive tract state multi-scale perception feature map; a digestive tract state multi-scale perception point convolution unit, which is used to perform point convolution processing on the normalized digestive tract state multi-scale perception feature map to obtain a digestive tract state multi-scale perception channel context association representation feature map; a digestive tract state multi-scale perception space association unit, which is used to perform convolution processing on the digestive tract state multi-scale perception channel context association representation feature map. The digestive tract state multi-scale perception channel context association representation feature map, the digestive tract state multi-scale perception space context association representation feature map and the digestive tract state multi-scale perception backup space context association representation feature map are respectively used to perform feature shape reshaping on the digestive tract state multi-scale perception channel context association representation feature map, the digestive tract state multi-scale perception space context association representation feature map and the digestive tract state multi-scale perception backup space context association representation feature map to obtain a digestive tract state multi-scale perception channel context association representation feature map, the digestive tract state multi-scale perception space context association representation feature map and the digestive tract state multi-scale perception backup space context association representation feature map to obtain a digestive tract state multi-scale perception channel context association representation feature map, the digestive tract state multi-scale perception space context association representation feature map and the digestive tract state multi-scale perception backup space context association representation feature map. A digestive tract state multi-scale perception channel context association representation feature matrix, a digestive tract state multi-scale perception space context association representation feature matrix and a digestive tract state multi-scale perception backup space context association representation feature matrix; a digestive tract state multi-scale perception cross-channel feature calculation unit, used to calculate the digestive tract state multi-scale perception cross-channel cross-covariance matrix between the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception space context association representation feature matrix; a digestive tract state multi-scale perception activation unit, used to use a Softmax function to activate the digestive tract state multi-scale perception cross-channel cross-covariance matrix to obtain a digestive tract state multi-scale perception feature global interactive attention matrix; a digestive tract state multi-scale perception attention reinforcement unit, used to calculate the product between the digestive tract state multi-scale perception backup space context association representation feature matrix and the digestive tract state multi-scale perception feature global interactive attention matrix to obtain a digestive tract state multi-scale perception attention reinforcement feature representation matrix; a feature shape reshaping unit, used to perform feature shape reshaping on the digestive tract state multi-scale perception attention reinforcement feature representation matrix to obtain the attention reinforcement digestive tract state multi-scale perception feature map.
[0013] In a possible implementation, the digestive tract state multi-scale perception cross-channel feature calculation unit is used to: calculate the matrix product between the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception space context association representation feature matrix to obtain the digestive tract state multi-scale perception channel-space context association representation feature matrix; calculate the positional division of the digestive tract state multi-scale perception channel-space context association representation feature matrix and a predetermined hyperparameter to obtain the digestive tract state multi-scale perception cross-channel cross-covariance matrix.
[0014] In a possible implementation, the recognition result generation module is used to: pass the attention-enhanced gastrointestinal state multi-scale perceptual feature map through a classifier-based gastrointestinal lesion identifier to obtain the recognition result, and the recognition result is used to represent the gastrointestinal abnormality type label of the examined patient; based on the recognition result, issue a warning prompt signal.
[0015] In a second aspect, the present invention provides a digestive endoscopy auxiliary method based on AI technology, comprising: obtaining a digestive endoscopic image of a patient being examined; passing the digestive endoscopic image through a digestive tract state feature extractor to obtain a digestive tract state feature map; passing the digestive tract state feature map through a digestive tract state enhancement expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception feature map; passing the digestive tract state multi-scale perception feature map through a cross-channel feature space self-attention enhancement adaptive feature module to obtain an attention-enhanced digestive tract state multi-scale perception feature map; obtaining a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issuing a warning prompt signal based on the recognition result;
[0016] The digestive tract state characteristic graph is passed through a digestive tract state enhanced expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception characteristic graph, including:
[0017] The digestive tract state characteristic map is subjected to convolution processing based on a multi-layer convolution layer to obtain a deep digestive tract state characteristic map, a middle digestive tract state characteristic map and a shallow digestive tract state characteristic map;
[0018] Performing convolution encoding on the deep digestive tract state characteristic map to obtain a deep digestive tract state convolution characteristic map;
[0019] Performing upsampling processing on the deep digestive tract state convolution feature map to obtain an upsampled deep digestive tract state feature map;
[0020] Performing multi-scale feature perception processing on the middle-layer digestive tract state characteristic map to obtain a first multi-scale perception digestive tract state characteristic map;
[0021] Performing multi-scale feature perception processing on the shallow digestive tract state characteristic map to obtain a second multi-scale perception digestive tract state characteristic map;
[0022] The deep digestive tract state convolution feature map, the first multi-scale perception digestive tract state feature map and the second multi-scale perception digestive tract state feature map are fused to obtain the digestive tract state multi-scale perception feature map.
[0023] The present application provides a digestive endoscopy auxiliary system and method based on AI technology, which obtains digestive tract endoscopic images of the patient being examined, and uses AI-based image recognition and analysis technology to extract digestive tract state features and multi-scale perception of the digestive tract endoscopic images, thereby intelligently evaluating the type of digestive tract abnormality of the patient being examined based on the features of the digestive tract endoscopic images under multi-scale enhancement, and issuing an early warning prompt signal. In this way, the system can analyze images at multiple scales at the same time, improving the ability to detect lesions of different sizes and shapes, including tiny lesions and large-scale abnormalities. At the same time, the deep learning model is used to automatically extract image features, reducing the reliance on manually designed features and improving the intelligence of the digestive endoscopy auxiliary system. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a schematic block diagram of a digestive endoscopy auxiliary system based on AI technology according to an embodiment of the present application.
[0025] Figure 2 This is a data flow diagram of the digestive endoscopy auxiliary system based on AI technology in an embodiment of the present application.
[0026] Figure 3 This is a schematic block diagram of a multi-scale feature perception module of the digestive tract state of a digestive endoscopy auxiliary system based on AI technology in an embodiment of the present application.
[0027] Figure 4 This is a schematic block diagram of a middle-level perception digestive tract state extraction unit of a digestive endoscopy auxiliary system based on AI technology in an embodiment of the present application.
[0028] Figure 5 This is a schematic block diagram of a multi-scale perception feature enhancement module of the digestive tract state of the AI-based digestive endoscopy auxiliary system in an embodiment of the present application.
[0029] Figure 6 This is a schematic flowchart of the AI-based digestive endoscopy assistance method of an embodiment of the present application. DETAILED DESCRIPTION
[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.
[0031] The digestive tract is an extremely complex environment that contains a variety of pathological manifestations such as polyps, tumors, and inflammation. Because these lesions may be highly similar in appearance, they can be easily confused with each other clinically. Traditional digestive endoscopy image analysis methods are limited to single-scale feature extraction technology, which means that their detection capabilities are limited when faced with lesions of different sizes and shapes. This single-scale analysis method makes the system appear stretched when dealing with lesions of various sizes and shapes in the digestive tract, especially when identifying tiny or imperceptible lesions. In addition, traditional systems rely on manually designed feature extraction methods. This experience-based and intuition-based approach may not fully capture all the details and complexities of the lesions, which may lead to the loss of important diagnostic information and reduce the accuracy and reliability of the diagnosis.
[0032] In response to the above technical problems, the present application proposes a digestive endoscopy auxiliary system based on AI technology, which obtains digestive tract endoscopic images of the patient being examined, and uses AI-based image recognition and analysis technology to extract digestive tract state features and multi-scale perception of the digestive tract endoscopic images, thereby intelligently evaluating the type of digestive tract abnormality of the patient being examined based on the features of the digestive tract endoscopic images under multi-scale enhancement, and issuing an early warning prompt signal. In this way, the system can analyze images at multiple scales at the same time, improving the ability to detect lesions of different sizes and shapes, including tiny lesions and large-scale abnormalities. At the same time, the deep learning model is used to automatically extract image features, reducing the reliance on manually designed features and improving the intelligence of the digestive endoscopy auxiliary system.
[0033] Figure 1 This is a schematic block diagram of a digestive endoscopy auxiliary system based on AI technology according to an embodiment of the present application. Figure 2 This is a data flow diagram of the digestive endoscopy auxiliary system based on AI technology in the embodiment of the present application. Figure 1 and Figure 2As shown, a digestive endoscopy auxiliary system 100 based on AI technology includes: a patient digestive tract endoscopy image acquisition module 110, used to acquire a digestive tract endoscopy image of the patient being examined; a digestive tract state feature processing module 120, used to pass the digestive tract endoscopy image through a digestive tract state feature extractor to obtain a digestive tract state feature map; a digestive tract state multi-scale feature perception module 130, used to pass the digestive tract state feature map through a digestive tract state enhancement expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception feature map; a digestive tract state multi-scale perception feature enhancement module 140, used to pass the digestive tract state multi-scale perception feature map through a cross-channel feature space self-attention enhancement adaptive feature module to obtain an attention-enhanced digestive tract state multi-scale perception feature map; a recognition result generation module 150, used to obtain a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issue a warning prompt signal based on the recognition result.
[0034] In the above-mentioned digestive endoscopy auxiliary system based on AI technology, the patient digestive tract endoscopy image acquisition module 110 is used to acquire digestive tract endoscopy images of the patient being examined. Optionally, in one embodiment of the present application, a real-time image of the inside of the digestive tract is captured by an endoscopic camera, and the image data is sent to the image acquisition module via a connecting line or wireless transmission. The image acquisition module collects and stores the image data for subsequent processing.
[0035] In the above-mentioned digestive endoscopy auxiliary system based on AI technology, the digestive tract state feature processing module 120 is used to pass the digestive tract endoscopy image through a digestive tract state feature extractor to obtain a digestive tract state feature map. It should be understood that, considering that the digestive tract endoscopy image contains state information of different scales such as texture and shape of the digestive tract, the void convolution can expand the receptive field without increasing the number of parameters, and capture a wider range of multi-scale contextual information of the image. Based on this, in the technical solution of the present application, the digestive tract endoscopy image is passed through a digestive tract state feature extractor based on a void convolutional neural network to obtain a digestive tract state feature map.
[0036] Optionally, in one embodiment of the present application, the digestive tract state feature processing module 120 is used to: pass the digestive tract endoscopic image through a digestive tract state feature extractor based on a void convolutional neural network to obtain the digestive tract state feature map.
[0037] In the above-mentioned digestive endoscopy auxiliary system based on AI technology, the digestive tract state multi-scale feature perception module 130 is used to pass the digestive tract state feature map through the digestive tract state enhanced expressor based on multi-scale feature perception to obtain the digestive tract state multi-scale perception feature map. It should be understood that considering that the digestive tract state feature map contains feature information of different scales for digestive tract abnormality judgment, such as small-sized features (such as small polyps or microvascular changes), large-scale features such as depressions or damaged areas on the mucosa, it will also be disturbed by complex backgrounds, such as wrinkles and blood vessels of the digestive tract wall. In order to better extract key features of different scales, so as to better extract and express digestive tract state information, so as to improve the perception ability of small-scale targets and complex backgrounds, in the technical solution of the present application, the digestive tract state feature map is passed through the digestive tract state enhanced expressor based on multi-scale feature perception to obtain the digestive tract state multi-scale perception feature map. That is to say, the digestive tract state enhanced expressor based on multi-scale feature perception enhances the expression ability and perception ability of small-scale targets in the digestive tract state feature map by introducing a multi-scale perception network to perform multi-scale target perception on features of different scales in the digestive tract state feature map. In detail, the digestive tract state feature map is first subjected to convolution processing based on different scales to obtain three feature maps of different scales. Then, the features of the shallow layer and the middle layer are respectively input into the multi-scale perception network for multi-scale perception feature capture. In particular, the multi-scale perception network first copies the initial digestive tract state feature map as input to two processing branches respectively. Then, one branch directly uses the original features, while the other branch captures the multi-scale feature dimensions through 1x1, 3x3 and 5x5 convolution kernels, and performs activation processing based on the ReLU function and standardized batch processing after each convolution layer to extract feature maps of different levels layer by layer. Subsequently, the output feature maps of each layer are superimposed and fused with the output of the previous stage in a cascade manner to realize multi-scale extraction and fusion of features, so as to enhance the model's ability to capture details and contextual information of the digestive tract state. Next, the hierarchical structure composed of these three features gradually digs deeper and learns the features extracted by the previous layer to capture richer semantic information and obtain a multi-scale perceptual feature map of the digestive tract state. In other words, compared with the independent multi-path parallel structure in the original network, this improved method can achieve more accurate detection performance, especially when detecting variable sizes and targets in the complex digestive tract.
[0038] Figure 3 Schematic block diagram of the digestive tract state multi-scale feature perception module of the digestive endoscopy auxiliary system based on AI technology in the embodiment of the present application. Figure 3As shown, optionally, in one embodiment of the present application, the digestive tract state multi-scale feature perception module 130 includes: a digestive tract state multi-layer convolution unit 131, which is used to perform convolution processing on the digestive tract state feature map based on a multi-layer convolution layer to obtain a deep digestive tract state feature map, a middle digestive tract state feature map and a shallow digestive tract state feature map; a deep digestive tract state convolution encoding unit 132, which is used to perform convolution encoding on the deep digestive tract state feature map to obtain a deep digestive tract state convolution feature map; a deep digestive tract state upsampling processing unit 133, which is used to perform upsampling processing on the deep digestive tract state convolution feature map to obtain an upsampled deep digestive tract state feature map. a digestive tract state characteristic map; a middle-layer perception digestive tract state extraction unit 134, used to perform multi-scale feature perception processing on the middle-layer digestive tract state characteristic map to obtain a first multi-scale perception digestive tract state characteristic map; a shallow-layer perception digestive tract state extraction unit 135, used to perform multi-scale feature perception processing on the shallow-layer digestive tract state characteristic map to obtain a second multi-scale perception digestive tract state characteristic map; a digestive tract state multi-scale feature fusion unit 136, used to fuse the deep-layer digestive tract state convolution feature map, the first multi-scale perception digestive tract state characteristic map and the second multi-scale perception digestive tract state characteristic map to obtain the digestive tract state multi-scale perception feature map.
[0039] Figure 4 Schematic block diagram of the middle-level digestive tract state extraction unit of the digestive endoscopy auxiliary system based on AI technology in the embodiment of the present application. Figure 4 As shown, optionally, in one embodiment of the present application, the middle-layer perception digestive tract state extraction unit 134 includes: a middle-layer multi-scale digestive tract state perception subunit 1341, which is used to input the middle-layer digestive tract state feature map into a multi-scale perception network to obtain a middle-layer multi-scale perception digestive tract state feature map; a digestive tract state feature fusion subunit 1342, which is used to perform feature fusion on the middle-layer multi-scale perception digestive tract state feature map and the upsampled deep-layer digestive tract state feature map to obtain a first digestive tract state fusion feature map, and then perform convolution encoding on the first digestive tract state fusion feature map to obtain the first multi-scale perception digestive tract state feature map.
[0040] Optionally, in one embodiment of the present application, the middle-level multi-scale digestive tract state perception subunit 1341 is used to: copy the middle-level digestive tract state feature map to obtain a middle-level digestive tract state copy feature map; first pass the middle-level digestive tract state feature map through a convolution layer with a convolution kernel of 1×1, and then sequentially perform activation processing and standardized batch processing based on the ReLU function on the obtained feature map to obtain a first-scale middle-level standardized digestive tract state feature map; first pass the first-scale middle-level standardized digestive tract state feature map through a convolution layer with a convolution kernel of 3×3, and then sequentially perform activation processing and standardized batch processing based on the ReLU function on the obtained feature map Processing is performed to obtain a second-scale middle-level standardized digestive tract state feature map; the second-scale middle-level standardized digestive tract state feature map is first passed through a convolution layer with a convolution kernel of 5×5, and then the obtained feature map is sequentially subjected to the activation processing based on the ReLU function and the standardized batch processing to obtain a third-scale middle-level standardized digestive tract state feature map; the middle-level digestive tract state replication feature map, the first-scale middle-level standardized digestive tract state feature map, the second-scale middle-level standardized digestive tract state feature map and the third-scale middle-level standardized digestive tract state feature map are cascaded through a feature fusion layer to obtain the middle-level multi-scale perception digestive tract state feature map.
[0041] Optionally, in one embodiment of the present application, the shallow perception digestive tract state extraction unit 135 is used to: input the shallow digestive tract state feature map into the multi-scale perception network to obtain a shallow multi-scale perception digestive tract state feature map; upsample the first multi-scale perception digestive tract state feature map to obtain an upsampled middle-layer multi-scale perception digestive tract state feature map; perform feature fusion of the shallow multi-scale perception digestive tract state feature map and the upsampled middle-layer multi-scale perception digestive tract state feature map to obtain a second digestive tract state fusion feature map, and then perform convolution encoding on the second digestive tract state fusion feature map to obtain the second multi-scale perception digestive tract state feature map.
[0042] In the above-mentioned digestive endoscopy auxiliary system based on AI technology, the digestive tract state multi-scale perception feature enhancement module 140 is used to obtain an attention-enhanced digestive tract state multi-scale perception feature map by self-attention enhancement adaptive feature module across channel feature space. It should be understood that considering that the digestive tract state multi-scale perception feature map contains both feature information on the spatial structure and there is an interactive relationship between features in different channels. Based on this, in order to be able to maintain the integrity of the spatial structure of the feature, the contextual connection information between channels is obtained through the interaction between feature channels to enhance the integrity and detail of the feature. In the technical solution of the present application, the digestive tract state multi-scale perception feature map is passed through a cross-channel feature space self-attention enhancement adaptive feature module to obtain an attention-enhanced digestive tract state multi-scale perception feature map. Specifically, the digestive tract state multi-scale perception feature map is first layer-normalized to normalize the distribution of features and reduce internal covariate shift. Then, the normalized features are sequentially subjected to point convolutional coding and convolutional coding to capture more complex and advanced information between channels of multi-scale perception features of digestive tract status, while enhancing the perception of spatial structural information of features. Next, the spatial context information in the original multi-scale perception feature map of digestive tract status is further retained by copying the spatial features. After that, the channel, space and backup features are transformed into a matrix form by reshaping, and the channel and spatial features are globally interacted across channels of multi-scale perception features of digestive tract status, so that the features of different channels can fully perceive and utilize each other's information, thereby generating a richer and more effective feature expression. Furthermore, the global interaction features are activated by using the Softmax function to automatically learn and adjust the importance of different features through this adaptive feature weighting mechanism, effectively improving the performance of key digestive tract status features. Finally, the backup spatial features and the activated features are weighted and reshaped to obtain the final attention-enhanced multi-scale perception feature map of digestive tract status.
[0043] Figure 5 Schematic block diagram of the digestive tract state multi-scale perception feature enhancement module of the digestive endoscopy auxiliary system based on AI technology in the embodiment of the present application. Figure 5As shown, optionally, in one embodiment of the present application, the digestive tract state multi-scale perception feature enhancement module 140 includes: a digestive tract state multi-scale perception normalization unit 141, which is used to perform layer normalization on the digestive tract state multi-scale perception feature map to obtain a normalized digestive tract state multi-scale perception feature map; a digestive tract state multi-scale perception point convolution unit 142, which is used to perform point convolution processing on the normalized digestive tract state multi-scale perception feature map to obtain a digestive tract state multi-scale perception channel context association representation feature map; a digestive tract state multi-scale perception space association unit 143, which is used to perform point convolution processing on the normalized digestive tract state multi-scale perception feature map to obtain a digestive tract state multi-scale perception channel context association representation feature map. The digestive tract state multi-scale perception channel context association representation feature map is convolutionally encoded to obtain a digestive tract state multi-scale perception space context association representation feature map; the digestive tract state multi-scale perception backup unit 144 is used to copy the digestive tract state multi-scale perception space context association representation feature map to obtain a digestive tract state multi-scale perception backup space context association representation feature map; the digestive tract state multi-scale perception feature shape reshaping unit 145 is used to perform feature shape reshaping on the digestive tract state multi-scale perception channel context association representation feature map, the digestive tract state multi-scale perception space context association representation feature map and the digestive tract state multi-scale perception backup space context association representation feature map respectively. The digestive tract state multi-scale perception channel context association representation feature matrix, the digestive tract state multi-scale perception space context association representation feature matrix and the digestive tract state multi-scale perception backup space context association representation feature matrix are reshaped to obtain the digestive tract state multi-scale perception channel context association representation feature matrix, the digestive tract state multi-scale perception space context association representation feature matrix and the digestive tract state multi-scale perception backup space context association representation feature matrix; the digestive tract state multi-scale perception cross-channel feature calculation unit 146 is used to calculate the digestive tract state multi-scale perception cross-channel cross covariance matrix between the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception space context association representation feature matrix; the digestive tract state multi-scale perception activation unit 147 is used to use the Softmax function to activate the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception space context association representation feature matrix. The scale-perceptual cross-channel cross-covariance matrix is activated to obtain the global interactive attention matrix of the multi-scale perception features of the digestive tract state; the multi-scale perception attention enhancement unit 148 of the digestive tract state is used to calculate the product between the multi-scale perception backup space context association representation feature matrix of the digestive tract state and the global interactive attention matrix of the multi-scale perception features of the digestive tract state to obtain the multi-scale perception attention enhancement feature representation matrix of the digestive tract state; the feature shape reshaping unit 149 is used to perform feature shape reshaping on the multi-scale perception attention enhancement feature representation matrix of the digestive tract state to obtain the attention-enhanced multi-scale perception feature map of the digestive tract state.
[0044] Optionally, in one embodiment of the present application, the digestive tract state multi-scale perception cross-channel feature calculation unit 146 is used to: calculate the matrix product between the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception spatial context association representation feature matrix to obtain the digestive tract state multi-scale perception channel-spatial context association representation feature matrix; calculate the positional division of the digestive tract state multi-scale perception channel-spatial context association representation feature matrix and a predetermined hyperparameter to obtain the digestive tract state multi-scale perception cross-channel cross-covariance matrix.
[0045] In summary, in the above embodiment, the multi-scale perceptual feature map of the digestive tract state is processed through a cross-channel feature space self-attention enhancement adaptive feature module to obtain an attention-enhanced multi-scale perceptual feature map of the digestive tract state, including: processing with the following self-attention enhancement formula to obtain the attention-enhanced multi-scale perceptual feature map of the digestive tract state; wherein the self-attention enhancement formula is:
[0046] ;
[0047] ;
[0048] ;
[0049] ;
[0050] ;
[0051] ;
[0052] ;
[0053] ;
[0054] ;
[0055] in, is the multi-scale perception feature map of the digestive tract state, Indicates the layer normalization operation on the feature map. is the normalized digestive tract state multi-scale perception feature map, is the point convolution operation, is a multi-scale perception channel context association representation feature map of the digestive tract state, The convolution kernel is The convolution operation, is a multi-scale perceptual space context association representation feature map of the digestive tract state, For copy operation, is a multi-scale perception backup space context association representation feature map of the digestive tract state, For the reshape operation, , and are respectively the digestive tract state multi-scale perception channel context association representation feature matrix, the digestive tract state multi-scale perception space context association representation feature matrix and the digestive tract state multi-scale perception backup space context association representation feature matrix, To predetermine the hyperparameters, for function, is the global interactive attention matrix of the multi-scale perception features of the digestive tract state, is matrix multiplication, A multi-scale perceptual feature map of the digestive tract state for attention enhancement.
[0056] In the above-mentioned digestive endoscopy auxiliary system based on AI technology, the recognition result generation module 150 is used to obtain a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issue a warning prompt signal based on the recognition result. It should be understood that the attention-enhanced digestive tract state multi-scale perception feature map is passed through a digestive tract lesion identifier based on a classifier to obtain a recognition result, and the recognition result is used to represent the digestive tract abnormality type label of the patient being examined, and based on the recognition result, an early warning prompt signal is issued. That is, the attention-enhanced digestive tract state multi-scale perception feature obtained by using the digestive tract state multi-scale perception feature map to strengthen attention is classified and processed, so as to intelligently evaluate the digestive tract abnormality type of the patient being examined, and issue a warning prompt signal. In this way, the system can analyze images at multiple scales at the same time, improving the detection ability of lesions of different sizes and shapes, including micro-lesions and large-scale abnormalities. At the same time, the deep learning model is used to automatically extract image features, reducing the dependence on artificially designed features.
[0057] Optionally, in one embodiment of the present application, the recognition result generation module 150 is used to: pass the attention-enhanced gastrointestinal state multi-scale perceptual feature map through a classifier-based gastrointestinal lesion identifier to obtain the recognition result, and the recognition result is used to represent the gastrointestinal abnormality type label of the examined patient; based on the recognition result, issue a warning prompt signal.
[0058] In particular, considering that the multi-scale perceptual feature map of the digestive tract state represents the multi-scale image semantic features of the digestive tract endoscopy image, when the cross-channel feature space self-attention enhancement adaptation is performed on it, there will be an image semantic adaptation offset caused by the imbalance of self-attention enhancement of the spatial distribution of the image semantic features of each scale across channels, so it is expected to further improve the image semantic adaptive regression comprehensibility of the attention-enhanced multi-scale perceptual feature map of the digestive tract state, thereby improving the accuracy of the recognition results obtained by the classifier-based digestive tract lesion identifier.
[0059] Therefore, in a preferred embodiment, when the attention-enhanced digestive tract state multi-scale perception feature map is passed through a classifier-based digestive tract lesion identifier to obtain a recognition result, the present application optimizes the attention-enhanced digestive tract state multi-scale perception feature map to obtain an optimized attention-enhanced digestive tract state multi-scale perception feature vector, including the following steps:
[0060] Expanding the attention-enhanced digestive tract state multi-scale perception feature map into an attention-enhanced digestive tract state multi-scale perception feature vector;
[0061] Subtract the zero norm of the attention-enhanced digestive tract state multi-scale perception feature vector from the length of the attention-enhanced digestive tract state multi-scale perception feature vector to obtain an attention-enhanced digestive tract state multi-scale perception isolated representation value;
[0062] Calculate the logarithm of the square of the attention-enhanced digestive tract state multi-scale perception isolated representation value and the sum of the attention-enhanced digestive tract state multi-scale perception isolated representation value with base 2 to obtain the attention-enhanced digestive tract state multi-scale perception information order value;
[0063] Calculate a power function with each eigenvalue of the attention-enhanced digestive tract state multi-scale perception eigenvector as the base, the difference of the attention-enhanced digestive tract state multi-scale perception isolated representation value minus one as the exponent, and perform a point multiplication with the attention-enhanced digestive tract state multi-scale perception information order value to obtain the attention-enhanced digestive tract state multi-scale perception leading vector;
[0064] After performing a dot multiplication of the difference between the attention-enhanced digestive tract state multi-scale perception feature vector and the attention-enhanced digestive tract state multi-scale perception isolated representation value minus one, the dot multiplication is performed with the reciprocal of the attention-enhanced digestive tract state multi-scale perception isolated representation value to obtain an attention-enhanced digestive tract state multi-scale perception field constraint vector;
[0065] Calculate an exponential function with a natural constant as the base and each eigenvalue of the attention-enhanced digestive tract state multi-scale perception field constraint vector as the exponent to obtain the attention-enhanced digestive tract state multi-scale perception field bias vector;
[0066] The attention-enhanced digestive tract state multi-scale perception leading vector is dot-added with the attention-enhanced digestive tract state multi-scale perception field bias vector to obtain an optimized attention-enhanced digestive tract state multi-scale perception feature vector. Finally, the optimized attention-enhanced digestive tract state multi-scale perception feature vector is passed through a classifier-based digestive tract lesion identifier to obtain a recognition result.
[0067] The optimization process of the attention-enhanced digestive tract state multi-scale perception feature vector is expressed as:
[0068] ;
[0069] ;
[0070] in, represents the multi-scale perceptual feature vector of the attention-enhanced digestive tract state, represents the zero norm of a vector, represents the length of the multi-scale perceptual feature vector of the attention-enhanced digestive tract state, represents the attention-enhanced digestive tract state multi-scale perception isolated representation value, It is the point multiplication by position. is added by position, A power function whose exponent is the difference between the value of the attention-enhanced multi-scale perception isolated representation of the digestive tract state and one, representing the calculation of the feature value of each position in the feature vector, Represents the optimized attention-enhanced multi-scale perceptual feature vector of the digestive tract state.
[0071] Therefore, the high-dimensional feature manifold of the multi-scale perception feature vector of the state of the digestive tract is represented by the feature value of the feature set as the vector field of the aggregation dimension, and the superposition value of the vector field of the multi-scale perception feature vector of the state of the digestive tract is used as the order information to fix the local position of the feature value of the feature set, and add the bias of the reversibility of the feature regression distribution field of the multi-scale perception feature vector of the state of the digestive tract as a reward to achieve the mapping target tracking of the regression distribution of the multi-scale perception feature vector of the state of the digestive tract for the feature value position, so that the feature set of the multi-scale perception feature vector of the state of the digestive tract perceives the mapping migration to the aggregate distribution, thereby improving the image semantic adaptive regression comprehensibility of the multi-scale perception feature vector of the state of the digestive tract, and improving the accuracy of the recognition result obtained by the multi-scale perception feature vector of the state of the digestive tract based on the classifier. In this way, the system can analyze images at multiple scales at the same time, and improve the detection ability of lesions of different sizes and shapes, including micro-lesions and large-scale abnormalities. At the same time, the deep learning model is used to automatically extract image features, reducing the dependence on manually designed features.
[0072] In summary, according to the embodiment of the present application, a digestive endoscopy auxiliary system based on AI technology is explained, which obtains digestive tract endoscopic images of the patient being examined, and uses AI-based image recognition and analysis technology to extract digestive tract state features and multi-scale perception of the digestive tract endoscopic images, so as to intelligently evaluate the type of digestive tract abnormality of the patient being examined according to the features of the digestive tract endoscopic images under multi-scale enhancement, and issue a warning prompt signal. In this way, the system can analyze images at multiple scales at the same time, improving the ability to detect lesions of different sizes and shapes, including tiny lesions and large-scale abnormalities. At the same time, the deep learning model is used to automatically extract image features, reducing the reliance on manually designed features and improving the intelligence of the digestive endoscopy auxiliary system.
[0073] Figure 6 FIG. 1 is a schematic flow chart of a digestive endoscopy auxiliary method based on AI technology according to an embodiment of the present application. Figure 6As shown, the AI-based digestive endoscopy auxiliary method includes: S110, obtaining a digestive endoscopic image of the patient being examined; S120, passing the digestive endoscopic image through a digestive tract state feature extractor to obtain a digestive tract state feature map; S130, passing the digestive tract state feature map through a digestive tract state enhancement expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception feature map; S140, passing the digestive tract state multi-scale perception feature map through a cross-channel feature space self-attention enhancement adaptive feature module to obtain an attention-enhanced digestive tract state multi-scale perception feature map; S150, obtaining a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issuing a warning prompt signal based on the recognition result.
[0074] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned AI-based digestive endoscopy auxiliary method have been referred to above. Figures 1 to 5 The description of the AI technology-based digestive endoscopy assistance system has been introduced in detail, and therefore, its repeated description will be omitted.
[0075] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.
[0076] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0077] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0078] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.
[0079] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the device claim can also be implemented by one unit through software or hardware.
[0080] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A digestive endoscopy auxiliary system based on AI technology, characterized in that: include: A patient digestive tract endoscopic image acquisition module, used to acquire a digestive tract endoscopic image of the patient being examined; A digestive tract state feature processing module, used for passing the digestive tract endoscopic image through a digestive tract state feature extractor to obtain a digestive tract state feature map; A digestive tract state multi-scale feature perception module, used for obtaining a digestive tract state multi-scale perception feature map by passing the digestive tract state feature map through a digestive tract state enhanced expressor based on multi-scale feature perception; A digestive tract state multi-scale perception feature enhancement module, used for obtaining an attention-enhanced digestive tract state multi-scale perception feature map by self-attention-enhanced adaptive feature module through the cross-channel feature space multi-scale perception feature map of the digestive tract state; A recognition result generation module, used to obtain a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issue a warning prompt signal based on the recognition result; Wherein, the digestive tract state multi-scale feature perception module includes: A digestive tract state multi-layer convolution unit, used for performing convolution processing on the digestive tract state characteristic map based on a multi-layer convolution layer to obtain a deep digestive tract state characteristic map, a middle digestive tract state characteristic map and a shallow digestive tract state characteristic map; A deep digestive tract state convolutional coding unit, used for performing convolutional coding on the deep digestive tract state characteristic map to obtain a deep digestive tract state convolutional characteristic map; A deep digestive tract state upsampling processing unit, used for upsampling the deep digestive tract state convolution feature map to obtain an upsampled deep digestive tract state feature map; a middle-level perception digestive tract state extraction unit, configured to perform multi-scale feature perception processing on the middle-level digestive tract state feature map to obtain a first multi-scale perception digestive tract state feature map; A shallow-layer perception digestive tract state extraction unit, used for performing multi-scale feature perception processing on the shallow-layer digestive tract state feature map to obtain a second multi-scale perception digestive tract state feature map; The digestive tract state multi-scale feature fusion unit is used to fuse the deep digestive tract state convolution feature map, the first multi-scale perception digestive tract state feature map and the second multi-scale perception digestive tract state feature map to obtain the digestive tract state multi-scale perception feature map.
2. The AI-based digestive endoscopy auxiliary system according to claim 1 is characterized in that: The digestive tract state feature processing module is used to: pass the digestive tract endoscopic image through a digestive tract state feature extractor based on a void convolutional neural network to obtain the digestive tract state feature map.
3. The AI-based digestive endoscopy auxiliary system according to claim 2 is characterized in that: The middle-level digestive tract state extraction unit comprises: A middle-level multi-scale digestive tract state perception subunit, used for inputting the middle-level digestive tract state characteristic map into a multi-scale perception network to obtain a middle-level multi-scale perception digestive tract state characteristic map; The digestive tract state feature fusion subunit is used to perform feature fusion on the middle-layer multi-scale perception digestive tract state feature map and the upsampled deep-layer digestive tract state feature map to obtain a first digestive tract state fusion feature map, and then perform convolution encoding on the first digestive tract state fusion feature map to obtain the first multi-scale perception digestive tract state feature map.
4. The AI-based digestive endoscopy auxiliary system according to claim 3 is characterized in that: The middle-level multi-scale digestive tract state sensing subunit is used to: Copying the middle digestive tract state characteristic diagram to obtain a middle digestive tract state copy characteristic diagram; The middle-level digestive tract state feature map is first passed through a convolution layer with a convolution kernel of 1×1, and then the obtained feature map is sequentially subjected to activation processing based on a ReLU function and standardized batch processing to obtain a first-scale middle-level standardized digestive tract state feature map; The first-scale middle-layer standardized digestive tract state feature map is first passed through a convolution layer with a convolution kernel of 3×3, and then the obtained feature map is sequentially subjected to the activation processing based on the ReLU function and the standardized batch processing to obtain a second-scale middle-layer standardized digestive tract state feature map; The second-scale middle-layer standardized digestive tract state feature map is first passed through a convolution layer with a convolution kernel of 5×5, and then the obtained feature map is sequentially subjected to the activation processing based on the ReLU function and the standardized batch processing to obtain a third-scale middle-layer standardized digestive tract state feature map; The middle-level digestive tract state replication feature map, the first-scale middle-level standardized digestive tract state feature map, the second-scale middle-level standardized digestive tract state feature map and the third-scale middle-level standardized digestive tract state feature map are cascaded through a feature fusion layer to obtain the middle-level multi-scale perception digestive tract state feature map.
5. The AI-based digestive endoscopy auxiliary system according to claim 4 is characterized in that: The shallow layer sensing digestive tract state extraction unit is used to: Inputting the shallow digestive tract state characteristic map into the multi-scale perception network to obtain a shallow multi-scale perception digestive tract state characteristic map; Performing upsampling processing on the first multi-scale perception digestive tract state feature map to obtain an upsampled middle-layer multi-scale perception digestive tract state feature map; After feature fusion of the shallow multi-scale perception digestive tract state feature map and the upsampled middle-layer multi-scale perception digestive tract state feature map to obtain a second digestive tract state fusion feature map, convolution encoding is performed on the second digestive tract state fusion feature map to obtain the second multi-scale perception digestive tract state feature map.
6. The AI-based digestive endoscopy auxiliary system according to claim 5 is characterized in that: The digestive tract state multi-scale perception feature enhancement module includes: A digestive tract state multi-scale perception normalization unit, used for performing layer normalization on the digestive tract state multi-scale perception feature map to obtain a normalized digestive tract state multi-scale perception feature map; A digestive tract state multi-scale perception point convolution unit, used for performing point convolution processing on the normalized digestive tract state multi-scale perception feature map to obtain a digestive tract state multi-scale perception channel context association representation feature map; A digestive tract state multi-scale perception space association unit, used for performing convolution encoding on the digestive tract state multi-scale perception channel context association representation feature map to obtain a digestive tract state multi-scale perception space context association representation feature map; A digestive tract state multi-scale perception backup unit, used to copy the digestive tract state multi-scale perception space context association representation feature map to obtain a digestive tract state multi-scale perception backup space context association representation feature map; A digestive tract state multi-scale perception feature shape reshaping unit is used to reshape the digestive tract state multi-scale perception channel context association representation feature map, the digestive tract state multi-scale perception space context association representation feature map and the digestive tract state multi-scale perception backup space context association representation feature map to obtain a digestive tract state multi-scale perception channel context association representation feature matrix, a digestive tract state multi-scale perception space context association representation feature matrix and a digestive tract state multi-scale perception backup space context association representation feature matrix; A digestive tract state multi-scale perception cross-channel feature calculation unit, used to calculate a digestive tract state multi-scale perception cross-channel cross covariance matrix between the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception space context association representation feature matrix; A digestive tract state multi-scale perception activation unit is used to activate the digestive tract state multi-scale perception cross-channel cross-covariance matrix using a Softmax function to obtain a digestive tract state multi-scale perception feature global interactive attention matrix; A digestive tract state multi-scale perception attention enhancement unit, used to calculate the product between the digestive tract state multi-scale perception backup space context association representation feature matrix and the digestive tract state multi-scale perception feature global interaction attention matrix to obtain a digestive tract state multi-scale perception attention enhancement feature representation matrix; The feature shape reshaping unit is used to perform feature shape reshaping on the digestive tract state multi-scale perception attention enhancement feature representation matrix to obtain the attention enhancement digestive tract state multi-scale perception feature map.
7. The AI-based digestive endoscopy auxiliary system according to claim 6 is characterized in that: The digestive tract state multi-scale perception cross-channel feature calculation unit is used to: Calculating the matrix product between the digestive tract state multi-scale perception channel context association representation feature matrix and the digestive tract state multi-scale perception space context association representation feature matrix to obtain the digestive tract state multi-scale perception channel-space context association representation feature matrix; The position-wise division of the digestive tract state multi-scale perception channel-spatial context association representation feature matrix and a predetermined hyperparameter is calculated to obtain the digestive tract state multi-scale perception cross-channel cross-covariance matrix.
8. The AI-based digestive endoscopy auxiliary system according to claim 7 is characterized in that: The recognition result generating module is used for: The attention-enhanced digestive tract state multi-scale perception feature map is passed through a digestive tract lesion identifier based on a classifier to obtain the recognition result, and the recognition result is used to represent the digestive tract abnormality type label of the examined patient; Based on the recognition result, a warning signal is issued.
9. A digestive endoscopy auxiliary method based on AI technology, characterized in that: include: Obtaining endoscopic images of the digestive tract of the patient being examined; Passing the digestive tract endoscopy image through a digestive tract state feature extractor to obtain a digestive tract state feature map; The digestive tract state characteristic graph is passed through a digestive tract state enhanced expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception characteristic graph; The multi-scale perceptual feature map of the digestive tract state is subjected to a self-attention-enhanced adaptive feature module across the channel feature space to obtain an attention-enhanced multi-scale perceptual feature map of the digestive tract state; Obtaining a recognition result based on the attention-enhanced digestive tract state multi-scale perception feature map, and issuing a warning prompt signal based on the recognition result; The digestive tract state characteristic graph is passed through a digestive tract state enhanced expressor based on multi-scale feature perception to obtain a digestive tract state multi-scale perception characteristic graph, including: The digestive tract state characteristic map is subjected to convolution processing based on a multi-layer convolution layer to obtain a deep digestive tract state characteristic map, a middle digestive tract state characteristic map and a shallow digestive tract state characteristic map; Performing convolution encoding on the deep digestive tract state characteristic map to obtain a deep digestive tract state convolution characteristic map; Performing upsampling processing on the deep digestive tract state convolution feature map to obtain an upsampled deep digestive tract state feature map; Performing multi-scale feature perception processing on the middle-layer digestive tract state characteristic map to obtain a first multi-scale perception digestive tract state characteristic map; Performing multi-scale feature perception processing on the shallow digestive tract state characteristic map to obtain a second multi-scale perception digestive tract state characteristic map; The deep digestive tract state convolution feature map, the first multi-scale perception digestive tract state feature map and the second multi-scale perception digestive tract state feature map are fused to obtain the digestive tract state multi-scale perception feature map.
