Lesion feature recognition system for gastrointestinal endoscopy

Abstract

Disclosed in the present invention is a lesion feature recognition system for gastrointestinal endoscopy. By utilizing the system, the accuracy and efficiency of lesion feature recognition under gastrointestinal endoscopy can be improved, and the shortcomings including limited recognition capability, poor anti-interference performance, high demand for computing power, high misdiagnosis rate and the like under conventional AI-assisted endoscope inspection technology are overcome.

Description

Gastrointestinal endoscopic lesion feature recognition system

[0001] Cross-reference instructions

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office as early as August 17, 2023, with application number 202311040222.1, and invention name “Gastrointestinal endoscopic lesion feature identification system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present invention relates to the field of artificial intelligence, and in particular to a system for recognizing features of lesions under a gastrointestinal endoscope. Background Art

[0004] The application of artificial intelligence (AI) in clinical medicine has become a research hotspot. The main approach involves building a multi-layered deep learning model. This model collects a large number of images of digestive endoscopic lesions from hospitals. Doctors are required to annotate the lesions represented in the images, which serve as training data. AI then learns from these data features, ultimately identifying specific gastrointestinal diseases. Software prompts are often used to assist endoscopists in their examinations in real time. Upon discovery of a lesion, a prompt is generated on the screen, prompting the doctor to observe and perform the procedure, as well as providing a preliminary diagnosis. This type of technology is being researched and applied in various areas, including esophageal cancer, gastric cancer, and colorectal adenomatous polyps. It has proven clinically valuable, improving endoscopists' examination efficiency and reducing missed diagnoses. In addition to localized lesions, AI-assisted diagnosis of lesions such as chronic atrophic gastritis and ulcerative colitis is also making progress. Although AI has reached the level of doctors in recognizing static images of some lesions, and some technologies can even make diagnoses of benign and malignant risks, overall, such applications have not yet reached the level of mature products required for clinical practice, have a high false positive rate, and are still some distance away from their widespread promotion and application. The main achievements are mostly in scientific research and clinical trials.

[0005] Currently, leading AI technologies primarily focus on identifying and diagnosing endoscopic features of a single disease and lack the ability to differentiate across multiple diseases. Expanding disease diagnostic capabilities requires rebuilding the model, training it with new samples, and then running it in parallel with the existing model before diagnosis can be achieved. Clinically, dozens of diseases are commonly diagnosed during digestive endoscopy. Assisting physicians requires the ability to diagnose and differentiate these multiple diseases simultaneously. This places high demands on both computing power and accuracy on AI technologies using single-disease models. Current minicomputer hardware makes it difficult to achieve both low latency and versatility. Secondly, AI technology requires a large number of samples as a prerequisite for model training. Each additional disease requires increased sample collection and annotation costs, which limits the rapid expansion of such AI applications' diagnostic capabilities. Thirdly, current solutions have significant limitations in their ability to diagnose diseases with complex and variable features. Due to technical limitations, the more varied and complex the presentation of a lesion, the more difficult it is for AI to reduce false positives and false negatives to a reasonable level.

[0006] In the field of endoscopic diagnosis of gastrointestinal diseases, various AI technologies and methods currently exist, such as deep learning, convolutional neural networks, support vector machines, and Bayesian networks. While these technologies and methods are theoretically applicable to endoscopic diagnosis of gastrointestinal diseases, they present limitations or challenges in achieving the goal of distinguishing multiple diseases. For example: 1) Identification of a single disease or lesion: Some AI technologies may only be able to identify a single disease or lesion, and cannot simultaneously identify multiple lesions and disease types. 2) High sample requirements: Some AI technologies require a large amount of labeled data and the knowledge and experience of professional physicians to train high-quality models. 3) High model complexity: For some AI technologies, building multiple models or retraining them can increase model complexity and training time. 4) High misdiagnosis rate: Some AI technologies may not consider the interactions and similarities between different lesions, which may lead to misdiagnosis. Therefore, to address these technical issues, a new solution is needed to ensure widespread adoption of AI-assisted gastrointestinal endoscopic diagnosis.

[0007] Summary of the Invention

[0008] Technical issues solved:

[0009] Traditional AI models are typically trained and optimized for a single disease or lesion, requiring extensive labeled data and the knowledge and experience of specialized physicians. Furthermore, in practice, different lesions may interact and exhibit similarities, which traditional single-disease training methods cannot effectively address. Furthermore, developing separate models for each disease increases model complexity, sample processing costs, and training time, limiting their efficiency and feasibility in practical applications.

[0010] Specifically, one aspect of the present invention aims to solve the problems existing in traditional AI models in diagnosing gastrointestinal diseases under assisted endoscopy, and proposes a new system. That is, the characteristics of different types of lesions are analyzed, and multiple feature points that meet the clinical diagnostic basis are disassembled, and they are classified and summarized as the recognition targets of the AI ​​model. In this way, different lesions and diseases can be automatically identified at the same time, avoiding the problem of establishing multiple models and retraining. A secondary judgment is made on the identified feature combination to obtain the correct disease diagnosis. For the needs of unpredictable patient conditions in clinical work and the need to identify dozens of lesions, this method can improve the accuracy and efficiency of diagnosis, provide doctors with a truly usable auxiliary diagnostic tool, and make up for the shortcomings of traditional AI-assisted endoscopic examination technology, such as single diagnostic ability, poor anti-interference ability, large computing power requirements and high misdiagnosis rate.

[0011] Technical solution:

[0012] A digestive tract endoscopic lesion feature recognition system, comprising:

[0013] 1) Image acquisition module; and

[0014] 2) a gastrointestinal endoscopic multi-level feature recognition and training module, comprising a first multi-task classification neural network for identifying n preset different categories of gastrointestinal endoscopic feature groups and label values ​​in the gastrointestinal endoscopic feature groups, where 2≤n≤10; and

[0015] 3) a feature result collection module, used to collect the recognition results output by the multi-level feature recognition and training module under the digestive tract endoscopy;

[0016] The digestive tract endoscopic feature group includes a local lesion feature group, and the label value of the local lesion feature group includes a local depressed lesion, a local raised lesion, or a local flat lesion;

[0017] The downstream network of the local lesion feature group includes a local lesion multi-level feature recognition and training module, which includes a second multi-task classification neural network for identifying m different categories of preset local lesion features and supplementary identification values ​​corresponding to the categories, where 1≤m≤5.

[0018] Another aspect of the present invention is to provide an electronic device, which includes an electronic device body and the above-mentioned gastrointestinal endoscopic lesion feature recognition system, and the gastrointestinal endoscopic lesion feature recognition system is installed on the electronic device body.

[0019] Beneficial effects of the present invention:

[0020] This invention uses a new AI model based on feature point classification and aggregation, which can simultaneously identify multiple different lesions and diseases, avoiding the problem of building multiple models and retraining, and improving the accuracy and efficiency of endoscopic diagnosis of gastrointestinal diseases. Specifically:

[0021] 1) Improved diagnostic accuracy: The AI ​​model used in this paper can automatically identify and categorize multiple lesions and diseases simultaneously, eliminating the need for multiple models and retraining. By performing secondary assessments on identified feature combinations, more accurate disease diagnoses can be achieved, reducing misdiagnosis rates and improving diagnostic accuracy.

[0022] 2) Improved diagnostic efficiency: Traditional AI models require a separate model for each disease, increasing model complexity, sample processing costs, and training time, limiting their efficiency and feasibility in practical applications. However, the AI ​​model used in this invention, which is based on feature point classification and aggregation, can simultaneously identify multiple different lesions and diseases, avoiding the need for multiple model creation and retraining, and improving the efficiency of endoscopic diagnosis of gastrointestinal diseases.

[0023] 3) Reduce the workload of manual labeling: The AI ​​model used in the present invention can automatically perform feature analysis and classification summary of different lesions and diseases, reducing the workload of manual labeling.

[0024] 4) Improving clinical application value: The AI ​​model of the present invention can simultaneously identify multiple different lesions and diseases, providing doctors with a truly usable auxiliary diagnostic tool, improving the examination efficiency of endoscopists, reducing missed diagnoses, expanding the value of AI in clinical applications, and making the application of AI technology accessible to the public. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a gastrointestinal endoscopic lesion feature recognition system constructed in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention discloses a system for identifying lesion features under gastrointestinal endoscopy. Those skilled in the art may refer to the contents of this document and appropriately improve the process parameters to implement the system. It should be noted that all similar substitutions and modifications that would be obvious to those skilled in the art are considered to be included in the present invention, and relevant persons may obviously modify or appropriately change and combine the contents described herein without departing from the content, spirit, and scope of the present invention to implement and apply the technology of the present invention.

[0027] In the present invention, unless otherwise specified, scientific and technical terms used herein have the meanings commonly understood by those skilled in the art. Unless otherwise expressly indicated, throughout the specification and claims, the term "comprise" or variations thereof such as "comprises" or "comprising" will be understood to include the elements or components recited without excluding other elements or other components. The terms "such as" and "for example" are intended to refer to exemplary embodiments and are not intended to limit the scope of the present disclosure.

[0028] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is further described in detail below with reference to specific embodiments.

[0029] As shown in FIG1 , this embodiment discloses a digestive tract endoscopic lesion feature recognition system of the present invention, which includes:

[0030] S1. Image acquisition module.

[0031] The image acquisition module is used to acquire images in real time or pre-stored on a medium. The image acquisition module may include a video acquisition module and / or an image acquisition module. In certain embodiments of the present invention, the image acquisition module may utilize a video capture card to transmit digital / analog video signals such as HDMI, DVI, SDI, or S-Video to a computer, read the video signals using OpenCV, and transcode them frame by frame into an RGB image format.

[0032] In certain embodiments of the present invention, when the image acquisition module obtains the real-time video image signal output by the endoscope device, the image acquisition module performs interval sampling on the obtained frame-by-frame images, for example, the interval sampling rate is 1 second / time, obtains the image to be identified, and records the frame number of the image to be identified.

[0033] S2. Multi-level feature recognition and training module for gastrointestinal endoscopy.

[0034] The gastrointestinal endoscopic multi-level feature recognition and training module includes a first multi-task classification neural network for identifying n preset different categories of gastrointestinal endoscopic feature groups and label values ​​in the gastrointestinal endoscopic feature groups, wherein 2≤n≤10. The gastrointestinal endoscopic feature group includes a local lesion feature group. In certain embodiments of the present invention, n can be 2, 3, 4, 5, 6, 7, 8, 9 or 10. In one embodiment of the present invention, n=5. That is, the gastrointestinal endoscopic feature group includes 5 different categories, namely, a mucosal feature group, a vascular texture feature group, a bleeding feature group, a mucus feature group and a local lesion feature group. In some other embodiments of the present invention, at least one of the mucosal feature group, vascular texture feature group, bleeding feature group or mucus feature group, or other related gastrointestinal endoscopic feature groups, can be selected. The above-mentioned mucosal features, vascular texture features, bleeding features, and mucus features all belong to diffuse lesion features.

[0035] In certain embodiments of the present invention, the label values ​​corresponding to the above-mentioned different categories of gastrointestinal endoscopic feature groups may include, for example, the label values ​​of the mucosal feature group include mucosal thinning, mucosal strip redness, mucosal flaky redness, mucosal edema, island mucosa, residual glandular mucosa, gastric mucosa, intestinal mucosa or intestinal metaplasia; the label values ​​of the vascular texture feature group include vascular opacity, strip vessels, dendritic vessels or dot-flaky vessels; the label values ​​of the bleeding feature group include no bleeding, dot-flaky bleeding, small amount of bleeding or large amount of bleeding; the label values ​​of the mucus feature group include yellow mucus, viscous mucus, bile reflux mucus, clear mucus or mucus lake; the label values ​​of the local lesion feature group include local depressed lesions, local raised lesions or local flat lesions.

[0036] To achieve one of the objectives of the present invention, in certain embodiments of the present invention, the first multi-task classification neural network includes a set of main feature extraction networks for identifying n preset different categories of gastrointestinal endoscopic feature groups and n sets of branch feature extraction networks located downstream of the main feature extraction network for respectively identifying label values ​​in the gastrointestinal endoscopic feature groups. In one embodiment of the present invention, n = 5. The method for constructing the first multi-task classification neural network includes:

[0037] Step A) The main feature extraction network is composed of at least 3 groups (e.g., 4 groups, 6 groups, 8 groups, 9 groups, 10 groups) of multi-branch parallel feature extraction units, and a first main feature extraction network feature map is obtained through calculation;

[0038] Step B) using at least 3 groups (e.g., 4 groups, 5 groups, 6 groups, 8 groups) of multi-branch parallel feature extraction units to respectively form an n-th branch feature extraction network; inputting the first main feature extraction network feature graphs respectively, and finally obtaining the n-th branch feature extraction network feature graphs through multi-branch parallel convolution operations;

[0039] The design method of the multi-branch parallel convolution operation unit is as follows:

[0040] a) constructing a slicing unit for dividing the original input feature map into four groups to obtain a first slice feature group, a second slice feature group, a third slice feature group, and a fourth slice feature group;

[0041] b) constructing a group feature extraction unit using three depthwise separable convolutions to extract features from the first, second, and third slice feature groups, respectively. The three depthwise separable convolutions have sizes of 3*3, 1*13, and 13*1, respectively, for extracting spatial features, vertical channel features, and horizontal channel features. Finally, the extracted spatial features, vertical channel features, and horizontal channel feature maps are reconcatenated with the fourth slice feature group and passed through a regularization layer to obtain the first output.

[0042] c) constructing a feature enhancement unit, wherein the feature enhancement unit is composed of a 1*1 convolution layer, an activation function layer, a 1*1 convolution layer and a residual connection, and obtaining a second output by serially performing a 1*1 convolution, an activation function, and a 1*1 convolution operation. The residual connection is used to horizontally concatenate the original input feature map with the second output to obtain a third output, which is the output result of the multi-branch parallel convolution operation unit;

[0043] The 1*1 convolutional layer is constructed by conventional techniques in the art, and the activation function is GeLU;

[0044] Step C) uses the fully connected layer and the Sigmoid function as a combination unit to construct the first, second, third, fourth, and fifth prediction layers. Specifically, the first, second, third, fourth, and fifth branch feature maps are input into the fully connected layer. The network layer compresses the high-dimensional feature map into a 1-dimensional vector form, and finally obtains the first, second, third, fourth, and fifth feature group label recognition results through the Sigmoid function operation. The Sigmoid operation formula is:

[0045] p=Sigmoid(W T ·F+b)

[0046] Among them, p refers to the probability of solving for category k, W T is a trainable hyperparameter for category k, and b is a trainable bias hyperparameter for category k.

[0047] In certain embodiments of the present invention, training sets are input into the first multi-task classification neural network in batches for training.

[0048] The source of the training set is: based on the preset multiple groups of multi-label feature results, digestive tract gastrointestinal image data are collected, and feature labels are annotated one by one to obtain the first feature label data set, and then the training set and validation set are split according to the ratio of 8:2; the feature groups and corresponding label values ​​are shown in Table 1.

[0049] Table 1

[0050] In order to achieve one object of the present invention, the method for training the first multi-task classification neural network includes:

[0051] Step D) predicting the original label value outputted corresponding to each of the digestive tract endoscopic feature groups, using the BCE cross entropy loss function to calculate, obtaining the nth group loss function, and calculating the total network loss using a weighted summation method, the specific formula is:

[0052] Among them, k is the number of feature group branches; L k That is, the loss derivative of the k feature branch; N is the number of pictures in the training batch; i is the number of corresponding categories; y i is the true label value of category i in the image, that is, if the image is category i, then y i is 1, not category i, then y i is 0; p i is the prediction confidence of the image for category i;

[0053] L=w1×L1+w2×L2+......+w k ×L k

[0054] Among them, L is the total loss, w is the weight of the loss function; the total loss is obtained by summing;

[0055] The adjustment strategy for the loss function weight in the present invention is stage attenuation, that is, the fixed weights are [2.5, 2.5, 2.5, 2.5] at the beginning of training, and are adjusted every 10 rounds. The adjustment basis is: if L t <L (t-10) , then w t =w t ×θ; if L t >L (t-10) , then w t =w t×(2-θ). Wherein, θ is the attenuation coefficient, θ=0.85;

[0056] Step E) Based on the value of the total network loss, back propagation technology is used to update the neural network model parameters. When the final loss function stabilizes to a preset value or reaches a set maximum number of iterations, training is stopped to obtain the pre-trained first multi-task classification neural network.

[0057] A first multi-task classification neural network that has been constructed and trained is used to identify and predict features of lesions under gastrointestinal endoscopy. 1) The frame-by-frame images obtained by the image acquisition module are sampled at an interval of 1 second per time to obtain an image to be identified, and the frame sequence number of the image to be identified is recorded; 2) The image to be identified is subjected to a pre-constructed first multi-task classification neural network to perform multi-group multi-label feature recognition on the image to be identified, obtaining a first mucosal feature group recognition result, a first vascular texture feature group recognition result, a first bleeding feature group recognition result, a first mucus feature group recognition result, and a first local lesion feature group recognition result.

[0058] The first mucosal feature group identification results, the first vascular texture feature group identification results, the first bleeding feature group identification results, the first mucus feature group identification results, and the first local lesion feature group identification results obtained above are input into the feature result collection module for appropriate purposes, including but not limited to auxiliary diagnosis of gastrointestinal diseases.

[0059] To achieve one of the purposes of the present invention, in one embodiment of the present invention, the downstream network of the above-mentioned local lesion feature group may further include a local lesion multi-level feature recognition and training module, which includes a second multi-task classification neural network for identifying m different categories of preset local lesion features and supplementary identification values ​​corresponding to the categories, wherein 1≤m≤5. In certain embodiments of the present invention, m may be 1, 2, 3, 4 or 5. In one embodiment of the present invention, m=4. That is, the local lesion features include 4 different categories, for example, lesion morphological features, lesion area features, lesion surface features or lesion edge features.

[0060] Furthermore, S4. Local lesion multi-level feature recognition and training module.

[0061] The local lesion multi-level feature recognition and training module includes a second multi-task classification neural network for identifying four different categories of preset local lesion features and the supplementary identification values ​​corresponding to the categories. The second multi-task classification neural network can be obtained using the same construction and training method as the first multi-task classification neural network. In certain embodiments of the present invention, the construction method of the second multi-task classification neural network includes:

[0062] Step F) constructing a lesion segmentation artificial intelligence neural network based on the DeepLabV3+ algorithm, predicting the image of the local lesion feature group, and obtaining a first lesion segmentation map. Specifically,

[0063] i) Construct the required dataset. Mark the contours of the lesions, including raised lesion targets and eroded lesion targets, to obtain the first segmented lesion dataset;

[0064] Secondly, mark the multi-level labels of the lesions and construct the second label feature dataset

[0065] ii) training a lesion segmentation model based on the first segmented lesion data set to obtain a first lesion segmentation model, wherein the input data is an image and the output is a mask image containing only the segmented area.

[0066] Step G) A second multi-task classification neural network algorithm is used to predict the multi-level feature labels of lesions. The first lesion segmentation map is input to obtain the prediction results of the first lesion morphological feature group, the first lesion area feature group, the first lesion surface feature group, and the first lesion edge feature group. Specifically,

[0067] i) The network is constructed using the same construction method as the first multi-task classification neural network mentioned above. Since there are only 4 feature groups here, the number of branches is set to 4, and the subsequent loss function is also 4.

[0068] ii) After training, a pre-built second multi-task classification neural network algorithm is obtained, which has the ability to recognize the lesion feature group label, that is,

[0069] When the identification labels in the first lesion feature recognition group include "local erosive lesion" or "local raised lesion," a pre-built second multi-task classification neural network algorithm is used to identify the supplementary identification values ​​of the feature group. This yields a first supplementary lesion morphological feature group, a first supplementary lesion area feature group, a first supplementary lesion surface feature group, and a first supplementary lesion edge feature group.

[0070] 1) In the first supplementary lesion morphological feature group, the corresponding labels for "local erosive lesions" include: redness and swelling, no redness and swelling; the corresponding labels for "local raised lesions" include: flat, lateral, pedunculated, and irregular;

[0071] 2) In the first supplementary lesion area feature group, the corresponding labels for "local erosion lesions" include: dot-shaped, shallow and small, deep and large; the corresponding labels for "local raised lesions" include: multiple and single;

[0072] 3) In the first supplementary lesion surface feature group, the corresponding labels for "local erosive lesions" include: no white fur, grayish white, and yellowish white; the corresponding labels for "local raised lesions" include: light white, light pink, and dark red;

[0073] 4) In the first supplementary lesion edge feature group, the corresponding labels for "local erosion lesions" include: smooth, burred; the corresponding labels for "local raised lesions" include: smooth, rough;

[0074] Step H) constructing a label conversion module to fuse the supplementary identification value obtained in step G) with the label value of the local lesion feature group to obtain the label value of a new local lesion feature group (second local lesion feature group).

[0075] The above-mentioned second local lesion feature group identification results can be input into the feature result collection module together with the above-mentioned first mucosal feature group identification results, the first vascular texture feature group identification results, the first bleeding feature group identification results, and the first mucus feature group identification results for appropriate purposes, including but not limited to auxiliary diagnosis of gastrointestinal diseases.

[0076] To achieve one objective of the present invention, in one embodiment of the present invention, the above-mentioned gastrointestinal endoscopic lesion feature recognition system further includes a mucosal location recognition module, which is configured to convert the label values ​​in the mucosal feature group according to the mucosal location and output the recognition results to the feature result collection module. The locations include the esophagus, gastric body, gastric fundus, gastric antrum, duodenum, cecum, ascending colon, transverse colon, descending colon, sigmoid colon, or rectum.

[0077] Further, S3. Mucosal location recognition module.

[0078] The method for constructing the mucosal position recognition module includes:

[0079] Step I) Use the temporal segment classification model to classify and identify the parts of the current video / image to obtain a position recognition result.

[0080] i) Collecting video clips of any appropriate length during digestive tract endoscopy;

[0081] ii) Data preprocessing: using interval sampling technology to reduce the dimension and scale the image size to a suitable size according to the total number of frames in the acquisition time;

[0082] iii) Use ResNet-3D to input and train the preprocessed data. During the training process, backpropagation technology is used to update the parameters to obtain a preset time series segment classification model, which has the ability to recognize locations.

[0083] Step J) converting the feature recognition results of the first mucosal feature group image to be identified after sampling obtained by the multi-level feature recognition and training module under the digestive tract endoscope into the result of the first mucosal feature group, specifically,

[0084] i) accumulating the images read frame by frame, and processing the set of segments after reaching a suitable length using the above-mentioned preprocessing module;

[0085] ii) Use the preset time series segment classification model to perform position prediction and obtain the prediction results corresponding to the frame number range and position;

[0086] iii) According to the frame number range, reversely search for the first mucosal feature group corresponding to the frame number in the interval. If the position corresponding to the frame number range interval is "gastric body", "gastric fundus" or "gastric antrum", and the first feature group in the interval contains the "intestinal mucosa" label, remove the "intestinal mucosa" label in the first mucosal feature group and add the "intestinal metaplasia" label.

[0087] iv) adding the label mucosa position recognition module output result to the first mucosa feature group according to the frame segment position corresponding recognition result as the second mucosa feature group recognition result.

[0088] The above-mentioned second mucosal feature group identification results can be input into the feature result collection module together with the above-mentioned first vascular texture feature group identification results, the first bleeding feature group identification results, the first mucus feature group identification results, and the first local lesion feature group identification results for appropriate purposes, including but not limited to auxiliary diagnosis of gastrointestinal diseases.

[0089] In some other embodiments of the present invention, the recognition results output by the above-mentioned multi-level feature recognition and training module under gastrointestinal endoscopy may include: the first / second mucosal feature group recognition results, the first vascular texture feature group recognition results, the first bleeding feature group recognition results, the first mucus feature group recognition results, and the first / second local lesion feature group recognition results, which are input into the feature result collection module for appropriate purposes, including but not limited to auxiliary diagnosis of gastrointestinal diseases.

[0090] In order to achieve one object of the present invention, in one embodiment of the present invention, the gastrointestinal endoscopic lesion feature recognition system further includes a disease diagnosis decision module for obtaining a disease diagnosis result using the recognition result.

[0091] Furthermore, S5. Disease diagnosis decision module.

[0092] The method for constructing the disease diagnosis decision module includes:

[0093] Step K) performs disease diagnosis annotation on the medical images in the first label data set to obtain a first disease data set, wherein the input of the data set is, for example, the second mucosal feature group recognition result, the first vascular texture feature group recognition result, the first bleeding feature group recognition result, the first mucus feature group recognition result, and the second local lesion feature group recognition result (i.e., label value), and the output is the disease type. In certain embodiments of the present invention, the disease type includes, but is not limited to, ulcerative colitis and atrophic gastritis. In other embodiments of the present invention, the output can also be the nature of the disease, for example, including but not limited to local high-risk lesions, local low-risk lesions, diffuse lesions, and classification.

[0094] Step L) constructing a preset disease diagnosis decision algorithm. Specifically, based on the XGBOOST algorithm, the diseases contained in the medical images are labeled and the data is iteratively trained batch by batch to obtain the pre-constructed disease diagnosis decision algorithm.

[0095] Step M) The disease diagnosis decision module may further include a preprocessing module for aligning the second mucosal feature group identification results, the first vascular texture feature group identification results, the first bleeding feature group identification results, the first mucus feature group identification results, and the second local lesion feature group identification results (i.e., label values) to obtain an aggregated feature array.

[0096] Step N) uses a pre-built disease diagnosis decision algorithm to input the feature set and perform algorithmic reasoning to obtain a disease diagnosis result.

[0097] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. The gastrointestinal endoscopic lesion feature recognition system is characterized by: The system comprises: 1) Image acquisition module; and 2) a digestive endoscopic multi-level feature recognition and training module, comprising a first multi-task classification neural network for identifying preset n different categories of digestive endoscopic feature groups and label values ​​in the digestive endoscopic feature groups, wherein 1≤n≤10; and 3) a feature result collection module, used to collect the recognition results output by the multi-level feature recognition and training module under the digestive tract endoscopy; The digestive tract endoscopic feature group includes a local lesion feature group, and the label value of the local lesion feature group includes a local depressed lesion, a local raised lesion or a local flat lesion; The downstream network of the local lesion feature group includes a local lesion multi-level feature recognition and training module, which includes a second multi-task classification neural network for identifying m different categories of preset local lesion features and supplementary identification values ​​corresponding to the categories, where 1≤m≤5.

2. The digestive tract endoscopic lesion feature recognition system according to claim 1, characterized in that: The image acquisition module includes a video acquisition module and / or a picture acquisition module; Wherein, the video acquisition module is used to read the video signal output by the endoscope device to obtain a video frame image; Preferably, the video acquisition module performs interval sampling on the obtained frame-by-frame images to obtain the image to be identified, and records the frame sequence number of the image to be identified.

3. The digestive tract endoscopic lesion feature recognition system according to claim 1, characterized in that: The digestive tract endoscopic feature group further includes at least one selected from a mucosal feature group, a vascular texture feature group, a bleeding feature group or a mucus feature group; Preferably, the label values ​​of the mucosal feature group include mucosal thinning, mucosal strip redness, mucosal flaky redness, mucosal edema, island mucosa, residual glandular mucosa, gastric mucosa, intestinal mucosa or intestinal metaplasia; the label values ​​of the vascular texture feature group include vascular opacity, strip-shaped blood vessels, dendritic blood vessels or dot-shaped blood vessels; the label values ​​of the bleeding feature group include no bleeding, dot-shaped bleeding, small amount of bleeding or large amount of bleeding; the label values ​​of the mucus feature group include yellow mucus, viscous mucus, bile reflux mucus, clear mucus or mucus lake.

4. The digestive tract endoscopic lesion feature recognition system according to claim 1, characterized in that: The first multi-task classification neural network includes a main feature extraction network for identifying n preset different categories of gastrointestinal endoscopic feature groups and n groups of branch feature extraction networks located downstream of the main feature extraction network and respectively used to identify label values ​​in the gastrointestinal endoscopic feature groups. The construction method of the first multi-task classification neural network includes: Step A) The main feature extraction network is composed of at least 3 groups of multi-branch parallel feature extraction units, and after calculation, a first main feature extraction network feature graph is obtained; Step B) using at least 3 groups of multi-branch parallel feature extraction units to respectively form an n-th branch feature extraction network; inputting the first main feature extraction network feature graphs respectively, and finally obtaining the n-th branch feature extraction network feature graphs respectively through multi-branch parallel convolution operations; The design method of the multi-branch parallel convolution operation unit is as follows: a) constructing a slicing unit for dividing the original input feature map into four groups to obtain a first slicing feature group, a second slicing feature group, a third slicing feature group, and a fourth slicing feature group; b) constructing a group feature extraction unit by three depth-wise separable convolutions, and performing feature extraction on the first slice feature group, the second slice feature group, and the third slice feature group, respectively, wherein the sizes of the three depth-wise separable convolutions are 3*3, 1*13, and 13*1, respectively, for extracting spatial features, vertical channel features, and horizontal channel features, and finally rejoining the extracted spatial features, the vertical channel features, and the horizontal channel feature maps with the fourth slice feature group, and passing through a regularization layer to obtain a first output; c) constructing a feature enhancement unit, wherein the feature enhancement unit is composed of a 1*1 convolution layer, an activation function layer, a 1*1 convolution layer and a residual connection, and obtaining a second output by serially performing 1*1 convolution, activation function and 1*1 convolution operations, and using the residual connection to perform horizontal feature map splicing of the original input feature map and the second output to obtain a third output, which is the output result of the multi-branch parallel convolution operation unit; Step C) uses the fully connected layer and the Sigmoid function as a combination unit to construct the nth prediction layer, and obtains the label recognition result of the nth gastrointestinal endoscopic feature group through Sigmoid function operation, wherein the Sigmoid operation formula is: p=Sigmoid(W T ·F+b) Among them, p refers to the probability of solving for category k, W T is a trainable hyperparameter for category k, and b is a trainable bias hyperparameter for category k.

5. The digestive tract endoscopic lesion feature recognition system according to claim 1 or 4, characterized in that: The training method of the first multi-task classification neural network is to input a training set into the first multi-task classification neural network in batches for training, including: Step D) for each of the digestive tract endoscopic feature groups, the original label value corresponding to the output is Prediction, using the BCE cross entropy loss function to calculate, get the nth group of loss functions, and use the weighted summation method to calculate the total network loss. The specific formula is: Among them, k is the number of feature group branches; L k That is, the loss derivative of the k feature branch; N is the number of images in the training batch; i is the number of corresponding categories; y i is the true label value of category i in the image, that is, if the image is category i, then y i is 1, not category i, then y i is 0; p i is the prediction confidence of the image for category i; L=w1×L1+w2×L2+......+w k ×L k Among them, L is the total loss, w is the weight of the loss function; the total loss is obtained by summing; Step E) according to the value of the total network loss, the neural network model parameters are updated using back propagation technology. When the final loss function stabilizes to a preset value or reaches a set maximum number of iterations, the training is stopped to obtain the pre-trained first multi-task classification neural network.

6. The digestive tract endoscopic lesion feature recognition system according to claim 3, characterized in that: The digestive tract endoscopic lesion feature recognition system also includes a mucosal position recognition module, which is used to convert the label value in the mucosal feature group according to the mucosal position and output the recognition result to the feature result collection module; Wherein, the location includes esophagus, gastric body, gastric fundus, gastric antrum, duodenum, cecum, ascending colon, transverse colon, descending colon, sigmoid colon or rectum.

7. The digestive tract endoscopic lesion feature recognition system according to claim 1, characterized in that: The local lesion feature includes at least one selected from lesion morphological features, lesion area features, lesion surface features or lesion edge features.

8. The digestive tract endoscopic lesion feature recognition system according to claim 1 or 7, characterized in that: The method for constructing the second multi-task classification neural network includes: Step F) constructing a local lesion segmentation artificial intelligence neural network, wherein the local lesion segmentation artificial intelligence neural network is used to classify and predict the input image of the local lesion feature group to obtain a first lesion segmentation map; Step G) constructing a local feature multi-label recognition network based on the segmented region, wherein the local feature multi-label recognition network is used to identify m different categories of the preset local lesion features and the supplementary identification values ​​corresponding to the categories; Step H) constructing a label conversion module, the label conversion module is used to convert the label obtained in step G) The supplementary identification value is fused with the label value of the local lesion feature group to obtain a new label value of the local lesion feature group.

9. The digestive tract endoscopic lesion feature recognition system according to claim 1, characterized in that: The digestive tract endoscopic lesion feature recognition system also includes a disease diagnosis decision module for obtaining a disease diagnosis result using the recognition result.

10. An electronic device, characterized in that: The electronic device comprises an electronic device body and a gastrointestinal endoscopic lesion feature recognition system as described in any one of claims 1 to 9, wherein the gastrointestinal endoscopic lesion feature recognition system is installed on the electronic device body.

Images