Digestive tract lesion assessment method, system, device and readable storage medium

Abstract

The present invention discloses a method, system, device and readable storage medium for evaluating digestive tract lesions. The system includes: an acquisition module for acquiring digestive tract images; a first network module for extracting features from digestive tract images to obtain multiple trunk feature maps; a second network module for performing background feature enhancement operations based on the trunk feature map multiple times to generate background feature maps and calculate mucosal background classification results based on the trunk feature map; a third network module for fusing the trunk feature map and the background feature map, reinforcing multiple times to enhance the focal lesion information, suppress the background features, and obtain the focal lesion target detection results through regression prediction. The present invention accurately determines the mucosal background classification results through background enhancement, and then repeatedly enhances the target and suppresses the non-target so that local important lesions can be detected. The present invention can be applied to digestive tract mucosal images under endoscopic examination, and can further set the targets required for identification and classification to better guide doctors' work.

Description

Technical Field

[0001] The present invention relates to the technical field of lesion assessment, and in particular to a method, system, device and readable storage medium for assessing digestive tract lesions. Background Art

[0002] At present, digestive endoscopy has become one of the standard methods for diagnosing gastrointestinal lesions. With the development of digestive endoscopy technology, doctors can more accurately identify and evaluate gastrointestinal mucosal lesions. However, in actual operation, mucosal lesions are complex and diverse, that is, diffuse lesions exist, and focal lesions may also coexist, which brings certain challenges to doctors' accurate diagnosis under endoscopy. Diffuse lesions often cover a large area, such as multiple patchy redness, mucosal atrophy or extensive inflammatory reactions in the gastric mucosa, large areas of mucosal roughness changes in the esophageal mucosa, large areas of esophagitis and other reactions, while focal lesions such as esophageal and gastric ulcers, erosions or polyps may be hidden in these large lesions and difficult to be effectively identified.

[0003] Existing technologies usually rely on a single classification or detection model to process gastrointestinal endoscopic images. Although these methods can provide a certain degree of lesion recognition and localization, they have the following disadvantages when dealing with the coexistence of complex lesions:

[0004] (1) Lack of processing capabilities for the coexistence of large-area lesions (global lesions) and focal lesions: Existing classification models mainly focus on the classification of global lesions, while target detection models focus on the identification of local areas. This separation approach can easily cause the classification model to ignore local severe lesions when dealing with large-area lesions and focal lesions, or the detection model to generate too many useless detection frames in the large-area lesion area, thereby interfering with the doctor's identification of key lesions.

[0005] (2) In areas covered by large lesions such as macular redness, existing detection methods are prone to misidentifying background information as local lesions, resulting in over-labeling and increasing the diagnostic burden on doctors.

[0006] (3) Existing technologies usually fail to effectively distinguish and process background and foreground features, especially in complex lesion backgrounds, which can easily lead to the model's insufficient ability to detect important local lesions and cause the omission of key lesion areas.

[0007] Therefore, in order to solve the above-mentioned problem that the detection capability of local important lesions needs to be improved, it is urgent to develop a method, system, device and readable storage medium for evaluating gastrointestinal lesions. Summary of the invention

[0008] In view of the problem that the ability to detect gastrointestinal lesions needs to be improved, the present invention provides a method, system, device and readable storage medium for evaluating gastrointestinal lesions.

[0009] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0010] In a first aspect, the present invention provides a digestive tract lesion assessment system, comprising:

[0011] An acquisition module, used for acquiring digestive tract images;

[0012] The first network module is used to extract features from the digestive tract image to obtain a plurality of main feature maps;

[0013] The second network module is used to perform background feature enhancement operations multiple times based on the trunk feature map to generate a background feature map, and calculate the mucosal background classification result according to the background feature map;

[0014] The third network module is used to fuse the backbone feature map and the background feature map to obtain a fusion map, perform multiple enhancements based on the fusion map to enhance the focal lesion information and suppress the background features, and perform regression prediction based on the enhancement results to obtain the focal lesion target detection results.

[0015] In a preferred embodiment, the first network module includes five feature extraction units arranged in sequence, and the feature extraction units are used to extract image feature information step by step to obtain a global feature map, a first trunk feature map, a second trunk feature map, a third trunk feature map and a fourth trunk feature map in sequence.

[0016] In a preferred embodiment, the second network module includes:

[0017] A fourth background enhancement unit, used for receiving the fourth trunk feature map and performing operations on the fourth trunk feature map to enhance the background features, thereby obtaining a fourth background feature map;

[0018] a third background enhancement unit, configured to convolute and splice the fourth background feature map and the third backbone feature map to obtain a third spliced ​​map, and perform operations on the third spliced ​​map to enhance background features to obtain a third background feature map;

[0019] A second background enhancement unit is used for convolutionally splicing the third background feature map and the second backbone feature map to obtain a second spliced ​​map, and performing operation on the second spliced ​​map to enhance background features to obtain a second background feature map;

[0020] A first background enhancement unit is used for convolutionally splicing the second background feature map and the first backbone feature map to obtain a first spliced ​​map, and performing operation on the first spliced ​​map to enhance background features to obtain a first background feature map;

[0021] The decision unit is used to perform a full convolution operation on the first background feature map, and perform mucosal background classification after the full convolution operation to obtain a mucosal background classification result.

[0022] In a preferred embodiment, the classification categories of the mucosal background classification include diffuse lesion mucosa and normal mucosa, or include macular redness mucosa, atrophic mucosa, chicken skin-like change mucosa and normal mucosa.

[0023] In a preferred embodiment, the fourth background enhancement unit, the third background enhancement unit, the second background enhancement unit and the first background enhancement unit all include a convolutional layer, a batch normalization layer, an activation layer, an expanded convolutional layer and a reverse attention weighted layer.

[0024] In a preferred embodiment, the third network module includes:

[0025] a fourth reverse enhancement unit, configured to concatenate and fuse the fourth trunk feature map and the fourth background feature map to obtain a fourth fusion map, and to calculate the fourth fusion map so as to enhance the focal lesion information and suppress the background features, thereby obtaining a fourth reverse enhancement feature map;

[0026] A third reverse enhancement unit is used to splice and fuse the third trunk feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fusion map, and calculate the third fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a third reverse enhancement feature map;

[0027] The second reverse enhancement unit is used to splice and fuse the second trunk feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fusion map, and calculate the second fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a second reverse enhancement feature map;

[0028] A first reverse enhancement unit is used to splice and fuse the first trunk feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fusion map, and calculate the first fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a first reverse enhancement feature map;

[0029] The position regression unit is used to perform regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map using the anchor frame method to obtain focal lesion target detection results.

[0030] In a preferred embodiment, the fourth inverse reinforcement unit, the third inverse reinforcement unit, the second inverse reinforcement unit and the first inverse reinforcement unit each include a convolution layer, a Sigmoid activation layer, a ReLu activation layer, and a residual convolution block.

[0031] In a second aspect, the present invention provides a method for evaluating digestive tract lesions, comprising:

[0032] S1, obtaining digestive tract images;

[0033] S2, extracting features from the digestive tract image to obtain multiple trunk feature maps;

[0034] S3, performing background feature enhancement operations multiple times based on the trunk feature map to generate a background feature map, and calculating the mucosal background classification result according to the background feature map;

[0035] S4. The main feature map and the background feature map are fused to obtain a fusion map. Multiple enhancements are performed based on the fusion map to enhance the focal lesion information and suppress the background features. Regression prediction is performed based on the enhancement results to obtain the focal lesion target detection result.

[0036] In a preferred embodiment, S2 specifically includes: extracting image feature information step by step based on the digestive tract image to sequentially obtain a global feature map, a first trunk feature map, a second trunk feature map, a third trunk feature map, and a fourth trunk feature map;

[0037] The S3 specifically includes: receiving a fourth main feature map, and performing operations on the fourth background feature map to strengthen the background features, so as to obtain a fourth background feature map; convolutionally splicing the fourth background feature map with the third main feature map to obtain a third spliced ​​map, and performing operations on the third spliced ​​map to strengthen the background features, so as to obtain a third background feature map; convolutionally splicing the third background feature map with the second main feature map to obtain a second spliced ​​map, and performing operations on the second spliced ​​map to strengthen the background features, so as to obtain a second background feature map; convolutionally splicing the second background feature map with the first main feature map to obtain a first spliced ​​map, and performing operations on the first spliced ​​map to strengthen the background features, so as to obtain a first background feature map; performing full convolution operations on the first background feature map, and performing mucosal background classification after the full convolution operations to obtain a mucosal background classification result;

[0038] The S4 specifically includes: splicing and fusing the fourth trunk feature map and the fourth background feature map to obtain a fourth fused map, and calculating the fourth fused map so as to enhance the focal lesion information and suppress the background features to obtain a fourth reverse enhancement feature map; splicing and fusing the third trunk feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fused map, and calculating the third fused map so as to enhance the focal lesion information and suppress the background features to obtain a third reverse enhancement feature map; splicing and fusing the second trunk feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fused map, and calculating the second fused map so as to enhance the focal lesion information and suppress the background features to obtain a second reverse enhancement feature map; splicing and fusing the first trunk feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fused map, and calculating the first fused map so as to enhance the focal lesion information and suppress the background features to obtain a first reverse enhancement feature map; performing regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map using the anchor frame method to obtain a focal lesion target detection result.

[0039] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer executes the method for evaluating gastrointestinal lesions as described in the second aspect.

[0040] In a fourth aspect, the present invention provides a digestive tract lesion assessment device for storing and / or running the digestive tract lesion assessment system as described in the first aspect.

[0041] The digestive tract lesion assessment method, system, device and readable storage medium of the present invention are used to extract image features through the first network module to obtain multiple trunk feature maps. The trunk feature map serves as the image basis with obvious features for the second network module and the third network module. The background of the image features is enhanced by the second network module, and the mucosal background is classified after enhancement. The properties of the mucosal background can be accurately known, so that the mucosal background lesions can be accurately classified. The third network module is based on the trunk feature map with all the main image features and the background feature map with enhanced background, and further weakens the focal lesion background and strengthens the focal lesion information, so that the location of the focal lesion can be accurately determined. The present invention firstly enhances the background features to accurately determine the type of large-area lesions, and then further weakens the background of focal lesions and strengthens the information of focal lesions so that local serious lesions will not be ignored when large-area lesions and focal lesions exist at the same time. The present invention avoids generating too many useless detection frames in large-area lesion areas; by first strengthening the background features, and then weakening the background and strengthening the target according to the strengthened background features to determine the focal lesions, the probability of misidentifying the background information as a local lesion and causing excessive labeling is reduced; this method effectively processes the background, and finally weakens the background and strengthens the local lesions, thereby improving the detection capability of local important lesions. Even in a complex lesion background, local important lesions can be detected, which greatly reduces the occurrence of missing key lesion areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 It is a framework diagram of the digestive tract lesion evaluation system of the present invention.

[0044] Figure 2 This is a schematic diagram of the principle of implementing the digestive tract lesion assessment according to the present invention.

[0045] Figure 3The present invention is a flow chart of the method for evaluating digestive tract lesions. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0048] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0049] Figure 1 It is a framework diagram of a digestive tract lesion evaluation system in one embodiment of the present invention.

[0050] See also Figure 1 , the digestive tract lesion assessment system includes an acquisition module 10, a first network module 20, a second network module 30 and a third network module 30;

[0051] The acquisition module 10 is used to acquire the digestive tract image;

[0052] The first network module 20 is used to extract features from the digestive tract image to obtain a plurality of main feature maps;

[0053] The second network module 30 is used to enhance the background features based on the trunk feature map multiple times to generate a background feature map, and calculate the mucosal background classification result according to the background feature map;

[0054] The third network module 30 is used to fuse the main feature map and the background feature map to obtain a fusion map, perform multiple enhancements based on the fusion map to enhance the focal lesion information and suppress the background features, and perform regression prediction based on the enhancement results to obtain the focal lesion target detection result.

[0055] The target of the digestive tract lesion assessment system is set as the focal lesion area. The background described in this article is the background of the target, that is, the background is a non-target. The background features, background, background classification, background information, etc. described in this article all refer to other image areas except the focal lesion area.

[0056] The first network module 20 is used to extract image features. It can be understood that the features include background information and focal lesion information. The first network module 20 extracts image features to pave the way for the second network module 30 and the third network module 30, and serves as the basic information of the second network module 30 and the third network module 30. The second network module 30 is used to enhance the background of image features, and the second network module 30 is used to determine whether there is a large area of ​​lesions. The third network module 30 is used to weaken the background of focal lesions and strengthen the focal lesion information, so as to accurately obtain the focal lesion target frame. For digestive tract images with normal mucosal areas and lesion mucosal areas, the normal area and the large area of ​​lesions are known through the second network module 30, including determining the large area of ​​lesions in the large area of ​​lesions. The trunk feature map and the background feature map are used as the input of the third network module 30, and the background of the focal lesions is weakened to strengthen the focal lesion information, so as to accurately determine the location of the focal lesions.

[0057] In this embodiment, the acquisition module 10 is used to obtain the digestive tract video, that is, to read the digestive tract video signal output by the endoscope device, and then obtain frame-by-frame video frame images, that is, to obtain the digestive tract image.

[0058] like Figure 2, the first network module 20 contains 5 sets of feature extraction units arranged in sequence, and the feature extraction units are used to extract image feature information step by step to obtain a global feature map, a first trunk feature map, a second trunk feature map, a third trunk feature map and a fourth trunk feature map. The first feature extraction unit outputs a global feature map as an input of the second feature extraction unit, and the second to fifth feature extraction units respectively perform convolution operations step by step downward, and the input size is gradually reduced in the process to extract higher-level image feature information. That is, the first network module 20 includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit and a fifth feature extraction unit arranged in sequence, the first feature extraction unit is used to extract the global features of the digestive tract image to obtain a global feature map, the second feature extraction unit is used to extract the image feature information of the global feature map to obtain a second trunk feature map, the third feature extraction unit is used to extract the image feature information of the second trunk feature map to obtain a third trunk feature map, the fourth feature extraction unit is used to extract the image feature information of the third trunk feature map to obtain a fourth trunk feature map, and the fifth feature extraction unit is used to extract the image feature information of the fourth trunk feature map to obtain a fifth trunk feature map. In this embodiment, the first network module 20 uses Res2Net Block to perform feature extraction.

[0059] The second network module 30 is composed of a multi-scale background enhancement unit and a decision unit, which is used to perform reverse attention feature enhancement on the first to fourth trunk feature maps output by the first network module 20, and to fuse the feature maps from bottom to top step by step, and finally use the decision unit to obtain the mucosal background classification result.

[0060] Specifically, the second network module 30 includes four groups of background enhancement units, namely, the fourth background enhancement unit, the third background enhancement unit, the second background enhancement unit, the first background enhancement unit and the decision unit. Each group of background enhancement units consists of a 3×3 convolution layer (first convolution layer), a batch normalization layer (BN layer), an activation layer, a dilated convolution layer, and a reverse attention weighted layer.

[0061] like Figure 2 The starting point of the second network module 30 is the fourth background enhancement unit, which receives the fourth backbone feature map, and performs calculations by the fourth background enhancement unit to enhance its background, thereby obtaining a fourth background feature map.

[0062] The third background enhancement unit receives the fourth background feature map and the third trunk feature map, and obtains a third spliced ​​map through convolution. The third spliced ​​map is operated by the third background enhancement unit to enhance its background, and obtains a third background feature map.

[0063] The second background enhancement unit receives the third background feature map and the second trunk feature map, and obtains a second spliced ​​map through convolution. The second spliced ​​map is operated by the second background enhancement unit to enhance its background, thereby obtaining a second background feature map.

[0064] The first background enhancement unit receives the second background feature map and the first trunk feature map, and obtains a first spliced ​​map through convolution. The first spliced ​​map is operated by the first background enhancement unit to enhance its background, and obtains a first background feature map.

[0065] The decision unit inputs the first background feature map, performs a full convolution operation inside the decision unit, and performs mucosal background classification after the full convolution operation (through a fully connected layer) to obtain a mucosal background classification result. As an example but not a limitation, the classification categories of the mucosal background classification include diffuse lesion mucosa and normal mucosa, or include patchy red mucosa, atrophic mucosa, chicken skin-like mucosa and normal mucosa. The mucosal background classification result is preset during the specific implementation of network training, and is not limited or exhaustive here.

[0066] The third network module 30 is composed of reverse reinforcement units and position regression units of multiple scales, and is used to: fuse the backbone feature map and the background feature map, use the reverse reinforcement unit to enhance the focal lesion information and suppress the background features of the fused map, and in the reverse reinforcement process, perform multi-scale fusion step by step from the reverse reinforcement unit; finally, use the position regression unit to obtain the focal lesion target detection result.

[0067] Specifically, the third network module 30 includes a fourth reverse reinforcement unit, a third reverse reinforcement unit, a second reverse reinforcement unit, and a first reverse reinforcement unit. Each reverse reinforcement unit has a splicing fusion subunit and a reverse reinforcement subunit. As an example but not a limitation, the reverse reinforcement subunit includes a convolution layer (second convolution layer), a Sigmoid activation layer, a ReLu activation layer, a 3 3 residual convolution blocks.

[0068] like Figure 2 The starting point of the third network module 30 starts from the fourth reverse enhancement unit. In detail, the fourth reverse enhancement unit receives the fourth trunk feature map and the fourth background feature map, and its splicing and fusion subunit performs splicing and fusion to obtain a fourth fusion map, which is then input to its reverse enhancement subunit for reverse enhancement, that is, focal lesion information enhancement and background feature suppression to obtain a fourth reverse enhancement feature map;

[0069] The third reverse enhancement unit receives the third trunk feature map, the third background feature map and the fourth reverse enhancement feature map, performs splicing and fusion to obtain a third fusion map, and then performs reverse enhancement, that is, enhances the focal lesion information and suppresses the background features to obtain a third reverse enhancement feature map;

[0070] The second reverse enhancement unit receives the second trunk feature map, the second background feature map and the third reverse enhancement feature map, performs splicing and fusion to obtain a second fusion map, and then performs reverse enhancement, that is, enhances the focal lesion information and suppresses the background features to obtain a second reverse enhancement feature map;

[0071] The first reverse enhancement unit receives the first trunk feature map, the first background feature map and the second reverse enhancement feature map, performs splicing and fusion to obtain a first fusion map, and then performs reverse enhancement, that is, enhances the focal lesion information and suppresses the background features to obtain a first reverse enhancement feature map;

[0072] The position regression unit sequentially inputs the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map into the position regression unit, and the position regression unit uses the anchor frame method to perform regression prediction to obtain the focal lesion target detection result.

[0073] See also Figure 3 The present invention provides a method for evaluating digestive tract lesions, comprising the following steps:

[0074] S1, obtaining digestive tract images;

[0075] S2, extracting features from the digestive tract image to obtain multiple trunk feature maps;

[0076] S3, performing background feature enhancement operations multiple times based on the trunk feature map to generate a background feature map, and calculating the mucosal background classification result according to the background feature map;

[0077] S4. The main feature map and the background feature map are fused to obtain a fusion map. Multiple enhancements are performed based on the fusion map to enhance the focal lesion information and suppress the background features. Regression prediction is performed based on the enhancement results to obtain the focal lesion target detection result.

[0078] In this embodiment, S2 specifically includes: extracting image feature information step by step based on the digestive tract image to sequentially obtain a global feature map, a first trunk feature map, a second trunk feature map, a third trunk feature map and a fourth trunk feature map.

[0079] In this embodiment, S3 specifically includes: receiving a fourth main feature map, and performing operations on it to strengthen the background features to obtain a fourth background feature map; convolutionally splicing the fourth background feature map and the third main feature map to obtain a third spliced ​​map, and performing operations on the third spliced ​​map to strengthen the background features to obtain a third background feature map; convolutionally splicing the third background feature map and the second main feature map to obtain a second spliced ​​map, and performing operations on the second spliced ​​map to strengthen the background features to obtain a second background feature map; convolutionally splicing the second background feature map and the first main feature map to obtain a first spliced ​​map, and performing operations on the first spliced ​​map to strengthen the background features to obtain a first background feature map; performing a full convolution operation on the first background feature map, and performing mucosal background classification after the full convolution operation to obtain a mucosal background classification result.

[0080] In this embodiment, the S4 specifically includes: splicing and fusing the fourth trunk feature map and the fourth background feature map to obtain a fourth fused map, and calculating the fourth fused map to enhance the focal lesion information and suppress the background features to obtain a fourth reverse enhancement feature map; splicing and fusing the third trunk feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fused map, and calculating the third fused map to enhance the focal lesion information and suppress the background features to obtain a third reverse enhancement feature map; splicing and fusing the second trunk feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fused map, and calculating the second fused map to enhance the focal lesion information and suppress the background features to obtain a second reverse enhancement feature map; splicing and fusing the first trunk feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fused map, and calculating the first fused map to enhance the focal lesion information and suppress the background features to obtain a first reverse enhancement feature map; using the anchor frame method to perform regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map to obtain a focal lesion target detection result.

[0081] When the digestive tract lesion assessment method is specifically implemented, reference may be made to the specific implementation of the digestive tract lesion assessment system in any of the above embodiments, which will not be described in detail here.

[0082] The present invention provides a digestive tract lesion assessment device, which is used to store and / or run the digestive tract lesion assessment system. The device described in this embodiment includes but is not limited to a storage medium, a computer, a server, a mobile device, and the like.

[0083] The present invention also provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes each step of the method for evaluating digestive tract lesions described in any of the above embodiments.

[0084] According to the method of the present invention, an electronic device can be implemented, which includes: a memory; one or more processors; one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing a method for evaluating gastrointestinal lesions according to any of the above embodiments.

[0085] The effects of the digestive tract lesion assessment method, system, device and readable storage medium of the present invention are as follows: the first network module 20 is used to extract image features to obtain multiple trunk feature maps, the trunk feature maps serve as the image basis with obvious features of the second network module 30 and the third network module 30, the background of the image features is enhanced by the second network module 30, and the mucosal background classification is performed after the background enhancement, so that the mucosal background properties can be accurately known, so that the mucosal background lesions are accurately classified, and accurate classification results of large-area lesions are obtained; the third network module 30 comprehensively utilizes the trunk feature map and the background feature map, and based on the trunk feature map with all the main image features and the background feature map with enhanced background, further weakens the focal lesion background and strengthens the focal lesion information, so that the location of the focal lesion can be accurately determined. The present invention first determines the type of large-area lesions based on feature extraction, and then further weakens the background of focal lesions and strengthens the information of focal lesions so that local serious lesions will not be ignored when large-area lesions and focal lesions exist at the same time. The present invention avoids generating too many useless detection frames in large-area lesion areas; by first strengthening the background information and then determining the information of focal lesions based on the strengthened background information, the probability of misidentifying the background information as local lesions and causing excessive labeling is reduced; this method effectively processes the background, and finally weakens the background to strengthen the local lesions, which improves the detection ability of local important lesions. Even in a complex lesion background, local important lesions can be detected, which greatly reduces the occurrence of missing key lesion areas. Compared with the prior art, the present invention can better enable doctors to work.

[0086] In addition, the present invention uses a specifically designed network module to achieve high accuracy in mucosal background classification results, and can more accurately detect focal lesion target detection results.

[0087] The method and system designed by the present invention can be applied to endoscopic examination of digestive tract mucosal images. The target and background classification required to be identified by the present invention can be set, such as mucosal background classification, which can be further set to: normal mucosa, mucosal edema, vascular transparency, patchy redness, ridged redness, chicken skin-like changes, gastric mucosal atrophy, intestinal metaplasia, etc.; focal target detection results can be further set to ulcer targets, erosion targets, polyp targets, etc., so as to further support doctors in clinical judgment of gastritis typing, HP infection grading, early cancer detection, etc., so that the present invention can better guide doctors' work.

[0088] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A digestive tract lesion assessment system, characterized in that: include: An acquisition module, used for acquiring digestive tract images; The first network module is used to extract features from the digestive tract image to obtain a plurality of main feature maps; The second network module is used to perform background feature enhancement operations multiple times based on the trunk feature map to generate a background feature map, and calculate the mucosal background classification result according to the background feature map; The third network module is used to fuse the main feature map and the background feature map to obtain a fusion map, perform multiple enhancements based on the fusion map to enhance the focal lesion information and suppress the background features, and perform regression prediction based on the enhancement results to obtain the focal lesion target detection result; The first network module comprises five feature extraction units arranged in sequence, and the feature extraction units are used to extract image feature information step by step to obtain a global feature map, a first trunk feature map, a second trunk feature map, a third trunk feature map and a fourth trunk feature map in sequence; The second network module includes: A fourth background enhancement unit, used for receiving the fourth trunk feature map and performing operations on the fourth trunk feature map to enhance the background features, thereby obtaining a fourth background feature map; a third background enhancement unit, configured to convolute and splice the fourth background feature map and the third backbone feature map to obtain a third spliced ​​map, and perform operations on the third spliced ​​map to enhance background features to obtain a third background feature map; A second background enhancement unit is used for convolutionally splicing the third background feature map and the second backbone feature map to obtain a second spliced ​​map, and performing operation on the second spliced ​​map to enhance background features to obtain a second background feature map; A first background enhancement unit is used for convolutionally splicing the second background feature map and the first backbone feature map to obtain a first spliced ​​map, and performing operation on the first spliced ​​map to enhance background features to obtain a first background feature map; A decision unit, configured to perform a full convolution operation on the first background feature map, and perform mucosal background classification after the full convolution operation to obtain a mucosal background classification result; The third network module includes: a fourth reverse enhancement unit, configured to concatenate and fuse the fourth trunk feature map and the fourth background feature map to obtain a fourth fusion map, and to calculate the fourth fusion map so as to enhance the focal lesion information and suppress the background features, thereby obtaining a fourth reverse enhancement feature map; A third reverse enhancement unit is used to splice and fuse the third trunk feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fusion map, and calculate the third fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a third reverse enhancement feature map; The second reverse enhancement unit is used to splice and fuse the second trunk feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fusion map, and calculate the second fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a second reverse enhancement feature map; A first reverse enhancement unit is used to splice and fuse the first trunk feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fusion map, and calculate the first fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a first reverse enhancement feature map; The position regression unit is used to perform regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map using the anchor frame method to obtain focal lesion target detection results.

2. The digestive tract lesion evaluation system according to claim 1, characterized in that: The fourth background enhancement unit, the third background enhancement unit, the second background enhancement unit and the first background enhancement unit all include a convolution layer, a batch normalization layer, an activation layer, an expansion convolution layer and a reverse attention weighted layer.

3. The digestive tract lesion evaluation system according to claim 1, characterized in that: The fourth reverse reinforcement unit, the third reverse reinforcement unit, the second reverse reinforcement unit and the first reverse reinforcement unit each include a convolution layer, a Sigmoid activation layer, a ReLu activation layer, and a residual convolution block.

4. A method for evaluating digestive tract lesions, characterized in that: include: S1, obtaining digestive tract images; S2, extracting features from the digestive tract image to obtain multiple trunk feature maps; S3, performing background feature enhancement operations multiple times based on the trunk feature map to generate a background feature map, and calculating the mucosal background classification result according to the background feature map; S4, fusing the main feature map and the background feature map to obtain a fusion map, performing multiple enhancements based on the fusion map to enhance the focal lesion information and suppress the background features, and performing regression prediction based on the enhancement results to obtain the focal lesion target detection result; The S2 specifically includes: extracting image feature information step by step based on the digestive tract image to sequentially obtain a global feature map, a first trunk feature map, a second trunk feature map, a third trunk feature map, and a fourth trunk feature map; The S3 specifically includes: receiving a fourth main feature map, and performing operations on the fourth background feature map to strengthen the background features, so as to obtain a fourth background feature map; convolutionally splicing the fourth background feature map with the third main feature map to obtain a third spliced ​​map, and performing operations on the third spliced ​​map to strengthen the background features, so as to obtain a third background feature map; convolutionally splicing the third background feature map with the second main feature map to obtain a second spliced ​​map, and performing operations on the second spliced ​​map to strengthen the background features, so as to obtain a second background feature map; convolutionally splicing the second background feature map with the first main feature map to obtain a first spliced ​​map, and performing operations on the first spliced ​​map to strengthen the background features, so as to obtain a first background feature map; performing full convolution operations on the first background feature map, and performing mucosal background classification after the full convolution operations to obtain a mucosal background classification result; The S4 specifically includes: splicing and fusing the fourth trunk feature map and the fourth background feature map to obtain a fourth fused map, and calculating the fourth fused map so as to enhance the focal lesion information and suppress the background features to obtain a fourth reverse enhancement feature map; splicing and fusing the third trunk feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fused map, and calculating the third fused map so as to enhance the focal lesion information and suppress the background features to obtain a third reverse enhancement feature map; splicing and fusing the second trunk feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fused map, and calculating the second fused map so as to enhance the focal lesion information and suppress the background features to obtain a second reverse enhancement feature map; splicing and fusing the first trunk feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fused map, and calculating the first fused map so as to enhance the focal lesion information and suppress the background features to obtain a first reverse enhancement feature map; performing regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map using the anchor frame method to obtain a focal lesion target detection result.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions, which, when executed on a computer, enable the computer to execute a method for evaluating digestive tract lesions as claimed in claim 4.

6. A digestive tract lesion assessment device, characterized in that: Used to store and / or run the digestive tract lesion assessment system as described in any one of claims 1 to 3.

Images