Gastrointestinal lesion assessment method, system and apparatus, and readable storage medium

Abstract

Disclosed in the present invention are a gastrointestinal lesion assessment method, system and apparatus, and a readable storage medium. The system comprises: an acquisition module configured to acquire a gastrointestinal tract image; a first network module configured to perform feature extraction on the gastrointestinal tract image to obtain a plurality of backbone feature maps; a second network module configured to perform a plurality of background feature enhancement operations on the basis of the backbone feature maps so as to generate background feature maps, and calculate a mucosal background classification result on the basis of the background feature maps; and a third network module configured to fuse the backbone feature maps and the background feature maps, perform a plurality of enhancement operations so as to enhance focal lesion information and suppress background features, and perform regression prediction to obtain a focal lesion target detection result. In the present invention, mucosal background classification results can be accurately determined by means of background enhancement, and focal significant lesions can be detected by means of a plurality of target enhancement and non-target suppression operations. The present invention can be applied to gastrointestinal mucosal imaging under endoscopy, and allows for further configuration of targets needing to be recognized and classified, providing better guidance for doctors in work.

Description

Method, system, device and readable storage medium for evaluating lesions in digestive tract

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese patent application No. 202411223074.1, filed with the China Patent Office on September 3, 2024, and entitled “Method, system, device and readable storage medium for evaluating lesions in digestive tract”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the technical field of lesion evaluation, in particular to a method, system, device and readable storage medium for evaluating lesions in digestive tract. BACKGROUND

[0004] At present, digestive endoscopy has become one of the standard methods for diagnosing lesions in the digestive tract. With the development of digestive endoscopy technology, doctors can more accurately identify and evaluate lesions in the mucosa of the digestive tract. However, in actual operation, mucosal lesions are complex and diverse, i.e., there are diffuse lesions and possibly coexisting focal lesions, which poses certain challenges for doctors to accurately diagnose under endoscopy. Diffuse lesions often cover a large area, for example, there are multiple patchy redness, mucosal atrophy or extensive inflammatory response in the gastric mucosa, and there are large area mucosal rough changes and extensive esophagitis in the esophageal mucosa, while focal lesions such as esophageal and gastric ulcers, erosions or polyps may be hidden in these large area lesions and are difficult to be effectively identified.

[0005] The prior art usually relies on a single classification or detection model to process endoscopic images of the digestive tract. Although these methods can provide a certain degree of lesion identification and positioning, they have the following shortcomings when dealing with the coexistence of complex lesions:

[0006] (1) Lack of processing capability for the coexistence of large area lesions (global lesions) and focal lesions: existing classification models mainly focus on the classification of global lesions, while object detection models focus on the identification of local regions. This separation approach can easily cause the classification model to ignore local severe lesions or the detection model to generate too many useless detection boxes in the large area lesion region, thereby interfering with the identification of key lesions by doctors.

[0007] (2) In the area covered by large area lesions such as patchy redness, existing detection methods are prone to misidentifying background information as local lesions, leading to excessive labeling and increasing the diagnostic burden on doctors.

[0008] (3) The prior art generally fails to effectively distinguish and process background and foreground features, especially in a complex lesion background, which easily leads to insufficient detection capability of the model for local important lesions, resulting in omission of key lesion regions.

[0009] Therefore, in view of the above problem that the detection capability of the local important lesions needs to be improved, there is an urgent need to develop a digestive tract lesion evaluation method, system, device and readable storage medium. SUMMARY

[0010] In view of the above problem that the detection capability of the digestive tract lesions needs to be improved, the present application provides a digestive tract lesion evaluation method, system, device and readable storage medium.

[0011] The technical solution adopted by the present application to solve the technical problems is as follows:

[0012] In a first aspect, the present application provides a digestive tract lesion evaluation system, comprising:

[0013] An acquisition module for acquiring a digestive tract image;

[0014] A first network module for performing feature extraction on the digestive tract image to obtain a plurality of backbone feature maps;

[0015] A second network module for performing background feature enhancement operations multiple times based on the backbone feature maps to generate a background feature map, and calculating a mucosal background classification result according to the background feature map;

[0016] A third network module for fusing the backbone feature maps and the background feature map to obtain a fusion map, performing multiple enhancements based on the fusion map to enhance local lesion information and suppress background features, and performing regression prediction according to the enhancement result to obtain a local lesion target detection result.

[0017] In a preferred embodiment, the first network module comprises five feature extraction units arranged in sequence, which are used to sequentially extract image feature information to obtain a global feature map, a first backbone feature map, a second backbone feature map, a third backbone feature map and a fourth backbone feature map.

[0018] In a preferred embodiment, the second network module comprises:

[0019] A fourth background enhancement unit for receiving the fourth backbone feature map and performing operations thereon to enhance the background features to obtain a fourth background feature map;

[0020] A third background enhancement unit for convolving and splicing the fourth background feature map and the third backbone feature map to obtain a third splicing map, and performing operations on the third splicing map to enhance the background features to obtain a third background feature map;

[0021] a second background strengthening unit configured to perform convolutional concatenation on the third background feature map and the second backbone feature map to obtain a second concatenated map, and perform operation on the second concatenated map to strengthen the background feature, so as to obtain a second background feature map;

[0022] a first background strengthening unit configured to perform convolutional concatenation on the second background feature map and the first backbone feature map to obtain a first concatenated map, and perform operation on the first concatenated map to strengthen the background feature, so as to obtain a first background feature map;

[0023] a decision unit configured to perform full convolution operation on the first background feature map, and perform mucosa background classification on the first background feature map after the full convolution operation to obtain a mucosa background classification result.

[0024] In a preferred embodiment, the classification categories of the mucosa background classification include diffuse lesion mucosa and normal mucosa, or include patchy erythema mucosa, atrophic mucosa, chicken skin-like change mucosa and normal mucosa.

[0025] In a preferred embodiment, the fourth background strengthening unit, the third background strengthening unit, the second background strengthening unit and the first background strengthening unit each include a convolutional layer, a batch normalization layer, an activation layer, a dilated convolutional layer and a reverse attention weighting layer.

[0026] In a preferred embodiment, the third network module includes:

[0027] a fourth reverse strengthening unit configured to perform concatenation fusion on the fourth backbone feature map and the fourth background feature map to obtain a fourth fusion map, and perform calculation on the fourth fusion map to enhance the focal lesion information and suppress the background feature, so as to obtain a fourth reverse strengthening feature map;

[0028] a third reverse strengthening unit configured to perform concatenation fusion on the third backbone feature map, the third background feature map and the fourth reverse strengthening feature map to obtain a third fusion map, and perform calculation on the third fusion map to enhance the focal lesion information and suppress the background feature, so as to obtain a third reverse strengthening feature map;

[0029] a second reverse strengthening unit configured to perform concatenation fusion on the second backbone feature map, the second background feature map and the third reverse strengthening feature map to obtain a second fusion map, and perform calculation on the second fusion map to enhance the focal lesion information and suppress the background feature, so as to obtain a second reverse strengthening feature map;

[0030] a first reverse strengthening unit configured to perform concatenation fusion on the first backbone feature map, the first background feature map and the second reverse strengthening feature map to obtain a first fusion map, and perform calculation on the first fusion map to enhance the focal lesion information and suppress the background feature, so as to obtain a first reverse strengthening feature map;

[0031] A position regression unit is configured to perform regression prediction on the first, second and third inverse reinforced feature maps by using an anchor box method to obtain a focal lesion target detection result.

[0032] In a preferred embodiment, the fourth inverse reinforcement unit, the third inverse reinforcement unit, the second inverse reinforcement unit and the first inverse reinforcement unit each include a convolution layer, a Sigmoid activation layer, a ReLu activation layer and a residual convolution block.

[0033] In a second aspect, the present application provides a method for evaluating digestive tract lesions, comprising:

[0034] S1, obtaining a digestive tract image;

[0035] S2, performing feature extraction on the digestive tract image to obtain a plurality of backbone feature maps;

[0036] S3, performing background feature enhancement operation based on the backbone feature maps multiple times to generate a background feature map, and calculating a mucosal background classification result according to the background feature map;

[0037] S4, fusing the backbone feature map and the background feature map to obtain a fusion map, performing multiple reinforcements based on the fusion map to enhance focal lesion information and suppress background features, and performing regression prediction according to the reinforcement result to obtain a focal lesion target detection result.

[0038] In a preferred embodiment, the S2 specifically comprises sequentially obtaining a global feature map, a first backbone feature map, a second backbone feature map, a third backbone feature map and a fourth backbone feature map based on the image feature information extracted from the digestive tract image at different levels;

[0039] The S3 specifically comprises receiving the fourth backbone feature map and performing operation thereon to enhance the background features to obtain a fourth background feature map; convoluting and splicing the fourth background feature map and the third backbone feature map to obtain a third splicing map, performing operation on the third splicing map to enhance the background features to obtain a third background feature map; convoluting and splicing the third background feature map and the second backbone feature map to obtain a second splicing map, performing operation on the second splicing map to enhance the background features to obtain a second background feature map; convoluting and splicing the second background feature map and the first backbone feature map to obtain a first splicing map, performing operation on the first splicing map to enhance the background features to obtain a first background feature map; and performing full convolution operation on the first background feature map, and performing mucosal background classification after the full convolution operation to obtain a mucosal background classification result.

[0040] The S4 specifically comprises: splicing and fusing the fourth main backbone feature map and the fourth background feature map to obtain a fourth fusion map, and performing calculation on the fourth fusion map to enhance the focal lesion information and suppress the background features, to obtain a fourth reverse enhancement feature map; splicing and fusing the third main backbone feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fusion map, and performing calculation on the third fusion map to enhance the focal lesion information and suppress the background features, to obtain a third reverse enhancement feature map; splicing and fusing the second main backbone feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fusion map, and performing calculation on the second fusion map to enhance the focal lesion information and suppress the background features, to obtain a second reverse enhancement feature map; splicing and fusing the first main backbone feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fusion map, and performing calculation on the first fusion map to enhance the focal lesion information and suppress the background features, to obtain a first reverse enhancement feature map; and performing regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map by using an anchor box method, to obtain a focal lesion target detection result.

[0041] In a third aspect, the present application provides a computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method for evaluating a digestive tract lesion according to the second aspect.

[0042] In a fourth aspect, the present application provides a digestive tract lesion evaluation device for storing and / or running the digestive tract lesion evaluation system according to the first aspect.

[0043] The digestive tract lesion evaluation method, system, device and readable storage medium extract image features through the first network module to obtain a plurality of backbone feature maps, the backbone feature maps are used as the image basis with obvious features of the second network module and the third network module, the background of the image features is enhanced through the second network module, and after the enhancement, the mucosa background classification is performed, so that the mucosa background properties can be accurately known, and the mucosa background lesion classification is accurate; the third network module is based on the backbone feature map with all main image features and the background feature map with the enhanced background, and further weakens the background of the focal lesion and strengthens the focal lesion information, so that the position of the focal lesion can be accurately determined. The background feature is first strengthened to accurately determine the large-area lesion type, and then the background of the focal lesion is further weakened and the focal lesion information is strengthened, so that the local serious lesion will not be ignored when the large-area lesion and the focal lesion exist at the same time, the application avoids generating too many useless detection boxes in the large-area lesion area; through the way of strengthening the background feature first and then weakening the background and strengthening the target according to the strengthened background feature to determine the focal lesion, the probability of misidentifying the background information as the local lesion and then leading to excessive labeling is reduced; this way effectively processes the background, and finally weakens the background and strengthens the local lesion, improves the detection ability of the local important lesion, and even in a complex lesion background, the local important lesion can be detected, and the situation of missing the key lesion area is greatly reduced. BRIEF DESCRIPTION OF DRAWINGS DETAILED DESCRIPTION OF THE INVENTION BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0045] Fig. 1 is a framework diagram of the digestive tract lesion evaluation system of the present application.

[0046] Fig. 2 is a principle diagram for realizing the digestive tract lesion evaluation of the present application.

[0047] Fig. 3 is a flowchart of the digestive tract lesion evaluation method of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-described drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so construed can be interchanged, under appropriate circumstances, to describe the embodiments of the present application described herein in other than the particular order illustrated or described herein. Moreover, the terms "comprising" and "including" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that comprises a list of steps or units can not necessarily be limited to those steps or units which are clearly recited, but can include other not expressly recited steps or units, or steps or units inherent to such process, method, product or apparatus.

[0050] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and for the same or similar concepts or processes, some examples may not be described again.

[0051] Figure 1 is a framework diagram of a digestive tract lesion evaluation system in an embodiment of the present application.

[0052] Referring to Figure 1, the digestive tract lesion evaluation system comprises an acquisition module 10, a first network module 20, a second network module 30 and a third network module 30;

[0053] The acquisition module 10 is used to acquire digestive tract images;

[0054] The first network module 20 is used to extract features from the digestive tract images to obtain a plurality of backbone feature maps;

[0055] The second network module 30 is used to enhance the background features based on the backbone feature maps multiple times to generate a background feature map, and calculate a mucosal background classification result according to the background feature map;

[0056] The third network module 30 is used to fuse the backbone feature map and the background feature map to obtain a fusion map, and to perform multiple enhancements based on the fusion map to enhance the focal lesion information and suppress the background features, and to perform regression prediction according to the enhanced result to obtain a focal lesion target detection result.

[0057] The target of the digestive tract lesion evaluation system is set to the focal lesion area, and the background described herein is the background of the target, i.e. the background is the non-target, and the background features, background, background classification, background information, etc. described herein all refer to other image areas other than the focal lesion area.

[0058] The first network module 20 is configured to extract image features, which can include background information and focal lesion information. The first network module 20 extracts image features to lay a foundation for the second network module 30 and the third network module 30, and provide basic information for the second network module 30 and the third network module 30. The second network module 30 is configured to enhance the background of the image features, and determine whether there is a large area lesion through the second network module 30. The third network module 30 is configured to weaken the background of the focal lesion, and strengthen the focal lesion information, so that the position of the focal lesion is accurately determined. For a digestive tract image having a normal mucosa region and a lesion mucosa region, the second network module 30 is configured to determine the normal region and the large area lesion region, and determine the large area lesion category of the large area lesion region. The main feature map and the background feature map are input into the third network module 30, the background of the focal lesion is weakened, the focal lesion information is strengthened, and the position of the focal lesion is accurately determined.

[0059] In this embodiment, the acquisition module 10 is configured to obtain a digestive tract video, i.e., read a digestive tract video signal output by an endoscope device, and obtain a video frame image frame by frame, i.e., obtain a digestive tract image.

[0060] As shown in FIG. 2, the first network module 20 includes five groups of feature extraction units arranged in sequence. The feature extraction units are configured to sequentially extract image feature information to obtain a global feature map, a first main feature map, a second main feature map, a third main feature map, and a fourth main feature map. The first feature extraction unit outputs the global feature map as input of the second feature extraction unit. The second to fifth feature extraction units are configured to perform convolution operations step by step, and the input size is reduced step by step in the process to extract more advanced image feature information. That is, the first network module 20 includes the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, the fourth feature extraction unit, and the fifth feature extraction unit arranged in sequence. The first feature extraction unit is configured to extract global features of the digestive tract image to obtain the global feature map. The second feature extraction unit is configured to extract image feature information of the global feature map to obtain the second main feature map. The third feature extraction unit is configured to extract image feature information of the second main feature map to obtain the third main feature map. The fourth feature extraction unit is configured to extract image feature information of the third main feature map to obtain the fourth main feature map. The fifth feature extraction unit is configured to extract image feature information of the fourth main feature map to obtain the fifth main feature map. In this embodiment, the first network module 20 uses Res2Net Block to extract features.

[0061] The second network module 30 is composed of multi-scale background enhancement units and a decision unit, and is used for performing feature enhancement of the first to fourth backbone feature maps output by the first network module 20 through reverse attention, and fusing the feature maps step by step from bottom to top, and finally obtaining a mucosa background classification result through the decision unit.

[0062] Specifically, the second network module 30 includes four groups of background enhancement units, i.e., a fourth background enhancement unit, a third background enhancement unit, a second background enhancement unit, a first background enhancement unit and a decision unit arranged in sequence. Each group of background enhancement units is composed of a 3x3 convolution layer (first convolution layer), a batch normalization layer (BN layer), an activation layer, an expansion convolution layer and a reverse attention weighting layer.

[0063] As shown in FIG. 2, the starting point of the second network module 30 is the fourth background enhancement unit, which receives the fourth backbone feature map, and obtains a fourth background feature map through the operation of the fourth background enhancement unit to enhance the background thereof.

[0064] The third background enhancement unit receives the fourth background feature map and the third backbone feature map, obtains a third splicing map through convolution splicing, and obtains a third background feature map through the operation of the third background enhancement unit to enhance the background of the third splicing map.

[0065] The second background enhancement unit receives the third background feature map and the second backbone feature map, obtains a second splicing map through convolution splicing, and obtains a second background feature map through the operation of the second background enhancement unit to enhance the background of the second splicing map.

[0066] The first background enhancement unit receives the second background feature map and the first backbone feature map, obtains a first splicing map through convolution splicing, and obtains a first background feature map through the operation of the first background enhancement unit to enhance the background of the first splicing map.

[0067] The decision unit inputs the first background feature map, and performs full convolution operation inside. After the full convolution operation (through a full connection layer), mucosa background classification is performed to obtain a mucosa background classification result. As an example but not limitation, the classification categories of the mucosa background classification include diffuse lesion mucosa and normal mucosa, or include patchy redness mucosa, atrophic mucosa, chicken skin-like change mucosa and normal mucosa. The mucosa background classification result is determined according to the preset during network training, and is not limited and exhaustive here.

[0068] The third network module 30 is composed of multiple scales of reverse enhancement units and a position regression unit, and is used for: fusing the backbone feature map and the background feature map, performing focal lesion information enhancement and background feature suppression on the fused map through the reverse enhancement unit, and performing multi-scale fusion step by step from the reverse enhancement unit in the reverse enhancement process; and finally obtaining a focal lesion target detection result through the position regression unit.

[0069] Specifically, the third network module 30 comprises a fourth reverse reinforcement unit, a third reverse reinforcement unit, a second reverse reinforcement unit and a first reverse reinforcement unit. Each reverse reinforcement unit has a splicing and fusion subunit and a reverse reinforcement subunit. As an example but not limited, the reverse reinforcement subunit internally comprises a convolution layer (second convolution layer), a Sigmoid activation layer, a ReLu activation layer and a 3*3 residual convolution block.

[0070] As shown in FIG. 2, the third network module 30 starts from the fourth reverse reinforcement unit. In detail, the fourth reverse reinforcement unit receives the fourth backbone feature map and the fourth background feature map, and after the splicing and fusion subunit of the fourth reverse reinforcement unit performs splicing and fusion to obtain a fourth fusion map, the fourth fusion map is input to the reverse reinforcement subunit of the fourth reverse reinforcement unit for reverse reinforcement, i.e. focal lesion information enhancement and background feature suppression, to obtain a fourth reverse reinforcement feature map;

[0071] The third reverse reinforcement unit receives the third backbone feature map, the third background feature map and the fourth reverse reinforcement feature map, performs splicing and fusion to obtain a third fusion map, and then performs reverse reinforcement, i.e. focal lesion information enhancement and background feature suppression, to obtain a third reverse reinforcement feature map;

[0072] The second reverse reinforcement unit receives the second backbone feature map, the second background feature map and the third reverse reinforcement feature map, performs splicing and fusion to obtain a second fusion map, and then performs reverse reinforcement, i.e. focal lesion information enhancement and background feature suppression, to obtain a second reverse reinforcement feature map;

[0073] The first reverse reinforcement unit receives the first backbone feature map, the first background feature map and the second reverse reinforcement feature map, performs splicing and fusion to obtain a first fusion map, and then performs reverse reinforcement, i.e. focal lesion information enhancement and background feature suppression, to obtain a first reverse reinforcement feature map;

[0074] The position regression unit sequentially inputs the first reverse reinforcement feature map, the second reverse reinforcement feature map and the third reverse reinforcement feature map into the position regression unit, and the position regression unit uses the anchor box method to perform regression prediction to obtain a focal lesion target detection result.

[0075] Referring to FIG. 3, the present application provides a digestive tract lesion evaluation method, comprising the following steps:

[0076] S1, obtaining a digestive tract image;

[0077] S2, performing feature extraction on the digestive tract image to obtain a plurality of backbone feature maps;

[0078] S3, performing background feature strengthening operations multiple times based on the backbone feature maps to generate background feature maps, and calculating a mucosa background classification result according to the background feature maps;

[0079] S4, fusing the backbone feature maps and the background feature maps to obtain a fusion map, performing multiple strengthening based on the fusion map to enhance the focal lesion information and suppress the background features, and performing regression prediction according to the strengthened result to obtain a focal lesion target detection result.

[0080] In this embodiment, the S2 specifically includes: sequentially obtaining a global feature map, a first backbone feature map, a second backbone feature map, a third backbone feature map and a fourth backbone feature map based on the image features of the digestive tract image extracted in stages.

[0081] In this embodiment, the S3 specifically includes: receiving the fourth backbone feature map and performing operations thereon to strengthen the background features, to obtain a fourth background feature map; convoluting and splicing the fourth background feature map and the third backbone feature map to obtain a third splicing map, and performing operations on the third splicing map to strengthen the background features, to obtain a third background feature map; convoluting and splicing the third background feature map and the second backbone feature map to obtain a second splicing map, and performing operations on the second splicing map to strengthen the background features, to obtain a second background feature map; convoluting and splicing the second background feature map and the first backbone feature map to obtain a first splicing map, and performing operations on the first splicing map to strengthen the background features, to obtain a first background feature map; and performing full convolution operations on the first background feature map, and performing mucosa background classification after the full convolution operations to obtain a mucosa background classification result.

[0082] In this embodiment, the S4 specifically includes: splicing and fusing the fourth backbone feature map and the fourth background feature map to obtain a fourth fusion map, and performing calculations on the fourth fusion map to enhance the focal lesion information and suppress the background features, to obtain a fourth reverse strengthening feature map; splicing and fusing the third backbone feature map, the third background feature map and the fourth reverse strengthening feature map to obtain a third fusion map, and performing calculations on the third fusion map to enhance the focal lesion information and suppress the background features, to obtain a third reverse strengthening feature map; splicing and fusing the second backbone feature map, the second background feature map and the third reverse strengthening feature map to obtain a second fusion map, and performing calculations on the second fusion map to enhance the focal lesion information and suppress the background features, to obtain a second reverse strengthening feature map; splicing and fusing the first backbone feature map, the first background feature map and the second reverse strengthening feature map to obtain a first fusion map, and performing calculations on the first fusion map to enhance the focal lesion information and suppress the background features, to obtain a first reverse strengthening feature map; and performing regression prediction on the first reverse strengthening feature map, the second reverse strengthening feature map and the third reverse strengthening feature map by using an anchor box method, to obtain a focal lesion target detection result.

[0083] The digestive tract lesion evaluation method in the specific implementation can refer to the specific implementation mode of the digestive tract lesion evaluation system in any of the above embodiments, and details are not repeated here.

[0084] The present application provides a digestive tract lesion evaluation device for storing and / or running the digestive tract lesion evaluation system. The device of the present embodiment includes but is not limited to storage media, computers, servers, mobile devices, etc.

[0085] The present application also provides a computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the steps of the digestive tract lesion evaluation method described in any of the above embodiments.

[0086] The method according to the present application can realize an electronic device, which comprises a memory, one or more processors, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing a digestive tract lesion evaluation method according to any of the above embodiments.

[0087] The digestive tract lesion evaluation method, system, device and readable storage medium of the present application have the following effects: the first network module 20 is used to extract image features to obtain a plurality of backbone feature maps, the backbone feature maps are used as the basis of images with obvious features of the second network module 30 and the third network module 30, the second network module 30 is used to enhance the background of the image features, the mucosa background classification is performed after the background enhancement, the mucosa background properties can be accurately obtained, the mucosa background lesion classification is accurate, and the classification result of the large-area lesion is accurate; the third network module 30 comprehensively utilizes the backbone feature maps and the background feature maps, based on the backbone feature maps with all main image features and the background feature maps with enhanced background, the focal lesion background is further weakened and the focal lesion information is further strengthened, so that the position of the focal lesion can be accurately determined. The present application first determines the large-area lesion type based on feature extraction, and then further weakens the focal lesion background and strengthens the focal lesion information, so that the local serious lesion will not be ignored when the large-area lesion and the focal lesion exist at the same time. The present application avoids generating too many useless detection boxes in the large-area lesion area. By strengthening the background information first and then determining the focal lesion information according to the strengthened background information, the probability of misidentifying the background information as a local lesion and then leading to excessive labeling is reduced. This way effectively processes the background, and finally weakens the background and strengthens the local lesion, improves the detection ability of the local important lesion, and even in a complex lesion background, the local important lesion can be detected, and the situation of missing the key lesion area is greatly reduced. Compared with the prior art, the present application can better assist doctors in work.

[0088] In addition, the mucosa background classification result is accurate, and the local lesion target detection result can be detected more accurately.

[0089] The method and system designed in the application can be applied to the digestive tract mucosa image under endoscopy. The target and background classification required by the application can be set, such as mucosa background classification, which can be further set as normal mucosa, mucosal edema, vascular transparency, macular erythema, ridge erythema, chicken skin-like change, gastric mucosa atrophy, intestinal epithelial metaplasia, etc. The local target detection result can be further set as ulcer target, erosion target, polyp target, etc. Such setting can further support the doctor to judge the gastric inflammation type, HP infection classification, early cancer detection, etc. in the clinic, so that the application can better guide the doctor work.

[0090] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0091] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the application, and not to limit them; under the idea of the application, the technical features in the above examples or different examples can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the application as described above. In order to be brief, they are not provided in details; although the application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: the technical solutions recorded in the foregoing examples can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.

Claims

1. A digestive tract lesion evaluation system characterized by comprising: The method comprises the following steps: An acquisition module is configured to acquire a digestive tract image; A first network module is configured to perform feature extraction on the digestive tract image to obtain a plurality of backbone feature maps; A second network module is configured to perform background feature enhancement operations multiple times based on the backbone feature maps to generate a background feature map, and calculate a mucosal background classification result based on the background feature map; A third network module is configured to fuse the backbone feature maps and the background feature map to obtain a fusion map, perform multiple enhancements based on the fusion map to enhance focal lesion information and suppress background features, and perform regression prediction based on the enhanced result to obtain a focal lesion target detection result.

2. The digestive tract pathology assessment system of claim 1, wherein, The first network module comprises five feature extraction units arranged in sequence, which are configured to sequentially extract image feature information to obtain a global feature map, a first backbone feature map, a second backbone feature map, a third backbone feature map, and a fourth backbone feature map.

3. The digestive pathologies assessment system of claim 2, wherein, The second network module comprises: A fourth background enhancement unit is configured to receive the fourth backbone feature map and perform operations thereon to enhance background features, thereby obtaining a fourth background feature map; A third background enhancement unit is configured to convolve and splice the fourth background feature map and the third backbone feature map to obtain a third spliced map, and perform operations on the third spliced map to enhance background features, thereby obtaining a third background feature map; A second background enhancement unit is configured to convolve and splice the third background feature map and the second backbone feature map to obtain a second spliced map, and perform operations on the second spliced map to enhance background features, thereby obtaining a second background feature map; A first background enhancement unit is configured to convolve and splice the second background feature map and the first backbone feature map to obtain a first spliced map, and perform operations on the first spliced map to enhance background features, thereby obtaining a first background feature map; A decision unit is configured to perform full convolution operations on the first background feature map, and perform mucosal background classification after the full convolution operations to obtain a mucosal background classification result.

4. The digestive pathologies assessment system of claim 3, wherein, The fourth background enhancement unit, the third background enhancement unit, the second background enhancement unit, and the first background enhancement unit each comprise a convolution layer, a batch normalization layer, an activation layer, a dilated convolution layer, and an inverse attention weighting layer.

5. The digestive tract lesion evaluation system according to claim 3, wherein The third network module comprises: A fourth inverse reinforcement unit is configured to splice and fuse the fourth backbone feature map and the fourth background feature map to obtain a fourth fusion map, and perform calculations on the fourth fusion map to enhance focal lesion information and suppress background features, thereby obtaining a fourth inverse reinforcement feature map; A third inverse reinforcement unit is configured to splice and fuse the third backbone feature map, the third background feature map, and the fourth inverse reinforcement feature map to obtain a third fusion map, and perform calculations on the third fusion map to enhance focal lesion information and suppress background features, thereby obtaining a third inverse reinforcement feature map; A second inverse reinforcement unit is configured to splice and fuse the second backbone feature map, the second background feature map, and the third inverse reinforcement feature map to obtain a second fusion map, and perform calculations on the second fusion map to enhance focal lesion information and suppress background features, thereby obtaining a second inverse reinforcement feature map; The first reverse reinforcement unit is configured to splice and fuse the first backbone feature map, the first background feature map and the second reverse reinforcement feature map to obtain a first fusion map, and perform calculation on the first fusion map to enhance the focal lesion information and suppress the background features, so as to obtain a first reverse reinforcement feature map; The position regression unit is configured to perform regression prediction on the first reverse reinforcement feature map, the second reverse reinforcement feature map and the third reverse reinforcement feature map by using an anchor box method, so as to obtain a focal lesion target detection result.

6. The digestive pathologies assessment system of claim 5, wherein, The fourth reverse reinforcement unit, the third reverse reinforcement unit, the second reverse reinforcement unit and the first reverse reinforcement unit each include a convolution layer, a Sigmoid activation layer, a ReLu activation layer and a residual convolution block.

7. A method of evaluating a digestive tract lesion, characterized by, The method comprises the following steps: S1, acquiring a digestive tract image; S2, performing feature extraction on the digestive tract image to obtain a plurality of backbone feature maps; S3, performing background feature enhancement operation based on the backbone feature maps multiple times to generate a background feature map, and calculating a mucosa background classification result according to the background feature map; S4, fusing the backbone feature maps and the background feature map to obtain a fusion map, performing multiple reinforcement based on the fusion map to enhance the focal lesion information and suppress the background features, and performing regression prediction according to the reinforcement result to obtain a focal lesion target detection result.

8. A method of assessing a digestive tract lesion according to claim 7, wherein The S2 specifically comprises sequentially obtaining a global feature map, a first backbone feature map, a second backbone feature map, a third backbone feature map and a fourth backbone feature map based on the image feature information of the digestive tract image extracted level by level; The S3 specifically comprises receiving the fourth backbone feature map and performing operation thereon to enhance the background features, so as to obtain a fourth background feature map; convoluting and splicing the fourth background feature map and the third backbone feature map to obtain a third splicing map, performing operation on the third splicing map to enhance the background features, so as to obtain a third background feature map; convoluting and splicing the third background feature map and the second backbone feature map to obtain a second splicing map, performing operation on the second splicing map to enhance the background features, so as to obtain a second background feature map; convoluting and splicing the second background feature map and the first backbone feature map to obtain a first splicing map, performing operation on the first splicing map to enhance the background features, so as to obtain a first background feature map; and performing full convolution operation on the first background feature map, and performing mucosa background classification after the full convolution operation to obtain a mucosa background classification result; The S4 specifically comprises: splicing and fusing the fourth main stem feature map and the fourth background feature map to obtain a fourth fusion map, and performing calculation on the fourth fusion map to enhance the focal lesion information and suppress the background features, to obtain a fourth reverse enhancement feature map; splicing and fusing the third main stem feature map, the third background feature map and the fourth reverse enhancement feature map to obtain a third fusion map, and performing calculation on the third fusion map to enhance the focal lesion information and suppress the background features, to obtain a third reverse enhancement feature map; splicing and fusing the second main stem feature map, the second background feature map and the third reverse enhancement feature map to obtain a second fusion map, and performing calculation on the second fusion map to enhance the focal lesion information and suppress the background features, to obtain a second reverse enhancement feature map; splicing and fusing the first main stem feature map, the first background feature map and the second reverse enhancement feature map to obtain a first fusion map, and performing calculation on the first fusion map to enhance the focal lesion information and suppress the background features, to obtain a first reverse enhancement feature map; and performing regression prediction on the first reverse enhancement feature map, the second reverse enhancement feature map and the third reverse enhancement feature map by using an anchor box method, to obtain a focal lesion target detection result.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises instructions which, when executed on a computer, cause the computer to perform a digestive tract lesion evaluation method as claimed in claim 7 or 8.

10. A digestive tract lesion assessment device, characterized by, A computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform a digestive tract lesion evaluation method as claimed in claim 7 or 8. A computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform a digestive tract lesion evaluation method as claimed in claim 7 or 8.

Images