Intestine and stomach focus positioning method and device based on artificial intelligence

Abstract

The invention discloses a gastrointestinal focus positioning method and device based on artificial intelligence, and relates to the technical field of image processing. Extracting local features of the first image to obtain a local feature map, extracting global features of the first image, and obtaining a global feature map according to the local feature map and the global features; performing multi-scale feature extraction on the local feature map to obtain a first feature map set, and performing multi-scale feature extraction on the global feature map to obtain a second feature map set; obtaining a target feature map according to the first feature map set and the second feature map set; and substituting the target feature map into a preset database to determine a lesion type, and positioning a lesion area according to the lesion type and the target feature map. By extracting the local features and the global features respectively, context information can be provided when the subtle change of the lesion is captured, the position of the lesion in the whole image is helped to be understood, and then by capturing the features under different scales, the accuracy and precision of gastrointestinal focus positioning are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and device for locating gastrointestinal lesions based on artificial intelligence. Background Art

[0002] With the increasing incidence of digestive tract diseases, gastroenteroscopy, as an important diagnostic method, has gradually become popular. Many patients choose to undergo gastroenteroscopy due to symptoms such as indigestion, abdominal pain, and bleeding, in order to detect potential lesions early. However, traditional gastroenteroscopy relies on the doctor's experience and professional skills. Doctors need to concentrate and process a large amount of video data during a long endoscopic examination, which not only easily leads to visual fatigue, but also may affect the accurate judgment of lesion location, increasing the risk of missed diagnosis and misdiagnosis.

[0003] In order to solve the above problems, technologies based on artificial intelligence and machine learning have begun to be applied to the identification and location of lesions. These emerging technologies can be trained on a large amount of endoscopic image data, extract effective features and identify lesion areas, thereby reducing the workload of doctors and improving the accuracy and efficiency of examinations. Although existing machine learning methods have advantages in theory, the accuracy of lesion location is still challenged in practical applications, especially when facing fuzzy boundaries, the differentiation of feature information is small and the redundancy is too much, resulting in inaccurate lesion location, which reduces the accuracy of gastrointestinal lesion location. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that when facing fuzzy boundaries, the differentiation of feature information is small and the redundancy is excessive, resulting in inaccurate lesion positioning and reducing the accuracy of gastrointestinal lesion positioning, and to propose a gastrointestinal lesion positioning method and device based on artificial intelligence.

[0005] In a first aspect of the present invention, a method for locating gastrointestinal lesions based on artificial intelligence is first proposed, the method comprising: Acquire a captured image of the gastrointestinal tract of a target patient, perform image preprocessing on the captured image to obtain a first image, extract local features of the first image to obtain a local feature map, extract global features of the first image, and obtain a global feature map based on the local feature map and the global features; Performing multi-scale feature extraction on the local feature map to obtain a first feature map set, and performing multi-scale feature extraction on the global feature map to obtain a second feature map set; the number of images in the first feature map set and the number of images in the second feature map set are the same and the scales correspond one to one; Obtain a first target feature map and a second target feature map according to the first feature map set and the second feature map set, and perform image fusion on the first target feature map and the second target feature map to obtain a target feature map; The target feature map is substituted into a preset database to determine the lesion type, and the lesion area is located according to the lesion type and the target feature map.

[0006] Optionally, extracting local features of the first image to obtain a local feature map includes: Substituting the first image into a preset convolutional neural network model to obtain a first local feature map and a second local feature map; Extracting a high-frequency feature map of the first local feature map to obtain a first high-frequency feature map, and superimposing features of the first high-frequency feature map and the first local feature map to obtain a first target local feature map; Performing multi-scale feature extraction on the second local feature map to obtain a second-scale local feature map set, performing a non-local operation on each second-scale local feature map in the second-scale local feature map set to obtain a non-local feature map, and superimposing all non-local feature maps to obtain a second target local feature map; The first target local feature map and the second target local feature map are superimposed to obtain a local feature map.

[0007] Optionally, extracting the global features of the first image to obtain a global feature map includes: The first image is cut to obtain a first sub-image set, and a target sub-image is substituted into an attention mechanism to obtain a target query vector, a target key vector, and a target value vector corresponding to the target sub-image; the target sub-image is any one of the first sub-image set; Calculate the dot product of the target query vector and the key vector of each sub-image in the second sub-image set to obtain a weight value set corresponding to the target sub-image; the second sub-image set is a sub-image set after removing the target sub-image from the first sub-image set; obtain an output vector corresponding to the target sub-image according to the weight value set and the target value vector of each sub-image in the second sub-image set; The global feature is obtained by performing weighted summation on the output vectors corresponding to all sub-images in the first sub-image set.

[0008] Optionally, obtaining a first target feature map and a second target feature map according to the first feature map set and the second feature map set includes: The number of times the local feature map is obtained for multi-scale feature extraction is recorded as the extraction number K, the number of loops C is obtained by subtracting one from the extraction number K, and the target loop operation is performed according to the loop number C; The target cycle operation includes: Step 1: Through the image and images For images Processing to obtain image Go to step 2; Step 2: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 3: If i is not equal to K plus 1, then add 1 to i and repeat step 2 until the image is passed. and images For images Get the image Go to step 4; Step 4: Through the Image and images For images Processing to obtain image Go to step 5; Step 5: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 6: If i is not equal to K plus 1, then i is increased by 1 and step 5 is repeated until the image is passed. and images For images Get the image Step 7: If j is not equal to C, add 1 to j and repeat step 4 until j is equal to C, then go to step 8 Step 8: If the sum of the superscript value and the subscript value of image A is K plus 1, then the image is recorded as the target overlay map, and all target overlay maps are superimposed to obtain the final target feature map; If image A is an image in the first feature atlas, then image B is an image in the second feature atlas, and the final target feature map is recorded as the first target feature map; is the image A obtained when the image of the i-th channel is fused for the j-1th time; is the image B obtained when the image of the i-th channel is fused for the j-1th time; On the contrary, if image A is an image in the second feature atlas, then image B is an image in the first feature atlas, and the final target feature map is recorded as the first target feature map.

[0009] Optional, via image and images For images Processing to obtain image Includes: Image and images Superimposing to obtain a target corrected image, performing a first multi-core convolution on the target corrected image to obtain a first corrected convolution image, a second corrected convolution image and a third corrected convolution image; For images Performing a second multi-core convolution to obtain a first convolution image, a second convolution image, and a third convolution image; the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels of each other; The first rectified convolution image is subtracted from the first convolution image to obtain a first target image; the second rectified convolution image is subtracted from the second convolution image to obtain a second target image; the third rectified convolution image is subtracted from the third convolution image to obtain a third target image; The first target image, the second target image and the third target image are superimposed to obtain an image

[0010] In a second aspect of the present invention, a gastrointestinal lesion localization device based on artificial intelligence is provided, comprising: an image preprocessing module, for acquiring a captured image of the gastrointestinal tract of a target patient, performing image preprocessing on the captured image to obtain a first image, extracting local features of the first image to obtain a local feature map, extracting global features of the first image, and obtaining a global feature map according to the local feature map and the global features; A scale feature extraction module, configured to perform multi-scale feature extraction on the local feature map to obtain a first feature map set, and perform multi-scale feature extraction on the global feature map to obtain a second feature map set; the number of images in the first feature map set is the same as that in the second feature map set, and the scales thereof correspond one to one; A feature map fusion module is used to obtain a first target feature map and a second target feature map based on the first feature map set and the second feature map set, and to perform image fusion on the first target feature map and the second target feature map to obtain a target feature map; a lesion localization module is used to substitute the target feature map into a preset database to determine the lesion type, and to locate the lesion area based on the lesion type and the target feature map.

[0011] Optionally, the image preprocessing module includes: A local feature map generation module, used for substituting the first image into a preset convolutional neural network model to obtain a first local feature map and a second local feature map; A feature superposition module, used for extracting a high-frequency feature map of the first local feature map to obtain a first high-frequency feature map, and performing feature superposition on the first high-frequency feature map and the first local feature map to obtain a first target local feature map; a non-local feature map superposition module, configured to perform multi-scale feature extraction on the second local feature map to obtain a second-scale local feature map set, perform a non-local operation on each second-scale local feature map in the second-scale local feature map set to obtain a non-local feature map, and superimpose all non-local feature maps to obtain a second target local feature map; A local feature superposition module is used to superimpose the first target local feature map and the second target local feature map to obtain a local feature map.

[0012] Optionally, the image preprocessing module further includes: An image cutting module, configured to cut the first image to obtain a first sub-image set, and substitute a target sub-image into an attention mechanism to obtain a target query vector, a target key vector, and a target value vector corresponding to the target sub-image; the target sub-image is any one of the first sub-image set; A weight value calculation module, used for calculating the dot product of the target query vector and the key vector of each sub-image in a second sub-image set to obtain a weight value set corresponding to the target sub-image; the second sub-image set is a sub-image set after removing the target sub-image from the first sub-image set; an output vector determination module, configured to obtain an output vector corresponding to the target sub-image according to the weight value set and a target value vector of each sub-image in the second sub-image set; The weighted summation module is used to perform weighted summation on the output vectors corresponding to all sub-images in the first sub-image set to obtain global features.

[0013] Optionally, the feature map fusion module includes: A multi-scale feature extraction times determination module is used to obtain the times of multi-scale feature extraction of the local feature map as the extraction times K, and the extraction times K minus one to obtain the cycle times C, and the target cycle operation is performed according to the cycle times C; a cycle module, which is used for the target cycle operation, includes: Step 1: Through the image and images For images Processing to obtain image Go to step 2; Step 2: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 3: If i is not equal to K plus 1, then add 1 to i and repeat step 2 until the image is passed. and images For images Get the image Go to step 4; Step 4: Through the Image and images For images Processing to obtain image Go to step 5; Step 5: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 6: If i is not equal to K plus 1, then i is increased by 1 and step 5 is repeated until the image is passed. and images For images Get the image Step 7: If j is not equal to C, add 1 to j and repeat step 4 until j is equal to C, then go to step 8 Step 8: If the sum of the superscript value and the subscript value of image A is K plus 1, then the image is recorded as the target overlay map, and all target overlay maps are superimposed to obtain the final target feature map; If image A is an image in the first feature atlas, then image B is an image in the second feature atlas, and the final target feature map is recorded as the first target feature map; is the image A obtained when the image of the i-th channel is fused for the j-1th time; is the image B obtained when the image of the i-th channel is fused for the j-1th time; On the contrary, if image A is an image in the second feature atlas, then image B is an image in the first feature atlas, and the final target feature map is recorded as the first target feature map.

[0014] Optionally, the circulation module includes: The first multi-core convolution module is used to and images The target rectified image is superimposed, and the target rectified image is subjected to a first multi-core convolution to obtain a first rectified convolution image, a second rectified convolution image, and a third rectified convolution image; a second multi-core convolution module is used to perform convolution on the image Perform a second multi-core convolution to obtain a first convolution image, a second convolution image, and a third convolution image; the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels of each other; a convolution image fusion module is used to obtain a first target image by subtracting the first convolution image from the first rectified convolution image; obtain a second target image by subtracting the second convolution image from the second rectified convolution image; obtain a third target image by subtracting the third convolution image from the third rectified convolution image; A target image superposition module is used to superimpose the first target image, the second target image and the third target image to obtain an image

[0015] Beneficial effects of the present invention: The present invention proposes a gastrointestinal lesion localization method based on artificial intelligence, which comprises the following steps: obtaining a captured image of the gastrointestinal tract of a target patient, performing image preprocessing on the captured image to obtain a first image, extracting local features of the first image to obtain a local feature map, extracting global features of the first image, and obtaining a global feature map based on the local feature map and the global features; performing multi-scale feature extraction on the local feature map to obtain a first feature map set, and performing multi-scale feature extraction on the global feature map to obtain a second feature map set; the number of images in the first feature map set is the same as that in the second feature map set, and the scales correspond one to one; obtaining a first target feature map and a second target feature map based on the first feature map set and the second feature map set, and performing image fusion on the first target feature map and the second target feature map to obtain a target feature map; substituting the target feature map into a preset database to determine the lesion type, and locating the lesion area based on the lesion type and the target feature map. By extracting local features and global features separately, we can provide contextual information while capturing subtle changes in lesions, helping to understand the position of the lesion in the entire image. Multi-scale feature extraction of local and global feature maps can effectively capture features at different scales, especially for lesions with blurred boundaries. The target feature map is obtained by image fusion, which can integrate local and global information and improve the ability to describe the lesions. The lesion type is determined by combining historical data, so that the lesion area can be accurately located, improving the accuracy and precision of gastrointestinal lesion location. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below in conjunction with the accompanying drawings.

[0017] Figure 1 A flowchart of a method for locating gastrointestinal lesions based on artificial intelligence is provided for an embodiment of the present invention; Figure 2 A structural schematic diagram of a gastrointestinal lesion localization device based on artificial intelligence is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the present invention, the description of "first", "second", etc. is only used for descriptive purposes, and cannot be understood as indicating or implying its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0019] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0020] The embodiment of the present invention provides a method for locating gastrointestinal lesions based on artificial intelligence. Figure 1 , Figure 1 A flowchart of a method for locating gastrointestinal lesions based on artificial intelligence is provided in an embodiment of the present invention. The method comprises the following steps: S101, obtaining a captured image of the gastrointestinal tract of a target patient, performing image preprocessing on the captured image to obtain a first image, extracting local features of the first image to obtain a local feature map, extracting global features of the first image, and obtaining a global feature map based on the local feature map and the global features; S102, performing multi-scale feature extraction on the local feature map to obtain a first feature map set, and performing multi-scale feature extraction on the global feature map to obtain a second feature map set; S103, obtaining a first target feature map and a second target feature map according to the first feature map set and the second feature map set, and performing image fusion on the first target feature map and the second target feature map to obtain a target feature map; S104, substituting the target feature map into a preset database to determine the lesion type, and locating the lesion area according to the lesion type and the target feature map.

[0021] The number of images in the first feature atlas and the number of images in the second feature atlas are the same and their scales correspond one to one.

[0022] An artificial intelligence-based gastrointestinal lesion localization method provided by an embodiment of the present invention can provide contextual information while capturing subtle changes in the lesion by extracting local features and global features respectively, thereby helping to understand the position of the lesion in the entire image. Multi-scale feature extraction is performed on local and global feature maps, which can effectively capture features at different scales. In particular, for lesions with blurred boundaries, the target feature map is obtained by image fusion, which can integrate local and global information, improve the ability to describe the lesion, and then determine the lesion type in combination with historical data, so that the lesion area can be accurately located, thereby improving the accuracy and precision of gastrointestinal lesion localization.

[0023] In one implementation, multi-scale feature extraction can more comprehensively capture lesions of different sizes and shapes, improving the accuracy of lesion localization, especially when the boundaries are blurred; the combination of local features and global features enables the model to comprehensively consider details and overall structure during analysis, thereby improving the ability to identify lesions and reducing the risk of missed diagnosis.

[0024] In one implementation, the automated image analysis process reduces the visual fatigue of doctors during long examinations, helping to improve work efficiency and accuracy; the captured images of the gastrointestinal tract of the target patient can be obtained through endoscopy, CT scanning or MRI and other technologies, and the target patient is the patient who comes for treatment.

[0025] In one implementation, the global feature map is obtained by combining the enhanced global features with the local features of the CNN branch. The combination of global features and local features can capture the overall structure and detail information of the image at the same time, making the global feature map more comprehensive in subsequent lesion identification.

[0026] In one implementation, a preset database is used to store images corresponding to different lesions, on which lesion types and target feature maps corresponding to different lesions are marked. The cosine similarity between the target feature map and the target feature maps corresponding to different lesions in the preset database is calculated to find the lesion type closest to the target feature. The lesion area is obtained by applying bounding box regression based on the target feature map and the identified lesion type.

[0027] In one embodiment, extracting local features of the first image to obtain a local feature map includes: Substituting the first image into a preset convolutional neural network model to obtain a first local feature map and a second local feature map; Extracting a high-frequency feature map of the first local feature map to obtain a first high-frequency feature map, and superimposing features of the first high-frequency feature map and the first local feature map to obtain a first target local feature map; Perform multi-scale feature extraction on the second local feature map to obtain a second-scale local feature map set, perform a non-local operation on each second-scale local feature map in the second-scale local feature map set to obtain a non-local feature map, and superimpose all non-local feature maps to obtain a second target local feature map; The first target local feature map and the second target local feature map are superimposed to obtain a local feature map.

[0028] In one implementation, the preset convolutional neural network model uses ResNet-34 as an encoder; the first local feature map is a feature map obtained by a convolution kernel of 3*3, and the second local feature map is a feature map obtained by a convolution kernel of 7*7.

[0029] In one implementation, extracting the high-frequency feature map of the first local feature map to obtain the first high-frequency feature map specifically involves performing Fourier transform on the first local feature map to extract the high-frequency features to obtain the first high-frequency features, and performing inverse Fourier transform on the first high-frequency features to obtain the first high-frequency feature map.

[0030] In one implementation, a non-local operation is an image processing and feature extraction technique used to capture global relationships between pixels in an image.

[0031] In one implementation, by extracting high-frequency feature maps and superimposing them, the detail expression capability of local feature maps can be improved, making them more sensitive in identifying lesions; non-local operations can take into account global contextual information, which helps to improve the correlation between features, thereby enhancing the model's ability to recognize similar structures, especially in complex backgrounds. Superimposing feature maps from different convolution kernels can effectively integrate a variety of information and reduce the possibility of misjudgment; by combining high-frequency and non-local features, the boundaries and shapes of lesions can be captured more accurately, thereby improving positioning accuracy, especially in cases where the boundaries are blurred.

[0032] In one embodiment, extracting the global features of the first image to obtain a global feature map includes: The first image is cut to obtain a first sub-image set, and the target sub-image is substituted into the attention mechanism to obtain a target query vector, a target key vector, and a target value vector corresponding to the target sub-image; the target sub-image is any one in the first sub-image set; the dot product of the target query vector and the key vector of each sub-image in the second sub-image set is calculated to obtain a weight value set corresponding to the target sub-image; the second sub-image set is the sub-image set after the target sub-image is removed from the first sub-image set; The output vector corresponding to the target sub-image is obtained according to the weight value set and the target value vector of each sub-image in the second sub-image set; and the global feature is obtained by weighted summing of the output vectors corresponding to all sub-images in the first sub-image set.

[0033] In one implementation, each sub-image in the first sub-image set carries a position code; the attention mechanism can more effectively extract features related to the target sub-image and improve the model's sensitivity to important information.

[0034] In one implementation, after calculating the dot product of the target query vector and the key vector, a softmax function is applied to all dot product values ​​to ensure that the sum of the similarity values ​​of all keys corresponding to each query is 1, forming a set of probability distributions to prepare for the subsequent calculation of the output vector.

[0035] In one implementation, for each sub-image in the second sub-image set, the product of the weight value of the sub-image and the target value vector is calculated, and then the products calculated for all sub-images are summed to obtain an output vector.

[0036] In one embodiment, obtaining the first target feature map and the second target feature map according to the first feature map set and the second feature map set includes: The number of times the local feature map is obtained for multi-scale feature extraction is recorded as the extraction number K, the number of loops C is obtained by subtracting one from the extraction number K, and the target loop operation is performed according to the loop number C; The target cycle operations include, Step 1: Through the image and images For images Processing to obtain image Go to step 2; Step 2: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 3: If i is not equal to K plus 1, then add 1 to i and repeat step 2 until the image is passed. and images For images Get the image Go to step 4; Step 4: Through the Image and images For images Processing to obtain image Go to step 5; Step 5: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 6: If i is not equal to K plus 1, then i is increased by 1 and step 5 is repeated until the image is passed. and images For images Get the image Step 7: If j is not equal to C, add 1 to j and repeat step 4 until j is equal to C, then go to step 8 Step 8: If the sum of the superscript value and the subscript value of image A is K plus 1, then the image is recorded as the target overlay map, and all target overlay maps are superimposed to obtain the final target feature map; If image A is an image in the first feature atlas, then image B is an image in the second feature atlas, and the final target feature map is recorded as the first target feature map; is the image A obtained when the image of the i-th channel is fused for the j-1th time; is the image B obtained when the image of the i-th channel is fused for the j-1th time; On the contrary, if image A is an image in the second feature atlas, then image B is an image in the first feature atlas, and the final target feature map is recorded as the first target feature map.

[0037] In one implementation, through multiple cycles of processing, features of different scales can be fully extracted and fused, so that the final feature map contains more information and can better represent the image content; fusing the features of different channels allows the model to utilize the characteristics of each channel to generate a more comprehensive and effective feature representation, thereby improving the accuracy of lesion localization.

[0038] In one implementation, the order in which images are saved in the first feature atlas is in ascending order of scale, and the order in which images are saved in the second feature atlas is in ascending order of scale. When image A is an image in the first feature atlas, the set of images A is Similarly, the set of images B is For example, when the local feature map is subjected to multi-scale feature extraction three times, namely 16*16, 32*32, and 64*64, the set of images A is The set of images B is Represents a 16*16 image in the first feature atlas, Represents a 32*32 image in the first feature atlas, Represents a 64*64 image in the first feature atlas. The image in the B set has the same meaning as above. and images For images Correct the image Then the image This is the image obtained by the first fusion of the 16*16 images in the first feature atlas.

[0039] In one embodiment, by using an image and images For images Processing to obtain image include: For images and images Superimposing the target corrected image, performing a first multi-core convolution on the target corrected image to obtain a first corrected convolution image, a second corrected convolution image, and a third corrected convolution image; For images Performing a second multi-core convolution to obtain a first convolution image, a second convolution image, and a third convolution image; the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels of each other; The first rectified convolution image is subtracted from the first convolution image to obtain a first target image; the second rectified convolution image is subtracted from the second convolution image to obtain a second target image; the third rectified convolution image is subtracted from the third convolution image to obtain a third target image; The first target image, the second target image and the third target image are superimposed to obtain an image

[0040] In one implementation, by superimposing images A and B to generate a target corrected image, the features of the two images can be effectively fused, thereby improving the quality and expression of the features; using multi-core convolution can extract features from different angles, generate multiple convolution images, and provide richer and more diverse feature representations.

[0041] In one implementation, the convolution kernels in the first multi-core convolution and the second multi-core convolution are set to be transposed of each other, which can effectively capture the spatial information of the image and enhance the learning ability of the model. By subtracting the convolution image, redundant or noise information can be effectively eliminated, so that the target image can focus more on important features and improve the accuracy of the model. Superimposing multiple target images can effectively integrate features of different levels and scales and enhance the expressiveness of the final feature map.

[0042] In one implementation, the image and images Superposition can integrate local features and global features, and then subtract the image Get the image Enhance image contours; through differential processing of the corrected convolution image and the original convolution image, redundant information and noise are effectively removed, making the target features more accurate and improving the clarity of image expression.

[0043] In one implementation, for example, if the first multi-core convolution is 1*1, 1*3, or 1*5, then the first corrected convolution image is the image obtained by passing the target corrected image through the 1*1 convolution kernel, the second corrected convolution image is the image obtained by passing the target corrected image through the 1*3 convolution kernel, and the third corrected convolution image is the image obtained by passing the target corrected image through the 1*5 convolution kernel, and the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels, then the second multi-core convolution is 1*1, 3*1, or 5*1, then the first convolution image is the image obtained by passing the 1*3 convolution kernel. The image obtained by the 1*1 convolution kernel and the second convolution image are obtained by the image The image obtained by the 3*1 convolution kernel and the third convolution image are obtained by the image Image obtained by 5*1 convolution kernel.

[0044] Based on the same inventive concept, the embodiment of the present invention also provides a gastrointestinal lesion positioning device based on artificial intelligence. Figure 2 , Figure 2 A schematic diagram of a gastrointestinal lesion localization device based on artificial intelligence provided by an embodiment of the present invention includes: An image preprocessing module, used to obtain a captured image of the gastrointestinal tract of a target patient, perform image preprocessing on the captured image to obtain a first image, extract local features of the first image to obtain a local feature map, extract global features of the first image, and obtain a global feature map based on the local feature map and the global features; A scale feature extraction module, used for performing multi-scale feature extraction on the local feature map to obtain a first feature map set, and performing multi-scale feature extraction on the global feature map to obtain a second feature map set; the number of images in the first feature map set is the same as that in the second feature map set, and the scales correspond one to one; A feature map fusion module is used to obtain a first target feature map and a second target feature map according to the first feature map set and the second feature map set, and perform image fusion on the first target feature map and the second target feature map to obtain a target feature map; The lesion localization module is used to substitute the target feature map into a preset database to determine the lesion type, and locate the lesion area according to the lesion type and the target feature map.

[0045] An artificial intelligence-based gastrointestinal lesion localization device provided based on an embodiment of the present invention can provide contextual information while capturing subtle changes in the lesion by extracting local features and global features respectively, thereby helping to understand the position of the lesion in the entire image. Multi-scale feature extraction is performed on local and global feature maps, which can effectively capture features at different scales. In particular, for lesions with blurred boundaries, the target feature map is obtained by image fusion, which can integrate local and global information, improve the ability to describe the lesion, and then determine the lesion type in combination with historical data, so that the lesion area can be accurately located, thereby improving the accuracy and precision of gastrointestinal lesion localization.

[0046] In one embodiment, the image preprocessing module includes: A local feature map generation module, used to substitute the first image into a preset convolutional neural network model to obtain a first local feature map and a second local feature map; A feature superposition module is used to extract a high-frequency feature map of the first local feature map to obtain a first high-frequency feature map, and to perform feature superposition on the first high-frequency feature map and the first local feature map to obtain a first target local feature map; A non-local feature map superposition module is used to perform multi-scale feature extraction on the second local feature map to obtain a second-scale local feature map set, perform non-local operations on each second-scale local feature map in the second-scale local feature map set to obtain a non-local feature map, and superimpose all non-local feature maps to obtain a second target local feature map; The local feature superposition module is used to superimpose the first target local feature map and the second target local feature map to obtain a local feature map.

[0047] In one embodiment, the image preprocessing module further includes: An image cutting module, used for cutting the first image to obtain a first sub-image set, substituting the target sub-image into the attention mechanism to obtain a target query vector, a target key vector and a target value vector corresponding to the target sub-image; the target sub-image is any one in the first sub-image set; A weight value calculation module is used to calculate the dot product of the target query vector and the key vector of each sub-image in the second sub-image set to obtain a weight value set corresponding to the target sub-image; the second sub-image set is the sub-image set after the target sub-image is removed from the first sub-image set; an output vector determination module is used to obtain an output vector corresponding to the target sub-image according to the weight value set and the target value vector of each sub-image in the second sub-image set; The weighted summation module is used to perform weighted summation on the output vectors corresponding to all sub-images in the first sub-image set to obtain the global feature.

[0048] In one embodiment, the feature map fusion module includes: A multi-scale feature extraction times determination module is used to obtain the local feature map and perform multi-scale feature extraction times, which is recorded as the extraction times K, and the cycle times C is obtained by subtracting one from the extraction times K, and the target cycle operation is performed according to the cycle times C; Loop module, used for target loop operations including, Step 1: Through the image and images For images Processing to obtain image Go to step 2; Step 2: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 3: If i is not equal to K plus 1, then add 1 to i and repeat step 2 until the image is passed. and images For images Get the image Go to step 4; Step 4: Through the Image and images For images Processing to obtain image Go to step 5; Step 5: Through the image and images For images Processing to obtain image Where i is greater than 2 and i is less than K, i is an integer; Step 6: If i is not equal to K plus 1, then i is increased by 1 and step 5 is repeated until the image is passed. and images For images Get the image Step 7: If j is not equal to C, add 1 to j and repeat step 4 until j is equal to C, then go to step 8 Step 8: If the sum of the superscript value and the subscript value of image A is K plus 1, then the image is recorded as the target overlay map, and all target overlay maps are superimposed to obtain the final target feature map; If image A is an image in the first feature atlas, then image B is an image in the second feature atlas, and the final target feature map is recorded as the first target feature map; is the image A obtained when the image of the i-th channel is fused for the j-1th time; is the image B obtained when the image of the i-th channel is fused for the j-1th time; On the contrary, if image A is an image in the second feature atlas, then image B is an image in the first feature atlas, and the final target feature map is recorded as the first target feature map.

[0049] In one embodiment, the loop module includes: The first multi-core convolution module is used to and images Superimposing the target corrected image, performing a first multi-core convolution on the target corrected image to obtain a first corrected convolution image, a second corrected convolution image, and a third corrected convolution image; The second multi-core convolution module is used to Performing a second multi-core convolution to obtain a first convolution image, a second convolution image, and a third convolution image; the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels of each other; A convolution image fusion module, for obtaining a first target image by subtracting the first convolution image from the first rectified convolution image; obtaining a second target image by subtracting the second convolution image from the second rectified convolution image; and obtaining a third target image by subtracting the third convolution image from the third rectified convolution image; A target image superposition module is used to superimpose the first target image, the second target image and the third target image to obtain an image

[0050] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A gastrointestinal lesion localization method based on artificial intelligence, characterized in that: The method comprises: Acquire a captured image of the gastrointestinal tract of a target patient, perform image preprocessing on the captured image to obtain a first image, extract local features of the first image to obtain a local feature map, extract global features of the first image, and obtain a global feature map based on the local feature map and the global features; Performing multi-scale feature extraction on the local feature map to obtain a first feature map set, and performing multi-scale feature extraction on the global feature map to obtain a second feature map set; the number of images in the first feature map set and the number of images in the second feature map set are the same and the scales correspond one to one; Obtain a first target feature map and a second target feature map according to the first feature map set and the second feature map set, and perform image fusion on the first target feature map and the second target feature map to obtain a target feature map; The target feature map is substituted into a preset database to determine the lesion type, and the lesion area is located according to the lesion type and the target feature map.

2. The method for locating gastrointestinal lesions based on artificial intelligence according to claim 1, characterized in that: Extracting the local features of the first image to obtain a local feature map includes: Substituting the first image into a preset convolutional neural network model to obtain a first local feature map and a second local feature map; Extracting a high-frequency feature map of the first local feature map to obtain a first high-frequency feature map, and superimposing features of the first high-frequency feature map and the first local feature map to obtain a first target local feature map; Performing multi-scale feature extraction on the second local feature map to obtain a second-scale local feature map set, performing a non-local operation on each second-scale local feature map in the second-scale local feature map set to obtain a non-local feature map, and superimposing all non-local feature maps to obtain a second target local feature map; The first target local feature map and the second target local feature map are superimposed to obtain a local feature map.

3. The method for locating gastrointestinal lesions based on artificial intelligence according to claim 1, characterized in that: Extracting the global features of the first image to obtain a global feature map includes: The first image is cut to obtain a first sub-image set, and a target sub-image is substituted into an attention mechanism to obtain a target query vector, a target key vector, and a target value vector corresponding to the target sub-image; the target sub-image is any one of the first sub-image set; Calculate the dot product of the target query vector and the key vector of each sub-image in the second sub-image set to obtain a weight value set corresponding to the target sub-image; the second sub-image set is a sub-image set after removing the target sub-image from the first sub-image set; obtain an output vector corresponding to the target sub-image according to the weight value set and the target value vector of each sub-image in the second sub-image set; The global feature is obtained by performing weighted summation on the output vectors corresponding to all sub-images in the first sub-image set.

4. The method for locating gastrointestinal lesions based on artificial intelligence according to claim 1, characterized in that: Obtaining a first target feature map and a second target feature map according to the first feature map set and the second feature map set includes: The number of times the local feature map is obtained for multi-scale feature extraction is recorded as the extraction number K, the number of loops C is obtained by subtracting one from the extraction number K, and the target loop operation is performed according to the loop number C; The target cycle operation includes: Step 1: Through the image and images For images Processing to obtain image , go to step 2; Step 2: Through the image and images For images Processing to obtain image ; Where i is greater than 2 and i is less than K, i is an integer; Step 3: If i is not equal to K plus 1, then add 1 to i and repeat step 2 until the image is passed. and images For images Get the image , go to step 4; Step 4: Through the Image and images For images Processing to obtain image , go to step 5; Step 5: Through the Image and images For images Processing to obtain image , where i is greater than 2 and i is less than K, i is an integer; Step 6: If i is not equal to K plus 1, then i is increased by 1 and step 5 is repeated until the image is passed. and images For images Get the image ; Step 7: If j is not equal to C, add 1 to j and repeat step 4 until j is equal to C, then go to step 8 Step 8: If the sum of the superscript value and the subscript value of image A is K plus 1, then the image is recorded as the target overlay map, and all target overlay maps are superimposed to obtain the final target feature map; If image A is an image in the first feature atlas, then image B is an image in the second feature atlas, and the final target feature map is recorded as the first target feature map; is the image A obtained when the image of the i-th channel is fused for the j-1th time; is the image B obtained when the image of the i-th channel is fused for the j-1th time; On the contrary, if image A is an image in the second feature atlas, then image B is an image in the first feature atlas, and the final target feature map is recorded as the first target feature map.

5. The method for locating gastrointestinal lesions based on artificial intelligence according to claim 4, characterized in that: via image and images For images Processing to obtain image include: For images and images Superimposing to obtain a target corrected image, performing a first multi-kernel convolution on the target corrected image to obtain a first corrected convolution image, a second corrected convolution image and a third corrected convolution image; For images Performing a second multi-core convolution to obtain a first convolution image, a second convolution image, and a third convolution image; the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels of each other; The first rectified convolution image is subtracted from the first convolution image to obtain a first target image; the second rectified convolution image is subtracted from the second convolution image to obtain a second target image; the third rectified convolution image is subtracted from the third convolution image to obtain a third target image; The first target image, the second target image and the third target image are superimposed to obtain an image .

6. A gastrointestinal lesion localization device based on artificial intelligence, characterized in that: The device comprises: An image preprocessing module, used to obtain a captured image of the gastrointestinal tract of a target patient, perform image preprocessing on the captured image to obtain a first image, extract local features of the first image to obtain a local feature map, extract global features of the first image, and obtain a global feature map based on the local feature map and the global features; A scale feature extraction module, configured to perform multi-scale feature extraction on the local feature map to obtain a first feature map set, and perform multi-scale feature extraction on the global feature map to obtain a second feature map set; the number of images in the first feature map set is the same as that in the second feature map set, and the scales thereof correspond one to one; A feature map fusion module is used to obtain a first target feature map and a second target feature map based on the first feature map set and the second feature map set, and to perform image fusion on the first target feature map and the second target feature map to obtain a target feature map; a lesion localization module is used to substitute the target feature map into a preset database to determine the lesion type, and to locate the lesion area based on the lesion type and the target feature map.

7. The gastrointestinal lesion localization device based on artificial intelligence according to claim 6, characterized in that: The image preprocessing module comprises: A local feature map generation module, used for substituting the first image into a preset convolutional neural network model to obtain a first local feature map and a second local feature map; A feature superposition module, used for extracting a high-frequency feature map of the first local feature map to obtain a first high-frequency feature map, and performing feature superposition on the first high-frequency feature map and the first local feature map to obtain a first target local feature map; a non-local feature map superposition module, configured to perform multi-scale feature extraction on the second local feature map to obtain a second-scale local feature map set, perform a non-local operation on each second-scale local feature map in the second-scale local feature map set to obtain a non-local feature map, and superimpose all non-local feature maps to obtain a second target local feature map; A local feature superposition module is used to superimpose the first target local feature map and the second target local feature map to obtain a local feature map.

8. The gastrointestinal lesion localization device based on artificial intelligence according to claim 6, characterized in that: The image preprocessing module also includes: An image cutting module, configured to cut the first image to obtain a first sub-image set, and substitute a target sub-image into an attention mechanism to obtain a target query vector, a target key vector, and a target value vector corresponding to the target sub-image; the target sub-image is any one of the first sub-image set; A weight value calculation module, used for calculating the dot product of the target query vector and the key vector of each sub-image in a second sub-image set to obtain a weight value set corresponding to the target sub-image; the second sub-image set is a sub-image set after removing the target sub-image from the first sub-image set; an output vector determination module, configured to obtain an output vector corresponding to the target sub-image according to the weight value set and a target value vector of each sub-image in the second sub-image set; The weighted summation module is used to perform weighted summation on the output vectors corresponding to all sub-images in the first sub-image set to obtain global features.

9. The gastrointestinal lesion localization device based on artificial intelligence according to claim 6, characterized in that: The feature map fusion module includes: A multi-scale feature extraction times determination module is used to obtain the times of multi-scale feature extraction of the local feature map as the extraction times K, and the extraction times K minus one to obtain the cycle times C, and the target cycle operation is performed according to the cycle times C; a cycle module, which is used for the target cycle operation, includes: Step 1: Through the image and images For images Processing to obtain image , go to step 2; Step 2: Through the image and images For images Processing to obtain image ; Where i is greater than 2 and i is less than K, i is an integer; Step 3: If i is not equal to K plus 1, then add 1 to i and repeat step 2 until the image is passed. and images For images Get the image , go to step 4; Step 4: Through the Image and images For images Processing to obtain image , go to step 5; Step 5: Through the Image and images For images Processing to obtain image , where i is greater than 2 and i is less than K, i is an integer; Step 6: If i is not equal to K plus 1, then i is increased by 1 and step 5 is repeated until the image is passed. and images For images Get the image ; Step 7: If j is not equal to C, add 1 to j and repeat step 4 until j is equal to C, then go to step 8 Step 8: If the sum of the superscript value and the subscript value of image A is K plus 1, then the image is recorded as the target overlay map, and all target overlay maps are superimposed to obtain the final target feature map; If image A is an image in the first feature atlas, then image B is an image in the second feature atlas. In this case, the final target feature map is recorded as the first target feature map. is the image A obtained when the image of the i-th channel is fused for the j-1th time; the QUOTE is the image B obtained when the image of the i-th channel is fused for the j-1th time; conversely, if image A is an image in the second feature atlas, then image B is an image in the first feature atlas, and the final target feature map is recorded as the first target feature map.

10. The gastrointestinal lesion localization device based on artificial intelligence according to claim 9, characterized in that: The circulation module comprises: The first multi-core convolution module is used to and images Superimposing to obtain a target corrected image, performing a first multi-kernel convolution on the target corrected image to obtain a first corrected convolution image, a second corrected convolution image and a third corrected convolution image; The second multi-core convolution module is used to Performing a second multi-core convolution to obtain a first convolution image, a second convolution image, and a third convolution image; the convolution kernels in the second multi-core convolution and the first multi-core convolution are transposed convolution kernels of each other; A convolution image fusion module, configured to obtain a first target image by subtracting the first convolution image from the first rectified convolution image; obtain a second target image by subtracting the second convolution image from the second rectified convolution image; and obtain a third target image by subtracting the third convolution image from the third rectified convolution image; A target image superposition module is used to superimpose the first target image, the second target image and the third target image to obtain an image .

Images