Gastrointestinal metaplasia lesion area segmentation method, system, device, medium and product

Abstract

The gastrointestinal metaplasia lesion area segmentation method, system, device, medium and product disclosed in the present invention belong to the field of medical image processing technology. The method divides the gastrointestinal metaplasia endoscopic image for training into a query set and a support set. When the query set and the support set are used to train a feature extractor, the query set features and the support set features are extracted by the feature extractor; the support set features and their masks are converted into the support set initial prototype by using an attention guidance mechanism; the query set features are segmented using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask; the query set features and their masks are converted into the query set initial prototype by using the attention guidance mechanism; the support set features are segmented using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask; the segmentation accuracy of the gastrointestinal metaplasia lesion area is improved, and the current problem of inaccurate segmentation of the gastrointestinal metaplasia lesion area is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method, system, device, medium and product for segmenting a gastrointestinal metaplasia lesion region. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Gastrointestinal metaplasia (GIM) is a precancerous lesion of gastric cancer. Currently, the gastrointestinal metaplasia segmentation model (GIM segmentation model) is mainly used to segment GIM endoscopic images and determine the lesion area of ​​GIM. In order to enable the GIM segmentation model to obtain better visual feature learning ability from images, the common practice is to use large-scale labeled data and multiple iterations to train the model.

[0004] However, the available data of gastrointestinal metaplasia (GIM) is small in scale and cannot support the training of effective GIM segmentation models, resulting in a significant decrease in segmentation performance. The main reason is that GIM images often have complex and diverse features, which are not easy to extract and understand. The model cannot effectively capture the subtle differences in these features, resulting in low quality of the generated prototypes; at the same time, the prototype alignment network, as a method under small sample segmentation, generates prototypes from support images to segment the query set images by matching the query feature map with the prototype. However, when only a limited number of support images are available to segment the query set images, the differences between GIM lesion forms of the same category will lead to inaccurate segmentation. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a method, system, device, medium and product for segmenting gastrointestinal metaplasia lesion area, thereby achieving accurate segmentation of gastrointestinal metaplasia lesion area.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] First, a gastrointestinal metaplasia lesion region segmentation method is proposed, including:

[0008] Acquire training endoscopic images of gastrointestinal metaplasia;

[0009] The training gastrointestinal metaplasia endoscopic images are divided into a query set and a support set;

[0010] Train the feature extractor using the query set and support set;

[0011] Using the trained feature extractor, the endoscopic images of gastrointestinal metaplasia were segmented;

[0012] Among them, during the feature extractor training process, the feature extractor extracts query set features and support set features; the support set features and the support set gastrointestinal metaplasia lesion area mask are converted into the support set initial prototype by using the attention guidance mechanism through the support-query branch; the query set features are segmented using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask; the query set features and the query set gastrointestinal metaplasia lesion area mask are converted into the query set initial prototype by using the attention guidance mechanism through the query-support branch; the query set features are segmented using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask; the attention guidance mechanism is to calculate the attention vector between the corresponding feature and the mask; the attention vector is multiplied with the corresponding feature to obtain the lesion area attention feature map; the attention feature map is average pooled to obtain the corresponding prototype.

[0013] Furthermore, the feature extractor is constructed using a fully convolutional neural network.

[0014] Furthermore, the initial prototype of the support set is used as a guide to segment the query set features, and the process of obtaining the gastrointestinal metaplasia lesion area mask of the query set is as follows:

[0015] The query set features are segmented using the initial prototype of the support set to obtain the initial mask of the gastrointestinal metaplasia lesion area in the query set; the initial mask of the gastrointestinal metaplasia lesion area in the query set and the query set features are converted into the query set prototype using the attention guidance mechanism; the query set features are segmented using the query set prototype to obtain the gastrointestinal metaplasia lesion area mask in the query set;

[0016] The process of segmenting the support set features using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask is as follows:

[0017] The initial prototype of the query set is used to segment the support set features to obtain the initial mask of the support set gastrointestinal metaplasia lesion area; the initial mask of the support set gastrointestinal metaplasia lesion area and the support set features are converted into the support set prototype using the attention guidance mechanism; the support set features are segmented using the support set prototype to obtain the support set gastrointestinal metaplasia lesion area mask.

[0018] Furthermore, the corresponding set features are segmented using the corresponding prototypes to obtain the corresponding mask, which includes:

[0019] Calculate the similarity between the corresponding prototype and the corresponding set features to obtain the segmentation probability of the gastrointestinal metaplasia lesion area;

[0020] The gastrointestinal metaplasia lesion region segmentation probability is converted into a corresponding mask.

[0021] Furthermore, during the feature extractor training process, the support set segmentation loss and the query set segmentation loss are calculated; the support set segmentation loss and the query set segmentation loss are weightedly fused to obtain the total loss; and based on the total loss, it is determined whether the feature extractor training is completed.

[0022] Furthermore, the process of segmenting the gastrointestinal metaplasia endoscopic image using the trained feature extractor includes:

[0023] Using the trained feature extractor, image features of gastrointestinal metaplasia endoscopic images are extracted;

[0024] The classifier is used to classify and identify the image features of the gastrointestinal metaplasia endoscopic image to obtain the initial prediction results of the gastrointestinal metaplasia endoscopic image;

[0025] According to the initial prediction results, a gastrointestinal metaplasia lesion region mask of the gastrointestinal metaplasia endoscopic image is obtained.

[0026] Secondly, a gastrointestinal metaplasia lesion area segmentation system is proposed, including:

[0027] An image acquisition module, used for acquiring training gastrointestinal metaplasia endoscopic images;

[0028] An image partitioning module, used for partitioning the training gastrointestinal metaplasia endoscopic images into a query set and a support set;

[0029] A model training module is used to train a feature extractor using a query set and a support set. During the feature extractor training process, the feature extractor extracts query set features and support set features; the support set features and the support set gastrointestinal metaplasia lesion area mask are converted into the support set initial prototype by using the attention guidance mechanism through the support-query branch; the query set features are segmented using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask; the query set features and the query set gastrointestinal metaplasia lesion area mask are converted into the query set initial prototype by using the attention guidance mechanism through the query-support branch; the support set features are segmented using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask; the attention guidance mechanism is to calculate the attention vector between the corresponding feature and the mask; the attention vector is multiplied with the corresponding feature to obtain the lesion area attention feature map; the attention feature map is average pooled to obtain the corresponding prototype;

[0030] The image segmentation module is used to segment the gastrointestinal metaplasia endoscopic images using the trained feature extractor.

[0031] In a third aspect, a computer device is provided, the device comprising:

[0032] a processor adapted to execute a computer program;

[0033] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the method for segmenting the gastrointestinal metaplasia lesion area proposed in the first aspect is implemented.

[0034] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the gastrointestinal metaplasia lesion area segmentation method proposed in the first aspect.

[0035] In a fifth aspect, a computer program product is proposed, which includes a computer program. When the computer program is executed by a processor, the method for segmenting the gastrointestinal metaplasia lesion area proposed in the first aspect is implemented.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention proposes a method, system, device, medium and product for segmenting gastrointestinal metaplasia lesion regions. When the method uses a query set and a support set to train a feature extractor, after the feature extractor extracts query set features and support set features, the support set features and the support set gastrointestinal metaplasia lesion region mask are converted into a support set initial prototype by using an attention guidance mechanism through a support-query branch; the query set features are segmented using the support set initial prototype as a guide to obtain a query set gastrointestinal metaplasia lesion region mask; the query set features and the query set gastrointestinal metaplasia lesion region mask are converted into a query set initial prototype by using an attention guidance mechanism through a query-support branch; the query set initial prototype is used as a guide The support set features are segmented to obtain the support set gastrointestinal metaplasia lesion area mask; the image features and their corresponding masks are converted into corresponding prototypes using the attention guidance mechanism, so that the generated prototype can pay more attention to the target area of ​​the lesion area, increasing the difference between the lesion area and the normal tissue. Through continuous training, the ability of the prototype to represent the image category features is improved, and the feature extractor is further guided to extract the desired features, thereby improving the feature extractor's ability to extract features. When the trained feature extractor is used to extract the gastrointestinal metaplasia endoscopic image features, and then the extracted image features are used to segment the gastrointestinal metaplasia endoscopic image, the accuracy of the gastrointestinal metaplasia lesion area segmentation is improved.

[0038] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0040] Figure 1 A training flow chart of the attention-guided prototype alignment network disclosed in the embodiment;

[0041] Figure 2 Schematic diagram of the attention-guiding prototype generation module disclosed in the embodiment. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0044] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0045] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0046] Example 1

[0047] In this embodiment, a method for segmenting a gastrointestinal metaplasia lesion region is disclosed, such as Figure 1 , Figure 2 As shown, including:

[0048] S1: Acquisition of training endoscopic images of gastrointestinal metaplasia.

[0049] Among them, the training gastrointestinal metaplasia endoscopic images have been masked with gastrointestinal metaplasia lesion areas.

[0050] In order to ensure the consistency of the input image of the attention-guided prototype alignment network, this embodiment first preprocesses the collected gastrointestinal metaplasia endoscopic images to unify the image size.

[0051] Preferably, the image size is unified to 224×224.

[0052] S2: Divide the training gastrointestinal metaplasia endoscopic images into query set and support set.

[0053] This embodiment uses the preprocessed images to construct the query set and the support set.

[0054] Preferably, the preprocessed image is divided into a support set and a query set by using a random partitioning method. Specifically:

[0055] Randomly select N categories from the preprocessed image, preferably, N is 3, and the three categories are: moderate lesions, severe lesions, and non-lesions;

[0056] Randomly select K samples from each selected category, and a total of N×K samples constitute the support set;

[0057] T samples are randomly selected from the remaining samples of each selected category, and a total of N×T samples constitute the query set.

[0058] S3: The feature extractor is trained using the query set and the support set, wherein during the feature extractor training process, the feature extractor extracts query set features and support set features; the support set features and the support set gastrointestinal metaplasia lesion area mask are converted into the support set initial prototype using the attention guidance mechanism through the support-query branch; the query set features are segmented using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask; the query set features and the query set gastrointestinal metaplasia lesion area mask are converted into the query set initial prototype using the attention guidance mechanism through the query-support branch; the support set features are segmented using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask; the attention guidance mechanism is to calculate the attention vector between the corresponding feature and the mask; the attention vector is multiplied by the corresponding feature to obtain the lesion area attention feature map; the attention feature map is average pooled to obtain the corresponding prototype.

[0059] Among them, the process of segmenting the query set features using the initial prototype of the support set as a guide to obtain the mask of the gastrointestinal metaplasia lesion area of ​​the query set is:

[0060] The query set features are segmented using the initial prototype of the support set to obtain the initial mask of the gastrointestinal metaplasia lesion area in the query set; the initial mask of the gastrointestinal metaplasia lesion area in the query set and the query set features are converted into the query set prototype using the attention guidance mechanism; the query set features are segmented using the query set prototype to obtain the gastrointestinal metaplasia lesion area mask in the query set;

[0061] The process of segmenting the support set features using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask is as follows:

[0062] The initial prototype of the query set is used to segment the support set features to obtain the initial mask of the support set gastrointestinal metaplasia lesion area; the initial mask of the support set gastrointestinal metaplasia lesion area and the support set features are converted into the support set prototype using the attention guidance mechanism; the support set features are segmented using the support set prototype to obtain the support set gastrointestinal metaplasia lesion area mask.

[0063] The process of segmenting the corresponding set features using the corresponding prototype and obtaining the corresponding mask includes:

[0064] Calculate the similarity between the corresponding prototype and the corresponding set features to obtain the segmentation probability of the gastrointestinal metaplasia lesion area;

[0065] The gastrointestinal metaplasia lesion region segmentation probability is converted into a corresponding mask.

[0066] In this embodiment, when training the feature extractor using the support set and the query set, the feature extractor is placed in the attention-guided prototype alignment network, and the training of the feature extractor is achieved by training the attention-guided prototype alignment network.

[0067] The attention-guided prototype alignment network consists of a feature extractor, a support-query branch, and a query-support branch.

[0068] The feature extractor is constructed using a fully convolutional neural network to extract query set features and support set features. Specifically:

[0069] Will support set images and query set image Input feature extractor, use feature extractor Get the support set features respectively and queryset features , the formula is as follows:

[0070]

[0071] in, D , H and W are the depth, height and width of the feature map respectively.

[0072] The support-query branch uses progressive segmentation to generate the query set gastrointestinal metaplasia lesion area mask based on the support set features and the support set gastrointestinal metaplasia lesion area mask; the query-support branch uses progressive segmentation to generate the support set gastrointestinal metaplasia lesion area mask based on the query set features and the query set gastrointestinal metaplasia lesion area mask.

[0073] Specifically, the support-query branch includes an attention-guided prototype generation module, an initial segmentation subunit and a self-supporting segmentation subunit; through the attention-guided prototype generation module in this branch, the support set features and the support set gastrointestinal metaplasia lesion area mask are converted into the support set initial prototype; through the initial segmentation subunit in this branch, the query set features are segmented using the support set initial prototype to obtain the query set gastrointestinal metaplasia lesion area initial mask; then through the attention-guided prototype generation module in this branch, the query set gastrointestinal metaplasia lesion area initial mask and the query set features are converted into the query set prototype; finally, through the self-supporting segmentation subunit in this branch, the query set features are segmented using the query set prototype to obtain the query set gastrointestinal metaplasia lesion area mask.

[0074] like Figure 2 As shown, the attention-guided prototype generation module includes an attention-guided feature aggregation module and a multi-prototype generation module. The attention-guided feature aggregation module calculates the attention vector between the feature input to the attention-guided prototype generation module and the mask; multiplies the attention vector with the feature input to the attention-guided prototype generation module to obtain an attention feature map of the lesion area; the multi-prototype generation module performs average pooling on the attention feature map to obtain the corresponding prototype and determine the prototype category.

[0075] The process of obtaining the gastrointestinal metaplasia lesion area mask of the query set through the support-query branch includes:

[0076] Using the attention-guided prototype generation module, the support set features are transformed into and support set gastrointestinal metaplasia lesion area mask The formula for converting to the initial prototype of the support set is:

[0077]

[0078] in, Support set features Gastrointestinal metaplasia lesion area mask with support set The attention vector between Represents a bilinear interpolation function, which is used to change The dimension will The height and width are set to H and , so that it is Keep the height and width consistent so that calculate. represents pixel-by-pixel multiplication, P represents the pooling operation, is a convolutional network. is an activation function.

[0079] Then the attention vector and support set features Using formula (3), we get the attention feature map of the lesion area with a representative support set: .

[0080]

[0081] For the multi-prototype generation module, the lesion area attention feature map of the representative support set is and support set gastrointestinal metaplasia lesion area mask As input, the foreground class is calculated or background local prototype.

[0082] Foreground For example, attention feature map of the lesion area in the support set Perform average pooling, and the pooling window is , to obtain the spatial position Local prototype , the specific method is as follows:

[0083]

[0084] in, , , and , represents the category of the local prototype under consideration, Represent the height and width of the pooling window respectively.

[0085] Here, each local prototype is determined as follows Semantic category .

[0086] First of all, The application window size is Average pooling to obtain the same spatial position The values ​​on are as follows:

[0087]

[0088] in, express In spatial position The average pooling value of .

[0089] Then, the prototype is determined according to the given threshold Corresponding category , as shown below:

[0090]

[0091] in, is to classify the prototype into foreground classes or background class The threshold is empirically set to 0.9.

[0092] Therefore, the initial prototype of the support set is expressed as , and Represents the total number of prototypes of the foreground class and the background class respectively. and background class The local prototypes are represented as and . Each prototype in corresponds to the feature a local area.

[0093] Initial segmentation subunit calculates query set features Initial prototype with support set The similarity between them is used to obtain the initial segmentation probability of the gastrointestinal metaplasia lesion area in the query set. .

[0094]

[0095] The initial segmentation probability of the gastrointestinal metaplasia lesion area based on the query set features Initial mask of gastrointestinal metaplasia lesion area converted into query set , as shown below:

[0096]

[0097] in, It is a function that converts the segmentation probability of gastrointestinal metaplasia lesion area into a mask.

[0098] Then, the query set features and the initial mask of gastrointestinal metaplasia lesion area of ​​the query set Input to the attention-guided prototype generation unit to obtain the query set prototype P .

[0099]

[0100] The self-supporting segmentation subunit computes query set features With the queryset prototype P The similarity between them is used to obtain the self-supporting segmentation probability of the gastrointestinal metaplasia lesion area in the query set , and further transform it into a query set self-support mask , the self-supporting mask is the final query set gastrointestinal metaplasia lesion area mask.

[0101] The query-support branch uses progressive segmentation to generate a support set of gastrointestinal metaplasia lesion region masks, such as Figure 1 As shown by the dotted line, its purpose is to achieve support and query prototype alignment during the training process of the attention-guided prototype alignment network. This unit only appears in training and does not appear in segmentation prediction. The query-support branch includes an attention-guided prototype generation module, an initial segmentation subunit and a self-support segmentation subunit. The query set features and the query set gastrointestinal metaplasia lesion area mask are converted into the query set initial prototype through the attention-guided prototype generation module in this branch. The support set features are segmented using the query set initial prototype through the initial segmentation subunit in this branch to obtain the support set gastrointestinal metaplasia lesion area initial mask. Then, the support set gastrointestinal metaplasia lesion area initial mask and support set features are converted into support set prototypes through the attention-guided prototype generation module in this branch. Finally, the support set features are segmented using the support set prototype through the self-support segmentation subunit in this branch to obtain the support set gastrointestinal metaplasia lesion area mask.

[0102] Since the processing flow of input data through the query-support branch is consistent with that through the support-query branch, each subunit in the query-support branch will not be described in detail. Only a brief description of the data processing process through the query-support branch is given, specifically:

[0103] Using the attention-guided prototype generation module, the query set generated by the support-query branch is masked with gastrointestinal metaplasia lesion regions. Generate the initial prototype of the query set with the query set features ;

[0104] Will With support set features Perform similarity calculation to obtain the initial mask of the gastrointestinal metaplasia lesion area in the support set ;

[0105] Using the attention-guided prototype generation module, the initial mask of the gastrointestinal metaplasia lesion area in the support set is and support set features , generate support set prototype ;

[0106] Will With support set features Perform similarity calculation to obtain the support set gastrointestinal metaplasia lesion area mask .

[0107] This embodiment calculates the support set segmentation loss and the query set segmentation loss during the feature extractor training process; performs weighted fusion of the support set segmentation loss and the query set segmentation loss to obtain the total loss; and determines whether the feature extractor training is completed based on the total loss.

[0108] Among them, the query set segmentation loss for:

[0109]

[0110] in, is the initial segmentation loss of the query set, is the query set self-support segmentation loss, , To balance the two loss terms, the empirical values ​​are 0.6 and 0.4 respectively.

[0111] Specifically, the initial segmentation probability of the gastrointestinal metaplasia lesion area according to the query set is calculated. and the probability of self-supporting segmentation of gastrointestinal metaplasia lesion area , calculate the query set initial segmentation loss and query set self-support segmentation loss, and the relevant losses are defined as follows:

[0112]

[0113]

[0114] in, is the annotation mask of the images in the query set.

[0115] Support set segmentation loss for:

[0116]

[0117] in, is the initial segmentation loss of the support set, is the support set self-support segmentation loss, and are weighting coefficients, and their empirical values ​​are 0.6 and 0.4 respectively.

[0118] Initial segmentation probability of gastrointestinal metaplasia lesion area according to the support set The probability of self-support segmentation of gastrointestinal metaplasia lesion area based on the support set , calculate the support set initial segmentation loss and the support set self-support segmentation loss, and the relevant losses are defined as follows:

[0119]

[0120]

[0121] in, is the annotation mask of the image in the support set.

[0122] Finally, the total loss between the gastrointestinal metaplasia lesion area mask generated during the entire model training process and the annotated mask is calculated , use the total loss to train the entire model. The training objective is defined as follows:

[0123]

[0124] in, and are weight coefficients, which are set to 0.5 based on experience.

[0125] S4: Use the trained feature extractor to segment the endoscopic images of gastrointestinal metaplasia.

[0126] The process of segmenting gastrointestinal metaplasia endoscopic images using the trained feature extractor includes:

[0127] Using the trained feature extractor, image features of gastrointestinal metaplasia endoscopic images are extracted;

[0128] The classifier is used to classify and identify the image features of the gastrointestinal metaplasia endoscopic image to obtain the initial prediction results of the gastrointestinal metaplasia endoscopic image;

[0129] According to the initial prediction results, a gastrointestinal metaplasia lesion region mask of the gastrointestinal metaplasia endoscopic image is obtained.

[0130] The classifier here is used to convert each pixel point of the image feature into a prediction value of the corresponding category to obtain an initial prediction result; the categories here include moderate lesions, severe lesions, and non-lesions.

[0131] The initial prediction result of the gastrointestinal metaplasia endoscopic image is interpolated to the size of the gastrointestinal metaplasia endoscopic image using bilinear interpolation technology to ensure that the output is consistent with the size of the gastrointestinal metaplasia endoscopic image, and finally the gastrointestinal metaplasia lesion area mask of the gastrointestinal metaplasia endoscopic image is obtained.

[0132] The gastrointestinal metaplasia lesion area segmentation method disclosed in this embodiment extracts lesion area related information from a small number of samples through an attention guidance mechanism, and guides the network to focus on the target area, thereby increasing the difference between the lesion area and normal tissue, so as to generate a high-quality prototype and improve the representativeness of the prototype; when performing segmentation prediction, based on the progressive segmentation method, through initial segmentation and self-supporting segmentation, it effectively utilizes the characteristics of the image itself and realizes refined segmentation, thereby effectively improving the accuracy of gastrointestinal metaplasia segmentation; thereafter, through continuous training, the ability of the prototype to represent the image category characteristics is improved, and the feature extractor is further guided to extract the desired features, thereby improving the ability of the feature extractor to extract features. When the trained feature extractor is used to extract the features of the gastrointestinal metaplasia endoscopic image, and then the extracted image features are used to perform gastrointestinal metaplasia endoscopic image segmentation, the accuracy of gastrointestinal metaplasia lesion area segmentation is improved.

[0133] It should be noted that all data is obtained in compliance with laws and regulations and user consent, and the data is used legally.

[0134] Example 2

[0135] In this embodiment, a gastrointestinal metaplasia lesion region segmentation system is disclosed, comprising:

[0136] An image acquisition module, used for acquiring training gastrointestinal metaplasia endoscopic images;

[0137] An image partitioning module, used for partitioning the training gastrointestinal metaplasia endoscopic images into a query set and a support set;

[0138] The model training module is used to train the feature extractor using the query set and the support set. During the feature extractor training process, the feature extractor extracts query set features and support set features; the support set features and the support set gastrointestinal metaplasia lesion area mask are converted into the support set initial prototype by using the attention guidance mechanism through the support-query branch; the query set features are segmented using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask; the query set features and the query set gastrointestinal metaplasia lesion area mask are converted into the query set initial prototype by using the attention guidance mechanism through the query-support branch; the support set features are segmented using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask; the attention guidance mechanism is to calculate the attention vector between the corresponding feature and the mask; the attention vector is multiplied with the corresponding feature to obtain the lesion area attention feature map; the attention feature map is average pooled to obtain the corresponding prototype;

[0139] The image segmentation module is used to segment the gastrointestinal metaplasia endoscopic images using the trained feature extractor.

[0140] The present invention also discloses a computer device, which includes:

[0141] a processor adapted to execute a computer program;

[0142] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the method for segmenting the gastrointestinal metaplasia lesion area disclosed in Example 1 is implemented.

[0143] The present invention also discloses a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the gastrointestinal metaplasia lesion region segmentation method disclosed in Example 1.

[0144] The present invention also discloses a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for segmenting the gastrointestinal metaplasia lesion area disclosed in Example 1 is implemented.

[0145] The method disclosed in Example 1 can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for segmenting gastrointestinal metaplasia lesion regions, characterized in that: include: Acquire training endoscopic images of gastrointestinal metaplasia; The training gastrointestinal metaplasia endoscopic images are divided into a query set and a support set; Train the feature extractor using the query set and support set; Using the trained feature extractor, the endoscopic images of gastrointestinal metaplasia were segmented; Among them, during the feature extractor training process, the feature extractor extracts query set features and support set features; the support set features and the support set gastrointestinal metaplasia lesion area mask are converted into the support set initial prototype by using the attention guidance mechanism through the support-query branch; the query set features are segmented using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask; the query set features and the query set gastrointestinal metaplasia lesion area mask are converted into the query set initial prototype by using the attention guidance mechanism through the query-support branch; the query set initial prototype is used as a guide to segment the support set features to obtain the support set gastrointestinal metaplasia lesion area mask; the processing flow of the input data through the query-support branch is consistent with that through the support-query branch; the attention guidance mechanism is to calculate the attention vector between the corresponding feature and the mask; the attention vector is multiplied with the corresponding feature to obtain the lesion area attention feature map; the attention feature map is average pooled to obtain the corresponding prototype; The process of segmenting the query set features using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask is as follows: The query set features are segmented using the initial prototype of the support set to obtain the initial mask of the gastrointestinal metaplasia lesion area in the query set; the initial mask of the gastrointestinal metaplasia lesion area in the query set and the query set features are converted into the query set prototype using the attention guidance mechanism; the query set features are segmented using the query set prototype to obtain the gastrointestinal metaplasia lesion area mask in the query set; The process of segmenting the corresponding set features using the corresponding prototype and obtaining the corresponding mask includes: Calculate the similarity between the corresponding prototype and the corresponding set features to obtain the segmentation probability of the gastrointestinal metaplasia lesion area; Converting the gastrointestinal metaplasia lesion region segmentation probability into a corresponding mask; The process of segmenting the support set features using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask is as follows: The initial prototype of the query set is used to segment the support set features to obtain the initial mask of the support set gastrointestinal metaplasia lesion area; the initial mask of the support set gastrointestinal metaplasia lesion area and the support set features are converted into the support set prototype using the attention guidance mechanism; the support set features are segmented using the support set prototype to obtain the support set gastrointestinal metaplasia lesion area mask.

2. The method for segmenting gastrointestinal metaplasia lesion region according to claim 1, characterized in that: The feature extractor is constructed using a fully convolutional neural network.

3. The method for segmenting gastrointestinal metaplasia lesion region according to claim 1, characterized in that: Calculate the support set segmentation loss and the query set segmentation loss during the feature extractor training process; perform weighted fusion of the support set segmentation loss and the query set segmentation loss to obtain the total loss; Based on the total loss, determine whether the feature extractor training is complete.

4. The method for segmenting gastrointestinal metaplasia lesion region according to claim 1, characterized in that: The process of segmenting gastrointestinal metaplasia endoscopic images using the trained feature extractor includes: Using the trained feature extractor, image features of gastrointestinal metaplasia endoscopic images are extracted; The classifier is used to classify and identify the image features of the gastrointestinal metaplasia endoscopic image to obtain the initial prediction results of the gastrointestinal metaplasia endoscopic image; According to the initial prediction results, a gastrointestinal metaplasia lesion region mask of the gastrointestinal metaplasia endoscopic image is obtained.

5. A gastrointestinal metaplasia lesion region segmentation system, characterized in that: include: An image acquisition module is used to acquire a training gastrointestinal metaplasia endoscopic image, where the training gastrointestinal metaplasia endoscopic image has been masked with gastrointestinal metaplasia lesion areas; An image partitioning module, used for partitioning the training gastrointestinal metaplasia endoscopic images into a query set and a support set; A model training module is used to train a feature extractor using a query set and a support set. During the feature extractor training process, the feature extractor extracts query set features and support set features. The support-query branch uses the attention-guided mechanism to transform the support set features and the support set gastrointestinal metaplasia lesion area mask into the support set initial prototype; The initial prototype of the support set is used as a guide to segment the query set features and obtain the query set gastrointestinal metaplasia lesion area mask; The query-support branch uses the attention-guided mechanism to transform the query set features and the query set gastrointestinal metaplasia lesion area mask into the query set initial prototype; The initial prototype of the query set is used as a guide to segment the support set features and obtain the gastrointestinal metaplasia lesion area mask of the support set; The processing flow of input data through the query-support branch is consistent with that through the support-query branch; The attention guidance mechanism is to calculate the attention vector between the corresponding feature and the mask; multiply the attention vector with the corresponding feature to obtain the attention feature map of the lesion area; perform average pooling on the attention feature map to obtain the corresponding prototype; The process of segmenting the query set features using the support set initial prototype as a guide to obtain the query set gastrointestinal metaplasia lesion area mask is as follows: The query set features are segmented using the initial prototype of the support set to obtain the initial mask of the gastrointestinal metaplasia lesion area in the query set; The initial mask of the gastrointestinal metaplasia lesion area in the query set and the query set features are transformed into the query set prototype using the attention-guided mechanism; The query set features are segmented using the query set prototype to obtain the query set gastrointestinal metaplasia lesion area mask; The process of segmenting the corresponding set features using the corresponding prototype and obtaining the corresponding mask includes: Calculate the similarity between the corresponding prototype and the corresponding set features to obtain the segmentation probability of the gastrointestinal metaplasia lesion area; Converting the gastrointestinal metaplasia lesion region segmentation probability into a corresponding mask; The process of segmenting the support set features using the query set initial prototype as a guide to obtain the support set gastrointestinal metaplasia lesion area mask is as follows: The initial prototype of the query set is used to segment the support set features to obtain the initial mask of the gastrointestinal metaplasia lesion area in the support set; the initial mask of the gastrointestinal metaplasia lesion area in the support set and the support set features are converted into the support set prototype using the attention guidance mechanism; the support set features are segmented using the support set prototype to obtain the gastrointestinal metaplasia lesion area mask in the support set; The image segmentation module is used to segment the gastrointestinal metaplasia endoscopic images using the trained feature extractor.

6. An electronic device, characterized in that: The device comprises: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the method for segmenting the gastrointestinal metaplasia lesion area according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the gastrointestinal metaplasia lesion region segmentation method according to any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method for segmenting a gastrointestinal metaplasia lesion region according to any one of claims 1 to 4 is implemented.

Images