A method for constructing a colonoscopy-based tumor recognition model for ulcerative colitis

Abstract

The present invention relates to the field of medical image processing technology, and discloses a method for constructing a colonoscopy-related ulcerative colitis tumor recognition model, comprising: obtaining a colonoscopy image containing UCAN, the colonoscopy image containing UCAN being annotated with UCAN boundaries and UCAN lesion property labels; according to the colonoscopy image containing UCAN, a U-shaped deep neural network is used to train a colonoscopy-related ulcerative colitis tumor recognition model. The present invention uses a U-shaped deep neural network to train a colonoscopy-related ulcerative colitis tumor recognition model, which can recognize and extract the UCAN boundary of the colonoscopy image containing UCAN and generate a UCAN boundary image, and can also recognize the lesion property of the UCAN, effectively reducing the difficulty for doctors to recognize the UCAN boundary and lesion property by naked eyes, improving the accuracy and efficiency of UCAN recognition, and assisting doctors to quickly and flexibly formulate appropriate treatment plans.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model. Background Art

[0002] The incidence of ulcerative colitis-associated neoplasia (UCAN) is 2 times higher than that of colon tumors in the general population, and its associated mortality risk is 1.5 to 2 times higher than that of colon cancer in the general population. More importantly, although undifferentiated carcinoma in colon cancer is relatively rare, it is relatively high in patients with ulcerative colitis. Therefore, early detection of UCAN is very important, and it is generally recommended to start related tumor screening 8-10 years after the onset of ulcerative colitis.

[0003] However, unlike colon tumors in the general population, early ulcerative colitis-related tumors have some characteristics, including: 1. The lesions are mostly superficial, and many are even flat lesions, and the lesion boundaries may be fuzzy, which makes it very difficult to find the lesions; 2. Ulcerative colitis-related tumors may have multiple lesions at the same time; 3. The active phase of ulceration may affect the judgment of UCAN; these characteristics bring great challenges to the discovery of lesions. In the past, the academic community believed that many early UCANs could only be found through random biopsies. At present, the clinical judgment of endoscopic UCAN generally requires professionally trained endoscopists, which is inefficient, inaccurate, and costly.

[0004] Therefore, there is an urgent need for a colonoscopy-based ulcerative colitis-related tumor recognition model that can assist doctors in identifying colonoscopy-based UCAN and determining the benign or malignant nature of the lesions. Summary of the invention

[0005] The present invention provides a method for constructing a colonoscopy-based ulcerative colitis-related tumor recognition model, so as to solve the defects of low efficiency, poor accuracy and high cost in the prior art of colonoscopy-based ulcerative colitis-related tumor recognition by endoscopists.

[0006] The present invention provides a method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model, comprising:

[0007] A colonoscopic image containing UCAN is obtained, where the colonoscopic image containing UCAN is marked with a UCAN boundary and a UCAN lesion property label;

[0008] Based on the annotated colonoscopic images containing UCAN, a U-shaped deep neural network was used to train a colonoscopic ulcerative colitis-related tumor recognition model.

[0009] Among them, the colonoscopy-induced ulcerative colitis-related tumor recognition model includes an input layer, an encoder, an intermediate connection layer, a decoder, and an output layer connected in sequence. The output layer performs multi-task processing of two branches, one of which is used to generate a boundary image of the identified UCAN based on the colonoscopy image to be detected, and the other branch is used to identify the lesion properties of the UCAN on the colonoscopy image to be detected.

[0010] According to a method for constructing a colonoscopy-related ulcerative colitis tumor recognition model provided by the present invention, the input layer includes a convolution block with a receptive field of 3×3, the encoder includes five first-stage layers connected in sequence, each of the five first-stage layers includes a multi-scale feature fusion module, the first of the five first-stage layers is connected to the input layer, the last of the five first-stage layers is connected to the intermediate connection layer, the intermediate connection layer includes a 12-layer Transformer module, and the decoder includes five second-stage layers connected in sequence, a deformable attention module, and an integrated attention module.

[0011] According to a method for constructing a colonoscopy-related ulcerative colitis tumor recognition model provided by the present invention, a multi-scale feature fusion module includes a first branch, a second branch, and a third branch. The first branch, the second branch, and the third branch all include a convolution block, a normalization layer, and an activation layer. The receptive field of the convolution block of the first branch is 1×1, the receptive field of the convolution block of the second branch is 3×3, and the receptive field of the convolution block of the third branch is 5×5.

[0012] According to a method for constructing a colonoscopy-related ulcerative colitis tumor recognition model provided by the present invention, the deformable attention module includes two 3×3 branches, one of which is used to convert a colonoscopy image containing UCAN into a 2×k×k×H×W offset matrix using a 3×3 convolution kernel, where k represents a constant, H and W represent the height and width of the colonoscopy image containing UCAN, respectively, and the other branch is used to extract a two-dimensional tensor of attention weights from the colonoscopy image containing UCAN using a 3×3 convolution kernel, so as to learn the importance of features of different regions in the colonoscopy image containing UCAN.

[0013] According to a method for constructing a colonoscopy-related ulcerative colitis tumor recognition model provided by the present invention, the comprehensive attention module includes a channel attention module and a spatial attention module, and the output of the channel attention module is the input of the spatial attention module, wherein the channel attention module includes a first pooling layer, a multi-layer perceptron, and a first softmax layer connected in sequence, and the spatial attention module includes a second pooling layer, a convolution block with a receptive field of 1×1, and a second softmax layer connected in sequence, and the first pooling layer and the second pooling layer both include a maximum pooling layer and an average pooling layer.

[0014] According to a method for constructing a colonoscopy-related ulcerative colitis tumor recognition model provided by the present invention, the colonoscopy-related ulcerative colitis tumor recognition model is obtained by training a U-shaped deep neural network based on a colonoscopy image containing UCAN, comprising:

[0015] Preprocess the colonoscopy images containing UCAN;

[0016] The initial learning rate of the colonoscopy ulcerative colitis-related tumor recognition model is set to 0.005, the momentum is 0.9, the weight decay is 0.0001, the neural network batch is 16, and the initial learning rate is updated using the update expression;

[0017] The first loss function for identifying the UCAN boundary and the second loss function for identifying the properties of UCAN lesions in the colonoscopy-based ulcerative colitis-related tumor recognition model are set respectively.

[0018] According to a method for constructing a colonoscopy-based ulcerative colitis-related tumor recognition model provided by the present invention, the updated expression is:

[0019]

[0020] 1. 0 =0.005, Epoch=200,

[0021] In the update expression, lr represents the initial learning rate after the update, lr 0 Represents the initial learning rate, epoch represents the current training period, and Epoch represents the total training period. The value is between [0, 1].

[0022] According to a method for constructing a colonoscopy-based ulcerative colitis-related tumor recognition model provided by the present invention, the first loss function is:

[0023] Loss seg = lambda1 × Loss dice +lambda2×Loss bce

[0024]

[0025]

[0026] In the first loss function, Loss seg Indicates the first loss of the colonoscopy ulcerative colitis-related tumor recognition model to identify the UCAN boundary, Loss diceIndicates the dice loss of the colonoscopy-based ulcerative colitis-related tumor recognition model to identify the UCAN boundary, Loss bce represents the bce loss of the UCAN boundary identified by the colonoscopy ulcerative colitis-related tumor recognition model, lambda1 and lambda2 represent the loss dice and Loss bce The corresponding weight coefficient is lambda1 = lambda2 = 0.5, and y represents the true value of the UCAN boundary identified by the colonoscopy-based ulcerative colitis-related tumor recognition model. Represents the predicted value of the colonoscopy-based ulcerative colitis-related tumor recognition model in identifying the UCAN boundary.

[0027] According to a method for constructing a colonoscopy-based ulcerative colitis-related tumor recognition model provided by the present invention, the second loss function is:

[0028]

[0029] In the second loss function, Loss class represents the second loss of the ulcerative colitis-related tumor recognition model under colonoscopy to identify the UCAN lesion properties, y′ represents the true value of the ulcerative colitis-related tumor recognition model under colonoscopy to identify the UCAN lesion properties, It represents the predictive value of the colonoscopic ulcerative colitis-associated tumor recognition model in identifying the nature of UCAN lesions.

[0030] The present invention also provides a system for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model, comprising:

[0031] A data acquisition module is used to: acquire a colonoscopic image containing UCAN, wherein the colonoscopic image containing UCAN is annotated with a UCAN boundary and a UCAN lesion property label;

[0032] The model training module is used to: train a colonoscopy-related ulcerative colitis tumor recognition model based on colonoscopy images containing UCAN using a U-shaped deep neural network;

[0033] Among them, the colonoscopy-induced ulcerative colitis-related tumor recognition model includes an input layer, an encoder, an intermediate connection layer, a decoder, and an output layer connected in sequence. The output layer performs multi-task processing of two branches, one of which is used to generate a boundary image of the identified UCAN based on the colonoscopy image to be detected, and the other branch is used to identify the lesion properties of the UCAN on the colonoscopy image to be detected.

[0034] The present invention also provides a colonoscopic ulcerative colitis-related tumor recognition system, comprising:

[0035] A data receiving module is used to: receive a colonoscopy image to be identified;

[0036] The recognition module is used to: obtain a recognition result of whether the colonoscopic image to be identified contains UCAN based on the colonoscopic ulcerative colitis-related tumor recognition model constructed by the colonoscopic ulcerative colitis-related tumor recognition model construction method described in any of the above items according to the colonoscopic image to be identified, and when it is identified that the colonoscopic image to be identified contains UCAN, obtain the boundary image and lesion properties of UCAN.

[0037] The present invention further provides an electronic device, comprising a processor and a memory storing a computer program, wherein the processor implements the following steps when executing the computer program:

[0038] receiving a colonoscopy image to be identified;

[0039] According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any of the above items is used to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the following steps when executed by a processor:

[0041] receiving a colonoscopy image to be identified;

[0042] According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any of the above items is used to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

[0043] The present invention also provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can perform the following steps:

[0044] receiving a colonoscopy image to be identified;

[0045] According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any of the above items is used to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

[0046] The present invention provides a method for constructing a colonoscopy-associated ulcerative colitis tumor recognition model, which adopts a U-shaped deep neural network to train a colonoscopy-associated ulcerative colitis tumor recognition model. The method can effectively identify and extract the UCAN boundary in the colonoscopy image to be detected and generate a UCAN boundary image, and can also identify the lesion properties of the UCAN in the colonoscopy image to be detected, effectively reducing the difficulty for doctors to identify the UCAN boundary and lesion properties with the naked eye, improving the recognition accuracy and efficiency of the UCAN, and helping to accurately identify the UCAN, and assisting doctors to quickly and flexibly formulate appropriate treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of a method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model provided by the present invention.

[0049] Figure 2 This is a schematic diagram of the structure of the ulcerative colitis-related tumor recognition model under colonoscopy.

[0050] Figure 3 Schematic diagram of the structure of the multi-scale feature fusion module.

[0051] Figure 4 Schematic diagram of the structure of the deformable attention module.

[0052] Figure 5 Schematic diagram of the structure of the comprehensive attention module.

[0053] Figure 6 This is the application effect of the colonoscopy-related ulcerative colitis tumor recognition model constructed according to the present invention in UCAN recognition. The first row is the colonoscopy image, the second row is the real boundary of UCAN, and the third row is the lesion boundary identified by the colonoscopy-related ulcerative colitis tumor recognition model.

[0054] Figure 7 This is a schematic diagram of the structure of a colonoscopy-assisted ulcerative colitis-related tumor recognition system provided by the present invention.

[0055] Figure 8 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0057] Combine the following Figure 1-Figure 7 The invention describes a method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model.

[0058] Figure 1 Schematic diagram of the process of constructing a colonoscopy-based ulcerative colitis-related tumor recognition model provided by the present invention. Figure 1 The present invention provides a method for constructing a colonoscopy-based ulcerative colitis-related tumor recognition model, which may include:

[0059] Step S110, obtaining a colonoscopic image containing UCAN, wherein the colonoscopic image containing UCAN is marked with a UCAN boundary and a UCAN lesion property label;

[0060] Step S120: Based on the colonoscopic image containing UCAN, a U-shaped deep neural network is used to train a colonoscopic ulcerative colitis-related tumor recognition model.

[0061] In one embodiment, step S110 can obtain a colonoscopic image containing UCAN from a hospital medical record system or an endoscopic image system, etc. The colonoscopic image in this embodiment is an image under white light endoscopy, and a professional endoscopist marks the UCAN boundary and UCAN lesion property label on the colonoscopic image containing UCAN. After obtaining a number of colonoscopic images containing UCAN, they can be divided into a training set, a validation set, and a test set in a ratio of 8:1:1 for training a colonoscopic ulcerative colitis-related tumor recognition model.

[0062] Furthermore, before training the colonoscopic ulcerative colitis-related tumor recognition model, the colonoscopic images containing UCAN in the training set can be preprocessed, such as data enhancement processing, to improve data diversity. The implementation of data enhancement processing includes random horizontal and vertical flipping, random Gaussian blurring, resizing in the range of [0.75, 1.25], random cropping, etc. All data in the training set are uniformly set to 256×256 size. No data enhancement processing is required for the validation set and the test set.

[0063] In one embodiment, a colonoscopy-induced ulcerative colitis-related tumor recognition model includes an input layer, an encoder, an intermediate connection layer, a decoder, and an output layer connected in sequence, which together constitute a U-shaped structure. The output layer performs multi-task processing of two branches, one of which is used to generate a boundary image of the identified UCAN based on the colonoscopy image to be detected, and the other branch is used to identify the pathological properties of the UCAN on the colonoscopy image to be detected.

[0064] Specifically, refer to Figure 2 The input layer of the colonoscopy-induced ulcerative colitis-related tumor recognition model may include a convolution block with a receptive field of 3×3, the encoder may include five first-stage layers (E1, E2, E3, E4, E5) connected in sequence, each of the five first-stage layers includes a multi-scale feature fusion module (MSFM), the first of the five first-stage layers is connected to the input layer, the last of the five first-stage layers is connected to the intermediate connection layer, the intermediate connection layer may include a 12-layer Transformer module, and the decoder may include five second-stage layers (D1, D2, D3, D4, D5) connected in sequence as well as a deformable attention module (DAM) and a comprehensive attention module (SCAM).

[0065] The colonoscopy image containing UCAN is input into the encoder after data processing at the input layer. The encoder consists of five stages, each of which has a multi-scale feature extractor. After 5 downsamplings, the semantic features of the image at different scales are extracted. The last stage of the encoder is connected to the middle connection layer. The features of the last stage of the encoder are flattened and input into the 12-layer Transformer module of the middle connection layer to extract the global information of the features and the long-tail information of the image. The features output by the 12-layer Transformer module are converted and input into the first stage of the decoder. The decoder is the same as the encoder, consisting of five stages, in which E5 is connected to the 12-layer Transformer module and is input to D5 after processing. E4 is input to the deformable attention module (DAM). The output features E4' and D5 are passed through the comprehensive attention module (SCAM) to obtain D5', which are then upsampled and added bitwise to obtain D4. Similarly, E3 is passed through DAM to obtain E3', D4 is passed through SCAM to obtain D4', which are then upsampled and added to obtain D3. E2 is passed through DAM to obtain E2', D3 is passed through SCAM to obtain D3', which are then upsampled and added to obtain D2. E1 is passed through DAM to obtain E1', D2 is passed through SCAM to obtain D2', which are then upsampled and input to the output layer.

[0066] The multi-scale feature fusion module (MSFM) is capable of processing tumor information of different shapes and sizes. Figure 3 The multi-scale feature fusion module may include a first branch, a second branch, and a third branch. The first branch, the second branch, and the third branch all include a convolution block, a normalization layer, and an activation layer. The receptive field of the convolution block of the first branch is 1×1, the receptive field of the convolution block of the second branch is 3×3, and the receptive field of the convolution block of the third branch is 5×5.

[0067] The input layer includes a convolutional block with a receptive field of 3×3. The colonoscopy image of UCAN is input into the input layer to obtain preliminary features and then input to the encoder. The encoder has 5 stage layers E1-E5. The size of the features extracted by each stage layer is 1 / 2 of the size of the features extracted by the previous stage layer. Among them, E2 is 1 / 2 of the size of E1, E3 is 1 / 2 of the size of E2, E4 is 1 / 2 of the size of E3, and E5 is 1 / 2 of the size of E4.

[0068] Each stage layer E1-E5 includes a multi-scale feature fusion module (MSFM) and performs downsampling. Downsampling reduces the feature size by half. The image is extracted with different receptive field features of the first branch, the second branch, and the third branch, respectively, F1, F2, and F3, and finally added to the input feature F0 to obtain F = F0 + F1 + F2 + F3. After downsampling (maximum pooling), the size of F becomes 1 / 2 and is passed to the next stage of the encoder. Among them, F represents the output feature of MSFM, F0 represents the input feature, F1 represents the output feature of the first branch, F2 represents the output feature of the second branch, and F3 represents the output feature of the third branch.

[0069] In one embodiment, referring to Figure 4 , the deformable attention module can use two 3×3 branches to obtain attention weights. The first branch uses a 3×3 convolution kernel to convert the colonoscopy image into a 2×k×k×H×W offset matrix (where k is 3, H and W represent the height and width of the colonoscopy image). The 2×k×k length vector corresponding to the (x, y) point in the offset matrix is ​​used to represent the k×k group of offsets (dx, dy); the second branch uses a 3×3 convolution kernel to extract the attention weight two-dimensional tensor, and the latitude size of the matrix is ​​H×W, which is used to learn the importance of different regional features in the colonoscopy image. The second branch combines the offset matrix obtained by the first branch to superimpose the relevant weights of each pixel to enhance the attention of the concerned area. The normalized attention weight is obtained through sigmoid, and then multiplied with the original input to obtain the output feature.

[0070] Compared with traditional convolutional blocks, the deformable attention module can learn the offset of each pixel in the colonoscopy image containing UCAN to represent the surrounding related positions of the element point. The values ​​of more important positions are obtained from the positions that the deformable attention module pays attention to. The deformable attention module (DAM) can effectively identify and extract the UCAN boundary, improving the boundary feature expression ability of the colonoscopy-related ulcerative colitis tumor recognition model.

[0071] In one embodiment, referring to Figure 5 The comprehensive attention module includes a channel attention module and a spatial attention module, and the output of the channel attention module is the input of the spatial attention module. The channel attention module includes a first pooling layer, a multi-layer perceptron, and a first softmax layer connected in sequence, and the spatial attention module includes a second pooling layer, a convolution block with a receptive field of 1×1, and a second softmax layer connected in sequence. Both the first pooling layer and the second pooling layer include a maximum pooling layer and an average pooling layer.

[0072] The channel attention module performs the maximum pooling and average pooling algorithms after the convolution module, and then obtains the transformation results through three fully connected layers. Finally, it is applied to two channels respectively, and the sigmoid function is used to obtain the attention results of the channels to learn the importance weights of the channels, so as to learn the importance information of the target object. The spatial attention module first reduces the dimension of the channel itself, obtains the results of the maximum pooling and average pooling respectively, and then splices them into a feature map, and then uses a convolution layer to use the convolution operation to generate the final spatial attention feature map.

[0073] The comprehensive attention module (SCAM) performs attention-based feature fusion on the extracted features in space and channels, which can improve the overall fine characterization capability of the colonoscopy-based ulcerative colitis-related tumor recognition model.

[0074] In one embodiment, the initial learning rate of the colonoscopy-induced ulcerative colitis-related tumor recognition model can be set to 0.005, the momentum to 0.9, the weight decay to 0.0001, the neural network batch to 16, and the initial learning rate is updated using an update expression.

[0075] Update the expression to:

[0076]

[0077] 1. 0 =0.005, Epoch=200,

[0078] In the update expression, lr represents the initial learning rate after the update, lr 0 Represents the initial learning rate, epoch represents the current training period, and Epoch represents the total training period. The value is between [0, 1].

[0079] Then, the first loss function for identifying the UCAN boundary and the second loss function for identifying the properties of UCAN lesions of the colonoscopy-related ulcerative colitis tumor recognition model were set respectively.

[0080] The first loss function is:

[0081] Loss seg = lambda1 × Loss dice +lambda2×Loss bce

[0082]

[0083]

[0084] In the first loss function, Loss seg Indicates the first loss of the colonoscopy ulcerative colitis-related tumor recognition model to identify the UCAN boundary, Loss dice Indicates the dice loss of the colonoscopy-based ulcerative colitis-related tumor recognition model to identify the UCAN boundary, Loss bce represents the bce loss of the UCAN boundary identified by the colonoscopy ulcerative colitis-related tumor recognition model, lambda1 and lambda2 represent the loss dice and Loss bce The corresponding weight coefficient is lambda1 = lambda2 = 0.5, and y represents the true value of the UCAN boundary identified by the colonoscopy-based ulcerative colitis-related tumor recognition model. Represents the predicted value of the colonoscopy-based ulcerative colitis-related tumor recognition model in identifying the UCAN boundary.

[0085] The second loss function is:

[0086]

[0087] In the second loss function, Loss class represents the second loss of the ulcerative colitis-related tumor recognition model under colonoscopy to identify the UCAN lesion properties, y′ represents the true value of the ulcerative colitis-related tumor recognition model under colonoscopy to identify the UCAN lesion properties, It represents the predictive value of the colonoscopic ulcerative colitis-associated tumor recognition model in identifying the nature of UCAN lesions.

[0088] Figure 6 The application effect of the colonoscopy-associated ulcerative colitis tumor recognition model constructed according to the present invention on UCAN recognition is shown. The first row is the colonoscopy image, the second row is the real boundary of UCAN, and the third row is the lesion boundary identified by the colonoscopy-associated ulcerative colitis tumor recognition model, which is basically consistent with the real boundary.

[0089] The method for constructing a colonoscopy-associated ulcerative colitis tumor recognition model provided by the present invention adopts a U-shaped deep neural network to train a colonoscopy-associated ulcerative colitis tumor recognition model, learns the characteristics of UCAN on colonoscopy images, can effectively identify and extract the UCAN boundary on the colonoscopy image to be detected and generate a UCAN boundary image, and can also identify the lesion nature of the UCAN on the colonoscopy image to be detected that contains the UCAN, effectively reducing the difficulty for doctors to identify the UCAN boundary and lesion nature with the naked eye, and improving the recognition accuracy and efficiency of UCAN, assisting doctors to quickly and flexibly formulate appropriate treatment plans.

[0090] The following describes the system for constructing a colonoscopy-related ulcerative colitis tumor recognition model provided by the present invention. The system for constructing a colonoscopy-related ulcerative colitis tumor recognition model described below and the method for constructing a colonoscopy-related ulcerative colitis tumor recognition model described above can be referenced to each other.

[0091] The present invention provides a system for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model, which may include:

[0092] A data acquisition module is used to: acquire a colonoscopic image containing UCAN, wherein the colonoscopic image containing UCAN is annotated with a UCAN boundary and a UCAN lesion property label;

[0093] The model training module is used to: train a colonoscopy-related ulcerative colitis tumor recognition model based on colonoscopy images containing UCAN using a U-shaped deep neural network;

[0094] Among them, the colonoscopy-induced ulcerative colitis-related tumor recognition model includes an input layer, an encoder, an intermediate connection layer, a decoder, and an output layer connected in sequence. The output layer performs multi-task processing of two branches, one of which is used to generate a boundary image of the identified UCAN based on the colonoscopy image to be detected, and the other branch is used to identify the lesion properties of the UCAN on the colonoscopy image to be detected.

[0095] Reference Figure 7 The present invention also provides a colonoscopy-based ulcerative colitis-related tumor recognition system, comprising:

[0096] A data receiving module is used to: receive a colonoscopy image to be identified;

[0097] The recognition module is used to: obtain a recognition result of whether the colonoscopic image to be identified contains UCAN based on the colonoscopic ulcerative colitis-related tumor recognition model constructed by the colonoscopic ulcerative colitis-related tumor recognition model construction method described in any of the above items according to the colonoscopic image to be identified, and when it is identified that the colonoscopic image to be identified contains UCAN, obtain the boundary image and lesion properties of UCAN.

[0098] When it is necessary to identify the boundaries and lesion properties of UCAN, the doctor can first obtain the patient's colonoscopy image (the patient's colonoscopy image may or may not contain lesions), and then input the patient's colonoscopy image into the colonoscopy-related ulcerative colitis tumor recognition system. The data receiving module receives the patient's colonoscopy image, and uses the recognition module to identify whether the patient's colonoscopy image contains UCAN. When the recognition module identifies that the patient's colonoscopy image does not contain UCAN, it outputs a negative recognition result of the patient's colonoscopy image. When the recognition module identifies that the patient's colonoscopy image contains UCAN, it outputs a positive recognition result of the patient's colonoscopy image and outputs the boundary image and lesion properties of UCAN. The doctor can further diagnose the patient's UCAN based on the results output by the colonoscopy-related ulcerative colitis tumor recognition system to improve diagnostic efficiency and accuracy and formulate appropriate treatment plans for patients in a timely manner.

[0099] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to perform the following steps:

[0100] receiving a colonoscopy image to be identified;

[0101] According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any of the above items is used to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

[0102] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0103] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can perform the following steps:

[0104] receiving a colonoscopy image to be identified;

[0105] According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any of the above items is used to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

[0106] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented to perform the following steps when executed by a processor:

[0107] receiving a colonoscopy image to be identified;

[0108] According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any of the above items is used to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

[0109] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a colonoscopy-based ulcerative colitis-related tumor recognition model, characterized in that: include: A colonoscopic image containing UCAN is obtained, where the colonoscopic image containing UCAN is marked with a UCAN boundary and a UCAN lesion property label; Based on the annotated colonoscopic images containing UCAN, a U-shaped deep neural network was used to train a colonoscopic ulcerative colitis-related tumor recognition model. Among them, the colonoscopy-related ulcerative colitis tumor recognition model includes an input layer, an encoder, an intermediate connection layer, a decoder, and an output layer connected in sequence. The output layer performs multi-task processing of two branches, one of which is used to generate a boundary image of the identified UCAN according to the colonoscopy image to be detected, and the other branch is used to identify the lesion properties of the UCAN on the colonoscopy image to be detected; Among them, the input layer includes a convolution block with a receptive field of 3×3, the encoder includes five first-stage layers connected in sequence, each of the five first-stage layers includes a multi-scale feature fusion module, the first of the five first-stage layers is connected to the input layer, and the last of the five first-stage layers is connected to the intermediate connection layer, which includes a 12-layer Transformer module, and the decoder includes five second-stage layers connected in sequence, a deformable attention module, and a comprehensive attention module; Among them, the multi-scale feature fusion module includes a first branch, a second branch, and a third branch. The first branch, the second branch, and the third branch all include a convolution block, a normalization layer, and an activation layer. The receptive field of the convolution block of the first branch is 1×1, the receptive field of the convolution block of the second branch is 3×3, and the receptive field of the convolution block of the third branch is 5×5; The deformable attention module includes two 3×3 branches, one of which is used to convert the colonoscopy image containing UCAN into a 2×k×k×H×W offset matrix using a 3×3 convolution kernel, where k represents a constant, H and W represent the height and width of the colonoscopy image containing UCAN, respectively; the other branch is used to extract a two-dimensional tensor of attention weights from the colonoscopy image containing UCAN using a 3×3 convolution kernel to learn the importance of features of different regions in the colonoscopy image containing UCAN; The comprehensive attention module includes a channel attention module and a spatial attention module. The output of the channel attention module is the input of the spatial attention module. The channel attention module includes a first pooling layer, a multi-layer perceptron, and a first softmax layer connected in sequence. The spatial attention module includes a second pooling layer, a convolution block with a receptive field of 1×1, and a second softmax layer connected in sequence. The first pooling layer and the second pooling layer both include a maximum pooling layer and an average pooling layer.

2. The method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model according to claim 1, characterized in that: The method uses a U-shaped deep neural network to train a colonoscopy-related ulcerative colitis tumor recognition model based on colonoscopy images containing UCAN, including: Preprocess the colonoscopy images containing UCAN; The initial learning rate of the colonoscopy-related ulcerative colitis tumor recognition model is set to 0.005, the momentum is 0.9, the weight decay is 0.0001, the neural network batch is 16, and the initial learning rate is updated using the update expression; The first loss function for identifying the UCAN boundary and the second loss function for identifying the properties of UCAN lesions in the colonoscopy-based ulcerative colitis-related tumor recognition model are set respectively.

3. The method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model according to claim 2, characterized in that: Update the expression to: = 0.005,Epoch = 200, In the update expression, represents the updated initial learning rate, Represents the initial learning rate, epoch represents the current training period, and Epoch represents the total training period.

4. The method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model according to claim 2, characterized in that: The first loss function is: In the first loss function, It indicates the first loss of the UCAN boundary identified by the colonoscopy ulcerative colitis-related tumor recognition model. It represents the dice loss of the UCAN boundary identified by the colonoscopy-based ulcerative colitis-related tumor recognition model. It represents the bce loss of the UCAN boundary identified by the colonoscopy ulcerative colitis-related tumor recognition model. and express and The corresponding weight coefficient is, = =0.5, It indicates the true value of the UCAN boundary identified by the colonoscopy-based ulcerative colitis-related tumor recognition model. Represents the predicted value of the colonoscopy-based ulcerative colitis-related tumor recognition model in identifying the UCAN boundary.

5. The method for constructing a colonoscopy-assisted ulcerative colitis-related tumor recognition model according to claim 4, characterized in that: The second loss function is: In the second loss function, Indicates the second loss of the colonoscopy-based ulcerative colitis-related tumor recognition model to identify the properties of UCAN lesions, It represents the true value of the UCAN lesion properties identified by the colonoscopy-based ulcerative colitis-related tumor recognition model. It represents the predictive value of the colonoscopic ulcerative colitis-associated tumor recognition model in identifying the nature of UCAN lesions.

6. A colonoscopy-based ulcerative colitis-related tumor recognition system, characterized in that: include: A data receiving module is used to: receive a colonoscopy image to be identified; An identification module is used to: based on the colonoscopic image to be identified, obtain a recognition result of whether UCAN is contained in the colonoscopic image to be identified by using the colonoscopic ulcerative colitis-related tumor identification model constructed by the colonoscopic ulcerative colitis-related tumor identification model construction method described in any one of claims 1-5, and when it is identified that UCAN is contained in the colonoscopic image to be identified, obtain a boundary image and lesion properties of UCAN.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the following steps are implemented: receiving a colonoscopy image to be identified; According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model is constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any one of claims 1 to 5, to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the following steps are implemented: receiving a colonoscopy image to be identified; According to the colonoscopic image to be identified, a colonoscopic ulcerative colitis-related tumor recognition model is constructed by the method for constructing a colonoscopic ulcerative colitis-related tumor recognition model described in any one of claims 1 to 5, to obtain a recognition result of whether the colonoscopic image to be identified contains UCAN, and when it is identified that the colonoscopic image to be identified contains UCAN, a boundary image and lesion properties of UCAN are obtained.

Images