Image segmentation method, device, computer readable storage medium and computer equipment

Abstract

The present application relates to an image segmentation method, device, computer-readable storage medium and computer equipment, and the method relates to image segmentation technology, including: performing down-sampling interaction processing on high-pixel image features and low-pixel image features corresponding to the image to be segmented, and then fusing them to obtain high-pixel down-sampling interaction features and low-pixel down-sampling interaction features; performing convolution interaction processing on high-pixel down-sampling interaction features and low-pixel down-sampling interaction features, and then fusing them to obtain high-pixel convolution interaction features and low-pixel convolution interaction features; performing up-sampling interaction processing on high-pixel convolution interaction features and low-pixel convolution interaction features, and then fusing them to obtain high-pixel up-sampling interaction features and low-pixel up-sampling interaction features; and segmenting the target object from the image to be segmented according to the high-pixel up-sampling interaction features and the low-pixel up-sampling interaction features. The solution provided by the present application can improve the precision of image segmentation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image segmentation method, apparatus, computer-readable storage medium and computer equipment. Background Art

[0002] Image segmentation technology is an important research direction in the field of computer vision and an important part of image semantic understanding. In recent years, with the gradual deepening of deep learning technology, image segmentation technology has made rapid progress. Deep neural networks have been widely used in image segmentation tasks and have achieved remarkable results.

[0003] Traditionally, some classic algorithms such as FCN are used for segmentation. However, using FCN for segmentation requires direct upsampling by a large multiple. The upsampling result is fuzzy and smooth, insensitive to the details in the image, and the segmentation result has low precision. Summary of the invention

[0004] Based on this, it is necessary to provide an image segmentation method, apparatus, computer-readable storage medium and computer equipment to address the technical problem of low precision in the image segmentation technology in the prior art.

[0005] An image segmentation method, comprising:

[0006] Obtain the image to be segmented;

[0007] Acquire high-pixel image features and low-pixel image features corresponding to the image to be segmented;

[0008] Downsampling and interactively processing the high-pixel image features and the low-pixel image features is performed and then fused to obtain high-pixel downsampling interactive features and low-pixel downsampling interactive features;

[0009] The high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature are subjected to convolution interaction processing and then fused to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature;

[0010] Performing up-sampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature, and then fusing them to obtain a high-pixel up-sampling interaction feature and a low-pixel up-sampling interaction feature;

[0011] According to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, the probability that each pixel in the image to be segmented belongs to the target object is determined, and the target object is segmented from the image to be segmented according to the probability.

[0012] A lesion segmentation method for a medical image, comprising:

[0013] Acquiring medical images;

[0014] Acquire high-pixel image features and low-pixel image features corresponding to the medical image;

[0015] Downsampling and interactively processing the high-pixel image features and the low-pixel image features is performed and then fused to obtain high-pixel downsampling interactive features and low-pixel downsampling interactive features;

[0016] The high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature are subjected to convolution interaction processing and then fused to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature;

[0017] Performing up-sampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature, and then fusing them to obtain a high-pixel up-sampling interaction feature and a low-pixel up-sampling interaction feature;

[0018] The probability that each pixel in the medical image belongs to a lesion is determined based on the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and the lesion is segmented from the medical image based on the probability through the output layer of the image segmentation model.

[0019] An image segmentation device, comprising:

[0020] The image to be segmented acquisition module is used to acquire the image to be segmented;

[0021] An image feature acquisition module, used to acquire high-pixel image features and low-pixel image features corresponding to the image to be segmented;

[0022] A downsampling interaction module is used to perform downsampling interaction processing on the high-pixel image features and the low-pixel image features and then fuse them to obtain high-pixel downsampling interaction features and low-pixel downsampling interaction features;

[0023] A convolution interaction module, used for performing convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and then fusing them to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature;

[0024] An upsampling interaction module, used for performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature;

[0025] The target object segmentation module is used to determine the probability that each pixel in the image to be segmented belongs to the target object according to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and segment the target object from the image to be segmented according to the probability.

[0026] A medical image lesion segmentation device, the device comprising:

[0027] A medical image acquisition module, used for acquiring medical images;

[0028] An image feature acquisition module, used to acquire high-pixel image features and low-pixel image features corresponding to the medical image;

[0029] A downsampling interaction module is used to perform downsampling interaction processing on the high-pixel image features and the low-pixel image features and then fuse them to obtain high-pixel downsampling interaction features and low-pixel downsampling interaction features;

[0030] A convolution interaction module, used for performing convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and then fusing them to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature;

[0031] An upsampling interaction module, used for performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature;

[0032] A lesion segmentation module is used to determine the probability that each pixel in the medical image belongs to a lesion based on the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and to segment the lesion from the medical image based on the probability through the output layer of the image segmentation model.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to execute the steps of the above-mentioned image segmentation method or the medical image lesion segmentation method.

[0034] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned image segmentation method or the medical image lesion segmentation method.

[0035] The above-mentioned image segmentation method, device, computer-readable storage medium and computer equipment, after obtaining the image to be segmented, obtain the high-pixel image features and low-pixel image features of the image to be segmented at two scales, and then continue to maintain the information interaction of the two scales based on the image features of the two scales, that is, in the downsampling interaction process, the image features of the two scales are simultaneously downsampled and interacted, in the convolution interaction process, the downsampling interaction features of the two scales are simultaneously convolutionally interacted, and in the upsampling interaction process, the convolution interaction features of the two scales are simultaneously upsampled and interacted, and then according to the obtained upsampling interaction features of the two scales, jointly determine the probability that each pixel in the image to be segmented belongs to the target object, and segment the target object from the image to be segmented according to the probability. In the whole processing process, through continuous information interaction, and only maintaining the interaction of the two-scale features, the image information extracted by the interaction is more comprehensive, which can significantly improve the precision of the segmentation; further, since each information interaction is based on the interaction features obtained from the previous information interaction, it effectively avoids the error caused by the information loss caused by downsampling, and can segment more detailed information from the image, further improving the precision of the image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A diagram showing an application environment of an image segmentation method in an embodiment;

[0037] Figure 2 is a schematic flow chart of an image segmentation method in one embodiment;

[0038] Figure 3 A schematic diagram of the structure of a branch network in an embodiment;

[0039] Figure 4 A schematic diagram of the structure of a downsampling interaction network in one embodiment;

[0040] Figure 5 Schematic diagram of the structure of a convolutional interaction network in one embodiment;

[0041] Figure 6 A schematic diagram of the structure of an upsampling interaction network in one embodiment;

[0042] Figure 7 A schematic diagram of the structure of a converged network in one embodiment;

[0043] Figure 8 A network architecture diagram of an image segmentation model in one embodiment;

[0044] Fig. 9 is a flowchart of an image segmentation method in a specific embodiment;

[0045] Fig.10is a schematic flow chart of a method for segmenting a lesion in a medical image in one embodiment;

[0046] Fig.11 is a structural block diagram of an image segmentation device in one embodiment;

[0047] Fig.12 is a structural block diagram of a device for segmenting a lesion in a medical image in one embodiment;

[0048] Fig.13 FIG. 4 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The image segmentation method provided in the embodiment of the present application relates to the image segmentation technology in computer vision. Computer vision is a science that studies how to make computer devices "see". To put it more concretely, it means using computer devices to identify and measure targets instead of human eyes, and further performing image processing so that the computer devices process the images into images that are more suitable for human eye observation or transmission to instruments for detection. Image segmentation is a crucial preprocessing for image recognition and computer vision, which is specifically explained through the following embodiments.

[0051] Figure 1 FIG. 1 is an application environment diagram of an image segmentation method in an embodiment. Figure 1 , the image segmentation method is applied to an image segmentation system. The image segmentation system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network. The terminal 110 can obtain the image to be segmented, and send the image to be segmented to the server 120. After the server 120 obtains the image to be segmented, it obtains the high-pixel image features and the low-pixel image features corresponding to the image to be segmented, and then performs down-sampling interaction processing on the high-pixel image features and the low-pixel image features, and then fuses them to obtain high-pixel down-sampling interaction features and low-pixel down-sampling interaction features, and then performs convolution interaction processing on the high-pixel down-sampling interaction features and the low-pixel down-sampling interaction features, and then fuses them to obtain high-pixel convolution interaction features and low-pixel convolution interaction features, and then performs up-sampling interaction processing on the high-pixel convolution interaction features and the low-pixel convolution interaction features, and then fuses them to obtain high-pixel up-sampling interaction features and low-pixel up-sampling interaction features, and then determines the probability that each pixel in the image to be segmented belongs to the target object according to the high-pixel up-sampling interaction features and the low-pixel up-sampling interaction features, and segments the target object from the image to be segmented according to the probability.

[0052] It can be understood that in other embodiments, after the terminal 110 obtains the image to be identified, the terminal 110 may directly process the image to be segmented to segment the target object from the image to be segmented.

[0053] The terminal 110 may be a desktop terminal or a mobile terminal, and the mobile terminal may include at least one of a mobile phone, a tablet computer, a laptop computer, a personal digital assistant, a wearable device, a medical imaging device, etc. The server 120 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0054] like Figure 2 As shown, in one embodiment, an image segmentation method is provided. This embodiment mainly applies the method to the above Figure 1 The server 120 in FIG. Figure 2 , the image segmentation method specifically includes the following steps:

[0055] S202: Obtain an image to be segmented.

[0056] The image to be segmented is the image to be segmented. Image segmentation is the process of segmenting the target object from the image including the target object. Its essence is the process of assigning a label to each pixel in the image. The target object included in the image to be segmented can be a person, an animal, or a target object in a specific scene, etc.

[0057] Specifically, the server may obtain an image sent by the terminal and use the image as the image to be segmented. The server may also obtain an image locally and use the image as the image to be segmented.

[0058] In one embodiment, the image to be segmented may include a medical image. A medical image, also called a medical image, refers to an internal tissue image of a biological body or a part of a biological body obtained by a medical instrument in a non-invasive manner for medical treatment or medical research. Specifically, a medical image may include a CT image obtained by performing a computed tomography (CT) scan of biological tissue, an MRI image obtained by performing a magnetic resonance imaging (Magnetic Resonance Imaging) on ​​biological tissue, and may also include an ultrasonic image, an X-ray image, an electrocardiogram, an electroencephalogram, or an optical photography image, etc. If the image to be segmented is a medical image, then the target object included in the image to be segmented may be a lesion in a biological tissue, and the process of image segmentation is to segment the lesion area from the medical image.

[0059] In one embodiment, the image to be segmented may also be a video frame in a video, for example, a frame in a medical imaging video.

[0060] S204, obtaining high-pixel image features and low-pixel image features corresponding to the image to be segmented.

[0061] Among them, image features are features that can differentiate image features. The image features corresponding to different images are different. Image features are the form of representing image features in the form of spatial vectors or spatial matrices. High-pixel image features and low-pixel image features are spatial matrices used to preliminarily represent image features after different processing of each pixel in the image to be processed. High-pixel image features and low-pixel image features preliminarily express the image information of the image to be processed from two different scales. High-pixel image features are image features with a larger scale than low-pixel image features. The larger the feature scale obtained after processing, the higher the resolution of the feature. The high-resolution feature has more image detail information. The smaller the scale, the lower the resolution of the feature. The low-resolution feature has more image global information. In this way, further processing based on high-pixel image features and low-pixel image features can obtain more comprehensive image information. When the image features are represented in the form of a spatial matrix, the scale of the image features can be understood as the size of the spatial matrix. For example, the scale of high-pixel image features is M*N, and the scale of low-pixel image features is M / 2*N / 2.

[0062] It can be understood that "high pixel" and "low pixel" are relative concepts in terms of scale, and the resolution of high pixel is higher than that of low pixel. In one embodiment, the scale of low pixel image features is 1 / 2 of the scale of high pixel image features, and the scale of high pixel image features is the same as the scale of the image to be processed. High pixel image features and low pixel image features express the image information of the image to be processed at different scales.

[0063] In one embodiment, the server can process the image to be segmented through the trained image segmentation model to segment the target object from the image to be segmented. Therefore, the server can process the image to be segmented through the image segmentation model to obtain high-pixel image features and low-pixel image features corresponding to the image to be segmented.

[0064] In one embodiment, the image segmentation model may be a neural network model based on machine learning. The neural network model may learn through image samples, thereby having specific capabilities. In this embodiment, the image segmentation model is a pre-trained model with image segmentation capabilities. The neural network model may be implemented using a convolutional neural network (CNN).

[0065] In one embodiment, the server may construct an initial model based on the initial model parameters, and then train the initial model through image samples with annotated segmentation results, and obtain the trained model parameters when the training stop condition is met. When performing image segmentation on the image to be processed, the model parameters may be obtained, and then the model parameters may be imported into the initial model to obtain an image segmentation model that can perform image segmentation processing on the image to be processed. In the image segmentation process, there is no need to manually define the input features of the image segmentation model, and an end-to-end processing process is achieved.

[0066] Optionally, you can obtain image samples for training the initial model by following these steps:

[0067] The original image sample is obtained, and the pixels belonging to the target object and the pixels not belonging to the target object in the original image sample are respectively labeled to obtain the image sample labeled with the segmentation result. For example, the category of the pixels belonging to the target object can be labeled as 0, and the category of the pixels not belonging to the target object in the original image sample can be labeled as 1, or the category of the pixels belonging to the target object can be labeled as 1, and the category of the pixels not belonging to the target object in the original image sample can be labeled as 0.

[0068] In order to improve the accuracy of the trained image segmentation model, it is necessary to increase the number of image samples. The original image can be subjected to data augmentation operations such as flipping, rotating, enlarging, reducing or contrast enhancement to obtain new image samples. These new image samples have certain information value in terms of directionality and scale, and the image segmentation model trained based on these samples is more accurate. Optionally, in the process of training the image segmentation model, the gradient descent method can be used to update the model parameters.

[0069] In one embodiment, the image segmentation model used to perform image segmentation processing on the image to be segmented can be trained by the following steps:

[0070] Obtain image samples; input the image samples into the initial model, and after processing by the initial model, obtain the predicted probability that each pixel in the image sample belongs to the target object, and determine the predicted pixel area belonging to the target object according to the predicted probability; determine the real pixel area belonging to the target object in the image sample according to the labeled data of the image sample; construct a loss function according to the intersection area of ​​the predicted pixel area and the real pixel area and the sum of the areas of the predicted pixel area and the real pixel area; minimize the loss function to obtain the error gradient, and adjust the model parameters according to the error gradient; continue to train the model according to the adjusted model parameters and the image samples in the image sample set, until the trained model parameters are obtained when the training is completed.

[0071] In one embodiment, the loss function Loss can be expressed by the following formula:

[0072] Loss=1-2*(S1∩S2) / (S1+S2);

[0073] Wherein, S1 represents the area of ​​the predicted pixel region, S2 represents the area of ​​the real pixel region, S1∩S2 represents the intersection area of ​​the predicted pixel region and the real pixel region, and S1+S2 represents the union area of ​​the predicted pixel region and the real pixel region. In one embodiment, the area of ​​the pixel region can be represented by the number of pixels included in the pixel region.

[0074] In one embodiment, the image segmentation model includes an input layer, a branch network, a downsampling interaction network, a convolutional interaction network, an upsampling interaction network, a fusion network and an output layer. The server can input the image to be processed into the image segmentation model, and obtain the high-pixel image features and low-pixel image features corresponding to the image to be processed through the input layer and branch network of the image segmentation model.

[0075] In one embodiment, the step of obtaining high-pixel image features and low-pixel image features corresponding to the image to be segmented includes: obtaining the original image features corresponding to the image to be segmented through the input layer; performing ordinary convolution transformation on the original image features through the branch network to obtain the high-pixel image features corresponding to the image to be segmented; and performing downsampling processing and hole convolution processing on the original image features in sequence through the branch network to obtain the low-pixel image features corresponding to the image to be segmented.

[0076] Specifically, the input layer is connected to the branch network, and the input layer is used to obtain the input image to be processed, and the image to be processed is processed by the convolution kernel included in the input layer to obtain the original image features corresponding to the image to be processed, and then the original image features are processed differently by the branch network to obtain features of different scales. In this way, the small receptive field information of the image to be processed possessed by the high-pixel image features and the large receptive field information of the image to be processed possessed by the low-pixel image features can be used for information interaction, which can improve the accuracy of segmentation of the image to be processed.

[0077] Optionally, the image to be processed obtained by the input layer is the pixel data of the image to be processed on the four color channels of RGBA, and the input layer may include a convolution kernel, such as a 3*3 convolution kernel, through which the pixel data of the four channels are processed to obtain the original image features. The original image features are consistent with the scale of the image to be processed.

[0078] Optionally, the branch network includes two layers, and the convolution kernel of the first layer performs ordinary convolution transformation on the original image features to obtain high-pixel image features corresponding to the image to be segmented. The high-pixel image features are consistent with the scale of the original image features, that is, consistent with the scale of the image to be processed; the original image features are also downsampled and processed with a hole convolution kernel of the second layer in sequence to obtain low-pixel image features corresponding to the image to be segmented. Due to the downsampling process, the scale of the low-pixel image features is smaller than that of the original image features, and the number of feature channels is doubled. After the hole convolution process, the receptive field (Receptive Field) of any feature point can be made larger, and the obtained low-pixel image features have a larger receptive field, so that the information of the larger receptive field in the image to be processed can be represented.

[0079] like Figure 3 FIG. 1 is a schematic diagram of a branch network structure in an embodiment. Figure 3 , the original image features 304 of the image to be processed 303 are obtained through the convolution kernel 302 of the input layer, and then the original image features 304 are subjected to ordinary convolution processing through the convolution kernel 306a of the upper layer of the branch network 306 (including two convolutions) to obtain high-pixel image features 308. Assuming that the scale of the image to be processed is M*N, the scale of the high-pixel image features is also M*N. The original image features 304 are sequentially down-sampled and processed with dilated convolution through the lower layer convolution kernel 306b of the branch network 306 to obtain low-pixel image features 310 with a scale of M / 2*N / 2 and doubled number of channels.

[0080] S206, performing down-sampling interaction processing on the high-pixel image features and the low-pixel image features, and then fusing them to obtain high-pixel down-sampling interaction features and low-pixel down-sampling interaction features.

[0081] Among them, the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature are both spatial matrices obtained after downsampling interaction processing of the high-pixel image features and low-pixel image features corresponding to the image to be processed, which are used to further represent the context information of each pixel in the image. The high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature express the context information of each pixel in the image to be processed from two different scales. The high-pixel downsampling interaction feature is a feature with a larger scale than the low-pixel downsampling interaction feature. The high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature can be referred to as features obtained after encoding the image to be processed. Specifically, after obtaining the high-pixel image features and low-pixel image features corresponding to the image to be processed through the branch network, the downsampling interaction network in the image segmentation model continues to perform information interaction based on the high-pixel image features and the low-pixel image features, respectively, to further extract the features of the image to be processed. The downsampling interaction network is used to perform information interaction after downsampling the features, so as to obtain image information of the image to be processed at different scales. In one embodiment, the scale of the low-pixel downsampling interaction feature is 1 / 2 of the scale of the high-pixel downsampling interaction feature.

[0082] In one embodiment, the steps of performing downsampling interaction processing on high-pixel image features and low-pixel image features and then fusing them to obtain high-pixel downsampling interaction features and low-pixel downsampling interaction features include: performing downsampling processing on high-pixel image features and then fusing them with low-pixel image features through a downsampling interaction network to obtain high-pixel downsampling interaction features; performing downsampling processing on high-pixel image features and low-pixel image features respectively and then fusing them through a downsampling interaction network to obtain low-pixel downsampling interaction features.

[0083] Specifically, the downsampling interaction network includes two layers, which are respectively used to process and fuse high-pixel image features and high-pixel image features of different scales obtained by the branch network, wherein the first layer is used to downsample the high-pixel image features according to different scales, and obtain feature maps with scales of 1 / 2 and 1 / 4 of the high-pixel image features, that is, 1 / 2 and 1 / 4 of the scale of the image to be processed, and the number of channels is doubled after downsampling. The second layer is used to perform convolution operations and downsampling processing on the low-pixel image features, and obtain feature maps with the same scale as the low-pixel image features and a scale of 1 / 2 of the low-pixel image features, that is, 1 / 2 and 1 / 4 of the scale of the image to be processed, respectively. Then, the feature maps with the same scale obtained by the first layer and the second layer are fused to obtain downsampled interaction features, wherein the two feature maps of higher scale (i.e., the feature maps with a scale of 1 / 2 of the image to be processed) are fused to obtain high-pixel downsampled interaction features, and the two feature maps of relatively lower scale (i.e., the feature maps with a scale of 1 / 4 of the image to be processed) are fused to obtain low-pixel downsampled interaction features. The fusion process can be to cascade two feature maps of the same scale according to the number of channels of the feature maps.

[0084] Optionally, in order to increase the receptive field of the convolution kernel in the subsequent processing process, the fused feature map can be further convolved after fusion, and the down-sampled interactive features can be obtained after the convolution operation. In one embodiment, the features obtained by fusion of two feature maps of higher scale can be further subjected to ordinary convolution operations to obtain high-pixel down-sampled interactive features. The feature map obtained by fusion of two feature maps of lower scale can be further subjected to a hole convolution operation to obtain low-pixel down-sampled interactive features and increase the receptive field. In this way, the receptive fields of the high-pixel down-sampled interactive features and the low-pixel down-sampled interactive features obtained by further convolution processing after fusion are quite different. The high-pixel down-sampled interactive features can better express the detailed information of the image to be processed, and the low-pixel down-sampled interactive features can better express the global information of the image to be processed, so that the image information can be extracted more comprehensively. In one embodiment, the image segmentation model can include at least two down-sampled interactive networks, so as to obtain an image segmentation model with stronger segmentation ability through a deeper neural network.

[0085] like Figure 4 FIG. 1 is a schematic diagram of a downsampling interaction network in an embodiment. Figure 4, the scale of the image to be processed is M*N, the scale of the high-pixel image feature is M*N, and the scale of the low-pixel image feature is M / 2*N / 2. The high-pixel image feature 402 is downsampled at different scales by the first layer in the downsampling interactive network to obtain feature maps with scales of M / 2*N / 2 and M / 4*N / 4 respectively; the low-pixel image feature 404 is convolved and downsampled by the second layer in the downsampling interactive network to obtain feature maps with scales of M / 2*N / 2 and M / 4*N / 4 respectively. 2*N / 2 feature map and a feature map with a scale of M / 4*N / 4; then the feature maps with a scale of M / 2*N / 2 obtained from the first layer and the second layer are fused to obtain 406, and then ordinary convolution processing is performed to obtain high-pixel down-sampling interactive features 408 with a scale of M / 2*N / 2, and the feature maps with a scale of M / 4*N / 4 obtained from the first layer and the second layer are fused to obtain 410, and then hole convolution processing is performed to obtain low-pixel down-sampling interactive features 412 with a scale of M / 4*N / 4.

[0086] S208, performing convolution interaction processing on the high-pixel downsampling interaction features and the low-pixel downsampling interaction features, and then fusing them to obtain high-pixel convolution interaction features and low-pixel convolution interaction features.

[0087] Among them, the high-pixel convolution interaction feature and the low-pixel convolution interaction feature are spatial matrices for representing image characteristics obtained after convolution interaction processing of the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature. The high-pixel convolution interaction feature and the low-pixel convolution interaction feature express the depth image information of the image to be processed from two different scales respectively. The high-pixel convolution interaction feature is a feature with a larger scale than the low-pixel convolution interaction feature. Specifically, after obtaining the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature corresponding to the image to be processed through the downsampling interaction network, the convolution interaction network in the image segmentation model continues to perform information interaction based on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature to continue to extract the features of the image to be processed. The convolution interaction network is used to perform information interaction after convolution processing on the image features to further obtain image information of the image to be processed at different scales. In one embodiment, the scale of the low-pixel convolution interaction feature is 1 / 2 of the scale of the high-pixel convolution interaction feature.

[0088] In one embodiment, high-pixel downsampling interaction features and low-pixel downsampling interaction features are convolutionally interactively processed and then fused to obtain high-pixel convolution interaction features and low-pixel convolution interaction features, including: upsampling the low-pixel downsampling interaction features through a convolutional interactive network and then fusing them with the high-pixel downsampling interaction features, and then performing ordinary convolution processing to obtain high-pixel convolution interaction features; downsampling the high-pixel downsampling interaction features through a convolutional interactive network and then fusing them with the low-pixel downsampling interaction features, and then performing hole convolution processing to obtain low-pixel convolution interaction features.

[0089] Specifically, the convolutional interaction network includes two layers, which are used to process and fuse the high-pixel down-sampling interaction features and low-pixel down-sampling interaction features of different scales obtained by the down-sampling interaction network, respectively. The second layer is used to perform up-sampling and dilated convolution processing on the low-pixel down-sampling interaction features, and fuse the features obtained by the up-sampling processing with the features of the first layer that perform ordinary convolution processing on the high-pixel down-sampling interaction features. The first layer is also used to perform down-sampling processing on the high-pixel down-sampling interaction features, and fuse the features after the down-sampling processing with the features of the second layer that perform dilated convolution processing on the low-pixel down-sampling interaction features.

[0090] Optionally, in order to increase the receptive field of the convolution kernel in the network during subsequent processing, the fused feature map can be further convolved after fusion, and the convolution interaction feature is obtained after the convolution operation. In one embodiment, the second layer is used to perform upsampling and hole convolution processing on the low-pixel downsampling interaction feature, and the properly upsampled feature is fused with the feature of the first layer that performs ordinary convolution processing on the high-pixel downsampling interaction feature, and then ordinary convolution processing is performed to obtain high-pixel convolution interaction features. In another embodiment, the first layer is also used to downsample the high-pixel downsampling interaction feature, and the downsampled feature is fused with the feature of the second layer that performs hole convolution processing on the low-pixel downsampling interaction feature, and then hole convolution processing is performed to obtain low-pixel convolution interaction features to increase the receptive field. Similarly, the receptive field between the high-pixel convolution interaction feature obtained by further convolution processing after fusion and the low-pixel convolution interaction feature is quite different, and the high-pixel convolution interaction feature can better express the detail information of the image to be processed, and the low-pixel convolution interaction feature can better express the global information of the image to be processed, so that the image information can be extracted more comprehensively.

[0091] like Figure 5 FIG. 1 is a schematic diagram of a convolutional interaction network structure in one embodiment. Figure 5, the scale of the image to be processed is M*N, the scale of the high-pixel downsampling interactive feature is M / 4*N / 4, and the scale of the low-pixel downsampling interactive feature is M / 8*N / 8. The first layer in the convolutional interaction network performs ordinary convolution processing and downsampling processing on the high-pixel down-sampled interaction feature 502 to obtain a feature map with a scale of M / 4*N / 4 and a feature map with a scale of M / 8*N / 8. The second layer in the convolutional interaction network performs hole convolution processing and upsampling processing on the low-pixel down-sampled interaction feature 504 to obtain a feature map with a scale of M / 8*N / 8 and a feature map with a scale of M / 4*N / 4. Then, the feature maps with a scale of M / 4*N / 4 obtained by the first layer and the second layer are fused to obtain a feature map 506, and then ordinary convolution processing is performed to obtain a high-pixel convolution interaction feature 508 with a scale of M / 4*N / 4. The feature maps with a scale of M / 8*N / 8 obtained by the first layer and the second layer are fused to obtain a feature map 510, and then hole convolution processing is performed to obtain a low-pixel convolution interaction feature 512 with a scale of M / 8*N / 8.

[0092] S210, performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature, and then fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature.

[0093] Among them, the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature are both spatial matrices obtained after the high-pixel convolution interaction feature and the low-pixel convolution interaction feature are further upsampled and interactively processed to represent the classification tendency of each pixel in the image. The high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature respectively express the classification tendency of each pixel in the image to be processed from two different scales. The high-pixel upsampling interaction feature is a feature with a larger scale than the low-pixel upsampling interaction feature. The high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature can be referred to as decoding features corresponding to the image to be processed. Specifically, after obtaining the high-pixel convolution interaction feature and the low-pixel convolution interaction feature corresponding to the image to be processed through the convolution interaction network, the upsampling interaction network in the image segmentation model continues to perform information interaction based on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature. The upsampling interaction network is used to perform information interaction after the feature continues to be upsampled. In one embodiment, the scale of the low-pixel upsampling interaction feature is 1 / 2 of the scale of the high-pixel upsampling interaction feature.

[0094] In one embodiment, high-pixel convolution interaction features and low-pixel convolution interaction features are upsampled and interactively processed and then fused to obtain high-pixel up-sampling interaction features and low-pixel up-sampling interaction features, including: through an upsampling interaction network, the low-pixel convolution interaction features are upsampled and then fused with the high-pixel convolution interaction features to obtain low-pixel up-sampling interaction features; through an upsampling interaction network, the high-pixel convolution interaction features and the low-pixel convolution interaction features are upsampled and then fused to obtain high-pixel up-sampling interaction features.

[0095] Specifically, the upsampling interaction network includes two layers, which are respectively used to process and fuse the high-pixel convolution interaction features and low-pixel convolution interaction features of different scales obtained through the convolution interaction network, wherein the first layer is used to upsample the low-pixel convolution interaction features at different scales to obtain feature maps with scales of 1 / 2 and 1 / 4 of the low-pixel convolution interaction features, and the number of channels is halved after upsampling. The second layer is used to perform convolution operations and upsampling on the high-pixel convolution interaction features to obtain feature maps with the same scale as the high-pixel convolution interaction features and a scale of 1 / 2 of the high-pixel convolution interaction features; then the feature maps of the same scale obtained by the first layer and the second layer are fused to obtain the upsampled interaction features, wherein the two feature maps of the higher scale are fused to obtain the high-pixel upsampled interaction features, and the two feature maps of the relatively lower scale are fused to obtain the low-pixel upsampled interaction features. The fusion process can be to cascade the two feature maps of the same scale according to the number of channels of the feature maps.

[0096] Optionally, after fusion, the fused feature map can be further convolved to obtain upsampled interactive features. In one embodiment, the features obtained by fusion of two feature maps of higher scale can be further convolved to obtain high-pixel upsampled interactive features. The features obtained by fusion of two feature maps of lower scale can be further convolved to obtain low-pixel upsampled interactive features to increase the receptive field. In this way, the receptive field between the high-pixel upsampled interactive features and the low-pixel upsampled interactive features obtained by further convolution operation after fusion is further increased as the network goes deeper, so that image information can be extracted more comprehensively. In one embodiment, the image segmentation model can include at least two upsampled interactive networks, so as to obtain an image segmentation model with stronger segmentation ability through a deeper neural network.

[0097] Optionally, the upsampling interaction network can also obtain the high-pixel downsampling interaction features and low-pixel downsampling interaction features output by the downsampling interaction network, and jump-connect the high-pixel downsampling interaction features with two feature maps of higher scales through the first layer of the upsampling interaction network to obtain the high-pixel upsampling interaction features; jump-connect the low-pixel downsampling interaction features with two feature maps of lower scales through the second layer of the upsampling interaction network to obtain the low-pixel downsampling interaction features. Accordingly, the feature maps from three aspects obtained after the jump connection can be convolved to expand the receptive field.

[0098] In this embodiment, the down-sampling interaction features extracted by the down-sampling interaction network are jump-connected with the new feature map obtained by up-sampling through the up-sampling interaction network, which can retain the important feature information in the previous down-sampling process to the greatest extent.

[0099] like Figure 6 FIG. 1 is a schematic diagram of the structure of an upsampling interaction network in an embodiment. Figure 6 , the scale of the image to be processed is M*N, the scale of the high-pixel convolution interaction feature is M / 4*N / 4, and the scale of the low-pixel convolution interaction feature is M / 8*N / 8. The first layer in the upsampling interaction network performs ordinary convolution processing and upsampling processing on the high-pixel convolution interaction feature 602 to obtain a feature map with a scale of M / 4*N / 4 and a feature map with a scale of M / 2*N / 2. The second layer in the upsampling interaction network performs upsampling processing on the low-pixel convolution interaction feature 604 at different scales to obtain a feature map with a scale of M / 2*N / 2 and a feature map with a scale of M / 4*N / 4. Then, the feature maps with a scale of M / 2*N / 2 obtained by the first layer and the second layer are fused to obtain a feature map 606, and then ordinary convolution processing is performed to obtain a high-pixel upsampled interaction feature 608 with a scale of M / 2*N / 2. The feature maps with a scale of M / 4*N / 4 obtained by the first layer and the second layer are fused to obtain a feature map 610, and then a hole convolution processing is performed to obtain a low-pixel upsampled interaction feature 612 with a scale of M / 4*N / 4.

[0100] S212, determining the probability that each pixel in the image to be segmented belongs to the target object according to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and segmenting the target object from the image to be segmented according to the probability.

[0101] The image segmentation model provided in the present application is an end-to-end model, the input data is the image to be processed, the output data is the segmented image, the element value of the position corresponding to the pixel in the segmented image is 0, which indicates that the corresponding pixel belongs to the target object area, and the element value of the position corresponding to the pixel is 1, which indicates that the corresponding pixel does not belong to the target object area. Or, conversely, the element value of the position corresponding to the pixel in the segmented image is 1, which indicates that the corresponding pixel belongs to the target object area, and the element value of the position corresponding to the pixel is 0, which indicates that the corresponding pixel does not belong to the target object area.

[0102] Specifically, after obtaining the high-pixel upsampling interaction features and the low-pixel upsampling interaction features through the upsampling interaction network, the convergence network in the image segmentation model is continued to fuse the high-pixel upsampling interaction features and the low-pixel upsampling interaction features to obtain the prediction features for pixel-level classification, and the probability of each pixel in the image to be processed belonging to the target is determined based on the prediction features, and then the probability is binarized (0 or 1) according to the preset threshold to segment the target object from the image to be processed. The probability can reflect the possibility that the corresponding pixel belongs to the target object. For example, if the probability is greater than the preset threshold, it is determined that the pixel belongs to the target object. If the probability is less than the preset threshold, it is determined that the pixel does not belong to the target object or belongs to the background area in the image to be processed. The preset threshold can be 0.5, for example. The convergence network in the image segmentation model corresponds to the branch network, which is used to obtain the prediction features after fusing the image features. The convergence network is connected to the output layer of the image segmentation model, and the output layer performs pixel-level classification on the prediction features to determine whether each pixel in the image to be processed belongs to the target object area.

[0103] In one embodiment, the probability that each pixel in the image to be segmented belongs to the target object is determined based on the high-pixel upsampling interaction features and the low-pixel upsampling interaction features, including: obtaining the original image features corresponding to the image to be segmented through a fusion network; upsampling the low-pixel upsampling interaction features through the fusion network and fusing them with the high-pixel upsampling interaction features, and then jump-connecting the original image features to obtain the predicted features corresponding to the image to be segmented; and outputting the probability that each pixel in the image to be segmented belongs to the target object according to the predicted features through an output layer.

[0104] Specifically, the confluence network includes two layers. First, the confluence network obtains the original image features of the scale M*N obtained by the input layer. Then, the first layer of the confluence network performs ordinary convolution processing on the high-pixel upsampling interaction features of the scale M*N to obtain a feature map of the scale M*N. The second layer of the confluence network performs upsampling processing on the low-pixel upsampling interaction features of the scale M / 2*N / 2 to obtain a feature map of the scale M*N. The feature maps obtained by the first and second layers are fused and jump-connected with the original image features of the scale M*N, and then ordinary convolution processing is performed to obtain the predicted features of the scale M*N corresponding to the image to be segmented. Finally, the output layer of the image segmentation model predicts the probability that each pixel in the image to be segmented belongs to the target object according to the predicted features, and outputs a probability image. The target object can be segmented from the image to be processed according to the probability image.

[0105] like Figure 7 FIG. 1 is a schematic diagram of a structure of a converged network in an embodiment, referring to Figure 7 , the high-pixel upsampling interaction feature 702 and the low-pixel upsampling interaction feature 704 are further processed through the fusion network, and the high-pixel upsampling interaction feature 702 is subjected to ordinary convolution processing by the upper-layer convolution kernel 706, and then the low-pixel upsampling interaction feature 704 is subjected to upsampling processing by the lower-layer convolution kernel 708 to obtain the same-scale feature fusion, and then the convolution kernel 710 is used to perform a convolution operation on the fused features to obtain the prediction feature 712 corresponding to the image to be processed.

[0106] The above-mentioned image segmentation method, after obtaining the image to be segmented, obtains the high-pixel image features and low-pixel image features of the image to be segmented at two scales, and then based on the image features of the two scales, continuously maintains the information interaction of the two scales, that is, in the downsampling interaction process, the two-scale image features are simultaneously downsampled and interacted, in the convolution interaction process, the two-scale downsampling interaction features are simultaneously convolutionally interacted, and in the upsampling interaction process, the two-scale convolution interaction features are simultaneously upsampled and interacted, and then according to the obtained two-scale upsampling interaction features, jointly determine the probability that each pixel in the image to be segmented belongs to the target object, and segment the target object from the image to be segmented according to the probability. In the whole processing process, through continuous information interaction, and only maintaining the interaction of the two-scale features, the image information extracted interactively is more comprehensive, which can significantly improve the precision of segmentation; further, since each information interaction is based on the interaction features obtained from the previous information interaction, it effectively avoids the error caused by information loss caused by downsampling, can segment more detailed information from the image, and further improves the precision of image segmentation.

[0107] In the embodiments provided in the present application, the branch network, the downsampling interaction network, the convolution interaction network and the upsampling interaction network all include two layers, and the corresponding convolution operation also includes two layers. The convolution processing in the first layer is a normal convolution operation, and the convolution processing in the second layer is a dilated convolution operation. The dilated convolution can increase the receptive field of the convolution kernel. Each dilated convolution, coupled with the step size of the downsampling processing in the downsampling interaction network, can double the increased receptive field. As the network goes deeper, the receptive fields of the first layer and the second layer are very different after multiple dilated convolutions, and the global information of the image to be processed can be extracted.

[0108] Reference Figure 3 , assuming that the size of the convolution kernel 302 of the input layer is 3*3, then the receptive field of the convolution kernel is 3, and the receptive field of the obtained original image feature is 3. The convolution kernel size of the two convolutions in the upper layer of the branch network 306 is 3*3, then the receptive field of the first convolution kernel is 5, and the receptive field of the second convolution kernel is 7, so that the receptive field of the obtained high-pixel image feature 308 is 7 pixels, and the local information of the image can be obtained. Assuming that the convolution kernel size of the downsampling processing in the lower layer of the branch network 306 is 2*2, the step size is 2, and the receptive field of the convolution kernel is 4, the convolution kernel size of the lower layer hole convolution processing is 3*3, Dilation=1, then the receptive field of the convolution kernel is 12, indicating that the receptive field of the low-pixel image feature 310 is 12 pixels, which can obtain more image information than the upper layer. Among them, Dilation represents the number of spaces between each element in the convolution kernel. The number of spaces is 1, and 1 0 is inserted between the elements. It can be seen that after the ordinary convolution of the upper layer and the hole convolution of the lower layer, the receptive field difference between the high-pixel image feature 308 and the low-pixel image feature 310 output by the branch network is already very large. With the increase of downsampling times and the deepening of the network, this receptive field difference will become larger and larger. Then, the features obtained based on the huge difference in receptive fields are interacted and fused to obtain more comprehensive image information.

[0109] In one embodiment, the steps of downsampling high-pixel image features and fusing them with low-pixel image features through a downsampling interaction network to obtain high-pixel downsampling interaction features specifically include: downsampling high-pixel image features and fusing them with low-pixel image features through a first downsampling interaction network, and then performing ordinary convolution processing to obtain a first high-pixel downsampling interaction feature; downsampling high-pixel image features and low-pixel image features through the first downsampling interaction network, respectively, and then fusing them, and then performing hole convolution processing to obtain a first low-pixel downsampling interaction feature; downsampling the first high-pixel downsampling interaction feature and fusing it with the first low-pixel downsampling interaction feature through a second downsampling interaction network, and then performing ordinary convolution processing to obtain a high-pixel downsampling interaction feature.

[0110] In this embodiment, the image segmentation model includes two downsampling interaction networks, namely a first downsampling interaction network and a second downsampling interaction network. The first downsampling interaction network is connected to the second downsampling interaction network. The high-pixel image features and the low-pixel image features are first processed by the first downsampling interaction network, and then the processing is continued by the second downsampling interaction network to obtain downsampling interaction features. Of course, the image segmentation model can also include more downsampling interaction networks, so that feature information of multiple scales can be extracted from the image to be processed, and more accurate prediction features can be obtained after upsampling.

[0111] Specifically, each downsampling interaction network includes two layers. First, through the first layer in the first downsampling interaction network, the high-pixel image features with a scale of M*N are downsampled at different scales to obtain feature maps with scales of M / 2*N / 2 and M / 4*N / 4; through the second layer in the first downsampling interaction network, the low-pixel image features with a scale of M / 2*N / 2 are convolved and downsampled to obtain feature maps with scales of M / 2*N / 2 and M / 4*N / 4; then, the feature maps with a scale of M / 2*N / 2 obtained by the first layer and the second layer are fused and then subjected to ordinary convolution processing to obtain the first high-pixel downsampling interaction features with a scale of M / 2*N / 2 output by the first interaction network; the feature maps with a scale of M / 4*N / 4 obtained by the first layer and the second layer are fused and then subjected to hole convolution processing to obtain the first low-pixel downsampling interaction features with a scale of M / 4*N / 4 output by the first interaction network.

[0112] Next, the processing continues through the second downsampling interaction network, which also includes two layers. First, through the first layer in the second downsampling interaction network, the first high-pixel downsampling interaction feature with a scale of M / 2*N / 2 is downsampled at different scales to obtain feature maps with scales of M / 4*N / 4 and M / 8*N / 8; through the second layer in the second downsampling interaction network, the first low-pixel downsampling interaction feature with a scale of M / 4*N / 4 is convolved and downsampled to obtain feature maps with scales of M / 4*N / 4 and M / 8*N / 8; then the feature maps with a scale of M / 4*N / 4 obtained by the first layer and the second layer are fused and ordinary convolution is performed to obtain high-pixel downsampling interaction features with a scale of M / 4*N / 4 output by the second interaction network.

[0113] In one embodiment, a downsampling interaction network is used to downsample high-pixel image features and low-pixel image features and then fuse them to obtain low-pixel downsampling interaction features, including: downsampling a first high-pixel downsampling interaction feature and a first low-pixel downsampling interaction feature and then fuse them through a second downsampling interaction network, and then performing a dilated convolution process to obtain a low-pixel downsampling interaction feature.

[0114] Following the above processing, the feature maps with a scale of M / 8*N / 8 obtained by the first layer and the second layer of the second down-sampling interaction network are fused and then subjected to dilated convolution processing to obtain the low-pixel down-sampling interaction features with a scale of M / 8*N / 8 output by the second interaction network.

[0115] In the above embodiment, the image segmentation model includes at least two downsampling interaction networks, each of which includes two layers, and can obtain more comprehensive image information from the image to be processed by interacting with image information of different scales between the two layers.

[0116] In one embodiment, the upsampling interaction network includes a first upsampling interaction network and a second upsampling interaction network; through the upsampling interaction network, the low-pixel convolution interaction feature is upsampled and fused with the high-pixel convolution interaction feature to obtain the low-pixel upsampling interaction feature, including: through the first upsampling interaction network, the first high-pixel downsampling interaction feature and the first low-pixel downsampling interaction feature output by the first downsampling interaction network are obtained, and through the second upsampling interaction network, the high-pixel image feature and the low-pixel image feature are obtained; through the first upsampling interaction network, the low-pixel convolution interaction feature is upsampled and fused with the high-pixel convolution interaction feature The first low-pixel up-sampling interaction feature is obtained by performing a hole convolution process after fusion and then jump-connecting it with the first low-pixel down-sampling interaction feature; the low-pixel convolution interaction feature and the high-pixel convolution interaction feature are up-sampled and fused through the first up-sampling interaction network, and then jump-connecting the first high-pixel down-sampling interaction feature and then performing a normal convolution process to obtain the first high-pixel up-sampling interaction feature; the first low-pixel up-sampling interaction feature is up-sampled and fused with the first high-pixel up-sampling interaction feature through the second up-sampling interaction network, and then jump-connecting the low-pixel image feature and then performing a hole convolution process to obtain the low-pixel up-sampling interaction feature.

[0117] In this embodiment, the image segmentation model includes two upsampling interaction networks, namely the first upsampling interaction network and the second upsampling interaction network. The first upsampling interaction network is connected to the second upsampling interaction network. The high-pixel convolution interaction features and the low-pixel convolution interaction features are first processed by the first upsampling interaction network, and then jump-connected with the first high-pixel downsampling interaction features output by the first downsampling interaction network. Then, the processing is continued through the second upsampling interaction network, and jump-connected with the first low-pixel downsampling interaction features output by the first downsampling interaction network to obtain the upsampling interaction features. Accordingly, the image segmentation model can also include more upsampling interaction networks that are symmetrical to the downsampling interaction networks.

[0118] Specifically, each upsampling interaction network includes two layers. First, the first high-pixel downsampling interaction feature with a scale of M / 2*N / 2 and the first low-pixel downsampling interaction feature with a scale of M / 4*N / 4 output by the previous downsampling interaction network are obtained through the first upsampling interaction network; then, through the first layer of the first upsampling network, the high-pixel convolution interaction feature with a scale of M / 4*N / 4 is subjected to ordinary convolution processing and upsampling processing to obtain feature maps with scales of M / 4*N / 4 and M / 2*N / 2, respectively; through the second layer of the first upsampling interaction network, the low-pixel convolution interaction feature with a scale of M / 8*N / 8 is subjected to The interactive features are upsampled at different scales to obtain feature maps with scales of M / 4*N / 4 and M / 2*N / 2 respectively; the feature maps with scales of M / 4*N / 4 obtained from the first and second layers are fused and jump-connected with the first low-pixel down-sampling interactive features, and then a hole convolution process is performed to obtain the first low-pixel up-sampling interactive features with a scale of M / 4*N / 4; the feature maps with scales of M / 2*N / 2 from the first and second layers are fused and jump-connected with the first high-pixel down-sampling interactive features, and then a normal convolution process is performed to obtain the first high-pixel up-sampling interactive features with a scale of M / 2*N / 2.

[0119] Next, the second upsampling interaction network is used for processing, and the second upsampling interaction network also includes two layers. First, the high-pixel image features with a scale of M*N and the low-pixel image features with a scale of M / 2*N / 2 output by the previous branch network are obtained through the second upsampling interaction network; then, the first high-pixel upsampling interaction features with a scale of M / 2*N / 2 are subjected to ordinary convolution processing and upsampling processing through the first layer of the second upsampling interaction network, and feature maps with scales of M / 2*N / 2 and M*N are obtained respectively; through the second layer of the second upsampling interaction network, the first low-pixel upsampling interaction features with a scale of M / 4*N / 4 are subjected to upsampling processing according to different scales, and feature maps with scales of M / 2*N / 2 and M*N are obtained respectively; then, the feature maps with a scale of M / 2*N / 2 obtained from the first layer and the second layer are fused and jump-connected with the low-pixel image features with a scale of M / 2*N / 2, and then a hole convolution processing is performed to obtain a low-pixel upsampling interaction feature with a scale of M / 2*N / 2.

[0120] In one embodiment, through an upsampling interaction network, high-pixel convolution interaction features and low-pixel convolution interaction features are upsampled and fused respectively to obtain high-pixel upsampling interaction features, including: through a second upsampling interaction network, a first high-pixel upsampling interaction feature and a first low-pixel upsampling interaction feature are upsampled and fused respectively, and then jump-connected to the high-pixel image features and then perform ordinary convolution processing to obtain high-pixel upsampling interaction features.

[0121] Following the above processing, the feature maps with a scale of M*N obtained by the first and second layers of the second upsampling interaction network are fused and jump-connected with the high-pixel image features with a scale of M*N, and then ordinary convolution processing is performed to obtain high-pixel upsampling interaction features with a scale of M*N.

[0122] In the above embodiment, the image segmentation model includes at least two upsampling interaction networks, each of which includes two layers, and can obtain more comprehensive image information from the image to be processed by interacting with image information of different scales between the two layers.

[0123] like Figure 8 As shown, it is a network architecture diagram of an image segmentation model in one embodiment, referring to Figure 8 , the network architecture of the image segmentation model is explained based on the image segmentation process of the image to be processed. Figure 8In the figure, the image to be processed is a collected picture including a heart, and after being processed by the image segmentation model, the lesion area can be segmented from the image. It can be understood that the image to be processed can be other images. The image segmentation model includes an input layer, a branch network, a first down-sampling interaction network, a second down-sampling interaction network, a convolution interaction network, a first up-sampling interaction network, a second up-sampling interaction network, a confluence network and an output layer.

[0124] After obtaining the image 801 to be processed with a scale of M*N, the image 801 to be processed is convolved by the convolution kernel 802 in the input layer to obtain the original image feature 803.

[0125] The two convolution kernels 804a and 804b of the first layer of the branch network are used to perform ordinary convolution processing on the original image feature 803 to obtain a high-pixel image feature 805a with a scale of M*N; the original image feature 803 is down-sampled by the downsampling convolution kernel 804c of the second layer in the branch network, and then a dilated convolution processing is performed by the convolution kernel 804d to obtain a low-pixel image feature 805b with a scale of M / 2*N / 2.

[0126] The downsampling convolution kernels 806a and 806b in the first layer of the first downsampling interactive network are used to downsample the high-pixel image feature 805a at different scales to obtain feature maps with scales of M / 2*N / 2 and M / 4*N / 4. The ordinary convolution kernel 806c and the downsampling convolution kernel 806d in the second layer of the first downsampling interactive network are used to perform ordinary convolution and downsampling processing on the low-pixel image feature 805b to obtain feature maps with scales of M / 2*N / 2 and M / 4*N / 4. Then, the feature maps with a scale of M / 2*N / 2 obtained from the first and second layers are fused and then subjected to ordinary convolution processing through the convolution kernel 806e to obtain a first high-pixel downsampling interactive feature 807a with a scale of M / 2*N / 2. The feature maps with a scale of M / 4*N / 4 obtained from the first and second layers are fused and then subjected to hole convolution processing through the convolution kernel 806f to obtain a first low-pixel downsampling interactive feature 807b with a scale of M / 4*N / 4.

[0127] The first high-pixel down-sampled interactive feature 807a is down-sampled at different scales by the down-sampling convolution kernels 808a and 808b in the first layer of the second down-sampling interactive network to obtain feature maps with scales of M / 4*N / 4 and M / 8*N / 8, and the first low-pixel down-sampled interactive feature 807b is down-sampled by the ordinary convolution kernel 808c and the down-sampling convolution kernel 808d in the second layer of the second down-sampling interactive network to obtain feature maps with scales of M / 4*N / 4 and M / 8*N / 8. N / 4, M / 8*N / 8 feature maps, and then the feature maps of the first layer and the second layer with a scale of M / 4*N / 4 are fused and then processed by ordinary convolution kernel 808e to obtain high-pixel down-sampling interactive features 809a with a scale of M / 4*N / 4; the feature maps of the first layer and the second layer with a scale of M / 8*N / 8 are fused and then processed by dilated convolution kernel 808f to obtain low-pixel down-sampling interactive features 809b with a scale of M / 8*N / 8.

[0128] The high-pixel down-sampled interaction feature 809a is convolved and down-sampled by the common convolution kernel 810a and the down-sampling convolution kernel 810b in the first layer of the convolutional interaction network, respectively, to obtain feature maps with scales of M / 4*N / 4 and M / 8*N / 8; the low-pixel down-sampled interaction feature 809b is convolved and up-sampled by the common convolution kernel 810c and the up-sampling convolution kernel 810d in the second layer of the convolutional interaction network 810, respectively, to obtain feature maps with scales of M / 8* N / 8, and M / 4*N / 4 feature maps; the feature maps of the first and second layers with a scale of M / 4*N / 4 are fused and then processed by ordinary convolution kernel 810e to obtain high-pixel convolution interaction features 811a with a scale of M / 4*N / 4; the feature maps of the first and second layers with a scale of M / 8*N / 8 are fused and then processed by dilated convolution kernel 810f to obtain low-pixel convolution interaction features 811b with a scale of M / 8*N / 8.

[0129] The ordinary convolution kernel 812a and the upsampling convolution kernel 812b in the first layer of the first upsampling interaction network are used to perform ordinary convolution processing and upsampling processing on the high-pixel convolution interaction feature 811a respectively, so as to obtain feature maps with scales of M / 4*N / 4 and M / 2*N / 2. The upsampling convolution kernels 812c and 812d in the second layer of the first upsampling interaction network 812 are used to perform upsampling processing on the low-pixel convolution interaction feature 811b according to different scales, so as to obtain feature maps with scales of M / 4*N / 4 and M / 2*N / 2. The feature maps with scales of M / 4*N / 4 and M / 2*N / 2 obtained in the first and second layers are respectively used to perform upsampling processing on the low-pixel convolution interaction feature 811b according to different scales. The feature map of 2*N / 2 is fused with the first high-pixel down-sampling interaction feature 807a obtained by the first down-sampling interaction network, and then ordinary convolution processing is performed through the convolution kernel 812e to obtain the first high-pixel up-sampling interaction feature 813a with a scale of M / 2*N / 2; the feature map of the scale of M / 4*N / 4 obtained by the first and second layers is fused with the first low-pixel down-sampling interaction feature 807b obtained by the first down-sampling interaction network, and then hole convolution processing is performed through the convolution kernel 812f to obtain the first low-pixel up-sampling interaction feature 813b with a scale of M / 4*N / 4.

[0130] The first high-pixel up-sampled interactive feature 813a is subjected to ordinary convolution processing and up-sampling processing respectively by the ordinary convolution kernel 814a and the up-sampling convolution kernel 814b in the first layer of the second up-sampling interactive network, and the feature maps with scales of M / 2*N / 2 and M*N are obtained. The first low-pixel up-sampled interactive feature 813b is subjected to up-sampling processing according to different scales by the up-sampling convolution kernels 814c and 814d in the second layer of the second up-sampling interactive network 814, and the feature maps with scales of M / 2*N / 2 and M*N are obtained. The feature maps with a scale of M*N obtained by the first and second layers are fused with the high-pixel image feature 805a obtained by the branch network and then subjected to ordinary convolution processing through the convolution kernel 814e to obtain a high-pixel up-sampled interactive feature 815a with a scale of M*N. The feature maps with a scale of M / 2*N / 2 obtained by the first and second layers are fused with the low-pixel image feature 805b obtained by the branch network and then subjected to dilated convolution processing through the convolution kernel 814f to obtain a low-pixel up-sampled interactive feature 815b with a scale of M / 2*N / 2.

[0131] The high-pixel upsampling interaction feature 815a is convolved by the convolution kernel 816a of the first layer of the fusion network, and the low-pixel upsampling interaction feature 815b is upsampled by the upsampling convolution kernel 816b of the second layer. The feature map with a scale of M*N obtained after processing the first and second layers is fused with the original image feature 803 obtained by the input layer and then convolved by the convolution kernel 816c to obtain the predicted feature 817. Finally, the convolution kernel 818 of the output layer outputs the probability map 820 according to the predicted feature 817.

[0132] In one embodiment, the detailed parameters of each network in the above-mentioned image segmentation model 800 are shown in the following table. In the following table, Kernel represents the size of the convolution kernel, MaxPooling represents the size of the downsampling convolution kernel, Stride represents the step size of the convolution kernel moving on the feature map, and Padding represents the peripheral padding of the feature map, so that the size of the feature map will not be lost after each convolution operation on the feature. UpSampling_Scalefactor represents the upsampling scale factor. channel_in represents the number of channels of the feature map before processing, and channel_out represents the number of channels of the feature map after processing. Assuming that the size of the feature map before the convolution operation is M*N, the size of the feature map after the convolution operation is (M+2×Padding-Kernel+1)*(N+2×Padding-Kernel+1).

[0133]

[0134] like Fig. 9 As shown, in a specific embodiment, the image segmentation method includes the following steps:

[0135] S902: Obtain an image to be segmented.

[0136] S904: Input the image to be segmented into the image segmentation model.

[0137] S906, obtaining original image features corresponding to the image to be segmented through the input layer of the image segmentation model.

[0138] S908, through the branch network of the image segmentation model, ordinary convolution transformation is performed on the original image features to obtain high-pixel image features corresponding to the image to be segmented; down-sampling processing and hole convolution processing are performed on the original image features in turn to obtain low-pixel image features corresponding to the image to be segmented.

[0139] S910, through the first downsampling interaction network of the image segmentation model, downsample the high-pixel image features and fuse them with the low-pixel image features, and then perform ordinary convolution processing to obtain the first high-pixel downsampling interaction features; downsample the high-pixel image features and the low-pixel image features respectively and fuse them, and then perform hole convolution processing to obtain the first low-pixel downsampling interaction features.

[0140] S912, through the second downsampling interaction network of the image segmentation model, downsample the first high-pixel downsampling interaction feature and fuse it with the first low-pixel downsampling interaction feature, and then perform ordinary convolution processing to obtain high-pixel downsampling interaction features; downsample the first high-pixel downsampling interaction feature and the first low-pixel downsampling interaction feature respectively, and then fuse them, and then perform hole convolution processing to obtain low-pixel downsampling interaction features.

[0141] S914, through the convolutional interaction network of the image segmentation model, the low-pixel downsampling interaction features are upsampled and fused with the high-pixel downsampling interaction features, and then ordinary convolution is performed to obtain high-pixel convolution interaction features; the high-pixel downsampling interaction features are downsampled and fused with the low-pixel downsampling interaction features, and then hole convolution is performed to obtain low-pixel convolution interaction features.

[0142] S916, through the first upsampling interaction network of the image segmentation model, upsample the low-pixel convolution interaction feature and fuse it with the high-pixel convolution interaction feature, then perform a hole convolution process on the jump connection with the first low-pixel downsampling interaction feature to obtain the first low-pixel upsampling interaction feature; upsample the low-pixel convolution interaction feature and the high-pixel convolution interaction feature respectively and fuse them, then perform a normal convolution process on the jump connection with the first high-pixel downsampling interaction feature to obtain the first high-pixel upsampling interaction feature.

[0143] S918, through the second upsampling interaction network of the image segmentation model, the first low-pixel upsampling interaction feature is upsampled and fused with the first high-pixel upsampling interaction feature, and then jump-connected to the low-pixel image feature and then subjected to hole convolution processing to obtain the low-pixel upsampling interaction feature; the first high-pixel upsampling interaction feature and the first low-pixel upsampling interaction feature are upsampled and fused respectively, and then jump-connected to the high-pixel image feature and then subjected to ordinary convolution processing to obtain the high-pixel upsampling interaction feature.

[0144] S920, obtaining original image features corresponding to the image to be segmented through the convergence network of the image segmentation model; upsampling the low-pixel upsampling interactive features and fusing them with the high-pixel upsampling interactive features, and then jump-connecting the original image features to obtain predicted features corresponding to the image to be segmented.

[0145] S922, outputting the probability that each pixel in the image to be segmented belongs to the target object according to the predicted features through the output layer of the image segmentation model, and segmenting the target object from the image to be segmented according to the probability.

[0146] Fig. 9 FIG. 1 is a flow chart of an image segmentation method in one embodiment. It should be understood that although Fig. 9The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Fig. 9 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0147] like Fig.10 As shown, in one embodiment, a method for segmenting lesions in a medical image is provided. This embodiment mainly applies the method to the above Figure 1 The server 120 in FIG. Fig.10 , the image segmentation method specifically includes the following steps:

[0148] S1002, acquiring a medical image.

[0149] Among them, medical images are also called medical images, which refer to images of internal tissues of an organism or a part of an organism obtained by medical instruments in a non-invasive manner for medical treatment or medical research. Specifically, medical images may include CT images obtained by performing electronic computed tomography (CT) on biological tissues, MRI images obtained by performing magnetic resonance imaging (MRI) on biological tissues, and may also include ultrasonic images, X-ray images, electrocardiograms, electroencephalograms, or optical photography images, etc. Medical images may include lesions, and the lesion area can be segmented from the medical image through the lesion segmentation method of the medical image provided in this embodiment.

[0150] S1004, obtaining high-pixel image features and low-pixel image features corresponding to the medical image.

[0151] Specifically, the server can process the medical image through the trained image segmentation model to segment the lesion from the medical image. In one embodiment, the image segmentation model includes an input layer, a branch network, a downsampling interaction network, a convolution interaction network, an upsampling interaction network, a confluence network and an output layer. The server can input the medical image into the image segmentation model, and obtain the high-pixel image features and low-pixel image features corresponding to the image to be processed through the input layer and branch network of the image segmentation model.

[0152] The specific implementation of step S1004 may refer to the implementation described above for step S204.

[0153] S1006, performing downsampling interaction processing on the high-pixel image features and the low-pixel image features, and then fusing them to obtain high-pixel downsampling interaction features and low-pixel downsampling interaction features.

[0154] After obtaining the high-pixel image features and low-pixel image features corresponding to the medical image through the branch network, the downsampling interaction network in the image segmentation model is used to interact with the high-pixel image features and the low-pixel image features to further extract the features of the medical image. The downsampling interaction network is used to downsample the features and then interact with them to obtain image information at different scales of the image to be processed.

[0155] The specific implementation of step S1006 may refer to the implementation described above for step S206.

[0156] S1008, performing convolution interaction processing on the high-pixel downsampling interaction features and the low-pixel downsampling interaction features and then fusing them to obtain high-pixel convolution interaction features and low-pixel convolution interaction features.

[0157] Specifically, after obtaining the high-pixel downsampling interaction features and low-pixel downsampling interaction features corresponding to the medical image through the downsampling interaction network, the convolutional interaction network in the image segmentation model continues to perform information interaction based on the high-pixel downsampling interaction features and the low-pixel downsampling interaction features to continue to extract the features of the medical image. The convolutional interaction network is used to perform information interaction after convolution processing on the image features to further obtain image information of the medical image at different scales.

[0158] For the specific implementation of step S1008, reference may be made to the implementation described above for step S208.

[0159] S1010, performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature, and then fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature.

[0160] Specifically, after obtaining the high-pixel convolution interaction features and low-pixel convolution interaction features corresponding to the medical image through the convolution interaction network, the upsampling interaction network in the image segmentation model continues to perform information interaction based on the high-pixel convolution interaction features and the low-pixel convolution interaction features. The upsampling interaction network is used to perform information interaction after further upsampling of the features.

[0161] The specific implementation of step S1010 may refer to the implementation described above for step S210.

[0162] S1012, determining the probability that each pixel in the medical image belongs to a lesion based on the high-pixel upsampling interaction features and the low-pixel upsampling interaction features, and segmenting the lesion from the medical image based on the probability through the output layer of the image segmentation model.

[0163] Specifically, after obtaining high-pixel upsampling interaction features and low-pixel upsampling interaction features through the upsampling interaction network, the fusion network in the image segmentation model is used to fuse the high-pixel upsampling interaction features and the low-pixel upsampling interaction features respectively to obtain prediction features for pixel-level classification. The probability that each pixel in the medical image belongs to the target is determined based on the prediction features, and then the probability is binarized (0 or 1) according to a preset threshold to segment the lesion from the medical image.

[0164] The specific implementation of step S1012 may refer to the implementation described above for step S212.

[0165] The above-mentioned method for segmenting lesions of medical images obtains high-pixel image features and low-pixel image features of the medical image at two scales after obtaining the medical image, and then continuously maintains dual-scale information interaction based on the image features of the two scales, that is, in the downsampling interaction process, the dual-scale image features are simultaneously downsampled and interacted, in the convolution interaction process, the dual-scale downsampling interaction features are simultaneously convolutionally interacted, and in the upsampling interaction process, the dual-scale convolution interaction features are simultaneously upsampled and interacted, and then according to the obtained dual-scale upsampling interaction features, the probability that each pixel in the medical image belongs to the target object is jointly determined, and the lesion is segmented from the medical image according to the probability. In the entire processing process, through continuous information interaction, and only maintaining the interaction of dual-scale features, the image information extracted interactively is more comprehensive, which can significantly improve the precision of segmentation; further, since each information interaction is based on the interaction features obtained from the previous information interaction, the error caused by information loss caused by downsampling is effectively avoided, and more detailed information can be segmented from the medical image, further improving the precision of image segmentation.

[0166] In one embodiment, Fig.11 As shown, an image segmentation device 1100 is provided, which includes a to-be-segmented image acquisition module 1102, an image feature acquisition module 1104, a downsampling interaction module 1106, a convolution interaction module 1108, an upsampling interaction module 1110 and a target object segmentation module 1112, wherein:

[0167] The image to be segmented acquisition module 1102 is used to acquire the image to be segmented;

[0168] The image feature acquisition module 1104 is used to acquire high-pixel image features and low-pixel image features corresponding to the image to be segmented;

[0169] Downsampling interaction module 1106, used for performing downsampling interaction processing on high-pixel image features and low-pixel image features and then fusing them to obtain high-pixel downsampling interaction features and low-pixel downsampling interaction features;

[0170] The convolution interaction module 1108 is used to perform convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and then fuse them to obtain the high-pixel convolution interaction feature and the low-pixel convolution interaction feature;

[0171] An upsampling interaction module 1110 is used to perform upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fuse them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature;

[0172] The target object segmentation module 1112 is used to determine the probability that each pixel in the image to be segmented belongs to the target object based on the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and segment the target object from the image to be segmented based on the probability.

[0173] In one embodiment, the image segmentation device 1100 is implemented by an image segmentation model, which includes an input layer, a branch network, a downsampling interaction network, a convolution interaction network, an upsampling interaction network, a fusion network and an output layer.

[0174] In one embodiment, the image feature acquisition module 1104 is specifically used to obtain the original image features corresponding to the image to be segmented through the input layer; perform ordinary convolution transformation on the original image features through the branch network to obtain high-pixel image features corresponding to the image to be segmented; and perform downsampling processing and hole convolution processing on the original image features in sequence through the branch network to obtain low-pixel image features corresponding to the image to be segmented.

[0175] In one embodiment, the downsampling interaction module 1106 is specifically used to downsample high-pixel image features through a downsampling interaction network and then fuse them with low-pixel image features to obtain high-pixel downsampling interaction features; and downsample high-pixel image features and low-pixel image features through a downsampling interaction network and then fuse them to obtain low-pixel downsampling interaction features.

[0176] In one embodiment, the downsampling interaction module 1106 is specifically used to downsample the high-pixel image features through a first downsampling interaction network, fuse them with the low-pixel image features, and then perform ordinary convolution processing to obtain a first high-pixel downsampling interaction feature; downsample the high-pixel image features and the low-pixel image features through the first downsampling interaction network, respectively, and then fuse them, and then perform hole convolution processing to obtain a first low-pixel downsampling interaction feature; downsample the first high-pixel downsampling interaction feature through a second downsampling interaction network, and then fuse them with the first low-pixel downsampling interaction feature, and then perform ordinary convolution processing to obtain a high-pixel downsampling interaction feature.

[0177] In one embodiment, the downsampling interaction module 1106 is specifically used to downsample and fuse the first high-pixel downsampling interaction feature and the first low-pixel downsampling interaction feature through a second downsampling interaction network, and then perform a dilated convolution process to obtain a low-pixel downsampling interaction feature.

[0178] In one embodiment, the convolution interaction module 1108 is specifically used to upsample the low-pixel downsampling interaction features through a convolution interaction network, and then fuse them with the high-pixel downsampling interaction features, and then perform ordinary convolution processing to obtain high-pixel convolution interaction features; through a convolution interaction network, downsample the high-pixel downsampling interaction features and then fuse them with the low-pixel downsampling interaction features, and then perform hole convolution processing to obtain low-pixel convolution interaction features.

[0179] In one embodiment, the upsampling interaction module 1110 is specifically used to upsample the low-pixel convolution interaction features through an upsampling interaction network and then fuse them with the high-pixel convolution interaction features to obtain low-pixel upsampling interaction features; through the upsampling interaction network, upsample the high-pixel convolution interaction features and the low-pixel convolution interaction features and then fuse them to obtain high-pixel upsampling interaction features.

[0180] In one embodiment, the upsampling interaction module 1110 is specifically used to obtain the first high-pixel downsampling interaction feature and the first low-pixel downsampling interaction feature output by the first downsampling interaction network through the first upsampling interaction network, and obtain the high-pixel image feature and the low-pixel image feature through the second upsampling interaction network; through the first upsampling interaction network, the low-pixel convolution interaction feature is upsampled and fused with the high-pixel convolution interaction feature, and then jump-connected with the first low-pixel downsampling interaction feature to perform a hole convolution process to obtain the first low-pixel upsampling interaction feature; through the first upsampling interaction network, the low-pixel convolution interaction feature and the high-pixel convolution interaction feature are upsampled and fused respectively, and then jump-connected with the first high-pixel downsampling interaction feature to perform a normal convolution process to obtain the first high-pixel upsampling interaction feature; through the second upsampling interaction network, the first low-pixel upsampling interaction feature is upsampled and fused with the first high-pixel upsampling interaction feature, and then jump-connected with the low-pixel image feature to perform a hole convolution process to obtain the low-pixel upsampling interaction feature.

[0181] In one embodiment, the upsampling interaction module 1110 is specifically used to upsample the first high-pixel upsampling interaction feature and the first low-pixel upsampling interaction feature through a second upsampling interaction network, and then fuse them, and then jump-connect the high-pixel image features and perform ordinary convolution processing to obtain high-pixel upsampling interaction features.

[0182] In one embodiment, the target object segmentation module 1112 is specifically used to obtain the original image features corresponding to the image to be segmented through the fusion network; through the fusion network, the low-pixel upsampling interaction features are upsampled and merged with the high-pixel upsampling interaction features, and then jump-connected to the original image features to obtain the predicted features corresponding to the image to be segmented; through the output layer, the probability of each pixel in the image to be segmented belonging to the target object is output according to the predicted features.

[0183] The above-mentioned image segmentation device 1100, after obtaining the image to be segmented, obtains the high-pixel image features and low-pixel image features of the image to be segmented at two scales, and then continuously maintains the information interaction of the two scales based on the image features of the two scales, that is, in the downsampling interaction process, the image features of the two scales are simultaneously downsampled and interacted, in the convolution interaction process, the downsampling interaction features of the two scales are simultaneously convolutionally interacted, and in the upsampling interaction process, the convolution interaction features of the two scales are simultaneously upsampled and interacted, and then according to the obtained upsampling interaction features of the two scales, jointly determine the probability that each pixel in the image to be segmented belongs to the target object, and segment the target object from the image to be segmented according to the probability. In the whole processing process, through continuous information interaction, and only maintaining the interaction of the two-scale features, the image information extracted by the interaction is more comprehensive, which can significantly improve the precision of the segmentation; further, since each information interaction is based on the interaction features obtained by the previous information interaction, the error caused by the missing information caused by downsampling is effectively avoided, and more detailed information can be segmented from the image, further improving the precision of the image segmentation.

[0184] In one embodiment, Fig.12 As shown, a medical image lesion segmentation device 1200 is provided, the device includes a medical image acquisition module 1202, an image feature acquisition module 1204, a downsampling interaction module 1206, a convolution interaction module 1208, an upsampling interaction module 1210 and a lesion segmentation module 1212, wherein:

[0185] A medical image acquisition module 1202, used to acquire medical images;

[0186] An image feature acquisition module 1204 is used to acquire high-pixel image features and low-pixel image features corresponding to the medical image;

[0187] A downsampling interaction module 1206 is used to perform downsampling interaction processing on the high-pixel image features and the low-pixel image features and then fuse them to obtain high-pixel downsampling interaction features and low-pixel downsampling interaction features;

[0188] The convolution interaction module 1208 is used to perform convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and then fuse them to obtain the high-pixel convolution interaction feature and the low-pixel convolution interaction feature;

[0189] An upsampling interaction module 1210 is used to perform upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fuse them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature;

[0190] The lesion segmentation module 1212 is used to determine the probability that each pixel in the medical image belongs to a lesion based on the high-pixel upsampling interaction features and the low-pixel upsampling interaction features, and to segment the lesion from the medical image based on the probability through the output layer of the image segmentation model.

[0191] The above-mentioned lesion segmentation device 1200 for medical images, after acquiring the medical image, acquires the high-pixel image features and low-pixel image features of the medical image at two scales, and then continuously maintains the information interaction of the two scales based on the image features of the two scales, that is, in the downsampling interaction process, the image features of the two scales are simultaneously downsampled and interacted, in the convolution interaction process, the downsampling interaction features of the two scales are simultaneously convolutionally interacted, and in the upsampling interaction process, the convolution interaction features of the two scales are simultaneously upsampled and interacted, and then according to the obtained upsampling interaction features of the two scales, jointly determine the probability that each pixel in the medical image belongs to the target object, and segment the lesion from the medical image according to the probability. In the whole processing process, through continuous information interaction, and only maintaining the interaction of the two-scale features, the image information extracted by the interaction is more comprehensive, which can significantly improve the precision of the segmentation; further, since each information interaction is based on the interaction features obtained from the previous information interaction, the error caused by the missing information caused by downsampling is effectively avoided, and more detailed information can be segmented from the medical image, further improving the precision of the image segmentation.

[0192] Fig.13 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be Figure 1 The server 120 in FIG. Fig.10 As shown, the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement an image segmentation method or a medical image lesion segmentation method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute and implement the image segmentation method or the medical image lesion segmentation method.

[0193] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0194] In one embodiment, the image segmentation device 1100 provided in the present application can be implemented in the form of a computer program. The computer program can be implemented in the form of a computer program. Fig.13 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the image segmentation device 1100, for example, Fig.11 The image acquisition module 1102 to be segmented, the image feature acquisition module 1104, the downsampling interaction module 1106, the convolution interaction module 1108, the upsampling interaction module 1110 and the target object segmentation module 1112 are shown. The computer program composed of each program module enables the processor to execute the steps of the image segmentation method of each embodiment of the present application described in this specification.

[0195] For example, Fig.13 The computer device shown can be Fig.11 The image acquisition module 1102 to be segmented in the image segmentation apparatus 1100 shown in the figure performs step S202. The computer device may perform step S204 through the image feature acquisition module 1104. The computer device may perform step S206 through the downsampling interaction module 1106. The computer device may perform step S208 through the convolution interaction module 1108. The computer device may perform step S210 through the upsampling interaction module 1110. The computer device may perform step S212 through the target object segmentation module 1112.

[0196] In one embodiment, the medical image lesion segmentation device 1200 provided in the present application can be implemented in the form of a computer program. The computer program can be used in Fig.13 The computer device shown in the figure is run on the computer device. The memory of the computer device can store various program modules constituting the lesion segmentation device 1200 of the medical image, for example, Fig.12 The medical image acquisition module 1202, image feature acquisition module 1204, downsampling interaction module 1206, convolution interaction module 1208, upsampling interaction module 1210 and lesion segmentation module 1212 are shown. The computer program composed of various program modules enables the processor to execute the steps of the lesion segmentation method of the medical image of each embodiment of the present application described in this specification.

[0197] For example, Fig.13 The computer device shown can be Fig.11The medical image acquisition module 1202 in the lesion segmentation device 1200 of the medical image shown in the figure performs step S1002. The computer device may perform step S1004 through the image feature acquisition module 1204. The computer device may perform step S1006 through the downsampling interaction module 1206. The computer device may perform step S1008 through the convolution interaction module 1208. The computer device may perform step S1010 through the upsampling interaction module 1210. The computer device may perform step S1012 through the lesion segmentation module 1212.

[0198] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned image segmentation method and / or the lesion segmentation method of a medical image. The steps of the image segmentation method here may be the steps of the image segmentation method of each of the above-mentioned embodiments. The steps of the lesion segmentation method of a medical image here may be the steps of the lesion segmentation method of each of the above-mentioned embodiments.

[0199] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned image segmentation method and / or the medical image lesion segmentation method. The steps of the image segmentation method here may be the steps of the image segmentation method of each of the above-mentioned embodiments. The steps of the medical image lesion segmentation method here may be the steps of the medical image lesion segmentation method of each of the above-mentioned embodiments.

[0200] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0201] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0202] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image segmentation method, comprising: Obtain the image to be segmented; Acquire high-pixel image features and low-pixel image features corresponding to the image to be segmented, wherein the high-pixel image features and the low-pixel image features are both spatial matrices obtained by performing different processing on each pixel in the image to be segmented and used to preliminarily represent image characteristics, and the high-pixel image features and the low-pixel image features respectively preliminarily express image information of the image to be segmented from two different scales, and the high-pixel image features are image features with a larger scale than the low-pixel image features; Downsampling the high-pixel image features and fusing them with the low-pixel image features to obtain high-pixel downsampling interaction features, and downsampling the high-pixel image features and the low-pixel image features and fusing them to obtain low-pixel downsampling interaction features; The high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature are subjected to convolution interaction processing and then fused to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature; Performing up-sampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature, and then fusing them to obtain a high-pixel up-sampling interaction feature and a low-pixel up-sampling interaction feature; According to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, the probability that each pixel in the image to be segmented belongs to the target object is determined, and the target object is segmented from the image to be segmented according to the probability.

2. The method according to claim 1, characterized in that The method is implemented by an image segmentation model, which includes an input layer, a branch network, a downsampling interaction network, a convolution interaction network, an upsampling interaction network, a confluence network and an output layer; Implementing the step of acquiring high-pixel image features and low-pixel image features corresponding to the image to be segmented through the input layer and the branch network; The step of obtaining the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature is realized by the downsampling interaction network; Implementing the step of performing convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and fusing them through the convolution interaction network to obtain the high-pixel convolution interaction feature and the low-pixel convolution interaction feature; Implementing the step of performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and fusing them to obtain the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature through the upsampling interaction network; The steps of determining the probability that each pixel in the image to be segmented belongs to the target object according to the high-pixel upsampling interaction features and the low-pixel upsampling interaction features, and segmenting the target object from the image to be segmented according to the probability are implemented through the fusion network and the output layer.

3. The method according to claim 2, characterized in that The obtaining of high-pixel image features and low-pixel image features corresponding to the image to be segmented includes: Obtaining original image features corresponding to the image to be segmented through the input layer; Through the branch network, ordinary convolution transformation is performed on the original image features to obtain high-pixel image features corresponding to the image to be segmented; Through the branch network, the original image features are sequentially subjected to downsampling processing and hole convolution processing to obtain low-pixel image features corresponding to the image to be segmented.

4. The method according to claim 1, characterized in that: The downsampling of the high-pixel image features and fusion with the low-pixel image features to obtain high-pixel downsampling interaction features includes: The high-pixel image feature is downsampled through a first downsampling interaction network and then fused with the low-pixel image feature, and then subjected to ordinary convolution processing to obtain a first high-pixel downsampling interaction feature; Through the first downsampling interaction network, the high-pixel image features and the low-pixel image features are downsampled and then fused, and then a dilated convolution process is performed to obtain a first low-pixel downsampling interaction feature; The first high-pixel down-sampling interaction feature is down-sampled through a second down-sampling interaction network and then fused with the first low-pixel down-sampling interaction feature, and then subjected to ordinary convolution processing to obtain a high-pixel down-sampling interaction feature.

5. The method according to claim 4, characterized in that The downsampling and fusing the high-pixel image features and the low-pixel image features to obtain low-pixel downsampling interaction features includes: The first high-pixel down-sampling interaction feature and the first low-pixel down-sampling interaction feature are respectively down-sampled and fused through the second down-sampling interaction network, and then a hole convolution process is performed to obtain a low-pixel down-sampling interaction feature.

6. The method according to claim 2, characterized in that The step of performing convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and fusing them to obtain the high-pixel convolution interaction feature and the low-pixel convolution interaction feature includes: Through the convolutional interaction network, the low-pixel down-sampling interaction features are up-sampled and then fused with the high-pixel down-sampling interaction features, and then ordinary convolution processing is performed to obtain high-pixel convolution interaction features; Through the convolutional interaction network, the high-pixel down-sampling interaction feature is down-sampled and fused with the low-pixel down-sampling interaction feature, and then a hole convolution process is performed to obtain a low-pixel convolution interaction feature.

7. The method according to claim 2, characterized in that The step of performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature includes: Through the upsampling interaction network, the low-pixel convolution interaction feature is upsampled and then fused with the high-pixel convolution interaction feature to obtain a low-pixel upsampling interaction feature; The high-pixel convolution interaction feature and the low-pixel convolution interaction feature are upsampled and fused through the upsampling interaction network to obtain a high-pixel upsampling interaction feature.

8. The method according to claim 7, characterized in that The upsampling interaction network is used to upsample the low-pixel convolution interaction feature and then fuse it with the high-pixel convolution interaction feature to obtain the low-pixel upsampling interaction feature, including: Obtaining, through a first upsampling interaction network, a first high-pixel downsampling interaction feature and a first low-pixel downsampling interaction feature output by the first downsampling interaction network, and obtaining, through a second upsampling interaction network, the high-pixel image feature and the low-pixel image feature; Through the first upsampling interaction network, the low-pixel convolution interaction feature is upsampled and then fused with the high-pixel convolution interaction feature, and then jump-connected with the first low-pixel down-sampling interaction feature and then subjected to hole convolution processing to obtain a first low-pixel up-sampling interaction feature; Through the first upsampling interaction network, the low-pixel convolution interaction feature and the high-pixel convolution interaction feature are upsampled and then fused, and then the first high-pixel downsampling interaction feature is jump-connected and then subjected to ordinary convolution processing to obtain a first high-pixel upsampling interaction feature; Through the second upsampling interaction network, the first low-pixel upsampling interaction feature is upsampled and fused with the first high-pixel upsampling interaction feature, and then the low-pixel image feature is jump-connected and then subjected to a hole convolution process to obtain a low-pixel upsampling interaction feature.

9. The method according to claim 8, characterized in that The upsampling interaction network is used to upsample the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fuse them to obtain a high-pixel upsampling interaction feature, including: Through the second upsampling interaction network, the first high-pixel upsampling interaction feature and the first low-pixel upsampling interaction feature are upsampled and fused respectively, and then the high-pixel image feature is jump-connected and ordinary convolution is performed to obtain a high-pixel upsampling interaction feature.

10. The method according to any one of claims 2 to 3 and 6 to 9, characterized in that: Determining the probability that each pixel in the image to be segmented belongs to the target object according to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature includes: Obtaining original image features corresponding to the image to be segmented through the fusion network; Through the fusion network, the low-pixel up-sampling interaction feature is up-sampled and then merged with the high-pixel up-sampling interaction feature, and then jump-connected with the original image feature to obtain the prediction feature corresponding to the image to be segmented; The output layer outputs the probability that each pixel in the image to be segmented belongs to the target object according to the prediction feature.

11. A method for segmenting a lesion in a medical image, comprising: Acquiring medical images; Acquire high-pixel image features and low-pixel image features corresponding to the medical image, wherein the high-pixel image features and the low-pixel image features are both spatial matrices obtained by performing different processing on each pixel in the medical image and used to preliminarily represent image characteristics, and the high-pixel image features and the low-pixel image features respectively preliminarily express image information of the medical image from two different scales, and the high-pixel image features are image features with a larger scale than the low-pixel image features; Downsampling the high-pixel image features and fusing them with the low-pixel image features to obtain high-pixel downsampling interaction features, and downsampling the high-pixel image features and the low-pixel image features and fusing them to obtain low-pixel downsampling interaction features; The high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature are subjected to convolution interaction processing and then fused to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature; Performing up-sampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature, and then fusing them to obtain a high-pixel up-sampling interaction feature and a low-pixel up-sampling interaction feature; The probability that each pixel in the medical image belongs to a lesion is determined according to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and the lesion is segmented from the medical image according to the probability.

12. An image segmentation device, characterized in that: The device comprises: The image to be segmented acquisition module is used to acquire the image to be segmented; An image feature acquisition module is used to acquire high-pixel image features and low-pixel image features corresponding to the image to be segmented, wherein the high-pixel image features and the low-pixel image features are spatial matrices obtained by performing different processing on each pixel in the image to be segmented and used to preliminarily represent image characteristics, and the high-pixel image features and the low-pixel image features respectively preliminarily express image information of the image to be segmented from two different scales, and the high-pixel image features are image features with a larger scale than the low-pixel image features; A downsampling interaction module is used to perform downsampling processing on the high-pixel image features and fuse them with the low-pixel image features to obtain high-pixel downsampling interaction features, and to perform downsampling processing on the high-pixel image features and the low-pixel image features and fuse them to obtain low-pixel downsampling interaction features; A convolution interaction module, used for performing convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and then fusing them to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature; An upsampling interaction module, used for performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature; The target object segmentation module is used to determine the probability that each pixel in the image to be segmented belongs to the target object according to the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and segment the target object from the image to be segmented according to the probability.

13. The device according to claim 12, characterized in that The device includes an image segmentation model, which includes an input layer, a branch network, a downsampling interaction network, a convolution interaction network, an upsampling interaction network, a confluence network and an output layer; The image feature acquisition module is used to obtain the original image features corresponding to the image to be segmented through the input layer; perform ordinary convolution transformation on the original image features through the branch network to obtain high-pixel image features corresponding to the image to be segmented; and perform downsampling processing and hole convolution processing on the original image features in sequence through the branch network to obtain low-pixel image features corresponding to the image to be segmented.

14. The device according to claim 12, characterized in that The downsampling interaction module is used to downsample the high-pixel image features through a first downsampling interaction network, fuse them with the low-pixel image features, and then perform ordinary convolution processing to obtain a first high-pixel downsampling interaction feature; downsample the high-pixel image features and the low-pixel image features through the first downsampling interaction network, fuse them, and then perform hole convolution processing to obtain a first low-pixel downsampling interaction feature; downsample the first high-pixel downsampling interaction feature through a second downsampling interaction network, fuse them with the first low-pixel downsampling interaction feature, and then perform ordinary convolution processing to obtain a high-pixel downsampling interaction feature.

15. The device according to claim 14, characterized in that The downsampling interaction module is used to downsample the first high-pixel downsampling interaction feature and the first low-pixel downsampling interaction feature through the second downsampling interaction network, fuse them together, and then perform hole convolution processing to obtain a low-pixel downsampling interaction feature.

16. The device according to claim 13, characterized in that The convolution interaction module is used to upsample the low-pixel downsampling interaction features through the convolution interaction network, fuse them with the high-pixel downsampling interaction features, and then perform ordinary convolution processing to obtain high-pixel convolution interaction features; downsample the high-pixel downsampling interaction features through the convolution interaction network, fuse them with the low-pixel downsampling interaction features, and then perform hole convolution processing to obtain low-pixel convolution interaction features.

17. The device according to claim 13, characterized in that The upsampling interaction module is used to upsample the low-pixel convolution interaction features through the upsampling interaction network and then fuse them with the high-pixel convolution interaction features to obtain low-pixel upsampling interaction features; and upsample the high-pixel convolution interaction features and the low-pixel convolution interaction features through the upsampling interaction network and then fuse them to obtain high-pixel upsampling interaction features.

18. The device according to claim 17, characterized in that The upsampling interaction module is used to obtain the first high-pixel downsampling interaction feature and the first low-pixel downsampling interaction feature output by the first downsampling interaction network through the first upsampling interaction network, and obtain the high-pixel image feature and the low-pixel image feature through the second upsampling interaction network; through the first upsampling interaction network, upsampling the low-pixel convolution interaction feature and then fusing it with the high-pixel convolution interaction feature, and then performing a hole convolution process with the first low-pixel downsampling interaction feature to obtain the first low-pixel upsampling interaction feature; through the first upsampling interaction network, upsampling the low-pixel convolution interaction feature and the high-pixel convolution interaction feature and then fusing them, and then performing a normal convolution process with the first high-pixel downsampling interaction feature after jump connection, to obtain the first high-pixel upsampling interaction feature; through the second upsampling interaction network, upsampling the first low-pixel upsampling interaction feature and then fusing it with the first high-pixel upsampling interaction feature, and then performing a hole convolution process with the low-pixel image feature to obtain the low-pixel upsampling interaction feature.

19. The device according to claim 18, characterized in that The upsampling interaction module is used to upsample the first high-pixel upsampling interaction feature and the first low-pixel upsampling interaction feature through the second upsampling interaction network, and then fuse them, and then jump connect the high-pixel image features and perform ordinary convolution processing to obtain high-pixel upsampling interaction features.

20. The device according to any one of claims 13, 16 to 19, characterized in that The target object segmentation module is used to obtain original image features corresponding to the image to be segmented through the fusion network; Through the fusion network, the low-pixel upsampling interaction feature is upsampled and fused with the high-pixel upsampling interaction feature, and then the original image feature is jump-connected to obtain the predicted feature corresponding to the image to be segmented; through the output layer, the probability of each pixel in the image to be segmented belonging to the target object is output according to the predicted feature.

21. A medical image lesion segmentation device, characterized in that: The device comprises: A medical image acquisition module, used for acquiring medical images; An image feature acquisition module, used to acquire high-pixel image features and low-pixel image features corresponding to the medical image, wherein the high-pixel image features and the low-pixel image features are both spatial matrices obtained by performing different processing on each pixel in the medical image and used to preliminarily represent image features, and the high-pixel image features and the low-pixel image features respectively preliminarily express image information of the medical image from two different scales, and the high-pixel image features are image features with a larger scale than the low-pixel image features; A downsampling interaction module is used to perform downsampling processing on the high-pixel image features and fuse them with the low-pixel image features to obtain high-pixel downsampling interaction features, and to perform downsampling processing on the high-pixel image features and the low-pixel image features and fuse them to obtain low-pixel downsampling interaction features; A convolution interaction module, used for performing convolution interaction processing on the high-pixel downsampling interaction feature and the low-pixel downsampling interaction feature and then fusing them to obtain a high-pixel convolution interaction feature and a low-pixel convolution interaction feature; An upsampling interaction module, used for performing upsampling interaction processing on the high-pixel convolution interaction feature and the low-pixel convolution interaction feature and then fusing them to obtain a high-pixel upsampling interaction feature and a low-pixel upsampling interaction feature; A lesion segmentation module is used to determine the probability that each pixel in the medical image belongs to a lesion based on the high-pixel upsampling interaction feature and the low-pixel upsampling interaction feature, and to segment the lesion from the medical image based on the probability.

22. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 11.

23. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 11.

Images