Image processing method and device, equipment, medium and product

Abstract

The embodiment of the invention provides an image processing method and device, equipment, a medium and a product. The method comprises the steps that a 2D image including a target focus part is acquired; inputting the 2D image into a focus segmentation model obtained by pre-training, and outputting a focus segmentation image corresponding to the 2D image; wherein the lesion segmentation model comprises an input layer, a first target network layer, a second target network layer, a residual convolutional network layer and an output layer, the first target network layer comprises the residual convolutional network layer, a global and local attention layer and a down-sampling layer, and the second target network layer comprises an up-sampling layer, the residual convolutional network layer and a global and local attention layer. According to the technical scheme of the embodiment of the invention, the focus segmentation image corresponding to the 2D image is obtained according to the focus segmentation model at least comprising the global attention layer and the local attention layer, the generalization ability and robustness of the model are improved, and the segmentation precision of the focus image is extremely high.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and more particularly to an image processing method, apparatus, device, medium, and product. Background Art

[0002] With the application of deep learning technology in the field of image processing, image segmentation processing using constructed segmentation models has gradually developed. For example, in medical image analysis, images acquired based on CT or MRI can be input into a pre-built and trained image segmentation model to identify and segment the pathological areas in the medical image, such as tumors and lesions.

[0003] Currently used image segmentation models primarily rely solely on the U-Net architecture, employing a symmetrical encoder-decoder structure connected by numerous skip connections to fuse information from different layers. Alternatively, they incorporate the concept of residual networks, introducing residual connections within the convolutional layers of the U-Net to help address the vanishing gradient problem in deep networks. However, segmentation models based solely on the U-Net architecture or in combination with residual networks suffer from poor robustness, insufficient processing of edge details, and limited multi-scale adaptability. Furthermore, when using these segmentation models for image segmentation, they suffer from low segmentation accuracy. Summary of the Invention

[0004] The embodiments of the present disclosure provide an image processing method, apparatus, device, medium, and product to obtain a lesion segmentation image corresponding to a 2D image based on a lesion segmentation model that includes at least a global plus local attention layer, thereby improving the generalization ability and robustness of the model and achieving extremely high segmentation accuracy of the lesion image.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising:

[0006] Acquiring at least one 2D image including a target lesion site, wherein the at least one 2D image is extracted based on a 3D image of the target lesion site;

[0007] Inputting the at least one 2D image into a pre-trained lesion segmentation model, and outputting a lesion segmentation image corresponding to the at least one 2D image;

[0008] Among them, the lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, a global plus local attention layer. The number of the at least one first target network layer is the same as the number of the at least one second target network layer, and the output of the first target network layer in the same processing order is the input of the second target network layer.

[0009] In a second aspect, an embodiment of the present invention further provides an image processing device, the device comprising:

[0010] a 2D image acquisition module, configured to acquire at least one 2D image including a target lesion site, wherein the at least one 2D image is extracted based on a 3D image of the target lesion site;

[0011] a lesion segmentation image output module, configured to input the at least one 2D image into a pre-trained lesion segmentation model and output a lesion segmentation image corresponding to the at least one 2D image;

[0012] Among them, the lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, a global plus local attention layer. The number of the at least one first target network layer is the same as the number of the at least one second target network layer, and the output of the first target network layer in the same processing order is the input of the second target network layer.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of the embodiments of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the image processing method as described in any one of the embodiments of the present invention.

[0018] In a fifth aspect, an embodiment of the present invention further provides a computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, it implements the image processing method as described in any one of the embodiments of the present invention.

[0019] The technical solution of the embodiment of the present disclosure is as follows: first, at least one 2D image including the target lesion site is obtained, wherein the at least one 2D image is extracted based on the 3D image of the target lesion site. Then, the at least one 2D image is input into the pre-trained lesion segmentation model, and the lesion segmentation image corresponding to the at least one 2D image is output. The lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer, and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer, and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, and a global plus local attention layer. The number of at least one first target network layer is the same as the number of at least one second target network layer. The output of the first target network layer in the same processing order is the input of the second target network layer. This solves the problems of poor robustness, insufficient edge detail processing, and limited multi-scale adaptability of the segmentation model based only on the U-Net architecture or in combination with the residual network in the prior art. It also solves the problem of low segmentation accuracy when using the existing segmentation model for image segmentation. The embodiment of the present invention obtains a lesion segmentation image corresponding to a 2D image based on a lesion segmentation model that includes at least a global plus local attention layer. The local attention layer of the model enhances the capture of details of the lesion area in the image and improves the segmentation accuracy of small lesions or areas with blurred edges. The global attention layer of the model identifies the correlation between different areas in the image, which helps to understand the structure and overall context of the entire image. The features extracted by the global and local attention layers are fused to achieve the integration of global vision and local detail information, thereby improving the generalization ability and robustness of the model and improving the segmentation accuracy of the lesion image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings introduced here only illustrate some of the embodiments to be described by the present invention, and are not exhaustive. A person skilled in the art can derive other drawings based on these drawings without inventive effort.

[0021] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0022] Figure 2is a flow chart of the lesion segmentation model provided in an embodiment of the present invention;

[0023] Figure 3 2 is a flow chart of the global plus local attention layer in the lesion segmentation model provided by an embodiment of the present invention;

[0024] Figure 4 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0025] Figure 5 is a structural diagram of an image processing device provided by an embodiment of the present invention;

[0026] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0028] Before introducing the technical solutions provided by the embodiments of the present disclosure, an example description of the application scenarios can be given first. The technical solutions provided by the embodiments of the present disclosure can be applied to the scene where the image of the target lesion site is segmented based on the lesion segmentation model to obtain a corresponding lesion segmentation image. For example, for a 3D image of the prostate site acquired based on Computed Tomography (CT) or Magnetic Resonance Imaging (MRI), at least one 2D image is extracted from the 3D image, and at least one 2D image is input into the lesion segmentation model, and the image is segmented to obtain a corresponding lesion segmentation image. Based on the technical solution of the embodiments of the present disclosure, the lesion segmentation model includes at least a global plus local attention layer. Based on the constructed and trained lesion segmentation model, a lesion segmentation image corresponding to the 2D image can be obtained, which improves the generalization ability and robustness of the model and achieves extremely high segmentation accuracy of the lesion image.

[0029] Example 1

[0030] Figure 1It is a flow chart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the situation where an image of a target lesion site is segmented based on a lesion segmentation model including at least a global plus local attention layer to obtain a corresponding lesion segmentation image. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. The hardware can be a mobile electronic device, which can execute the image processing method provided by this technical solution.

[0031] like Figure 1 As shown, the method includes:

[0032] S110 , acquiring at least one 2D image including the target lesion site.

[0033] Wherein, at least one 2D image is extracted based on a 3D image of the target lesion site.

[0034] It should be noted that a 3D image refers to an image of the target lesion site captured using 3D imaging technology. In this embodiment, the target lesion site is a prostate lesion site. A 3D image is a visual representation of the internal structure and organs of the human body in three dimensions. A 3D image can present images of the internal structure and organs of the human body in three spatial dimensions (length, width, and height). 3D images provide richer and more three-dimensional visual information, can display the three-dimensional relationship of the target site, avoid information loss, and improve the usability of the image.

[0035] The acquisition of 3D images can rely on a variety of medical imaging technologies, which capture detailed information about the human body through different physical principles. Optionally, 3D images can be based on images taken by CT. CT scans at different angles through X-rays, uses computer processing to generate volume data, and then reconstructs 3D images. When the target lesion is the prostate, for the CT image of the prostate, the CT image can show the prostate and its surrounding structures in detail; 3D images can be based on images taken by MRI. MRI uses strong magnetic fields and radio frequency pulses to obtain high-resolution images of the soft tissue inside the human body, and generates 3D views through reconstruction technology. When the target lesion is the prostate, for the MRI image of the prostate, the MRI image can generate a high-resolution 3D image of the prostate; 3D images can be based on images taken by 3D ultrasound. 3D ultrasound uses multiple ultrasonic probes to collect data from different angles, and combines computer processing to generate stereoscopic images. When the target lesion is the prostate, a 3D ultrasound image of the prostate can capture three-dimensional data of the prostate, providing more comprehensive anatomical information. A 3D image can also be based on positron emission tomography (PET), which uses radioactive tracer distribution and imaging technology to generate functional 3D images. When the target lesion is the prostate, a PET image of the prostate can provide a detailed display of the metabolic activity of the lesion. It should be noted that the 3D image can reveal the size, location, and morphology of lesions such as tumors, cysts, or inflammation within the target lesion. For example, a 3D image of the target lesion can be a CT-based image of a prostate containing a tumor.

[0036] It should be noted that as the basic building block of the 3D image of the target lesion site, the 2D image is an image extracted from the 3D image. By acquiring and combining multiple 2D images with spatial positioning, computer processing and reconstruction technology can be used to combine a 3D image with a three-dimensional sense. A 2D image refers to an image displayed on a plane, which contains only two spatial dimensions (length and width). In the field of medical imaging, 2D images are usually presented as cross-sectional, coronal or sagittal slice images of the internal structure of the human body. The continuity and consistency between slices are ensured by clear spatial positioning parameters (such as slice thickness, interval, position coordinates). These 2D images provide detailed views of tissues, organs and lesions at specific levels, and are the basic units for constructing 3D images.

[0037] It should be noted that there is a close layering and combination relationship between 3D images and 2D images. The construction of 3D images can be based on a series of continuous 2D images obtained from different positions. These 2D images usually have a certain thickness and interval, covering the entire or partial area of the human body. 3D images can be generated by combining and superimposing multiple 2D images according to their spatial position information through computer algorithms (such as reconstruction algorithms and volume rendering technology) to generate a 3D image with a three-dimensional sense. 2D images and 3D images can achieve complementary perspectives. 2D images provide high-resolution local detail information, which helps to accurately identify and analyze the structure of a specific area. 3D images integrate multiple 2D images to display the spatial relationship and overall morphology of the entire structure, which is convenient for comprehensive understanding and evaluation.

[0038] Specifically, 2D images can be extracted from 3D images captured using medical imaging technology. The number of 2D images that can be extracted is equal to the number of layers in the 3D image, so at least one 2D image including the target lesion can be obtained.

[0039] For example, assuming that the number of layers of a CT image obtained by photographing a prostate area using CT is 320, each of the 320 layers can be called a 2D image, and there are 320 2D images that can be extracted from the 3D image.

[0040] S120 , inputting at least one 2D image into a pre-trained lesion segmentation model, and outputting a lesion segmentation image corresponding to the at least one 2D image.

[0041] It should be noted that the lesion segmentation model, as a deep learning model, is designed to accurately locate and segment the lesion area of a 2D image from images taken based on medical images (such as CT, MRI, etc.). After a 2D image is input into the pre-trained lesion segmentation model, the output image corresponding to the 2D image is the lesion segmentation image. In the lesion segmentation image, the lesion area is clearly marked, usually represented by different colors or grayscale levels. Compared with the 2D image before segmentation, the lesion segmentation image can more intuitively identify the position, shape and size of the lesion in the image.

[0042] The lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer, and an output layer. Figure 2 The first target network layer includes a residual convolutional network layer, a global plus local attention layer, and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, and a global plus local attention layer. The number of at least one first target network layer is the same as the number of at least one second target network layer. The output of the first target network layer in the same processing order is the input of the second target network layer.

[0043] Generally speaking, the input layer's primary responsibility is to receive and preprocess 2D images, converting them into a format and size suitable for the lesion segmentation model. This step is crucial for ensuring that the subsequent first target network layer can effectively extract features. Residual convolutional network layers utilize residual connections and residual blocks. In each residual block, the input is directly added to the convolutional layer output. This makes it easier to learn the "residual" rather than directly learning the mapping between input and output, thus avoiding the training difficulties associated with increased depth. The global-plus-local attention layer combines local attention with global attention, first calculating the local attention of the image and then the global attention. This ensures that the lesion segmentation model captures both global and local information while maintaining low computational complexity. Downsampling layers can be used to reduce the spatial dimensions of an image (e.g., width and height) to extract more abstract features and reduce computational effort. Downsampling can be achieved through pooling or convolutional layers. Upsampling layers expand or restore the spatial dimensions of an image, typically used to generate high-resolution output or restore feature maps to their original size. Upsampling can be achieved through methods such as transposed convolution or interpolation. The output layer is the last layer of the lesion segmentation model, and its function is to convert the final feature map of the lesion segmentation model into the prediction result of the model. In the embodiment of the present invention, the output result of the output layer of the lesion segmentation model is the lesion segmentation image corresponding to the 2D image.

[0044] It should be noted that, according to the forward processing order of at least one first target network layer and the reverse processing order of at least one second target network layer, the output of the first target network layer in the same processing order is connected to the input of the second target network layer.

[0045] In this embodiment, for at least one 2D image, the 2D image is input into an image segmentation model, and the 2D image is processed based on an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer, and an output layer to output a lesion segmentation image of the 2D image.

[0046] In this embodiment, see Figure 3 The global plus local attention layer includes the local region partition layer, the size transformation layer, the residual convolutional network layer and the average pooling layer.

[0047] It should be noted that the main purpose of the local region partitioning layer is to divide the input feature map into multiple local regions, which can be different areas of the image. By dividing these local regions, the model can focus on local information in the feature map, rather than just the overall structure. This helps capture local spatial relationships and local features, enabling the model to better understand details in the image. For all local regions obtained after the local region partitioning layer, the resizing layer calculates the Q value, or query value, for each local region. The query value is a vector used to "ask" the model for certain information about the input data, indicating its current focus on other elements. The resizing layer also calculates the K value (key value) and V value (value) for each local region. The residual convolutional network layer uses a "residual connection" to directly add the input feature map to the output of the convolution operation, thereby restoring the channel structure to a certain extent. This means that it not only learns the output of the convolutional layer but also retains some information in the input feature map, helping to maintain channel diversity and avoid information loss. The average pooling layer performs dimensionality reduction on each local region by averaging the pixels within each local region and using this average as the pixel value for the corresponding local region. This means that after the average pooling layer, the input image of each local area is averaged over all pixels, and the final output image becomes a 1*1 image with only one pixel. After the average pooling layer performs dimensionality reduction in this way, it retains the average information of the input image.

[0048] Optionally, the output feature map of the j-1th target network layer is input to the jth target network layer, the output feature map is convolved based on the convolution layer, and a convolution feature map of a first size is output; the convolution feature map is divided into regions based on the local region division layer to obtain multiple local region maps to be processed, the local region map to be processed is resized based on the size transformation layer, and the local region map to be processed after the size transformation is average pooled; the local region map to be processed is obtained by performing global attention calculation on the local region map to be processed after the average pooling, and the output feature map to be input into the j+1 target network layer is determined based on the local region map to be applied and the output feature map of the j-1th target network layer.

[0049] The output feature map refers to the feature map output after passing through the j-1th target network layer, and is also the feature map input to the jth target network layer. The first size refers to the size of the image obtained after the output feature map is processed by the convolution layer, and the image obtained after the convolution layer is the convolution feature image.

[0050] It should be noted that after convolution processing is performed on the output feature map based on the convolution layer, useful feature information can be extracted, and a convolution feature map of the first size can be output. Specifically, after inputting the output feature map of the j-1th target network layer, a convolution kernel can be defined. Then, based on the convolution kernel, each part of the output feature map is element-wise multiplied and summed. After the convolution kernel slides over the output feature map, a new feature map is output. The obtained new feature map can be nonlinearly transformed by applying an activation function, and the convolution feature map of the first size is finally output.

[0051] The local area map to be processed refers to the local area map obtained by dividing the convolution feature map into regions through the local area partitioning layer. It should be noted that the size of the local area map to be processed is usually a hyperparameter and can be adjusted according to needs. For example, for detailed features in an image, a smaller area (such as 3*3) may be more effective in the image processing process; for large-scale objects or structural information, a larger area (such as 7*7 or 9*9) may be more appropriate.

[0052] Optionally, local area division can be accomplished through a sliding window mechanism. For the convolution feature map input to the local area division layer, a fixed-size window (such as h*h) can be slid and the window can be processed as a local area map to be processed. Each time it slides, the sliding window can cover a portion of the convolution feature map, so that the features of all convolution feature maps can be extracted and processed. When using the sliding window method for local area division, the step size determines the distance the sliding window slides each time. A larger step size can reduce the overlap between the local area maps to be processed, thereby further reducing the amount of calculation; a smaller step size can increase the overlap of the local area maps to be processed, allowing the model to extract features more finely, but it will also increase the amount of calculation.

[0053] It should be noted that the purpose of the local region partitioning layer is to enhance the focus on local details by dividing the convolutional feature map into several small local region maps to be processed. This approach enables the model to capture detailed information with lower computational complexity and improves feature extraction efficiency.

[0054] For example, for a convolutional feature map of size 10*10, a fixed-size window of 3*3 with a step size of 1 can be used to divide the local area, and (10-3+1)*(10-3+1)=8*8=64 local area maps to be processed will be obtained; if a fixed-size window of 3*3 with a step size of 2 is used to divide the local area, the local area maps to be processed obtained by division are ((10-3) / 2+1)*((10-3) / 2+1)=16 local areas.

[0055] It should be noted that the resizing of the local region map to be processed based on the resizing layer can be further described. For each local region map to be processed output by the local region partitioning layer, the K, Q, and V values of each local region map to be processed can be calculated. For a set of K, Q, and V values corresponding to a local region map to be processed, first, the correlation between Q and K is calculated, typically using a dot product to calculate similarity. Then, a weighted summation is performed, that is, the corresponding V is weighted by the similarity result, with K pairs with higher similarity being assigned a greater weight. Finally, the weighted value V is weighted and summed to obtain a new local region representation, which is the output result. After obtaining the output result of each local region map to be processed, all output results can be calculated through a residual convolution layer. The local region map to be processed obtained after calculation by the residual convolution layer has the same number of image channels and image size as the multiple local region maps to be processed.

[0056] Among them, average pooling processing refers to the processing process of the average pooling layer. For the local area map to be processed after average pooling, after global attention calculation, the image obtained is the local area map to be applied. It should be noted that global attention calculation is performed on the local area map to be processed after average pooling, specifically: after passing through the average pooling layer, for each local area map to be processed after average pooling output, there is only one pixel value, and this pixel value can be used as the pixel value of the corresponding local area map to be processed after average pooling. Based on all local area maps to be processed after average pooling, the pixel values of all local area maps to be processed after average pooling can be calculated. For all local area maps to be processed after average pooling, based on the pixel values of all local area maps to be processed after average pooling, the Q and K values of the entire local area map to be processed after average pooling can be calculated. The entire local area map to be processed after size transformation can be used as the V value. First, the correlation between Q and K is calculated, and the similarity is usually calculated using the dot product. Then, a weighted sum is performed, that is, the corresponding V is weighted by the similarity result, and the K pairs with higher similarity will be given a larger weight. Finally, the weighted value V is weighted summed to obtain a new representation of the entire area, which is the local area map to be applied.

[0057] It should be noted that the corresponding elements of the local area map to be applied and the output feature map of the j-1th target network layer can be summed. After the summation, the resulting image is the output feature map to be input to the j+1th target network layer.

[0058] Specifically, at least one 2D image can be acquired based on a 3D image of the prostate area. All acquired 2D images can then be input into a pre-trained lesion segmentation model. Based on the lesion segmentation model, lesion segmentation images corresponding to all input 2D images can be output.

[0059] The technical solution of the embodiment of the present disclosure is as follows: first, at least one 2D image including the target lesion site is obtained, wherein the at least one 2D image is extracted based on the 3D image of the target lesion site. Then, the at least one 2D image is input into the pre-trained lesion segmentation model, and the lesion segmentation image corresponding to the at least one 2D image is output. The lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer, and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer, and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, and a global plus local attention layer. The number of at least one first target network layer is the same as the number of at least one second target network layer. The output of the first target network layer in the same processing order is the input of the second target network layer. This solves the problems of poor robustness, insufficient edge detail processing, and limited multi-scale adaptability of the segmentation model based only on the U-Net architecture or in combination with the residual network in the prior art. It also solves the problem of low segmentation accuracy when using the existing segmentation model for image segmentation. The embodiment of the present invention obtains a lesion segmentation image corresponding to a 2D image based on a lesion segmentation model that includes at least a global plus local attention layer. The local attention layer of the model enhances the capture of details of the lesion area in the image and improves the segmentation accuracy of small lesions or areas with blurred edges. The global attention layer of the model identifies the correlation between different areas in the image, which helps to understand the structure and overall context of the entire image. The features extracted by the global and local attention layers are fused to achieve the integration of global vision and local detail information, thereby improving the generalization ability and robustness of the model and improving the segmentation accuracy of the lesion image.

[0060] Example 2

[0061] Figure 4 This is a flowchart of an image processing method provided by an embodiment of the present invention. Based on the previous embodiment, a detailed description of the lesion segmentation model obtained by training is provided. For its specific implementation, please refer to the technical solution of this embodiment. Technical terms that are the same or corresponding to those in the above embodiment are not repeated here.

[0062] like Figure 4 As shown, the method specifically includes the following steps:

[0063] S210: Acquire multiple training samples.

[0064] The training samples include at least two consecutive 2D sample images extracted from 3D images of the same object and the same lesion site, and lesion annotated images corresponding to the at least two consecutive 2D images.

[0065] Among them, multiple training samples refer to multiple samples for training the lesion segmentation model. A 2D sample image refers to an image extracted from a 3D image. It should be noted that when extracting a 2D sample image, a certain number of continuous 2D images can be extracted from the 3D image of the same object and the same lesion site as needed. A lesion annotation image refers to an image obtained after annotating the 2D sample image. It should be noted that for a certain 2D sample image, the area that needs to be annotated (such as a tumor or a blood vessel) can be determined, and then the 2D sample image can be annotated manually or by using an automatic threshold segmentation tool. The area that needs to be annotated is saved as 1, and the remaining background areas are set to 0, and finally a lesion annotation image corresponding to the 2D sample image is generated.

[0066] Optionally, after obtaining a 3D image of the same object and the same lesion site and a 3D annotated image corresponding to the 3D image, the same data augmentation processing can be performed on the 3D image and the 3D annotated image. The purpose is to expand the training samples and improve the generalization ability of the model by performing the same transformation on the 3D image and the 3D annotated image. Optionally, the 3D image and the 3D annotated image can be rotated, and new samples can be generated by rotating the image around the center point of the image. Rotation is usually a fixed-angle operation, and the rotation angle can be positive or negative, usually common angles such as 90°, 180°, 270°, or any arbitrary angle; the 3D image and the 3D annotated image can be translated to displace the entire image horizontally or vertically. Through the translation transformation, the position of the image will change, but the content of the image will not be changed. Optionally, you can first select the number of pixels to translate, for example, by randomly selecting the translation amount in the x- and y-axis directions, and then use the translation function to translate the 3D image and 3D annotated image. You can also perform elastic transformations on the 3D image and 3D annotated image, creating a more natural effect by nonlinearly deforming the image. This is achieved by adding a certain degree of random displacement to each pixel in the image. Elastic transformations are very effective for enhancing model robustness. Optionally, you can create a random field (usually Gaussian noise) and apply it to each pixel in the image. Then, you can adjust the position of each pixel in the image using the displacement field to perform elastic transformations. You can also add noise to the 3D image and 3D annotated image, adding noise to the image to increase the robustness of the model and prevent overfitting when processing real-world data. Common noise types include Gaussian noise and salt and pepper noise. Optionally, you can add random noise that follows a Gaussian distribution to the pixel values of the image, or randomly select some pixels in the image and set them to extreme black or white values. The above data augmentation methods can help improve the generalization ability of the model and avoid overfitting, especially when there are insufficient training samples.

[0067] Specifically, at least two consecutive 2D sample images and lesion annotated images corresponding to the at least two consecutive 2D images can be extracted from the 3D image as training samples for training the lesion segmentation model. The same data augmentation processing can also be performed on the 3D image and the 3D annotated images corresponding to the 3D image. At least two consecutive 2D sample images and lesion annotated images corresponding to the at least two consecutive 2D images can be extracted from the enhanced 3D image as training samples for training the lesion segmentation model.

[0068] S220 . For a plurality of training samples, input at least two consecutive 2D sample images in the training samples into a lesion segmentation model to be trained, and output predicted labeled images corresponding to the at least two consecutive 2D images.

[0069] The lesion segmentation model to be trained is a model whose model parameters are initial parameters or default parameters. The predicted annotated image is an annotated image output after at least two consecutive 2D sample images in the training sample are input into the lesion segmentation model to be trained.

[0070] It should be noted that the model parameters in the lesion segmentation model to be trained do not meet the expected requirements. Therefore, there is a certain difference between the predicted annotation image and the lesion annotation image output based on the model parameters at this time. Therefore, the corresponding error loss value can be determined based on the predicted annotation image and lesion annotation image corresponding to each 2D sample image.

[0071] S230 : Processing the lesion annotated images and corresponding predicted annotated images of at least two consecutive 2D images based on the loss function in the lesion segmentation model to be trained to determine a target loss value.

[0072] Among them, the loss function is a mathematical formula used to measure the difference between the predicted annotation image of the lesion segmentation model to be trained and the lesion annotation image. The loss function is usually based on the comparison of the predicted annotation image of the 2D image segmentation (usually a binary image, indicating whether the pixel belongs to the lesion area) with the lesion annotation image. The loss function in the lesion segmentation model to be trained can be a cross-entropy loss function. The target loss value refers to the loss value that the lesion segmentation model to be trained is expected to be optimized to during the training process of the lesion segmentation model to be trained. When optimizing the lesion segmentation model to be trained, the value of the loss function can be minimized as much as possible, that is, the predicted annotation image of the lesion segmentation model to be trained is made as close to the lesion annotation image as possible.

[0073] Specifically, we select an appropriate loss function and calculate the loss for each pair of 2D images, i.e., the lesion annotation image and the corresponding predicted annotation image. We then sum all the loss values and gradually optimize the weights of the lesion segmentation model to be trained based on the sum of the loss values. The ultimate goal is to minimize the target loss value while achieving good generalization performance and avoiding overfitting.

[0074] S240 , correcting the model parameters in the lesion segmentation model to be trained based on the target loss value, and using the model obtained when the loss function converges as the lesion segmentation model.

[0075] When training the lesion segmentation model to be trained, the model parameters in the model can be corrected based on the output results of the lesion segmentation model to be trained. That is, the lesion segmentation model can be obtained by correcting the loss function in the lesion segmentation model to be trained.

[0076] Specifically, after a 2D image is input into the lesion segmentation model to be trained, the lesion segmentation model to be trained can obtain a predicted annotated image corresponding to the 2D image. Based on the predicted annotated image and the lesion annotated image, the loss value corresponding to the 2D image can be determined. The model parameters of the lesion segmentation model to be trained can be modified using the backpropagation method. The model obtained when the loss function converges is used as the lesion segmentation model.

[0077] S250: Acquire at least one 2D image including the target lesion site.

[0078] S260: Input at least one 2D image into a pre-trained lesion segmentation model, and output a lesion segmentation image corresponding to the at least one 2D image.

[0079] The technical solution of the embodiment of the present disclosure obtains multiple training samples. For the multiple training samples, at least two continuous 2D sample images in the training samples are input into the lesion segmentation model to be trained, and the predicted annotation images corresponding to the at least two continuous 2D images are output. Furthermore, the lesion annotation images and the corresponding predicted annotation images of the at least two continuous 2D images are processed based on the loss function in the lesion segmentation model to be trained to determine the target loss value. Based on the target loss value, the model parameters in the lesion segmentation model to be trained are corrected, and the model obtained when the loss function converges is used as the lesion segmentation model. Training with continuous 2D images can help the model better understand the temporal relationship between images, thereby improving the accuracy of lesion segmentation. Continuous image input enables the model to capture the contextual information between images, so that it can better handle the complex structures and lesion locations in the image, and improve the performance of the model in different slices and image noise, making the model more stable in practical applications.

[0080] Example 3

[0081] As an optional embodiment of the above embodiment, its specific implementation can be combined with Figure 3 And the following text to understand.

[0082] In this embodiment, first input data, input the image data x of training set sample i (i) and its label y (i) , then, for x (i) Perform data enhancement, such as rotation, translation, elastic transformation, noise addition, etc. If x (i) Involving spatial transformation, y (i) The same spatial transformation is also applied to ensure that x (i) and y (i) Furthermore, a position is randomly selected from the z-axis of the 3D image and (i)Take out c consecutive 2D images at the position and convert y (i) The label image at the corresponding position in is also taken out. (i) Normalize the pixels so that their values are between 0 and 1. (i) and y (i) The image is resized to be the same size as c×w×d and the image and label are used as input to the model for training.

[0083] The model of this solution is based on the U-Net structure, and local attention and global attention are added to each convolutional network. The U-Net of this solution consists of an encoder and a decoder. The encoder has L encoding layers and the decoder has L-1 decoding layers. The encoding and decoding process of this solution is as follows: In the encoding process, the encoder consists of L encoding layers, and each encoding layer is composed of a residual convolutional network and local and global attention. Taking the jth encoding layer as an example, the encoding process is as follows: The input of the encoding layer, the input of the jth encoding layer is the output of the j-1th encoding layer, that is, the size of the input feature is c j-1 ×w j-1 ×d j-1 Convolution calculation, the jth encoding layer first uses the residual convolution network to calculate the input features, and the output size is c j ×w j ×d j , where w j It is w j-1 Half of d j It is d j-1 Divide the local area, then divide the result of convolution into several local areas, each of which has a size of h×h. Local attention calculation, then transform the divided features to facilitate attention calculation. The transformed size is That can be understood as batches, each batch has (h×h) pixels, each pixel is composed of c j dimensional feature representation. Then, for each batch, the self-attention between (h×h) pixels is calculated. The output after the calculation is So resize it again to restore its size to its original size This process only calculates attention within h×h, calculates the relationship between adjacent regions, that is, local attention, so that some can capture complex boundary information and subtle structural changes. By refining the characteristics of local regions, the segmentation accuracy of key anatomical regions such as the prostate peripheral zone is significantly improved. Average pooling calculates the average value of each local region, so after average pooling, the size of the feature becomes Global attention calculation, then Considered to have pixels, and each pixel is composed of c j dimensional feature representation, so we can calculate this The self-attention between pixels, the output size is Each row of the matrix represents the relationship between the current pixel and all other pixels. Applying global attention, the output The h×h in the image is considered as a whole pixel, and then and output Perform matrix multiplication and finally get the size of By calculating the relationship between all local regions, the global attention mechanism further aggregates global information based on the detailed features extracted by local attention, and establishes a contextual association between the prostate and surrounding tissues from a larger range. Convolution calculation, finally, the result of global attention is resized to obtain a size of c j ×w j ×d j The feature is finally input into the residual convolutional network to obtain a size of c j ×w j ×d j The output of the convolution is then added to the result of the convolution calculation, completing the encoding process. This process continues until the final encoding layer is reached. The output of the final encoding layer is then used as the input to the decoder.

[0084] During the decoding process, the decoder consists of L-1 decoding layers, each of which is composed of a residual convolutional network (RCN) and local and global attention. Taking the jth decoding layer as an example, the decoding process is as follows: The jth decoding layer inputs the output of the j+1th decoding layer and the output of the jth encoder layer (the order of the decoding layers is exactly the opposite of the encoding layer). Similar to the encoding layer, the decoding layer first performs residual convolution on the input features. Similar to the encoding layer, the decoding layer calculates local and global attention on the convolved features. Finally, similar to the encoding layer, the residual convolutional network is used to calculate the attended features, which are then accumulated with the initial input and output. This output becomes the final output of the decoding layer. This process continues until the final decoding output is the model's prediction. Loss calculation involves calculating the cross-entropy loss between the model's prediction and the true label, and updating the model parameters based on the loss.

[0085] Model selection: After model training is completed, the validation set is used to select the model with the best segmentation result from the trained models as the final model. Model prediction: input a new data into the final model and output the model prediction result.

[0086] The technical solution of the embodiment of the present disclosure is to import and pre-process the data, and then train the model based on the encoder and decoder. Furthermore, the model parameters are updated according to the loss value. Furthermore, a model with the best segmentation result is selected from the trained models using the validation set as the final model. Finally, a new data is input into the final model, and the result predicted by the model is output. The local and global attention mechanisms are introduced, and the characteristics of the prostate anatomy are combined to achieve refined segmentation. Dividing the self-attention into local and global makes the model's memory of local and global better, and reduces the amount of calculation.

[0087] Example 4

[0088] Figure 5 is a structural diagram of an image processing device provided by an embodiment of the present disclosure, such as Figure 5 As shown, the device includes: a 2D image acquisition module 310 and a lesion segmentation image output module 320.

[0089] A 2D image acquisition module is used to acquire at least one 2D image including a target lesion site, wherein the at least one 2D image is extracted based on a 3D image of the target lesion site; a lesion segmentation image output module is used to input the at least one 2D image into a pre-trained lesion segmentation model and output a lesion segmentation image corresponding to the at least one 2D image; wherein the lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer and an output layer, the first target network layer includes a residual convolutional network layer, a global plus local attention layer and a downsampling layer, the second target network layer includes an upsampling layer, a residual convolutional network layer, a global plus local attention layer, the number of the at least one first target network layer is the same as the number of the at least one second target network layer, and the output of the first target network layer with the same processing order is the input of the second target network layer.

[0090] The technical solution of the embodiment of the present disclosure is as follows: first, at least one 2D image including the target lesion site is obtained, wherein the at least one 2D image is extracted based on the 3D image of the target lesion site. Then, the at least one 2D image is input into the pre-trained lesion segmentation model, and the lesion segmentation image corresponding to the at least one 2D image is output. The lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer, and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer, and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, and a global plus local attention layer. The number of at least one first target network layer is the same as the number of at least one second target network layer. The output of the first target network layer in the same processing order is the input of the second target network layer. This solves the problems of poor robustness, insufficient edge detail processing, and limited multi-scale adaptability of the segmentation model based only on the U-Net architecture or in combination with the residual network in the prior art. It also solves the problem of low segmentation accuracy when using the existing segmentation model for image segmentation. The embodiment of the present invention obtains a lesion segmentation image corresponding to a 2D image based on a lesion segmentation model that includes at least a global plus local attention layer. The local attention layer of the model enhances the capture of details of the lesion area in the image and improves the segmentation accuracy of small lesions or areas with blurred edges. The global attention layer of the model identifies the correlation between different areas in the image, which helps to understand the structure and overall context of the entire image. The features extracted by the global and local attention layers are fused to achieve the integration of global vision and local detail information, thereby improving the generalization ability and robustness of the model and improving the segmentation accuracy of the lesion image.

[0091] Based on the above technical solutions, the lesion segmentation image output module 320 includes: a lesion segmentation image output submodule, which is used to input the at least one 2D image into the image segmentation model, and process the 2D image based on the input layer, at least one first target network layer, at least one second target network layer, the residual convolution network layer and the output layer, and output the lesion segmentation image of the 2D image.

[0092] Based on the above technical solutions, the global plus local attention layer includes a local area division layer, a size transformation layer, a residual convolutional network layer and an average pooling layer.

[0093] On the basis of the above technical solutions, the device further includes: a 2D image processing module. The 2D image processing module includes: a convolution feature map output submodule, an average pooling processing submodule, and a feature map output submodule.

[0094] A convolutional feature map output submodule, configured to input the output feature map of the j-1th target network layer into the jth target network layer, perform convolution processing on the output feature map based on the convolution layer, and output a convolutional feature map of a first size;

[0095] an average pooling processing submodule, configured to divide the convolution feature map into regions based on the local region division layer to obtain a plurality of local region maps to be processed, perform size transformation on the local region maps to be processed based on the size transformation layer, and perform average pooling processing on the size-transformed local region maps to be processed;

[0096] The feature map output submodule is used to obtain the local area map to be applied by performing global attention calculation on the local area map to be processed after average pooling, and determine the output feature map to be input to the j+1 target network layer based on the local area map to be applied and the output feature map of the j-1th target network layer.

[0097] Based on the above technical solutions, the device further includes a lesion segmentation model training module. The lesion segmentation model training module includes a training sample acquisition submodule, a predicted and labeled image output submodule, a target loss value determination submodule, and a lesion segmentation model determination submodule.

[0098] a training sample acquisition submodule, configured to acquire a plurality of training samples, wherein the training samples include at least two consecutive 2D sample images extracted from 3D images of the same subject and the same lesion site, and lesion annotated images corresponding to the at least two consecutive 2D images;

[0099] a predicted and annotated image output submodule, configured to input at least two consecutive 2D sample images from the plurality of training samples into a lesion segmentation model to be trained, and output predicted and annotated images corresponding to the at least two consecutive 2D images;

[0100] a target loss value determination submodule, configured to process the lesion annotated images and the corresponding predicted annotated images of the at least two consecutive 2D images based on the loss function in the lesion segmentation model to be trained to determine a target loss value;

[0101] The lesion segmentation model determination submodule is used to correct the model parameters in the lesion segmentation model to be trained based on the target loss value, and use the model obtained when the loss function converges as the lesion segmentation model.

[0102] On the basis of the above technical solutions, the target lesion site is a prostate lesion site.

[0103] The image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0104] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0105] Example 5

[0106] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 6 , which shows an electronic device (eg Figure 6 The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), an in-vehicle terminal (such as an in-vehicle navigation terminal), and the like. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0107] like Figure 6 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0108] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0109] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0110] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0111] The electronic device provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0112] Example 6

[0113] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the image processing method provided by the above embodiment is implemented.

[0114] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0115] In some embodiments, the server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0116] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0117] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0118] Acquiring at least one 2D image including a target lesion site, wherein the at least one 2D image is extracted based on a 3D image of the target lesion site;

[0119] Inputting the at least one 2D image into a pre-trained lesion segmentation model, and outputting a lesion segmentation image corresponding to the at least one 2D image;

[0120] Among them, the lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, a global plus local attention layer. The number of the at least one first target network layer is the same as the number of the at least one second target network layer, and the output of the first target network layer in the same processing order is the input of the second target network layer.

[0121] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0124] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0125] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0127] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0128] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: Acquiring at least one 2D image including a target lesion site, wherein the at least one 2D image is extracted based on a 3D image of the target lesion site; Inputting the at least one 2D image into a pre-trained lesion segmentation model, and outputting a lesion segmentation image corresponding to the at least one 2D image; Among them, the lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, a global plus local attention layer. The number of the at least one first target network layer is the same as the number of the at least one second target network layer, and the output of the first target network layer in the same processing order is the input of the second target network layer.

2. The method according to claim 1, characterized in that Inputting the at least one 2D image into a pre-trained image segmentation model and outputting a lesion segmentation image corresponding to the at least one 2D image includes: For the at least one 2D image, the 2D image is input into the image segmentation model, and the 2D image is processed based on the input layer, at least one first target network layer, the at least one second target network layer, the residual convolutional network layer and the output layer, and a lesion segmentation image of the 2D image is output.

3. The method according to claim 1, characterized in that The global plus local attention layer includes a local area division layer, a size transformation layer, a residual convolutional network layer and an average pooling layer.

4. The method according to claim 3, characterized in that Processing the 2D image based on the first target network layer includes: Inputting the output feature map of the j-1th target network layer into the jth target network layer, performing convolution processing on the output feature map based on the convolution layer, and outputting a convolution feature map of a first size; Dividing the convolution feature map into regions based on the local region division layer to obtain a plurality of local region maps to be processed, resizing the local region maps to be processed based on the resizing layer, and performing average pooling processing on the resized local region maps to be processed; By performing global attention calculation on the local area map to be processed after average pooling, a local area map to be applied is obtained, and based on the local area map to be applied and the output feature map of the j-1th target network layer, the output feature map to be input into the j+1 target network layer is determined.

5. The method according to claim 1, wherein The method further comprises: Acquiring a plurality of training samples, wherein the training samples include at least two consecutive 2D sample images extracted from 3D images of the same subject and the same lesion site, and lesion annotated images corresponding to the at least two consecutive 2D images; For the multiple training samples, input at least two consecutive 2D sample images in the training samples into the lesion segmentation model to be trained, and output predicted labeled images corresponding to the at least two consecutive 2D images; Processing the lesion annotated images and the corresponding predicted annotated images of the at least two consecutive 2D images based on the loss function in the lesion segmentation model to be trained to determine a target loss value; The model parameters in the lesion segmentation model to be trained are modified based on the target loss value, and the model obtained when the loss function converges is used as the lesion segmentation model.

6. The method according to claim 1, characterized in that The target lesion site is a prostate lesion site.

7. An image processing device, characterized in that: include: a 2D image acquisition module, configured to acquire at least one 2D image including a target lesion site, wherein the at least one 2D image is extracted based on a 3D image of the target lesion site; a lesion segmentation image output module, configured to input the at least one 2D image into a pre-trained lesion segmentation model and output a lesion segmentation image corresponding to the at least one 2D image; Among them, the lesion segmentation model includes an input layer, at least one first target network layer, at least one second target network layer, a residual convolutional network layer and an output layer. The first target network layer includes a residual convolutional network layer, a global plus local attention layer and a downsampling layer. The second target network layer includes an upsampling layer, a residual convolutional network layer, a global plus local attention layer. The number of the at least one first target network layer is the same as the number of the at least one second target network layer, and the output of the first target network layer in the same processing order is the input of the second target network layer.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by one or more processors, the one or more processors implement the image processing method according to any one of claims 1 to 6.

9. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the image processing method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the image processing method according to any one of claims 1 to 6.

Images