An image segmentation method and related apparatus

Abstract

The application provides an image segmentation method and related device, comprising: obtaining a to-be-processed image, the voxel spacing of the to-be-processed image being a first voxel spacing, the voxel spacing of a training image used for training an image segmentation network model being a second voxel spacing, the first voxel spacing being smaller than the second voxel spacing; referring to at least the second voxel spacing, obtaining all sub-images of the to-be-processed image, the value of each pixel in each sub-image being the same as the value of the pixel in the to-be-processed image, the voxel spacing of each sub-image being a third voxel spacing, the third voxel spacing matching the second voxel spacing; inputting each sub-image into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model; and restoring the segmentation result of each sub-image to the to-be-processed image to obtain the segmentation result of the to-be-processed image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image segmentation method and related apparatus. Background Technology

[0002] In the medical field, medical images (also known as medical imaging) are commonly used to assist in disease diagnosis and treatment planning. With the development of deep learning technology, segmenting medical images using deep learning has gradually become mainstream. For example, an image segmentation network model can be trained using training images labeled with segmentation results. This image segmentation network model can learn the human structural features in the segmentation results, enabling it to recognize human structures. After training the image segmentation network model, the image to be processed (referred to as the inference image) is input into the image segmentation network model. The image segmentation network model can automatically identify and segment the human structures in the image to be processed and output the segmentation results of the image to be processed.

[0003] Both the training images and the images to be processed have voxel spacing. Voxel spacing refers to the actual physical distance (e.g., mm) between adjacent voxels (the basic units of an image) in three-dimensional (x, y, z) or two-dimensional (x, y) space. Voxel spacing reflects the spatial resolution of the image. Because the voxel spacing between the training images and the images to be processed is different, the images to be processed need to be resampled before being input into the image segmentation network model to make the voxel spacing of the images to be processed the same as that of the training images. After obtaining the segmentation results, the segmentation results are resampled to obtain a segmented image that matches the size of the images to be processed. However, resampling will lose information and accumulate errors, resulting in a decrease in the accuracy of the segmentation results. Summary of the Invention

[0004] In view of the above problems, this application provides an image segmentation method and related apparatus to improve the accuracy of segmentation results. The specific solution is as follows:

[0005] The first aspect of this application provides an image segmentation method, comprising:

[0006] The image to be processed is obtained, wherein the voxel spacing of the image to be processed is the first voxel spacing, and the voxel spacing of the training images used to train the image segmentation network model is the second voxel spacing, wherein the first voxel spacing is less than the second voxel spacing.

[0007] At least with reference to the second voxel spacing, all sub-images of the image to be processed are obtained, the value of each pixel in each sub-image is the same as the value of the pixel in the image to be processed, and the voxel spacing of each sub-image is the third voxel spacing, which matches the second voxel spacing;

[0008] Each sub-image is input into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model;

[0009] The segmentation results of each sub-image are restored to the image to be processed to obtain the segmentation results of the image to be processed.

[0010] In one possible implementation, obtaining all sub-images of the image to be processed, at least with reference to the second voxel spacing, includes:

[0011] For any dimension of the first voxel spacing, the number of segments of the image to be processed in that dimension is determined according to the ratio of the second voxel spacing to the first voxel spacing in that dimension. The number of segments is an integer of the ratio and the number of segments is greater than or equal to 1.

[0012] The split grid of the image to be processed is determined based on the ratio of the second voxel spacing and the first voxel spacing in different dimensions;

[0013] Using a splitting network for the image to be processed, the image to be processed is split into multiple sub-images in a preset order, and the splitting network is used to indicate the size of the sub-images.

[0014] In one possible implementation, the segmentation result of the sub-image includes: annotation information for each pixel in the sub-image and the index of each pixel, wherein the annotation information is used to indicate whether the pixel is background or human structure; the step of restoring the segmentation result of each sub-image to the image to be processed to obtain the segmentation result of the image to be processed includes:

[0015] Determine the position of each pixel in each sub-image within the image to be processed;

[0016] The annotation information of each pixel is written into the position of that pixel in the image to be processed to obtain the segmentation result of the image to be processed.

[0017] In one possible implementation, the image to be processed is divided into multiple sub-images in a preset order using a split grid, and determining the position of each pixel in each sub-image within the image to be processed includes:

[0018] The sub-image is determined to be the i-th sub-image of the image to be processed, where i is any value in [1, N] and N is the number of all sub-images of the image to be processed;

[0019] Obtain the index of each pixel in each dimension of the i-th sub-image;

[0020] The index of each pixel in the i-th sub-image in each dimension is multiplied by i to obtain the target index of each pixel in each dimension. The target index of the pixel in each dimension is used to indicate the position of the pixel in the image to be processed.

[0021] In one possible implementation, the image to be processed includes a T1-weighted magnetic resonance image and a T2-weighted magnetic resonance image, wherein the voxel spacing of the T1-weighted magnetic resonance image and the voxel spacing of the T2-weighted magnetic resonance image are both the first voxel spacing.

[0022] The step of inputting each sub-image into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model includes: simultaneously inputting sub-images with the same position in the T1-weighted magnetic resonance image and the T2-weighted magnetic resonance image into the image segmentation network model to obtain the segmentation result output by the image segmentation network model.

[0023] A second aspect of this application provides an image segmentation apparatus, comprising:

[0024] The first acquisition module is used to acquire the image to be processed, wherein the voxel spacing of the image to be processed is the first voxel spacing, and the voxel spacing of the training image used to train the image segmentation network model is the second voxel spacing, wherein the first voxel spacing is smaller than the second voxel spacing.

[0025] The second acquisition module is used to obtain all sub-images of the image to be processed by referring at least to the second voxel spacing, wherein the value of each pixel in each sub-image is the same as the value of the pixel in the image to be processed, and the voxel spacing of each sub-image is the third voxel spacing, wherein the third voxel spacing matches the second voxel spacing.

[0026] The segmentation module is used to input each sub-image into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model;

[0027] The restoration module is used to restore the segmentation result of each sub-image to the image to be processed, so as to obtain the segmentation result of the image to be processed.

[0028] In one possible implementation, the second acquisition module is configured to determine, for any dimension of the first voxel spacing, the number of segments of the image to be processed in that dimension based on the ratio of the second voxel spacing to the first voxel spacing in that dimension, wherein the number of segments is an integer of the ratio and the number of segments is greater than or equal to 1; determine the segmentation grid of the image to be processed based on the ratio of the second voxel spacing to the first voxel spacing in different dimensions; and use the segmentation grid of the image to be processed to segment the image to be processed into multiple sub-images in a preset order, wherein the segmentation grid is used to indicate the size of the sub-images;

[0029] And / or,

[0030] The segmentation result of the sub-image includes: the annotation information of each pixel in the sub-image and the subscript of each pixel, wherein the annotation information is used to indicate whether the pixel is background or human body structure; the restoration module is used to determine the position of each pixel in each sub-image in the image to be processed; and the annotation information of each pixel is written into the position of the pixel in the image to be processed to obtain the segmentation result of the image to be processed.

[0031] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the image segmentation method described in the first aspect or any implementation thereof.

[0032] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program so that the electronic device can implement the image segmentation method of the first aspect or any implementation thereof.

[0035] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to perform the image segmentation method described in the first aspect or any implementation thereof.

[0036] By employing the above technical solution, the image segmentation method and related apparatus provided in this application, after acquiring the image to be processed, obtains all sub-images of the image to be processed by referring at least to the second voxel spacing. The value of each pixel in each sub-image is the same as the value of that pixel in the image to be processed. The voxel spacing of each sub-image is the third voxel spacing, which matches the second voxel spacing. Each sub-image is input into an image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model. The segmentation result of each sub-image is then restored to the image to be processed to obtain the segmentation result of the image to be processed. The matching of the third voxel spacing of the sub-images with the second voxel spacing of the training images allows the image segmentation network model to accurately identify human structural features in the sub-images, improving the accuracy of the segmentation results. Furthermore, the value of any pixel in a sub-image is the value of that pixel in the image to be processed, preserving the detailed information in the image to be processed. This allows the image segmentation network model to utilize the detailed information for recognition and segmentation, further improving the accuracy of the segmentation results. Attached Figure Description

[0037] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0038] Figure 1 A flowchart of an image segmentation method provided in this application;

[0039] Figure 2 A schematic diagram of the structure of an image segmentation device provided in this application;

[0040] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0041] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0042] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0043] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements but may include other elements not explicitly listed or inherent to such processes, methods, systems, products, or apparatus.

[0044] Currently, medical images are commonly used in the medical field to assist in disease diagnosis and treatment planning, such as CT (Computed Tomography) images and MR (Magnetic Resonance) images.

[0045] CT images are generated by scanning the human body layer by layer using X-rays, producing tomographic images that display the body's structures. These images can be combined into three-dimensional (3D) or two-dimensional (2D) sequences. Because CT images can clearly display human structures (such as organs and blood vessels), they are widely used in disease diagnosis (such as tumor localization) and treatment planning (such as surgical pathway design). Among these applications, "whole-body organ segmentation" in CT images refers to the precise delineation of the boundaries of organs such as the liver, heart, and blood vessels from CT images. Therefore, obtaining accurate segmentation results from CT images helps doctors determine the morphology, location, and potential lesions of organs (such as tumor size and vascular stenosis).

[0046] MR imaging uses magnetic fields and radiofrequency pulses to generate images of human structures, featuring no ionizing radiation and high contrast in soft tissues. Because MR images emit no ionizing radiation and can clearly display soft tissues (such as brain white matter, spinal cord, and internal organs), they are widely used for the diagnosis of systemic organ diseases (such as Alzheimer's disease and tumors) and treatment planning. Among these applications, "systemic organ segmentation" in MR imaging refers to the precise delineation of organ boundaries from MR images. Accurate segmentation results from MR images are the foundation for clinical quantitative analysis. For example, changes in hippocampal size can aid in the early diagnosis of Alzheimer's disease, and vascular segmentation results can assess the degree of arteriosclerosis.

[0047] In this process, CT and MR images awaiting processing are input into an image segmentation network model, which then outputs the segmentation results. In practical applications, the voxel spacing between the image to be processed and the training images used by the image segmentation network model differs due to factors such as equipment and imaging parameters. For example, the voxel spacing of CT images is affected by the scanning equipment (e.g., different manufacturers' CT scanners), the scanned area (e.g., chest, abdomen), and scanning parameters (e.g., slice thickness), resulting in significant differences in voxel spacing between different CT images. For instance, the voxel spacing of a training image might be (0.6mm, 0.6mm, 1.2mm), while the voxel spacing of an image awaiting processing might be (1.2mm, 1.2mm, 2.4mm), a difference of up to two times across all dimensions. The voxel spacing of MR images is significantly affected by the equipment model (e.g., 1.5T vs. 3.0TMR scanners) and sequence parameters (e.g., slice thickness, field of view). For example, the voxel spacing of a training image is (0.5mm, 0.5mm, 1.0mm), while the voxel spacing of an image to be processed may be (1.0mm, 1.0mm, 2.0mm), and the ratio between the two in each dimension can be up to 2 times.

[0048] The difference in voxel spacing between the training image and the image to be processed can degrade the segmentation performance of image segmentation network models. Image segmentation network models "learn" human structural features based on training images with specific voxel spacing. If the voxel spacing of the image to be processed does not match, feature recognition will be biased. One solution is resampling. Resampling can use interpolation or downsampling to make the voxel spacing of the image consistent with the target value (e.g., the voxel spacing of the image to be processed is consistent with the voxel spacing of the training image). However, resampling may alter the image size and details within the image. The specific steps are as follows:

[0049] 1) Determine the target voxel spacing of the training image. The target voxel spacing of the training image is the original voxel spacing of the training image, which can be denoted as (X,Y,Z).

[0050] 2) Determine the voxel spacing of the image to be processed as (x, y, z). Since the voxel spacing of the image to be processed is different from the target voxel spacing, the image to be processed is resampled to make the voxel spacing of the image to be processed (X, Y, Z). For example, the voxel arrangement of the image to be processed can be adjusted by interpolation algorithms (such as linear interpolation, cubic interpolation).

[0051] 3) Input the resampled image to be processed into the image segmentation network model to obtain the segmentation result output by the image segmentation network model;

[0052] 4) If it is necessary to restore the original size, resample the segmentation result to obtain a segmented image that matches the original size of the image to be processed.

[0053] For example, when the voxel spacing of the training image is (0.4mm, 0.4mm) and the voxel spacing of the image to be processed is (0.8mm, 0.8mm), downsampling is used to merge every 2×2 pixels in the image to be processed into 1 pixel, making the voxel spacing (0.4mm, 0.4mm). After downsampling, the downsampled image to be processed is input into the image segmentation network model for segmentation. However, resampling will lose information and accumulate errors, reducing the accuracy of the segmentation results. Taking the above CT and MR images as examples, we will explain why resampling technology reduces the accuracy of the segmentation results. The disadvantage of resampling technology stems from its nature of "changing the voxel distribution of the image to be processed," and the specific reasoning is as follows:

[0054] Loss of detail: During downsampling, multiple original pixels in the image to be processed are merged into one pixel, resulting in the loss of subtle differences between pixels. For example, blood vessels with diameters smaller than the resampling step size may be directly "erased," such as branches of pulmonary vessels or narrow sections of coronary arteries, leading to incomplete vascular structures in the segmentation result. Similarly, in MR images, structures such as the hippocampus and microvessels are close to or smaller than the resampling step size, and resampling will cause their details to be erased. For instance, a 0.6mm diameter renal artery branch may be misclassified as background after downsampling, resulting in incomplete segmentation. The signal intensity of MR images is related to tissue relaxation time (e.g., fat shows high signal in T1-weighted images). The resampling interpolation process alters voxel signal values, disrupting signal continuity and leading to blurred tissue features. For example, the signal difference between white matter and gray matter in the brain is weakened after resampling, making region confusion more likely during segmentation.

[0055] Blurred segmentation edges: During upsampling, new pixels need to be "created" through interpolation. The values ​​of these pixels are inferred from surrounding pixels, which can lead to a noticeable "pixelation effect" in the image (stepped edges or artifacts at the edges). For example, the edges of the liver may become blurred after resampling, making it difficult to determine whether there are lesions protruding based on the segmentation results. Similarly, artifacts exist at tumor boundaries, making it difficult to determine whether the boundaries are clear based on the segmentation results.

[0056] Secondary error accumulation: If the segmentation result is resampled to match the original size of the image to be processed, new interpolation errors will be introduced, further reducing the segmentation accuracy.

[0057] In summary, resampling techniques can lead to information loss and error accumulation, resulting in segmentation results that fail to accurately reflect the true shape and details of the human body structure, thus reducing the accuracy of the segmentation results.

[0058] Reference Figure 1 , Figure 1 An optional process for an image segmentation method provided in an embodiment of this application, such as... Figure 1As shown in the figure, the data processing method provided in this application embodiment may include steps S101 to S105, which are described in detail below.

[0059] S101. Obtain the image to be processed. The voxel spacing of the image to be processed is the first voxel spacing, and the voxel spacing of the training images used to train the image segmentation network model is the second voxel spacing. The first voxel spacing is less than the second voxel spacing.

[0060] The image to be processed is the image that needs to be segmented to obtain the segmentation result. For example, the image to be processed can be at least one of CT images and MR images. The human structural features in different types of images to be processed may be different. Therefore, in order to better segment different types of images to be processed, different image segmentation network models may be used for different types of images to be processed. Any image segmentation network model is trained using a type of historical image and the segmentation results labeled in that historical image.

[0061] In some examples, an image segmentation network model can be trained using multiple types of historical images and the segmentation results labeled in those historical images, enabling the model to handle various types of images and reduce costs.

[0062] The historical images used to train the image segmentation network model are the training images, and the voxel spacing of the training images is the second voxel spacing. The voxel spacing of the images to be processed is the first voxel spacing. This embodiment mainly targets the scenario where the first voxel spacing is smaller than the second voxel spacing.

[0063] S102. At least referencing the second voxel spacing, obtain all sub-images of the image to be processed. The value of each pixel in each sub-image is the same as the value of that pixel in the image to be processed. The voxel spacing of each sub-image is the third voxel spacing, and the third voxel spacing matches the second voxel spacing.

[0064] The matching of the third voxel spacing with the second voxel spacing can be either that the third voxel spacing and the second voxel spacing are the same, or that the third voxel spacing and the second voxel spacing are close to each other. The third voxel spacing and the second voxel spacing being close to each other can be defined as follows: the difference between the third voxel spacing and the second voxel spacing is very small, and the third voxel spacing is approximately equal to the second voxel spacing.

[0065] In this embodiment, each sub-image of the image to be processed can reuse the pixel values ​​in the image to be processed, so that the sub-image can retain the detailed information in the image to be processed and reduce the loss rate of detailed information. Furthermore, the voxel spacing of the sub-image matches the voxel spacing of the training image, enabling the image segmentation network model to accurately identify human structural features in the sub-image, thus making the segmentation result of the sub-image more accurate.

[0066] When obtaining all sub-images of the image to be processed, the image can be split according to a preset order with reference to at least the second voxel spacing to obtain all sub-images. If the image to be processed is a 2D image, the preset order can be, but is not limited to, splitting from top to bottom and from left to right; if the image to be processed is a 3D image, the preset order can be, but is not limited to, splitting from top to bottom, from left to right, and from front to back.

[0067] In one possible implementation, obtaining all sub-images of the image to be processed includes: splitting the image to be processed in a preset order, with the second voxel spacing as the voxel spacing of the sub-images. When splitting the image to be processed, there is no overlapping area between adjacent sub-images, or there is partial overlap between adjacent sub-images. If the remaining pixels in the image to be processed cannot form a sub-image when obtaining the last sub-image, pixels adjacent to the remaining pixels can be reused.

[0068] In one possible implementation, for any dimension of the first voxel spacing, the number of segments of the image to be processed in that dimension is determined according to the ratio of the second voxel spacing to the first voxel spacing in that dimension, wherein the number of segments is an integer of the ratio and is greater than or equal to 1; the segmentation grid of the image to be processed is determined according to the ratio of the second voxel spacing to the first voxel spacing in different dimensions; the image to be processed is segmented into multiple sub-images in a preset order using the segmentation grid of the image to be processed, wherein the segmentation grid is used to indicate the size of the sub-images.

[0069] For example, suppose the first voxel spacing is (x,y,z) or (x,y), where (x,y,z) is the 3D voxel spacing and (x,y) is the 2D voxel spacing, and the second voxel spacing is (X,Y,Z) (3D) or (X,Y) (2D). The ratio of the two voxel spacings in each dimension is as follows:

[0070] 3D Scene: Calculate the proportions of each dimension: n1 = X / x, n2 = Y / y, n3 = Z / z. If n1, n2, and n3 are positive integers, then n1 is directly used as the number of partitions in the x-dimensional, n2 as the number of partitions in the y-dimensional, and n3 as the number of partitions in the z-dimensional. If at least one of n1, n2, and n3 is a decimal, then round up, down, or to the nearest integer, and use the resulting integer as the number of partitions. For example, if n1 is a decimal, then round up n1, and use the resulting integer as the number of partitions in the x-dimensional. 2D Scene: Calculate n1 = X / x and n2 = Y / y, and process n1 and n2 as in the 3D scene.

[0071] After obtaining the ratios of the second voxel spacing and the first voxel spacing in different dimensions, the split grid of the image to be processed is the product of these ratios. For example, in a 3D scene, the split grid equals n1×n2×n3, and in a 2D scene, it equals n1×n2. The image to be processed is then split using this split grid in a preset order to obtain sub-images with a size equal to the split grid. Neither adjacent sub-images overlap, nor do they partially overlap.

[0072] Example: In a 3D scene, the image to be processed is a 4×4×4 image. The first voxel spacing is (0.6mm, 0.6mm, 1.2mm), and the second voxel spacing is (1.2mm, 1.2mm, 2.4mm). Then n1=2, n2=2, n3=2, and the split grid is equal to 2×2×2. Using this split grid, the image to be processed can be split in the order of top to bottom, left to right, and front to back, resulting in 8 sub-images of size 2×2×2. There is no overlap between the 8 sub-images.

[0073] The number of pixels in each dimension of the sub-image is half the number of pixels in the image to be processed in that dimension. Therefore, the voxel spacing of the sub-image in that dimension is twice the voxel spacing of the image to be processed in that dimension (equivalent to stretching deformation in that dimension). Thus, the relationship between the third voxel spacing of the sub-image and the first voxel spacing of the image to be processed in one dimension is equal to the ratio of the second voxel spacing to the first voxel spacing in that dimension. This makes the value of the third voxel spacing of the sub-image match the value of the second voxel spacing in that dimension, achieving the goal of matching the third voxel spacing of the sub-image with the second voxel spacing.

[0074] In summary, this embodiment achieves a match between the third voxel spacing of the sub-images and the second voxel spacing of the training images by sequentially splitting the image to be processed with reference to the second voxel spacing.

[0075] S103. Input each sub-image into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model.

[0076] S104. Restore the segmentation results of each sub-image to the image to be processed to obtain the segmentation results of the image to be processed.

[0077] In this embodiment, a sub-image is a small image obtained by sequentially splitting the image to be processed, and the sub-image is the input unit of the image segmentation network model. The image segmentation network model has been trained using training images with labeled segmentation results and a second voxel spacing. After obtaining at least one sub-image of the image to be processed, the sub-image is input into the image segmentation network model to obtain the segmentation result of the sub-image output by the image segmentation network model.

[0078] In some examples, the segmentation result of the sub-image includes: annotation information of each pixel in the sub-image and the index of each pixel. The annotation information is used to indicate whether the pixel is background or human structure. The human structure and background in the sub-image are segmented by the annotation information of each pixel.

[0079] The factor image is a partial image of the image to be processed. The segmentation result of the sub-image cannot be directly used as the segmentation result of the image to be processed. Therefore, after obtaining the segmentation result of the sub-image, it is restored to the image to be processed to obtain the segmentation result of the image to be processed.

[0080] The process of restoring the segmentation results of sub-images to the image to be processed can include: determining the position of each pixel in each sub-image in the image to be processed, and writing the annotation information of each pixel into the position of that pixel in the image to be processed, so as to obtain the segmentation result of the image to be processed.

[0081] In one possible implementation, restoring the segmentation result of a sub-image to the image to be processed is the reverse process of sequential splitting. The restoration process may include: determining the sub-image as the i-th sub-image of the image to be processed, where i is any value in [1, N] and N is the number of all sub-images of the image to be processed; obtaining the index of each pixel in the i-th sub-image in each dimension; multiplying the index of each pixel in the i-th sub-image in each dimension by i to obtain the target index of each pixel in each dimension, where the target index of the pixel in each dimension is used to indicate the position of the pixel in the image to be processed; and writing the annotation information of each pixel into the position of the pixel in the image to be processed to obtain the segmentation result of the image to be processed.

[0082] For example, if the sub-image is the second sub-image of the image to be processed, and the index of a pixel in this sub-image is (10, 20, 30), and the annotation information is an organ (a structure in the human body), the calculation process of the target index in different dimensions is as follows:

[0083] Given x = 10 × 2 = 20, y = 20 × 2 = 40, z = 30 × 2 = 60, the pixel's position in the image to be processed is (20, 40, 60). The annotation information for position (20, 40, 60) is an organ. After the annotation information for each pixel in each sub-image is written to the pixel's position in the image to be processed, the segmentation of the image to be processed is completed, and the segmentation result of the image to be processed is obtained.

[0084] When restoring the segmentation results of each sub-image to the image to be processed, the value of each pixel and the pixel labeling information are not changed, thus avoiding error accumulation and improving the accuracy of the segmentation results.

[0085] As can be seen from the above technical solution, the image segmentation method provided in this embodiment, after acquiring the image to be processed, at least refers to the second voxel spacing to obtain all sub-images of the image to be processed. The value of each pixel in each sub-image is the same as the value of that pixel in the image to be processed. The voxel spacing of each sub-image is the third voxel spacing, which matches the second voxel spacing. Each sub-image is input into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model. The segmentation result of each sub-image is restored to the image to be processed to obtain the segmentation result of the image to be processed. The third voxel spacing of the sub-image matches the second voxel spacing of the training image, enabling the image segmentation network model to accurately identify human structural features in the sub-image, thus improving the accuracy of the segmentation result. Furthermore, the value of any pixel in the sub-image is the value of that pixel in the image to be processed, preserving the detailed information in the image to be processed, allowing the image segmentation network model to utilize the detailed information for recognition and segmentation, further improving the accuracy of the segmentation result.

[0086] In some examples, the images to be processed include T1-weighted MR images and T2-weighted MR images, where the voxel spacing of both the T1-weighted MR images and the T2-weighted MR images is the first voxel spacing. Correspondingly, each sub-image is input into the image segmentation network model. Obtaining the segmentation result of each sub-image output by the image segmentation network model involves simultaneously inputting sub-images with the same position from both the T1-weighted MR images and the T2-weighted MR images into the image segmentation network model, obtaining the segmentation result output by the model. This segmentation result can be used as the segmentation result of the two input sub-images; that is, sub-images with the same position in the T1-weighted MR images and the T2-weighted MR images can share a single segmentation result. After obtaining the segmentation results of all sub-images, the segmentation results of the sub-images are restored to the T1-weighted MR images and the T2-weighted MR images to obtain the segmentation results of the T1-weighted MR images and the T2-weighted MR images, respectively.

[0087] If a sub-image from either a T1-weighted MR image or a T2-weighted MR image is input into an image segmentation network model, the segmentation result will be directly affected if noise or artifacts (such as motion artifacts) are present in the sub-image, and cannot be verified or corrected using other sub-images. However, when two sub-images with the same position from both the T1-weighted MR images are input into the image segmentation network model simultaneously, these two sub-images can be cross-validated. For example, if an artifact exists in a sub-image of the T1-weighted MR image, it can be verified and corrected using a sub-image of the T2-weighted MR image. Therefore, inputting two sub-images with the same position from both the T1-weighted MR images into the image segmentation network model simultaneously can improve the accuracy of the segmentation results.

[0088] In some examples, sub-images in the T1-weighted MR image and the T2-weighted MR image can be input into the image segmentation network model, respectively. The image segmentation network model outputs the segmentation result for each sub-image in the T1-weighted MR image, and the segmentation result for each sub-image in the T2-weighted MR image. The segmentation result of each sub-image in the T1-weighted MR image is then restored to the T1-weighted MR image to obtain the segmentation result of the T1-weighted MR image. Similarly, the segmentation result of each sub-image in the T2-weighted MR image is restored to the T2-weighted MR image to obtain the segmentation result of the T2-weighted MR image.

[0089] After obtaining the segmentation results of T1-weighted MR images and T2-weighted MR images, cross-validation is performed using the bone boundaries in the T1-weighted MR image segmentation results and the soft tissue boundaries in the T2-weighted MR image segmentation results. Cross-validation refers to using the bone boundaries in the T1-weighted MR image segmentation results to correct the bone boundaries in the T2-weighted MR image segmentation results, and using the soft tissue boundaries in the T2-weighted MR image segmentation results to correct the soft tissue boundaries in the T1-weighted MR image segmentation results, so that the bone boundaries are retained only from the T1-weighted MR image segmentation results and the soft tissue boundaries are retained only from the T2-weighted MR image segmentation results.

[0090] If, during cross-validation, the bone boundaries in the segmentation results of the T1-weighted MR image and the soft tissue boundaries in the segmentation results of the T2-weighted MR image overlap, then the segmentation result of one of the T1-weighted MR images is prioritized. The boundaries of the human structures in the overlapping region within that image are used to correct the boundaries of the human structures in the other image. For example, if the segmentation result of the T1-weighted MR image is prioritized, the segmentation result of the T2-weighted MR image in the overlapping region can be corrected using the segmentation result of the T1-weighted MR image in that overlapping region. This can be achieved by using the soft tissue boundaries of the T1-weighted MR image in the overlapping region to correct the soft tissue boundaries of the T2-weighted MR image in the overlapping region, or by using the bone boundaries of the T1-weighted MR image in the overlapping region to correct the soft tissue boundaries of the T2-weighted MR image in the overlapping region.

[0091] If the liver is located in an overlapping area, the correction method includes, but is not limited to, directly subtracting the liver contour from the T2-weighted MR image from the liver contour of the T1-weighted MR image, or directly taking the intersection of the liver contour of the T1-weighted MR image and the liver contour of the T2-weighted MR image, or subtracting the liver contour from the T2-weighted MR image from the bone boundary of the T1-weighted MR image.

[0092] In some examples, if there is overlap between sub-images of the image to be processed, the annotation information of each pixel in the overlapping area can be corrected by, but is not limited to, using a weighted average or mode, and the corrected annotation information is written into the image to be processed.

[0093] The above describes an image segmentation method provided by an embodiment of this application. The following describes the apparatus for performing the above image segmentation method.

[0094] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an image segmentation device provided in an embodiment of this application. The image segmentation device includes: a first acquisition module 10, a second acquisition module 20, a segmentation module 30, and a restoration module 40.

[0095] The first acquisition module 10 is used to acquire the image to be processed. The voxel spacing of the image to be processed is the first voxel spacing, and the voxel spacing of the training images used to train the image segmentation network model is the second voxel spacing. The first voxel spacing is smaller than the second voxel spacing.

[0096] The second acquisition module 20 is used to obtain all sub-images of the image to be processed by referring at least to the second voxel spacing. The value of each pixel in each sub-image is the same as the value of the pixel in the image to be processed. The voxel spacing of each sub-image is the third voxel spacing, and the third voxel spacing matches the second voxel spacing.

[0097] In one possible implementation, the second acquisition module 20 is used to determine the number of segments of the image to be processed in any dimension of the first voxel spacing based on the ratio of the second voxel spacing to the first voxel spacing in that dimension, wherein the number of segments is an integer of the ratio and is greater than or equal to 1; determine the segmentation grid of the image to be processed based on the ratio of the second voxel spacing to the first voxel spacing in different dimensions; and use the segmentation grid of the image to be processed to segment the image to be processed into multiple sub-images in a preset order, wherein the segmentation grid is used to indicate the size of the sub-images, as detailed in the relevant descriptions in the above method embodiments.

[0098] The segmentation module 30 is used to input each sub-image into the image segmentation network model and obtain the segmentation result of each sub-image output by the image segmentation network model.

[0099] The restoration module 40 is used to restore the segmentation result of each sub-image to the image to be processed, so as to obtain the segmentation result of the image to be processed.

[0100] The segmentation results of the sub-images include: the annotation information of each pixel in the sub-image and the index of each pixel. The annotation information is used to indicate whether the pixel is the background or a human body structure. The restoration module 40 is used to determine the position of each pixel in each sub-image in the image to be processed. The annotation information of each pixel is written into the position of the pixel in the image to be processed to obtain the segmentation results of the image to be processed.

[0101] In one possible implementation, the restoration module 40 is used to determine that the sub-image is the i-th sub-image of the image to be processed, where i is any value in [1, N] and N is the number of all sub-images of the image to be processed; obtain the index of each pixel in the i-th sub-image in each dimension; multiply the index of each pixel in the i-th sub-image in each dimension by i respectively to obtain the target index of each pixel in each dimension, the target index of the pixel in each dimension is used to indicate the position of the pixel in the image to be processed, and write the annotation information of each pixel into the position of the pixel in the image to be processed to obtain the segmentation result of the image to be processed.

[0102] In some examples, the images to be processed include a T1-weighted MR image and a T2-weighted MR image, where the voxel spacing of both the T1-weighted MR image and the T2-weighted MR image is the first voxel spacing. Correspondingly, the segmentation module 30 simultaneously inputs sub-images with the same position in both the T1-weighted MR image and the T2-weighted MR image into the image segmentation network model to obtain the segmentation result output by the image segmentation network model. This segmentation result can be used as the segmentation result for the two input sub-images; that is, sub-images with the same position in the T1-weighted MR image and the T2-weighted MR image can share a single segmentation result. After obtaining the segmentation results for all sub-images, the segmentation results of the sub-images are restored to the T1-weighted MR image and the T2-weighted MR image to obtain the segmentation results for the T1-weighted MR image and the T2-weighted MR image, respectively.

[0103] If a sub-image from either a T1-weighted MR image or a T2-weighted MR image is input into an image segmentation network model, the segmentation result will be directly affected if noise or artifacts (such as motion artifacts) are present in the sub-image, and cannot be verified or corrected using other sub-images. However, when two sub-images with the same position from both the T1-weighted MR images are input into the image segmentation network model simultaneously, these two sub-images can be cross-validated. For example, if an artifact exists in a sub-image of the T1-weighted MR image, it can be verified and corrected using a sub-image of the T2-weighted MR image. Therefore, inputting two sub-images with the same position from both the T1-weighted MR images into the image segmentation network model simultaneously can improve the accuracy of the segmentation results.

[0104] In some examples, sub-images in the T1-weighted MR image and the T2-weighted MR image can be input into the image segmentation network model, respectively. The image segmentation network model outputs the segmentation result for each sub-image in the T1-weighted MR image, and the segmentation result for each sub-image in the T2-weighted MR image. The segmentation result of each sub-image in the T1-weighted MR image is then restored to the T1-weighted MR image to obtain the segmentation result of the T1-weighted MR image. Similarly, the segmentation result of each sub-image in the T2-weighted MR image is restored to the T2-weighted MR image to obtain the segmentation result of the T2-weighted MR image.

[0105] After obtaining the segmentation results of the T1-weighted MR image and the T2-weighted MR image, cross-validation is performed using the bone boundaries in the segmentation results of the T1-weighted MR image and the soft tissue boundaries in the segmentation results of the T2-weighted MR image. For details, please refer to the relevant descriptions in the above method embodiments.

[0106] In some examples, if there is overlap between sub-images of the image to be processed, the restoration module 40 can correct the annotation information of each pixel in the overlapping area by means of, but not limited to, weighted average or mode, and the corrected annotation information is written into the image to be processed.

[0107] For a detailed description of each module in the above image segmentation device, please refer to the method section; it will not be repeated here.

[0108] This application also provides an electronic device in its embodiments. (See reference...) Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0109] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 202 or a program loaded from a storage device 208 into a random access memory (RAM) 203. When the electronic device is powered on, the RAM 203 also stores various programs and data required for the operation of the electronic device. The processing unit 201, ROM 202, and RAM 203 are interconnected via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204. The processing unit can be a processor in the electronic device, and the ROM, RAM, and storage device can be memory in the electronic device. The memory is used to store computer programs; the processor is used to execute the computer programs to enable the electronic device to implement any of the image segmentation methods provided in the embodiments of this application.

[0110] Typically, the following devices can be connected to I / O interface 205: input devices 206 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 207 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 208 including, for example, memory cards, hard drives, etc.; and communication devices 209. Communication device 209 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0111] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the image segmentation methods provided in this application.

[0112] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the image segmentation methods provided in this application.

[0113] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0115] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0116] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An image segmentation method, characterized in that, include: The image to be processed is obtained, wherein the voxel spacing of the image to be processed is the first voxel spacing, and the voxel spacing of the training images used to train the image segmentation network model is the second voxel spacing, wherein the first voxel spacing is less than the second voxel spacing. At least with reference to the second voxel spacing, all sub-images of the image to be processed are obtained, the value of each pixel in each sub-image is the same as the value of the pixel in the image to be processed, and the voxel spacing of each sub-image is the third voxel spacing, which matches the second voxel spacing; Each sub-image is input into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model; The segmentation results of each sub-image are restored to the image to be processed to obtain the segmentation results of the image to be processed; The step of obtaining all sub-images of the image to be processed by referring at least to the second voxel spacing includes: For any dimension of the first voxel spacing, the number of segments of the image to be processed in that dimension is determined according to the ratio of the second voxel spacing to the first voxel spacing in that dimension. The number of segments is an integer of the ratio and the number of segments is greater than or equal to 1. The split grid of the image to be processed is determined based on the ratio of the second voxel spacing and the first voxel spacing in different dimensions; Using the splitting grid of the image to be processed, the image to be processed is split into multiple sub-images in a preset order, and the splitting grid is used to indicate the size of the sub-images.

2. The method according to claim 1, characterized in that, The segmentation result of the sub-image includes: annotation information and index of each pixel in the sub-image, wherein the annotation information is used to indicate whether the pixel is background or human structure; the step of restoring the segmentation result of each sub-image to the image to be processed to obtain the segmentation result of the image to be processed includes: Determine the position of each pixel in each sub-image within the image to be processed; The annotation information of each pixel is written into the position of that pixel in the image to be processed to obtain the segmentation result of the image to be processed.

3. The method according to claim 2, characterized in that, Determining the position of each pixel in each sub-image within the image to be processed includes: The sub-image is determined to be the i-th sub-image of the image to be processed, where i is any value in [1, N] and N is the number of all sub-images of the image to be processed; Obtain the index of each pixel in each dimension of the i-th sub-image; The index of each pixel in the i-th sub-image in each dimension is multiplied by i to obtain the target index of each pixel in each dimension. The target index of the pixel in each dimension is used to indicate the position of the pixel in the image to be processed.

4. The method according to claim 1, characterized in that, The image to be processed includes a T1-weighted magnetic resonance image and a T2-weighted magnetic resonance image, wherein the voxel spacing of the T1-weighted magnetic resonance image and the voxel spacing of the T2-weighted magnetic resonance image are both the first voxel spacing. The step of inputting each sub-image into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model includes: simultaneously inputting sub-images with the same position in the T1-weighted magnetic resonance image and the T2-weighted magnetic resonance image into the image segmentation network model to obtain the segmentation result output by the image segmentation network model.

5. An image segmentation apparatus, characterized in that, include: The first acquisition module is used to acquire the image to be processed, wherein the voxel spacing of the image to be processed is the first voxel spacing, and the voxel spacing of the training image used to train the image segmentation network model is the second voxel spacing, wherein the first voxel spacing is smaller than the second voxel spacing. The second acquisition module is used to obtain all sub-images of the image to be processed by referring at least to the second voxel spacing, wherein the value of each pixel in each sub-image is the same as the value of the pixel in the image to be processed, and the voxel spacing of each sub-image is the third voxel spacing, wherein the third voxel spacing matches the second voxel spacing. The segmentation module is used to input each sub-image into the image segmentation network model to obtain the segmentation result of each sub-image output by the image segmentation network model; The restoration module is used to restore the segmentation result of each sub-image to the image to be processed, so as to obtain the segmentation result of the image to be processed; The second acquisition module is used to determine the number of segments of the image to be processed in any dimension of the first voxel spacing based on the ratio of the second voxel spacing to the first voxel spacing in that dimension, wherein the number of segments is an integer of the ratio and the number of segments is greater than or equal to 1; and to determine the segmentation grid of the image to be processed based on the ratio of the second voxel spacing to the first voxel spacing in different dimensions. Using the splitting grid of the image to be processed, the image to be processed is split into multiple sub-images in a preset order, and the splitting grid is used to indicate the size of the sub-images.

6. The apparatus according to claim 5, characterized in that, The segmentation result of the sub-image includes: the annotation information of each pixel in the sub-image and the subscript of each pixel, wherein the annotation information is used to indicate whether the pixel is background or human body structure; the restoration module is used to determine the position of each pixel in each sub-image in the image to be processed; and the annotation information of each pixel is written into the position of the pixel in the image to be processed to obtain the segmentation result of the image to be processed.

7. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the image segmentation method as described in any one of claims 1 to 4.

8. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the image segmentation method as described in any one of claims 1 to 4.

9. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the image segmentation method as described in any one of claims 1 to 4.

Images