Image processing method and device

Abstract

The invention relates to the field of image processing, in particular to an image processing method and device. The method comprises the steps of obtaining a first image; the first image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on the head of the human body. Obtaining n image sets according to the first image; wherein each image set comprises a two-dimensional image obtained by slicing the first image according to the corresponding direction. Respectively inputting the n image sets into n segmentation models to obtain a segmentation result set; wherein the n segmentation models are respectively used for outputting a result image for indicating an area corresponding to a skull part in a two-dimensional image after the two-dimensional image is input; the segmentation result set comprises result images output by the n segmentation models. Determining a three-dimensional segmentation result according to the segmentation result set; wherein the three-dimensional segmentation result is used for indicating an image obtained after the skull part of the first image is stripped.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method and device. Background Art

[0002] Currently, Magnetic Resonance Imaging (MRI) is being used more and more widely in medical diagnosis.

[0003] In order to facilitate diagnosis and analysis, it is often necessary to peel off the skull from the MRI image of the human head, so as to use the MRI image after removing the skull to perform brain morphology measurement, brain volume measurement, cortical surface reconstruction and other analyses.

[0004] Therefore, how to efficiently and accurately obtain images after stripping the skull from MRI images of the human head is a technical problem that needs to be solved at present. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides an image processing method and device.

[0006] In a first aspect, the present disclosure provides an image processing method, the method comprising: acquiring a first image; the first image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head. Based on the first image, n image sets are obtained; wherein the n image sets correspond one-to-one to n directions, and each image set includes: a two-dimensional image obtained by slicing the first image in the corresponding direction, and n is a positive integer greater than 1. The n image sets are respectively input into n segmentation models to obtain a segmentation result set; wherein the n segmentation models correspond one-to-one to the n image sets, and the n segmentation models are respectively used to output a result image indicating the area corresponding to the skull part in the two-dimensional image after inputting the two-dimensional image; the segmentation result set includes the result images output by the n segmentation models. Based on the segmentation result set, a three-dimensional segmentation result is determined; wherein the three-dimensional segmentation result is used to indicate the image after the skull part is stripped from the first image.

[0007] In the above method, the first image (i.e., the three-dimensional image obtained by MRI of the human head) is sliced ​​in n directions to obtain n image sets, and then the skull parts in the two-dimensional images in the n image sets are detected respectively to obtain a segmentation result set (including the result image indicating the area corresponding to the skull part in each two-dimensional image). Then, according to the segmentation result set, the image after the skull part is stripped from the first image is determined. In this way, on the one hand, the method does not need to rely on a high-precision segmentation model based on three-dimensional images, but mainly uses a segmentation model based on two-dimensional images, so the method can greatly reduce the data volume of the training data set and reduce the computing power overhead; on the other hand, compared with detecting the skull part based on a single-direction two-dimensional image, the above method of the present disclosure can use two-dimensional images in multiple directions as the detection basis, so it can fully utilize the high-resolution two-dimensional image in one direction, and can also fully explore the spatial information contained in the two-dimensional images in other directions (for example, the two-dimensional image in the horizontal plane direction generally has a relatively higher resolution, so the horizontal plane direction can be used as one of the n directions. In this way, the high-resolution information of the two-dimensional image in the horizontal plane direction can be fully utilized, and the spatial information contained in other directions can be fully explored).

[0008] In one implementation, determining a three-dimensional segmentation result according to the segmentation result set includes: determining a detection result corresponding to the target voxel according to a plurality of result images corresponding to the target voxel in the segmentation result set, the detection result being used to indicate whether the target voxel corresponds to a skull portion. Determining a three-dimensional segmentation result according to the detection result corresponding to the target voxel.

[0009] In one implementation, each of the n image sets further includes: a two-dimensional image obtained by fusing a plurality of two-dimensional images obtained by slicing the first image.

[0010] In the above implementation, the two-dimensional image obtained by fusing a plurality of two-dimensional images obtained by slicing the first image is included in the image set, thereby improving the smoothness of the segmentation result.

[0011] In one implementation, the two-dimensional images included in the n image sets are gradient images.

[0012] In this implementation, it is considered that: on the one hand, because the gradient map is an image obtained by deriving the original image, the gradient map can be used to represent the degree of change in image brightness. On the other hand, in the MRI image of the human head, the skull part belongs to the highlighted part of the image, and the brain tissue part is relatively low in brightness. Therefore, by using the gradient map as the input data of the segmentation model, the accuracy of the segmentation model can be further improved.

[0013] In one implementation, the n segmentation models are deep learning models constructed using an attention mechanism.

[0014] In the above implementation, it is taken into consideration that the attention mechanism can suppress irrelevant areas and highlight the significant features of local areas. Therefore, by utilizing the attention mechanism to construct the above n segmentation models, the accuracy of the segmentation edge of the area corresponding to the skull part in the result image can be improved.

[0015] In one implementation, the n directions include: a coronal plane direction, a sagittal plane direction, and a horizontal plane direction.

[0016] In one implementation, the method further includes: acquiring a second image; the second image is a three-dimensional image obtained by performing MRI on a human head; inputting the second image into a target segmentation model to obtain a target result image; the target segmentation model is used to output a result image indicating the area corresponding to the brain in the three-dimensional image after inputting the three-dimensional image; the target result image is used to indicate the area corresponding to the brain in the second image; and determining the first image from the second image according to the target result image.

[0017] In the above implementation, it is considered that the three-dimensional image acquired by MRI can be roughly segmented first, thereby excluding irrelevant areas. Then, the image excluding irrelevant areas is used to determine the three-dimensional segmentation result according to the method provided by the above disclosure. In this way, the influence of irrelevant areas on the three-dimensional segmentation result can be reduced, and then the target small area can be better focused for accurate segmentation.

[0018] In one implementation, based on a target result image, a first image including an area corresponding to the brain is segmented from a second image, including: expanding the area corresponding to the brain indicated by the target result image outward by a preset margin to obtain an extended image; wherein the extended image is used to indicate a target area, and the target area includes: an area corresponding to the brain indicated in the target result image and an area corresponding to the preset margin; based on the extended image, the first image is determined from the second image.

[0019] Through the above implementation, it is possible to avoid the situation in which the target result image may identify the brain corresponding area as the skull corresponding area, resulting in the first image not including the entire brain corresponding area.

[0020] In one implementation, the method further includes: acquiring a third image; the third image is a three-dimensional image obtained by performing MRI on a human head; preprocessing the third image to obtain a second image; the preprocessing includes at least one of the following: bias field correction and pixel value normalization.

[0021] In a second aspect, an image processing device is provided, comprising: an acquisition unit, for acquiring a first image; the first image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head. A processing unit, for obtaining n image sets based on the first image; wherein the n image sets correspond one-to-one to n directions, and each image set includes: a two-dimensional image obtained by slicing the first image in the corresponding direction, and n is a positive integer greater than 1. The processing unit is also used to input the n image sets into n segmentation models, respectively, to obtain a segmentation result set; wherein the n segmentation models correspond one-to-one to the n image sets, and the n segmentation models are respectively used to output a result image indicating the area corresponding to the skull part in the two-dimensional image after inputting the two-dimensional image; the segmentation result set includes the result images output by the n segmentation models. The processing unit is also used to determine a three-dimensional segmentation result based on the segmentation result set; wherein the three-dimensional segmentation result is used to indicate the image after the skull part is stripped from the first image.

[0022] In one implementation, the processing unit is further used to determine the three-dimensional segmentation result according to the segmentation result set, including: the processing unit is further used to determine the detection result corresponding to the target voxel according to the multiple result images corresponding to the target voxel in the segmentation result set, the detection result is used to indicate whether the target voxel corresponds to the skull part. The processing unit is further used to determine the three-dimensional segmentation result according to the detection result corresponding to the target voxel.

[0023] In one implementation, each of the n image sets further includes: a two-dimensional image obtained by fusing a plurality of two-dimensional images obtained by slicing the first image.

[0024] In one implementation, the two-dimensional images included in the n image sets are gradient images.

[0025] In one implementation, the n segmentation models are deep learning models constructed using an attention mechanism.

[0026] In one implementation, the n directions include: a coronal plane direction, a sagittal plane direction, and a horizontal plane direction.

[0027] In one implementation, the acquisition unit is also used to acquire a second image; the second image is a three-dimensional image obtained by performing MRI on a human head; the processing unit is also used to input the second image into a target segmentation model to obtain a target result image; the target segmentation model is used to output a result image indicating the area corresponding to the brain in the three-dimensional image after inputting the three-dimensional image; the target result image is used to indicate the area corresponding to the brain in the second image; the processing unit is also used to determine the first image from the second image based on the target result image.

[0028] In one implementation, the processing unit is further configured to segment, according to the target result image, the first image including the area corresponding to the brain from the second image, including:

[0029] The processing unit is further used to expand the area corresponding to the brain indicated by the target result image outward by a preset margin to obtain an extended image; wherein the extended image is used to indicate the target area, and the target area includes: the area corresponding to the brain indicated in the target result image and the area corresponding to the preset margin. The processing unit is further used to determine the first image from the second image according to the extended image.

[0030] In one implementation, the acquisition unit is further used to acquire a third image; the third image is a three-dimensional image obtained by performing MRI on a human head. The processing unit is further used to preprocess the third image to obtain the second image; the preprocessing includes at least one of the following: bias field correction and pixel value normalization.

[0031] In a third aspect, an image processing device is provided, including a processor and an interface, wherein the processor receives or sends data through the interface, and the processor is used to implement the method as in the first aspect or any implementation manner of the first aspect.

[0032] According to a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a processor, the method according to the first aspect or any implementation method of the first aspect is implemented.

[0033] According to a fifth aspect, a computer program product is provided. The computer program product includes instructions. When the instructions are executed on a processor, the method according to the first aspect or any implementation method of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 A schematic diagram of the structure of an MRI system provided in an embodiment of the present disclosure;

[0037] Figure 2 One of the flowcharts of an image processing method provided by an embodiment of the present disclosure;

[0038] Figure 3 A schematic diagram of generating an enhanced image provided by an embodiment of the present disclosure;

[0039] Figure 4 A schematic diagram of using a segmentation model to determine a region corresponding to a skull portion in a two-dimensional image provided by an embodiment of the present disclosure;

[0040] Figure 5 A second flowchart of an image processing method provided by an embodiment of the present disclosure;

[0041] Figure 6 A schematic diagram of the structure of a 3Dunet network provided in an embodiment of the present disclosure;

[0042] Figure 7 A third flowchart of an image processing method provided by an embodiment of the present disclosure;

[0043] Figure 8 A fourth flowchart of an image processing method provided by an embodiment of the present disclosure;

[0044] Fig. 9 A fifth flowchart of an image processing method provided by an embodiment of the present disclosure;

[0045] Fig.10 One of the structural schematic diagrams of an image processing device provided in an embodiment of the present disclosure;

[0046] Fig.11 The second structural schematic diagram of an image processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0049] The specific implementation process of the technical solution provided by the present disclosure is introduced below with reference to examples:

[0050] Skull stripping is an important data processing step for brain MRI analysis, and accurate skull stripping helps with subsequent diagnostic analysis. For example, brain morphology measurement, brain volume measurement, and cortical surface reconstruction all require images after the skull is removed.

[0051] The quality of skull stripping may be affected by many factors. For example, the presence of imaging artifacts, the type of imaging equipment used, and the different acquisition protocols may all affect the quality of skull stripping. In addition, the variability of anatomical structures, age, and brain atrophy may also affect the quality of skull stripping.

[0052] At present, traditional skull stripping is generally performed by professional doctors with rich experience in manual outlining and segmentation, which requires a lot of time and effort, and the segmentation results are overly dependent on subjective factors such as the doctor's experience.

[0053] In view of the above situation, some skull stripping methods based on computer image processing technology are provided in the related art. However, due to the complexity of the image in the actual processing process (for example, the image may have unclear skull boundaries, low image quality contrast, etc.) and the lack of image intensity standardization, it is difficult for the related technology to efficiently and accurately obtain the image after the skull is stripped from the MRI image of the human head.

[0054] For example, in a related technology, a method for stripping the skull is provided in the Brain Extraction Tool (BET) of the Oxford University Brain Functional Magnetic Resonance Imaging Center Software Library (FMRIB's Software Library, FSL), which includes the following: First, a deformable network model is used to fit the surface shape that adapts to the brain tissue structure. Among them, the model is initialized as a spherical grid around the center of gravity of the brain. Then, the model can be locally adaptively "inflated" to the brain boundary like a balloon, so as to finally obtain the regional range where the brain tissue is located.

[0055] Although the above method can quickly obtain MRI images of the skull-peeled part, it is easily affected by the accuracy of the initial center of gravity position estimation, especially in areas where the grayscale values ​​of brain tissue and adjacent tissues are similar, which are easily missing or over-segmented during extraction. In other words, the above method is greatly affected by image quality and has poor robustness.

[0056] In another related technology, a method for skull peeling based on deep learning is provided. For example, a deep iterative fusion learning model for skull segmentation can be constructed in the method. The model adopts an encoder and decoder structure, with short jump connections added between them to enhance the fusion of features between different layers. Among them, a residual convolution module is added to the encoder path to optimize the model's feature learning ability; a dual-path upsampling module is added to the decoder to restore more detailed features; the jump connection layer in the middle is formed by multiple upsampling iterative fusions, which speeds up the input of shallow semantic information into the deep network, thereby improving segmentation accuracy.

[0057] Although the above method can obtain MRI images of the peeled skull part more accurately, it requires the construction of a high-precision segmentation model based on three-dimensional images to perform skull peeling, which requires a large number of three-dimensional training data sets and a lot of computing power for training the model.

[0058] In view of the above situation, the present disclosure provides an image processing method, which takes into account that when it is necessary to peel off the skull part from an MRI image (i.e., a three-dimensional image obtained by performing MRI on a human head, hereinafter referred to as the "first image" for the sake of distinction), the first image can be firstly segmented into an image set of two-dimensional images according to different directions. For example, the first image can be sliced ​​according to the coronal plane direction to obtain multiple two-dimensional images, that is, to obtain an image set corresponding to the coronal plane direction; in addition, the first image can be sliced ​​according to the sagittal plane direction to obtain an image set corresponding to the sagittal plane direction, and so on.

[0059] Then, different segmentation models are used to process the image sets in different directions, respectively, to obtain result images of the area corresponding to the skull part in each two-dimensional image (hereinafter, the set of result images is referred to as the segmentation result set). For example, segmentation model a is used to process the two-dimensional images in the image set corresponding to the coronal plane direction, to obtain result images indicating the area corresponding to the skull part in each two-dimensional image; segmentation model b is used to process the two-dimensional images in the image set corresponding to the sagittal plane direction, to obtain result images of the area corresponding to the skull part in each two-dimensional image, and so on.

[0060] Then, the method determines the image after the skull part is stripped from the first image based on the above segmentation result set. For example, the region corresponding to the skull part in the three-dimensional image (i.e., the first image) can be determined by fusing the detection results of different directions in the segmentation result set, thereby determining the image after the skull part is stripped from the first image.

[0061] It can be seen that in the method provided by the present disclosure, a set of two-dimensional images is obtained by first slicing the first image (i.e., a three-dimensional image obtained by MRI of the human head), and then the skull parts in the two-dimensional images are detected respectively to obtain a set of segmentation results (including result images indicating the area corresponding to the skull parts in each two-dimensional image). Then, according to the set of segmentation results, the image after the skull part is stripped from the first image is determined. In this way, on the one hand, the method does not need to rely on a high-precision segmentation model based on three-dimensional images, but mainly uses a segmentation model of two-dimensional images, so the method can greatly reduce the amount of data in the training data set and reduce computing power overhead; on the other hand, compared with detecting the skull part based on a single-direction two-dimensional image, the above method of the present disclosure can use two-dimensional images in multiple directions as a detection basis, so it can fully utilize a high-resolution two-dimensional image in one direction, and can also fully explore the spatial information contained in the two-dimensional images in other directions (for example, the two-dimensional image in the horizontal plane direction generally has a relatively higher resolution, so the horizontal plane direction can be used as one of the n directions. In this way, the high-resolution information of the two-dimensional image in the horizontal plane direction can be fully utilized, and the spatial information contained in other directions can be fully explored).

[0062] The specific implementation process of the method provided by the present disclosure is introduced below with reference to examples. Figure 1 FIG. 1 is a schematic diagram of the structure of an MRI system provided by an embodiment of the present disclosure. The system 10 may include: an MRI device 11 and an image processing device 12 , and the system 10 may also include a display 13 .

[0063] The MRI device 11 is used to detect the monitored human body 115 to obtain an MRI image. The MRI device 11 may include: a static magnetic field generating magnet 111, a gradient magnetic field coil 112, a radio frequency (RF) coil 113 for transmission, a RF coil 114 for reception, a gradient magnetic field power supply 116, a transmitter 117 for driving the gradient magnetic field coil 112 and the high frequency coil 113, a receiver 118 for receiving nuclear magnetic resonance signals, and a sequencer 119 for making the gradient magnetic field power supply 112, the transmitter 117 and the receiver 118 operate according to a prescribed pulse sequence. In addition, the MRI device 11 may also include: a processor 110 for controlling each functional element.

[0064] The image processing device 12 is used to obtain an MRI image from the MRI device 11 and process the MRI image according to the method provided in the present disclosure, so as to obtain an image after the skull portion is stripped.

[0065] The display 13 is used to display images obtained by the MRI device 11 and the image processing device 12 .

[0066] Understandably, Figure 1 The schematic diagram of a possible structure of the MRI system is only exemplarily introduced. In actual application, the MRI system may also adopt other structural forms. For example, the image processing device 12 may be built into the MRI device 11. In this case, the function of the image processing device 12 may be implemented by software and / or hardware in the MRI device 11. The present disclosure may not limit the product form of the image processing device 12.

[0067] The following takes the operation process of the image processing device 12 as an example to introduce the image processing method provided by the embodiment of the present disclosure. Figure 2 As shown, the method may include:

[0068] S201: The image processing device 12 acquires a first image.

[0069] The first image is a three-dimensional image obtained by performing MRI on the human head.

[0070] For example, Figure 1 In the embodiment, the first image may be an MRI image acquired from the MRI device 11. In addition, the first image may also be a three-dimensional image pre-processed from the MRI image acquired by the MRI device 11. The specific implementation process of acquiring the first image will be described in detail below and will not be expanded here.

[0071] S202: The image processing device 12 obtains a set of n images based on the first image.

[0072] Wherein, n is a positive integer greater than 1. The n image sets correspond to the n directions one by one, and each image set includes: a two-dimensional image obtained by slicing the first image according to the corresponding direction.

[0073] For example, take the three directions including the coronal plane direction, the sagittal plane direction and the horizontal plane direction as an example: on the one hand, the first image can be sliced ​​according to the coronal plane direction to obtain multiple two-dimensional images, and the image set corresponding to the coronal plane direction (referred to as image set G1) can include the above multiple two-dimensional images; on the other hand, the first image can be sliced ​​according to the sagittal plane direction to obtain multiple two-dimensional images, and the image set corresponding to the sagittal plane direction (referred to as image set G2) can include the above multiple two-dimensional images; on the other hand, the first image can be sliced ​​according to the horizontal plane direction to obtain multiple two-dimensional images, and the image set corresponding to the horizontal plane direction (referred to as image set G3) can include the above multiple two-dimensional images. In this way, three image sets corresponding to the three directions can be obtained.

[0074] It is understandable that the above examples use the coronal plane direction, sagittal plane direction and horizontal plane direction as examples of n directions for illustration. In actual application, more or fewer directions other than the above three directions may be used to slice the first image, thereby obtaining a corresponding set of n images. The specific directions included in the n directions may not be limited in the embodiments of the present disclosure.

[0075] In one implementation, each of the n image sets mentioned above further includes: a two-dimensional image obtained by fusing multiple two-dimensional images obtained by slicing the first image (for ease of distinction, the two-dimensional image obtained by fusion is referred to as an enhanced image hereinafter).

[0076] Among them, in a possible design, the two-dimensional image obtained by fusing multiple two-dimensional images obtained by slicing the first image can specifically include: an enhanced image obtained by weighted averaging pixel values ​​at corresponding positions in the multiple two-dimensional images obtained by slicing the first image.

[0077] For example, take the image set G1 corresponding to the coronal plane as an example, where the image set G1 includes x two-dimensional images obtained by slicing the first image: image I 1 、Image I 2 、Image I 3 , ... Image I x-2 、Image I x-1 And image I x In addition, k consecutive images in the x two-dimensional images may be fused to obtain an enhanced image, and then the enhanced image may be added to the above image set.

[0078] For example, taking k as 3 as an example, the enhanced image can be obtained by referring to the following formula (1):

[0079]

[0080] Among them, I n-1 ,I n and I n+1 Respectively represent the images of the n-1th layer, nth layer and n+1th layer in x two-dimensional images, I gn represents the enhanced image corresponding to the nth layer, where 0<τ<1.

[0081] For example, Figure 3 As shown, referring to formula (1), image I n-1 、Image I n and image I n+1 After fusion, the image I gn like Figure 3 As shown in (d).

[0082] It can be understood that when formula (1) is used to generate enhanced images for each of k consecutive two-dimensional images in an image set, the x two-dimensional images obtained by slicing the first image can be expanded to 2x two-dimensional images.

[0083] S203 . The image processing device 12 inputs the n image sets into n segmentation models respectively to obtain a segmentation result set.

[0084] The n segmentation models correspond to the n image sets one by one. The n segmentation models are respectively used to output result images indicating the area corresponding to the skull part in the two-dimensional image after inputting the two-dimensional image. The segmentation result set includes the result images output by the n segmentation models.

[0085] In one implementation, the result image is specifically a mask image used to indicate the area corresponding to the skull part in the two-dimensional image.

[0086] For example, if n=3, and the three directions are respectively the coronal plane direction, the sagittal plane direction and the horizontal plane direction, and the coronal plane direction, the sagittal plane direction and the horizontal plane direction correspond to the image set G1, the image set G2 and the image set G3 respectively, the two-dimensional images in the image set G1 can be input into the segmentation model N1, the two-dimensional images in the image set G2 can be input into the segmentation model N2, and the two-dimensional images in the image set G3 can be input into the segmentation model N3. Thus, each result image in the segmentation result set is obtained.

[0087] It can be understood that, since each image set may include a two-dimensional image obtained by slicing the first image in the corresponding direction, and may also include a two-dimensional image obtained by fusing multiple two-dimensional images obtained by slicing the first image, the result image in the segmentation result set may include a result image corresponding to the two-dimensional image obtained by slicing the first image, and may also include a result image corresponding to the two-dimensional image obtained by the fusion.

[0088] For example, Figure 4 As shown, the two-dimensional image I in the image set G1 mc After inputting the segmentation model N1, the result image I is obtained mc_p ; In the enhanced image I in the image set G1 gmc After inputting the segmentation model N1, the result image I is obtained gmc_p ; And so on, we get the result image I ms_p ,I gms_p ,I mt_p and I gmt_p Among them, the result image I mc_p ,Igmc_p ,I ms_p ,I gms_p ,I mt_p and I gmt_p are respectively included in the segmentation result set.

[0089] In one implementation, the n segmentation models are respectively deep learning models constructed using an attention mechanism.

[0090] For example, the n segmentation models mentioned above may be an Attention-Unet network, wherein each segmentation model includes an encoder plus a decoder structure, wherein the decoder includes a self-attention (soft-attention) module.

[0091] In the above implementation, it is taken into consideration that the attention mechanism can suppress irrelevant areas and highlight the significant features of local areas. Therefore, by utilizing the attention mechanism to construct the above n segmentation models, the accuracy of the segmentation edge of the area corresponding to the skull part in the result image can be improved.

[0092] The training process of the n segmentation models will be described in detail below and will not be described in detail here.

[0093] In addition, in one implementation, the two-dimensional images included in the above n image sets are gradient images.

[0094] In this implementation, it is considered that: on the one hand, because the gradient map represents the rate of change of the original image pixel along the coordinate axis (compared with the adjacent pixels), the gradient map can be used to represent the degree of change of the image brightness. On the other hand, in the MRI image of the human head, the skull part belongs to the highlight part of the image, and the brain tissue part is relatively low in brightness. Therefore, by using the gradient map as the input data of the segmentation model, the accuracy of the segmentation model can be further improved.

[0095] S204: The image processing device 12 determines a three-dimensional segmentation result according to the segmentation result set.

[0096] The three-dimensional segmentation result is used to indicate an image obtained by stripping the skull portion of the first image.

[0097] In one implementation, S204 may specifically include:

[0098] S2041 . The image processing device 12 determines the detection result corresponding to the target voxel according to a plurality of result images corresponding to the target voxel in the segmentation result set.

[0099] The detection result is used to indicate whether the target voxel corresponds to the skull part.

[0100] For example, the multiple result images corresponding to the target voxel in the segmentation result set may specifically include: result images corresponding to the target voxel in the result images in different directions in the segmentation result set.

[0101] Taking the coronal plane direction, sagittal plane direction and horizontal plane direction as examples, the multiple result images corresponding to the target voxel in the segmentation result set may specifically include: the result image corresponding to the target voxel in the result image in the coronal plane direction in the segmentation result set, the result image corresponding to the target voxel in the result image in the sagittal plane direction in the segmentation result set, and the result image corresponding to the target voxel in the result image in the horizontal plane direction in the segmentation result set.

[0102] Furthermore, when the result images in the segmentation result set include both the result images corresponding to the two-dimensional images obtained by slicing the first image and the result images corresponding to the two-dimensional images obtained by the fusion, the target voxel can correspond to 6 result images.

[0103] After determining a plurality of result images corresponding to the target voxel, the final segmentation result corresponding to the target voxel may be determined through a preset strategy.

[0104] For example, when the result image is a mask image indicating an area corresponding to a skull portion in a two-dimensional image, the median of pixel values ​​of a target voxel in multiple mask images may be determined first, and then the segmentation result corresponding to the target voxel may be determined based on the median.

[0105] For another example, when the result image is a mask image, the pixel values ​​of the target voxel in multiple result images may be weighted averaged to obtain the numerical value of the target voxel, and then the segmentation result corresponding to the target voxel may be determined according to the numerical value.

[0106] S2042: The image processing device 12 determines a three-dimensional segmentation result according to the segmentation result corresponding to each target voxel.

[0107] Specifically, after determining the detection results corresponding to each voxel in the three-dimensional image according to the above process of determining the detection results corresponding to the target voxel, the image after the skull part is stripped off the first image can be determined according to the detection results corresponding to each voxel (i.e., the three-dimensional segmentation result).

[0108] In addition, in one implementation, before determining the three-dimensional segmentation result according to the method shown in S201-S204 above, the three-dimensional image obtained by MRI can be roughly segmented to exclude irrelevant areas. Then, the image excluding irrelevant areas is used to determine the three-dimensional segmentation result according to the method shown in S201-S204 above. In this way, the influence of irrelevant areas on the three-dimensional segmentation result can be reduced, and the target small area can be better focused for accurate segmentation. Therefore, Figure 5 As shown, before executing the above steps S201-S204, the method may further include:

[0109] S205, the image processing device 12 obtains a second image.

[0110] The second image is a three-dimensional image obtained by performing MRI on the human head.

[0111] For example, the second image may be an MRI image acquired from the MRI device 11. In addition, the second image may also be a three-dimensional image obtained by pre-processing the MRI image acquired by the MRI device 11. The specific implementation process of acquiring the second image will be described in detail below and will not be expanded here.

[0112] S206: The image processing device 12 inputs the second image into the target segmentation model to obtain a target result image.

[0113] The target result image is used to indicate the area corresponding to the brain in the second image. For example, the target result image may be a mask image used to indicate the area corresponding to the brain in the second image.

[0114] The target segmentation model is used to output a result image indicating the area corresponding to the brain in the three-dimensional image after inputting the three-dimensional image. For example, the target segmentation model can be as follows: Figure 6 The 3D unet network with a simplified network structure shown uses the first three encoder layers in the 3D unet network for feature extraction, and then connects to the corresponding decoder layers to achieve preliminary segmentation of the target area.

[0115] The training process of the target segmentation model will be described in detail below and will not be described in detail here.

[0116] S207 . The image processing device 12 determines the first image from the second image according to the target result image.

[0117] For example, when the target result image is a mask image, the first image may be obtained by segmenting the region corresponding to the mask in the target result image from the second image.

[0118] In a possible design, in order to avoid the situation in which the target result image may identify the brain corresponding region as the skull corresponding region, thereby causing the first image not to include the entire brain corresponding region, S207 may specifically include:

[0119] S2071. The image processing device 12 expands the area corresponding to the brain indicated by the target result image outward by a preset margin to obtain an expanded image.

[0120] The extended image is used to indicate the target area, and the target area includes: the area corresponding to the brain indicated in the target result image and the area corresponding to the preset margin.

[0121] For example, the extended image may be a mask image. The image processing device 12 may expand the area corresponding to the brain indicated by the target result image outward by p pixels to form a mask area of ​​the extended image (wherein the mask area may be understood as the target area).

[0122] S2072: The image processing device 12 determines the first image from the second image according to the extended image.

[0123] For example, the image processing device 12 may obtain the first image by segmenting the target area indicated by the extended image from the second image.

[0124] In addition, in one implementation, before determining the first image according to the method shown in S205-S207, the three-dimensional image obtained by MRI may be preprocessed to reduce the impact of image quality on the final result. Figure 7 As shown, the method may also include:

[0125] S208: The image processing device 12 acquires a third image.

[0126] The third image is a three-dimensional image obtained by performing MRI on the human head.

[0127] For example, the third image may be an MRI image acquired from the MRI device 11 .

[0128] S209: The image processing device 12 pre-processes the third image to obtain a second image.

[0129] Wherein, the preprocessing includes at least one of the following: bias field correction and pixel value normalization.

[0130] Specifically, due to factors such as the scanner itself and the deviation during the scanning process, the obtained MRI image may appear brighter or darker in part when visualized, which is the bias field effect. In the present disclosure, the influence of the bias field effect can be reduced or eliminated through bias field correction.

[0131] In addition, the image processing device 12 may refer to the following formula (2) to normalize the pixel values ​​of the image to obtain a second image:

[0132]

[0133] Among them, x 0 Represents the normalized image pixel value, x iRepresents the image pixel value of the original image, x min Represents the minimum pixel value in the original image, x max Indicates the maximum pixel value in the original image.

[0134] The following is an introduction to the training process of the target segmentation model used in S206. Figure 8 As shown, the method may also include:

[0135] S301 : The image processing device 12 obtains a first training data set.

[0136] The first training data set includes: an original three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head.

[0137] In one implementation, the first training data set also includes: a three-dimensional image obtained by performing a preset operation on the original three-dimensional image, wherein the preset operation includes: one or more of random translation, scaling, rotation, gamma change, and nonlinear deformation transformation.

[0138] In addition, the preset operation may also include: based on the affine transformation, randomly offsetting the voxel points in the original three-dimensional image by (Δx, Δy, Δz), thereby obtaining a random displacement field, where Δx, Δy, Δz∈(-1,1). Then, based on the assumption that the pixel distribution in the voxel conforms to the Gaussian distribution, the image affected by the random offset field is calculated.

[0139] In addition, the preset operation may also include: performing random cropping and downsampling processing on the original three-dimensional image data.

[0140] S302. The image processing device 12 trains the deep learning model according to the first training data set to obtain the above-mentioned target segmentation model.

[0141] Specifically, the image processing device 12 can use the images in the above-mentioned first training data set and the corresponding areas of the skull part to perform supervised learning training on the deep learning model, so as to obtain a target segmentation model.

[0142] Exemplarily, the deep learning model can be Figure 6 3DUnet network shown.

[0143] The following is an introduction to the training process of the n segmentation models used in S203. Specifically, the training process of one model (hereinafter referred to as the first segmentation model) among the n segmentation models is taken as an example. Fig. 9 As shown, the method may also include:

[0144] S401: The image processing device 12 obtains a second training data set.

[0145] The second training data set includes: a first image set. The first image set includes: a two-dimensional image obtained by slicing a sample image according to a target direction. The sample image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head.

[0146] The target direction is the direction corresponding to the first segmentation model. For example, when the first segmentation model is used to process a two-dimensional image in the coronal plane direction, the target direction is the coronal plane direction.

[0147] In addition, the second training data set also includes: a second image set. The second image set includes: a two-dimensional image obtained by fusing a plurality of two-dimensional images obtained by slicing the sample image.

[0148] In addition, the second training data set also includes: mask images corresponding to the first image set and the second image set. The mask images are used to indicate the area corresponding to the skull part in the two-dimensional image.

[0149] S402. The image processing device 12 trains the deep learning model according to the second training data set to obtain a first segmentation model.

[0150] Specifically, the image processing device 12 can use the images in the above-mentioned second training data set and the corresponding prediction results to perform supervised learning training on the deep learning model, so as to obtain the first segmentation model.

[0151] It can be understood that the training process of each segmentation model in the n segmentation models can refer to the above-mentioned process S401-S402.

[0152] Based on the above method embodiment, the device provided by the embodiment of the present disclosure is described below. Fig.10 FIG. 5 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Specifically, the image processing device 50 may be a chip or a system on a chip. The image processing device 50 may be used to implement Figure 1-Figure 9 Specifically, the image processing device 50 may include:

[0153] The acquisition unit 501 is used to acquire a first image; the first image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head.

[0154] The processing unit 502 is used to obtain n image sets according to the first image; wherein the n image sets correspond one-to-one to n directions, and each image set includes: a two-dimensional image obtained by slicing the first image according to the corresponding direction, and n is a positive integer greater than 1.

[0155] The processing unit 502 is also used to input the n image sets into n segmentation models respectively to obtain a segmentation result set; wherein the n segmentation models correspond one-to-one to the n image sets, and the n segmentation models are used to output result images indicating the area corresponding to the skull part in the two-dimensional image after inputting the two-dimensional image; the segmentation result set includes the result images output by the n segmentation models.

[0156] The processing unit 502 is further used to determine a three-dimensional segmentation result according to the segmentation result set; wherein the three-dimensional segmentation result is used to indicate an image after the skull portion is stripped from the first image.

[0157] In one implementation, the processing unit 502 is further configured to determine the three-dimensional segmentation result according to the segmentation result set, including:

[0158] The processing unit 502 is further configured to determine a detection result corresponding to the target voxel according to a plurality of result images corresponding to the target voxel in the segmentation result set, wherein the detection result is used to indicate whether the target voxel corresponds to a skull part.

[0159] The processing unit 502 is further configured to determine a three-dimensional segmentation result according to the detection result corresponding to the target voxel.

[0160] In one implementation, each of the n image sets further includes: a two-dimensional image obtained by fusing a plurality of two-dimensional images obtained by slicing the first image.

[0161] In one implementation, the two-dimensional images included in the n image sets are gradient images.

[0162] In one implementation, the n segmentation models are deep learning models constructed using an attention mechanism.

[0163] In one implementation, the n directions include: a coronal plane direction, a sagittal plane direction, and a horizontal plane direction.

[0164] In one implementation, the acquisition unit 501 is further used to acquire a second image; the second image is a three-dimensional image obtained by performing MRI on a human head.

[0165] The processing unit 502 is also used to input the second image into the target segmentation model to obtain a target result image; the target segmentation model is used to output a result image indicating the area corresponding to the brain in the three-dimensional image after inputting the three-dimensional image; the target result image is used to indicate the area corresponding to the brain in the second image.

[0166] The processing unit 502 is further configured to determine the first image from the second image according to the target result image.

[0167] In one implementation, the processing unit 502 is further configured to segment, according to the target result image, the first image including the area corresponding to the brain from the second image, including:

[0168] The processing unit 502 is also used to expand the area corresponding to the brain indicated by the target result image outward by a preset margin to obtain an extended image; wherein the extended image is used to indicate the target area, and the target area includes: the area corresponding to the brain indicated in the target result image and the area corresponding to the preset margin.

[0169] The processing unit 502 is further configured to determine the first image from the second image according to the extended image.

[0170] In one implementation, the acquisition unit 501 is further used to acquire a third image; the third image is a three-dimensional image obtained by performing MRI on a human head.

[0171] The processing unit 502 is further used to preprocess the third image to obtain the second image; the preprocessing includes at least one of the following: bias field correction and pixel value normalization.

[0172] The processing unit is a processor and the acquisition unit is a communication interface. Fig.11 As shown in , it includes part or all of the components in a processor 601, a communication line 602, a memory 603 and at least one communication interface 604.

[0173] The processor 601 is used to execute all or part of the steps in the method provided in this embodiment.

[0174] Specifically, the processor 601 may include a general-purpose central processing unit (CPU), and the processor 601 may also include a microprocessor, a field programmable gate array (Field Programmable Gate Array, FPGA), a digital signal processor (digital signal processing, DSP) or an application-specific integrated circuit (application-specific integrated circuit, ASIC), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0175] In a specific implementation, as an embodiment, the processor 601 may include one or more CPUs. In a specific implementation, as an embodiment, the device 60 may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing, for example, computer data (computer program instructions).

[0176] In addition, the memory 603 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM). Memory 603 can be independent and connected to processor 601 via communication line 602. Memory 603 can also be integrated with processor 601.

[0177] The memory 603 stores computer instructions. The processor 601 can execute the computer instructions stored in the memory 603 to perform all or part of the steps in the method provided in this embodiment.

[0178] Optionally, the computer-executable instructions in this embodiment may also be referred to as application program codes, which is not specifically limited in this embodiment.

[0179] In addition, the communication interface 604 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0180] In addition, the communication line 602 is used to connect the components in the device 60. Specifically, the communication line 602 may include a data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are marked as communication lines 602 in the figure.

[0181] In a specific implementation, as an embodiment, the apparatus 60 may further include an output device and an input device. The output device may communicate with the processor 601 and may display information in a variety of ways. The input device may communicate with the processor 601 and may receive user input in a variety of ways.

[0182] In addition, the device 60 may also include a storage medium. The storage medium is used to store computer instructions and various data for implementing the technical solution of this embodiment. When the device 60 executes the above method of this embodiment, the computer instructions and various data stored in the storage medium are loaded into the memory 603, so that the processor 601 can execute the computer instructions stored in the memory 603 to execute the method provided by this embodiment.

[0183] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0184] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image; the first image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head; According to the first image, n image sets are obtained; wherein the n image sets correspond to n directions one by one, and each of the image sets includes: a two-dimensional image obtained by slicing the first image according to the corresponding direction, and n is a positive integer greater than 1; Input the n image sets into n segmentation models respectively to obtain a segmentation result set; wherein the n segmentation models correspond to the n image sets one by one, and the n segmentation models are respectively used to output a result image indicating the area corresponding to the skull part in the two-dimensional image after inputting a two-dimensional image; the segmentation result set includes the result images output by the n segmentation models; A three-dimensional segmentation result is determined according to the segmentation result set; wherein the three-dimensional segmentation result is used to indicate an image after the skull portion is stripped from the first image.

2. The method according to claim 1, characterized in that: Determining the three-dimensional segmentation result according to the segmentation result set includes: Determine, according to the plurality of result images corresponding to the target voxel in the segmentation result set, a detection result corresponding to the target voxel, wherein the detection result is used to indicate whether the target voxel corresponds to a skull part; A three-dimensional segmentation result is determined according to the detection result corresponding to the target voxel.

3. The method according to claim 1, characterized in that Each of the n image sets further includes: a two-dimensional image obtained by fusing a plurality of two-dimensional images obtained by slicing the first image.

4. The method according to claim 1, characterized in that The two-dimensional images included in the n image sets are gradient images.

5. The method according to any one of claims 1 to 4, characterized in that: The n segmentation models are respectively deep learning models constructed using the attention mechanism.

6. The method according to any one of claims 1 to 4, characterized in that: The n directions include: a coronal plane direction, a sagittal plane direction and a horizontal plane direction.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire a second image; the second image is a three-dimensional image obtained by performing MRI on a human head; Inputting the second image into a target segmentation model to obtain a target result image; the target segmentation model is used to output a result image indicating the area corresponding to the brain in the three-dimensional image after inputting the three-dimensional image; the target result image is used to indicate the area corresponding to the brain in the second image; The first image is determined from the second image according to the target result image.

8. The method according to claim 7, characterized in that The step of segmenting the first image including the area corresponding to the brain from the second image according to the target result image comprises: Expanding the area corresponding to the brain indicated by the target result image outward by a preset margin to obtain an extended image; wherein the extended image is used to indicate a target area, and the target area includes: the area corresponding to the brain indicated in the target result image and the area corresponding to the preset margin; The first image is determined from the second image according to the extended image.

9. The method according to claim 7, characterized in that: The method further comprises: Acquire a third image; the third image is a three-dimensional image obtained by performing MRI on a human head; The third image is preprocessed to obtain the second image; the preprocessing includes at least one of the following: bias field correction and pixel value normalization.

10. An image processing device, characterized in that: include: An acquisition unit, configured to acquire a first image; The first image is a three-dimensional image obtained by performing magnetic resonance imaging (MRI) on a human head; A processing unit, configured to obtain n image sets according to the first image; wherein the n image sets correspond one-to-one to n directions, and each of the image sets comprises: a two-dimensional image obtained by slicing the first image according to the corresponding direction, and n is a positive integer greater than 1; The processing unit is further used to input the n image sets into n segmentation models respectively to obtain a segmentation result set; wherein the n segmentation models correspond to the n image sets one by one, and the n segmentation models are respectively used to output a result image indicating the area corresponding to the skull part in the two-dimensional image after inputting a two-dimensional image; the segmentation result set includes the result images output by the n segmentation models; The processing unit is further used to determine a three-dimensional segmentation result according to the segmentation result set; wherein the three-dimensional segmentation result is used to indicate an image after the skull portion is stripped from the first image.

Images