Method, device, equipment and storage medium for image processing

Abstract

The application provides a method, device and equipment for image processing and a storage medium. The method comprises: performing first segmentation on a brain magnetic resonance pre-processing image based on a brain structure to obtain a first segmentation image, wherein the first segmentation image comprises a plurality of brain sub-regions; performing second segmentation on the brain magnetic resonance pre-processing image based on a voxel block to obtain a second segmentation image, wherein the second segmentation image comprises a plurality of voxel blocks; fusing the first segmentation image and the second segmentation image to determine a voxel block contained in each brain sub-region; and obtaining a volume of the brain sub-region based on the voxel block contained in each brain sub-region. The application can improve the accuracy of the differential diagnosis result of PD and P-plus to a certain extent by accurately segmenting the brain sub-region.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of medicine, imaging examinations have become an important routine means of clinical diagnosis. For example, diagnostic results are determined through imaging techniques such as magnetic resonance imaging (MRI), computed tomography (CT), and three-dimensional ultrasound imaging.

[0003] Currently, accurate diagnosis relies heavily on the experience of professionals, which is highly subjective, lacks real-time accuracy, and is prone to misdiagnosis, leading to worsening of the patient's condition and missed opportunities for optimal treatment. Because the clinical symptoms of Parkinson's disease (PD) and Parkinson's syndrome (P-plus) are highly similar, the misdiagnosis rate for PD and P-plus exceeds 23.5%. However, PD and P-plus have fundamental differences in their pathogenesis, disease progression, and treatment methods.

[0004] With the development of artificial intelligence technology, deep learning technology is used to automatically analyze patients' medical images (e.g., MRI images) for auxiliary disease diagnosis. However, research on the differential diagnosis of PD and P-plus is still limited, and most studies rely on single specific features for differentiation, resulting in low accuracy. Therefore, how to efficiently and quickly differentiate between PD and P-plus while ensuring the accuracy of the diagnostic results is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] Some embodiments of this application provide an image processing method, apparatus, device, and storage medium that can at least partially solve the above-mentioned problems existing in the prior art.

[0006] According to one aspect of this application, an image processing method is provided, the method comprising: performing a first segmentation on a preprocessed brain MRI image based on brain structure to obtain a first segmented image, wherein the first segmented image includes a plurality of brain sub-regions; performing a second segmentation on the preprocessed brain MRI image based on voxel blocks to obtain a second segmented image, wherein the second segmented image includes a plurality of voxel blocks; fusing the first segmented image and the second segmented image to determine the voxel blocks included in each of the brain sub-regions; and obtaining the volume of the brain sub-region based on the voxel blocks included in each of the brain sub-regions.

[0007] In an embodiment of the present application, the fusing the first segmentation image and the second segmentation image to determine the voxel block contained in each of the brain sub-regions can further include: comparing the number of voxel blocks contained in each of the brain sub-regions with a threshold range of the number of voxel blocks; in response to the number of voxel blocks contained in the brain sub-region being within the threshold range of the number of voxel blocks, fusing the first segmentation image and the second segmentation image; and in response to the number of voxel blocks contained in the brain sub-region being beyond the threshold range of the number of voxel blocks, re-performing the second segmentation based on the voxels of the brain magnetic resonance preprocessing image.

[0008] In an embodiment of the present application, the fusing the first segmentation image and the second segmentation image to determine the voxel block contained in each of the brain sub-regions can include: determining the voxel label corresponding to the voxels in each of the brain sub-regions based on the first segmentation image; obtaining the number of the voxel labels corresponding to the voxels in the voxel block in combination with the second segmentation image; determining the label of the voxel block based on the number of the voxel labels; and determining the voxel block contained in each of the brain sub-regions based on the label of the voxel block.

[0009] In an embodiment of the present application, the method can further include: inputting the volume of the brain sub-region into a trained classification model, and obtaining a corresponding target classification result.

[0010] In an embodiment of the present application, the method can further include: preprocessing the obtained brain magnetic resonance image to obtain the brain magnetic resonance preprocessing image.

[0011] Another aspect of the present application provides an image processing device, which can include: a first segmentation module configured to perform first segmentation on a brain magnetic resonance preprocessing image based on brain structure to obtain a first segmentation image, wherein the first segmentation image includes a plurality of brain sub-regions; a second segmentation module configured to perform second segmentation on the brain magnetic resonance preprocessing image based on voxel blocks to obtain a second segmentation image, wherein the second segmentation image contains a plurality of voxel blocks; a fusion module configured to fuse the first segmentation image and the second segmentation image to determine the voxel block contained in each of the brain sub-regions; and a volume acquisition module configured to acquire the volume of each of the brain sub-regions based on the voxel block contained in each of the brain sub-regions.

[0012] In an embodiment of the present application, the second segmentation module can be configured to: compare the number of voxel blocks contained in each brain sub-region with a threshold range of the number of voxel blocks; in response to the number of voxel blocks contained in the brain sub-region being within the threshold range of the number of voxel blocks, fuse the first segmentation image and the second segmentation image; and in response to the number of voxel blocks contained in the brain sub-region being beyond the threshold range of the number of voxel blocks, re-segment the second segmentation based on the voxels of the brain magnetic resonance preprocessing image.

[0013] In an embodiment of the present application, the fusion module can be configured to: determine the voxel label corresponding to each voxel in the brain sub-region based on the first segmentation image; in combination with the second segmentation image, obtain the number of voxel labels corresponding to the voxel blocks; determine the label of the voxel block based on the number of voxel labels; and determine the voxel blocks contained in each brain sub-region based on the label of the voxel block.

[0014] In an embodiment of the present application, the apparatus can further include a classification module configured to input the volume of the brain sub-region into a trained classification model and obtain a corresponding target classification result.

[0015] In an embodiment of the present application, the apparatus can further include a preprocessing module configured to preprocess the obtained brain magnetic resonance image to obtain the brain magnetic resonance preprocessing image.

[0016] In yet another aspect of the present application, an electronic device is provided, which can include a processor adapted to execute a computer program, and a computer readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method of image processing of any of the above.

[0017] In yet another aspect of the present application, a computer readable storage medium is provided, which is configured to store a computer program, wherein the computer program causes a computer to execute the method of image processing of any of the above.

[0018] According to the exemplary embodiments of the present application, by fusing the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region, the gray scale, density, texture and other information of the brain sub-region are comprehensively utilized to accurately segment the brain sub-region and determine the volume of each brain sub-region, which can improve the accuracy of the PD and P-plus differential diagnosis results to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0019] Other features, objects, and advantages of the present application will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. In the drawings, which are not necessarily to scale, like reference characters often refer to similar items:

[0020] Figure 1 Flowchart of the method 1000 of image processing according to an embodiment of the present application;

[0021] Figure 2 Flowchart of forming voxel blocks in the second segmentation according to an exemplary embodiment of the present application;

[0022] Figure 3 Flowchart of determining voxel blocks contained in a sub-region of the brain according to an exemplary embodiment of the present application;

[0023] Figure 4 Flowchart of verifying the second segmented image according to an exemplary embodiment of the present application;

[0024] Figure 5 Schematic diagram of the apparatus 2000 of image processing according to an embodiment of the present application;

[0025] Figure 6 is a structural schematic diagram of an electronic device 700 suitable for use to implement embodiments of the present disclosure according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] For a better understanding of the present application, various aspects of the present application will be described in greater detail below with reference to the accompanying drawings. It is to be understood that the detailed description is merely descriptive in nature and is in no way intended to limit the scope of the present application. Throughout the specification, like reference numerals refer to like elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0027] In the drawings, the size, dimensions, and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are merely examples and are not strictly to scale. As used in this document, the terms "substantially", "approximately", and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent deviations in a measuring or computing process. In addition, in the present application, the order of the processing described in each step does not necessarily indicate the order in which the processes appear in actual operation, unless there is an explicit other limitation or can be derived from the context.

[0028] It should also be understood that all references to the application herein using expressions such as "including", "having", "including" or "containing" refer to the presence of stated features, elements or components, but do not preclude the presence or addition of one or more other features, elements, components or combinations thereof. In addition, when describing the application, the use of "or" means "and / or" unless clearly indicated otherwise. Furthermore, the words "a" or "an" shall not be construed as meaning only one, unless expressly so defined herein. Moreover, the words "may" and "can" are used interchangeably and mean "one or more implementations of the application." Furthermore, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an overly formal or overly literal sense unless expressly so defined herein.

[0030] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0031] Figure 1 A flowchart of a method 1000 of image processing according to an embodiment of the present application. As shown in Figure 1 The method 1000 of image processing can include:

[0032] Step S100: performing first segmentation on a brain magnetic resonance pre-processed image based on brain structure to obtain a first segmented image, wherein the first segmented image includes a plurality of brain sub-regions;

[0033] Step S200: performing second segmentation on the brain magnetic resonance pre-processed image based on voxel blocks to obtain a second segmented image, wherein the second segmented image includes a plurality of voxel blocks;

[0034] Step S300: fusing the first segmented image and the second segmented image to determine the voxel blocks included in each brain sub-region; and

[0035] Step S400: obtaining the volume of each brain sub-region based on the voxel blocks included in the brain sub-region.

[0036] The specific content of each step of the above-mentioned method 1000 of image processing will be described in detail below.

[0037] Brain magnetic resonance imaging (MRI) is essential for the diagnosis of PD and P-plus, and there are distinguishing features in brain magnetic resonance imaging (MRI) in Parkinson's disease (PD) and Parkinson's syndrome (P-plus). For example, multiple system atrophy (MSA) and progressive supranuclear palsy (PSP) in Parkinson's syndrome (P-plus) can show various features in P-plus but not in PD. For example, PSP patients show significant midbrain atrophy (hummingbird sign), increased pontine and third ventricle width; in MSA-P (Parkinson's symptoms) patients, putamen atrophy and flattened lateral margin (cleft sign) will occur; in MSA-C (cerebellar ataxia symptoms) patients, the cerebellum will atrophy, and the ratio of midbrain to pons will increase; and for the caudate nucleus, putamen and globus pallidus, which are the main components of the basal ganglia, it is generally believed that the occurrence of most neurodegenerative diseases is related to the changes in the structure of these parts. However, during the acquisition of brain MRI images, due to the difference in head size and placement position, there is a large difference between the acquired brain images, so it is necessary to preprocess the acquired brain images.

[0038] In the exemplary embodiments of the present application, the acquired brain magnetic resonance images are preprocessed to obtain brain magnetic resonance preprocessing images. Illustratively, brain magnetic resonance (MRI) images of different types of patients are acquired, and it is detected whether the data type of the brain MRI image is DICOM format. If the brain MRI image is not in DICOM format, the format of the brain MRI image can be converted, which is beneficial to the subsequent unified processing of the image. Then the brain MRI image is randomly divided into a training data set and a test data set, and a three-dimensional brain MRI image slice is converted into a plurality of two-dimensional brain magnetic resonance images. For example, the data set contains brain MRI images of 149 PD patients, 57 PSP patients and 86 MSA patients, which are divided into training set and test set according to a random ratio, and all brain MRI images are converted into DICOM format. The brain MRI image after format conversion is sequentially sliced, and the size of the slice is 256 pixels x 256 pixels, and the corresponding voxel size is 1.172 x 1.172 x 3.5 mm 3 .

[0039] In the exemplary embodiments of the present application, the preprocessing of the brain MRI image can further include: center cropping the two-dimensional brain magnetic resonance image, removing part of the background image, making the size of the brain MRI image consistent, reducing the interference of the background of the brain MRI image, and reducing the size of the brain MRI image. Then, the brain MRI image after center cropping can be corrected for bias field, mainly correcting the uneven low-frequency part of the brain MRI image, removing artifacts, and avoiding affecting the results of subsequent image processing. Finally, the image after bias field correction is normalized to obtain a brain magnetic resonance preprocessing image.

[0040] According to the exemplary embodiments of the present application, by preprocessing the brain magnetic resonance (MRI) image, the preprocessing process can include operations such as format conversion, sequence slicing, center cropping, bias field correction, and normalization processing on the brain magnetic resonance (MRI) image to obtain a brain magnetic resonance preprocessing image. To some extent, the accuracy of subsequent segmentation of the brain magnetic resonance preprocessing image can be improved, and the time for training the subsequent segmentation model can also be saved.

[0041] In the exemplary embodiments of the present application, after obtaining the brain magnetic resonance preprocessing image, the brain magnetic resonance preprocessing image is segmented based on brain structure to obtain a first segmentation image, wherein the first segmentation image includes a plurality of brain sub-regions. For example, a first segmentation model can be established and training parameters can be set to train the model. Taking the first segmentation model as a 3D-VNet model as an example, after establishing the 3D-VNet model, the training parameters of the 3D-VNet model can be set as follows: the base learning rate is set to 0.003, the learning rate decay frequency is set to 10 times every 40 rounds, the batch size is set to 16, the iteration period of training is set to 400, and the loss function is set to cross-entropy loss function and DICE loss function. After completing the parameter setting, the 3D-VNet model can be trained based on the brain magnetic resonance preprocessing images in the training data set, and a trained 3D-VNet model is obtained. Further, the brain magnetic resonance preprocessing images in the test data set can be input into the trained 3D-VNet model to obtain a first segmentation image, wherein the plurality of brain sub-regions of the first segmentation image can include: cerebellum, midbrain, pons, caudate nucleus, putamen, globus pallidus, and third ventricle. Those skilled in the art can know that the parameter setting of the first segmentation model of the present application is an exemplary description, and the parameters can be adjusted according to the actual situation, which is not limited by the present application.

[0042] According to the exemplary embodiments of the present application, by performing first segmentation on the brain magnetic resonance pre-processing image based on brain structures to obtain a plurality of first segmentation images of brain sub-regions, the second segmentation result can be better fused in the subsequent process to obtain a more accurate segmentation result.

[0043] In the exemplary embodiments of the present application, the brain magnetic resonance pre-processing image can also be second segmented based on voxel blocks to obtain a second segmentation image, wherein the second segmentation image contains a plurality of voxel blocks. The second segmentation can be performed on the brain magnetic resonance pre-processing image based on a 3D-SLIC model, and specifically, the voxels in the brain magnetic resonance pre-processing image can be iteratively clustered based on morphological features such as pixel values and textures to integrate similar voxels in the brain magnetic resonance pre-processing image into voxel blocks. Figure 2 A flowchart for forming voxel blocks in the second segmentation according to the exemplary embodiments of the present application is shown in FIG. 6. Figure 2 As shown in FIG. 6, the second segmentation can include the following steps:

[0044] Step S210: selecting a seed point in the brain magnetic resonance pre-processing image;

[0045] Step S220: setting a step range to determine the similarity between the voxel points and the seed point;

[0046] Step S230: classifying the voxel points based on the similarity to obtain a plurality of voxel blocks.

[0047] Exemplarily, if the brain magnetic resonance pre-processing image contains N voxel points, and K voxel blocks are expected to be obtained after the second segmentation, each voxel block contains N / K voxels, and the distance between adjacent seed points can be set to S = V N / K. When selecting the seed point, in order to reduce the seed point falling on the edge of the brain sub-region, a position with smaller gradient in a certain area of the original seed point can be selected as a new seed point. Then, based on the set step range, the similarity between the seed point and the voxel point within the step range is determined. Further, the voxel points are classified based on the similarity to obtain a plurality of voxel blocks. For example, a similarity threshold can be set, and if the similarity between the seed point and the voxel point exceeds the similarity threshold, the voxel point and the seed point are divided into the same voxel block.

[0048] In the exemplary embodiments of the present application, after obtaining the first segmentation image and the second segmentation image, the first segmentation image and the second segmentation image are fused to determine the voxel blocks contained in each brain sub-region. Figure 3 A flowchart for determining the voxel blocks contained in the brain sub-region according to the exemplary embodiments of the present application is shown in FIG. 7. Figure 3 As shown in FIG. 7, the determination of the voxel blocks contained in the brain sub-region can include the following steps:

[0049] Step S310: determining the voxel label corresponding to each voxel in each brain sub-region based on the first segmentation image;

[0050] Step S320: obtaining the number of voxel labels corresponding to each brain sub-region in the voxel block in combination with the second segmentation image;

[0051] Step S330: determining the label of the voxel block based on the number of voxel labels;

[0052] Step S340: determining the voxel block contained in each brain sub-region based on the label of the voxel block.

[0053] Exemplarily, first, the voxel label corresponding to each voxel in each brain sub-region is determined based on the first segmentation image, wherein the voxel label can represent the brain sub-region to which the voxel belongs. The voxel label can use numbers to represent the brain sub-region, for example, the voxel label in the cerebellum is set to 1, the voxel label in the midbrain is set to 2, the voxel label in the pontine is set to 3, the voxel label in the caudate nucleus is set to 4, the voxel label in the putamen is set to 5, the voxel label in the globus pallidus is set to 6, and the voxel label in the third ventricle is set to 7. Through the voxel label, the brain sub-region where the voxel is located can be confirmed. Then, in combination with the second segmentation image, the number of voxel labels corresponding to the voxel block is obtained. For example, based on the second segmentation image, the corresponding voxel in the voxel block is determined, and the voxel label corresponding to the voxel is determined. The number of voxel labels in each voxel block is counted, and the label with the most voxel labels is set as the label of the voxel block. Further, the voxel block contained in each brain sub-region can be determined based on the voxel block label.

[0054] According to the exemplary embodiments of the present application, by fusing the first segmentation image and the second segmentation image, the voxel block contained in each brain sub-region is determined, and the gray scale, density, texture and other information of the brain sub-region are comprehensively utilized to accurately segment the brain sub-region, so that the volume of each brain sub-region is more accurately determined in the subsequent process.

[0055] In the exemplary embodiments of the present application, after fusing the first segmentation image and the second segmentation image, the second segmentation image can also be verified. Figure 4 The flowchart for verifying the second segmentation image according to the exemplary embodiments of the present application is shown in FIG. 6. As shown in FIG. 6, verifying the second segmentation image can include the following steps: Figure 4

[0056] Step S350: comparing the number of voxel blocks contained in each brain sub-region with the voxel block number threshold range;

[0057] ​Step S360: in response to the number of voxel blocks contained in the brain sub-region being within the voxel block number threshold range, fusing the first segmentation image and the second segmentation image;

[0058] Step S370: in response to the number of voxel blocks contained in the brain sub-region being beyond the voxel block number threshold range, re-performing the second segmentation based on the voxels of the brain magnetic resonance preprocessing image.

[0059] In the example embodiments of the present application, since the size of each brain sub-region is different, the voxel block number threshold range can be set based on the size of each brain sub-region. If the number of voxel blocks contained in the brain sub-region is small, a large error will be generated when calculating the volume of the brain sub-region based on the voxel blocks, which may lead to subsequent disease type determination error; if the number of voxel blocks contained in the brain sub-region is too large, the requirement for hardware is high, and the time spent for calculating the volume of the brain sub-region is more, therefore, a reasonable voxel block number threshold range can be set based on the size of each brain sub-region. In the process of performing the second segmentation, the number of voxel blocks contained in each brain sub-region can be compared with the voxel block number threshold range, if the number of voxel blocks contained in the brain sub-region is within the voxel block number threshold range, the first segmentation image and the second segmentation image are fused; if the number of voxel blocks contained in the brain sub-region is beyond the voxel block number threshold range, re-performing the voxel block division and the second segmentation based on the voxels of the brain magnetic resonance preprocessing image.

[0060] According to the example embodiments of the present application, by detecting the number of voxel blocks contained in each brain sub-region, the rationality of the second segmentation result can be determined, and at the same time, the efficiency of the second segmentation can be improved to a certain extent.

[0061] In the example embodiments of the present application, after the fusion of the first segmentation image and the second segmentation image is completed, the volume of the brain sub-region can also be obtained based on the number of voxel blocks contained in each brain sub-region and the size of each pixel block. By fusing the first segmentation image and the second segmentation image, a more accurate volume of the brain sub-region can be obtained.

[0062] In the exemplary embodiments of the present application, after obtaining the volume of the brain sub-region, the volume of the brain sub-region can be input into the trained classification model to obtain the corresponding target classification result. For example, the classification model can be established and the training parameters of the model can be set to train the model. For example, the classification model can be based on the integrated learning algorithm (GBDT) model. After the classification model is established, the training parameters of the classification model can be set as follows: the number of decision trees is set to 50, the depth is 3, and the learning rate is 0.1. The decision tree is used as a learner to fit the residual, thereby reducing the bias and variance of the classification model, and obtaining more accurate PD and P-plus classification results.

[0063] Another aspect of the present application also provides an image processing device 2000. Figure 5 The schematic diagram of the image processing device 2000 according to the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the image processing device 2000 can include a first segmentation module 2100, a second segmentation module 2200, a fusion module 2300, and a volume acquisition module 2400. Figure 5

[0064] In the exemplary embodiments of the present application, the image processing device 2000 can further include a preprocessing module 2500, which can be used to preprocess the acquired brain magnetic resonance image to obtain a brain magnetic resonance preprocessing image. For example, the brain magnetic resonance (MRI) images of different types of patients are acquired, and it is detected whether the data type of the brain MRI image is in DICOM format. If the brain MRI image is not in DICOM format, the format of the brain MRI image can be converted, which is beneficial for later unified image processing. Then the brain MRI image is randomly divided into a training data set and a test data set, and a three-dimensional brain MRI image slice is converted into a plurality of two-dimensional brain magnetic resonance images. For example, the brain MRI images of 149 PD patients, 57 PSP patients, and 86 MSA patients are included in the data set, which are divided into a training set and a test set according to a random ratio, and all the brain MRI images are converted into DICOM format. The brain MRI image after format conversion is sequentially sliced, and the size of the slice is 256 pixels x 256 pixels, and the corresponding voxel size is 1.172 x 1.172 x 3.5 mm 3 .

[0065] ​In the exemplary embodiments of the present application, the preprocessing of the brain MRI image can further include: center cropping the two-dimensional brain magnetic resonance image, removing part of the background image, making the size of the brain MRI image consistent, reducing the interference of the background of the brain MRI image, and reducing the size of the brain MRI image. Then, the brain MRI image after center cropping can be corrected for bias field, mainly correcting the uneven low-frequency part of the brain MRI image, removing artifacts, and avoiding affecting the results of subsequent image processing. Finally, the image after bias field correction is normalized to obtain a brain magnetic resonance preprocessing image.

[0066] According to the exemplary embodiments of the present application, by preprocessing the brain magnetic resonance (MRI) image, the preprocessing process can include operations such as format conversion, sequence slicing, center cropping, bias field correction, and normalization processing on the brain magnetic resonance (MRI) image to obtain a brain magnetic resonance preprocessing image. To some extent, the accuracy of subsequent segmentation of the brain magnetic resonance preprocessing image can be improved, and the time for training the subsequent segmentation model can also be saved.

[0067] In the exemplary embodiments of the present application, the first segmentation module 2100 can be used to perform first segmentation on the brain magnetic resonance preprocessing image based on brain structure to obtain a first segmentation image, wherein the first segmentation image includes a plurality of brain sub-regions. For example, a first segmentation model can be established and training parameters can be set to train the model. Taking the first segmentation model as a 3D-VNet model as an example, after establishing the 3D-VNet model, the training parameters of the 3D-VNet model can be set as follows: the base learning rate is set to 0.003, the learning rate decay frequency is set to 10 times every 40 rounds, the batch size is set to 16, the iteration period of training is set to 400, and the loss function is set to cross-entropy loss function and DICE loss function. After completing the parameter setting, the 3D-VNet model can be trained based on the brain magnetic resonance preprocessing images in the training data set, and a trained 3D-VNet model can be obtained. Further, the brain magnetic resonance preprocessing images in the test data set can be input into the trained 3D-VNet model to obtain a first segmentation image, wherein the plurality of brain sub-regions of the first segmentation image can include: cerebellum, midbrain, pons, caudate nucleus, putamen, globus pallidus, and third ventricle. Those skilled in the art can know that the parameter setting of the first segmentation model of the present application is an exemplary description, and the parameters can be adjusted according to the actual situation, which is not limited by the present application.

[0068] According to the exemplary embodiments of the present application, by performing the first segmentation on the brain magnetic resonance pre-processing image based on the brain structure to obtain a plurality of first segmentation images of brain sub-regions, the second segmentation result can be better fused in the subsequent process to obtain a more accurate segmentation result.

[0069] In the exemplary embodiments of the present application, the second segmentation module 2200 can be used to perform the second segmentation on the brain magnetic resonance pre-processing image based on the voxel block to obtain a second segmentation image, wherein the second segmentation image contains a plurality of voxel blocks. The second segmentation can be performed on the brain magnetic resonance pre-processing image based on the 3D-SLIC model, specifically, the voxels in the brain magnetic resonance pre-processing image can be iteratively clustered based on the morphological features such as pixel value, texture, etc., and the similar voxels in the brain magnetic resonance pre-processing image can be integrated into a voxel block. The second segmentation can include selecting a seed point in the brain magnetic resonance pre-processing image, setting a step range to determine the similarity between the voxel point and the seed point, and classifying the voxel point based on the similarity to obtain a plurality of voxel blocks. For example, if the brain magnetic resonance pre-processing image contains N voxel points, and K voxel blocks are expected to be obtained after the second segmentation, each voxel block contains N / K voxels, and the distance between adjacent seed points can be set to S=√N / K. When selecting the seed point, in order to reduce the seed point falling on the edge position of the brain sub-region, a position with smaller gradient in a certain area of the original seed point can be selected as a new seed point. Then, based on the set step range, the similarity between the seed point and the voxel point within the step range is determined. Further, the voxel point is classified based on the similarity to obtain a plurality of voxel blocks. For example, a similarity threshold can be set, and if the similarity between the seed point and the voxel point exceeds the similarity threshold, the voxel point and the seed point are divided into the same voxel block.

[0070] In the example embodiment of the present application, the fusion module 2300 can be used to fuse the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region. Determining the voxel blocks contained in each brain sub-region can include: determining the voxel label corresponding to each voxel in each brain sub-region based on the first segmentation image; obtaining the number of voxel labels corresponding to each brain sub-region in the voxel block in combination with the second segmentation image; determining the label of the voxel block based on the number of voxel labels; and determining the voxel blocks contained in each brain sub-region based on the label of the voxel block. For example, first, the voxel label corresponding to each voxel in each brain sub-region is determined based on the first segmentation image, wherein the voxel label can represent the brain sub-region to which the voxel belongs. The voxel label can represent the brain sub-region using numbers, for example, the voxel label in the cerebellum is set to 1, the voxel label in the midbrain is set to 2, the voxel label in the pontine is set to 3, the voxel label in the caudate nucleus is set to 4, the voxel label in the putamen is set to 5, the voxel label in the globus pallidus is set to 6, and the voxel label in the third ventricle is set to 7. Through the voxel label, the brain sub-region where the voxel is located can be determined. Then, in combination with the second segmentation image, the number of voxel labels corresponding to the voxel block is obtained. For example, based on the second segmentation image, the corresponding voxel in the voxel block is determined, and the voxel label corresponding to the voxel is determined. The number of voxel labels in each voxel block is counted, and the label with the most voxel labels is set as the label of the voxel block. Further, the voxel blocks contained in each brain sub-region can be determined based on the voxel block label.

[0071] According to the example embodiment of the present application, by fusing the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region, the gray scale, density, texture and other information of the brain sub-region are comprehensively utilized to accurately segment the brain sub-region, so that the volume of each brain sub-region determined in the subsequent process is more accurate.

[0072] In the example embodiment of the present application, after fusing the first segmentation image and the second segmentation image, the second segmentation image can also be verified. Verifying the second segmentation image can include: comparing the number of voxel blocks contained in each brain sub-region with a voxel block number threshold range; in response to the number of voxel blocks contained in the brain sub-region being within the voxel block number threshold range, fusing the first segmentation image and the second segmentation image; and in response to the number of voxel blocks contained in the brain sub-region exceeding the voxel block number threshold range, re-performing the second segmentation based on the voxels of the brain magnetic resonance preprocessing image.

[0073] In the example embodiments of the present application, since the sizes of each brain sub-region are different, the threshold range of the number of voxel blocks can be set based on the size of each brain sub-region. If the number of voxel blocks contained in a brain sub-region is small, a large error will be generated when the volume of the brain sub-region is calculated based on the voxel blocks, which can lead to subsequent disease type determination errors; if the number of voxel blocks contained in a brain sub-region is too large, the requirement for hardware is high, and the time spent in calculating the volume of the brain sub-region is large, therefore, a reasonable threshold range of the number of voxel blocks can be set based on the size of each brain sub-region. In the process of the second segmentation, the number of voxel blocks contained in each brain sub-region can be compared with the threshold range of the number of voxel blocks, if the number of voxel blocks contained in the brain sub-region is within the threshold range of the number of voxel blocks, the first segmented image and the second segmented image are fused; if the number of voxel blocks contained in the brain sub-region exceeds the threshold range of the number of voxel blocks, the voxel blocks are re-divided based on the voxels of the brain magnetic resonance preprocessing image and the second segmentation is performed again.

[0074] According to the example embodiments of the present application, by detecting the number of voxel blocks contained in each brain sub-region, the rationality of the second segmentation result can be determined, and the efficiency of the second segmentation can be improved to a certain extent while ensuring the calculation of the volume of the brain sub-region.

[0075] In the example embodiments of the present application, the volume acquisition module 2400 can be configured to acquire the volume of the brain sub-region based on the voxel blocks contained in each brain sub-region. By fusing the first segmented image and the second segmented image, a more accurate volume of the brain sub-region can be obtained.

[0076] In the example embodiments of the present application, the image processing device 2000 can further include a classification module 2600, which can be configured to input the volume of the brain sub-region into a trained classification model and obtain a corresponding target classification result. For example, a classification model can be established and training parameters of the model can be set to train the model. For example, the classification model can be a GBDT model based on ensemble learning algorithm, after the classification model is established, the training parameters of the classification model can be set as follows: the number of decision trees is set to 50, the depth is 3, and the learning rate is 0.1. The decision tree is used as a learner to fit the residual, thereby reducing the bias and variance of the classification model, and obtaining more accurate PD and P-plus classification results.

[0077] The present application also provides an electronic device and a computer readable storage medium. Figure 6 FIG. 7 is a structural schematic diagram of an electronic device 700 according to an embodiment of the present disclosure.

[0078] The following will be described with reference to Figure 6The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The terminal device / server shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0079] like Figure 6 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0080] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0081] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0082] It should be noted that the computer readable medium described in embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In embodiments of the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to wire, cable, radio frequency (RF), and any suitable combination thereof.

[0083] The computer readable medium can be included in the electronic device, or can be separate from the electronic device. The computer readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform: first segmenting a brain magnetic resonance pre-processing image based on brain structure to obtain a first segmentation image, wherein the first segmentation image includes a plurality of brain sub-regions; second segmenting the brain magnetic resonance pre-processing image based on a voxel block to obtain a second segmentation image, wherein the second segmentation image includes a plurality of voxel blocks; fusing the first segmentation image and the second segmentation image to determine a voxel block included in each brain sub-region; and obtaining a volume of the brain sub-region based on the voxel block included in each brain sub-region.

[0084] Computer program code for carrying out operations of embodiments of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0085] The computer readable program code can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the code which implement the process is executed on the computer, other programmable apparatus or other device.

[0086] The specific embodiments described above are further explained in connection with the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of image processing, characterized by, The method comprises: performing first segmentation on a brain magnetic resonance pre-processing image based on brain structure to obtain a first segmentation image, wherein the first segmentation image comprises a plurality of brain sub-regions; performing second segmentation on the brain magnetic resonance pre-processing image based on voxel blocks to obtain a second segmentation image, wherein the second segmentation image comprises a plurality of voxel blocks; fusing the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region; and acquiring the volume of each brain sub-region based on the voxel blocks contained in each brain sub-region. The method further comprises: inputting the volume of each brain sub-region into a trained classification model to obtain a corresponding target classification result; The second segmentation step comprises: selecting a seed point in the brain magnetic resonance pre-processing image; setting a step range to determine the similarity between the voxel point and the seed point; classifying the voxel points based on the similarity to obtain a plurality of voxel blocks; fusing the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region, comprising: determining the voxel label corresponding to the voxel in each brain sub-region based on the first segmentation image; acquiring the number of corresponding voxel labels in the voxel block in combination with the second segmentation image; determining the label of the voxel block based on the number of voxel labels; determining the voxel blocks contained in each brain sub-region based on the label of the voxel block.

2. The method of image processing according to claim 1, characterized in that, Fusing the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region further comprises: comparing the number of voxel blocks contained in each brain sub-region with a voxel block number threshold range; in response to the number of voxel blocks contained in the brain sub-region being within the voxel block number threshold range, fusing the first segmentation image and the second segmentation image; in response to the number of voxel blocks contained in the brain sub-region exceeding the voxel block number threshold range, re-performing the second segmentation based on the voxels of the brain magnetic resonance pre-processing image.

3. The method of image processing according to claim 1, characterized in that, The method further comprises: pre-processing the acquired brain magnetic resonance image to obtain the brain magnetic resonance pre-processing image.

4. An apparatus for image processing, characterized by, The device comprises: a first segmentation module configured to perform first segmentation on a brain magnetic resonance pre-processing image based on brain structure to obtain a first segmentation image, wherein the first segmentation image comprises a plurality of brain sub-regions; a second segmentation module configured to perform second segmentation on the brain magnetic resonance pre-processing image based on voxel blocks to obtain a second segmentation image, wherein the second segmentation image comprises a plurality of voxel blocks; and a fusion module configured to fuse the first segmentation image and the second segmentation image to determine the voxel blocks contained in each brain sub-region; a volume acquisition module configured to acquire the volume of each brain sub-region based on the voxel blocks contained in each brain sub-region; a classification module configured to input the volume of each brain sub-region into a trained classification model to obtain a corresponding target classification result; the second segmentation module is configured to: Selecting a seed point in a brain magnetic resonance preprocessing image; Setting a step range to determine the similarity between the voxel point and the seed point; Classifying the voxel points based on the similarity to obtain a plurality of voxel blocks; The fusion module is configured to: Determine the voxel label corresponding to each voxel in each brain sub-region based on the first segmentation image; Obtain the number of voxel labels corresponding to the voxel blocks in the second segmentation image; Determine the label of the voxel block based on the number of voxel labels; Determine the voxel blocks contained in each brain sub-region based on the label of the voxel block.

5. The apparatus for image processing of claim 4, wherein, The second segmentation module is configured to: Compare the number of voxel blocks contained in each brain sub-region with a voxel block number threshold range; In response to the number of voxel blocks contained in the brain sub-region being within the voxel block number threshold range, fuse the first segmentation image and the second segmentation image; In response to the number of voxel blocks contained in the brain sub-region exceeding the voxel block number threshold range, re-segment the second segmentation based on the voxels of the brain magnetic resonance preprocessing image.

6. An electronic device, comprising: Comprise: A processor adapted to execute a computer program; And A computer readable storage medium having a computer program stored therein, wherein the computer program is executed by the processor to implement the image processing method of any one of claims 1 to 3.

7. A computer readable storage medium characterized in that, A computer program for storing a computer program, which enables a computer to execute the image processing method of any one of claims 1 to 3.

Images