Medical Image Processing Method, Apparatus, and Storage Medium

Abstract

The present application provides a medical image processing method, apparatus, and storage medium. The medical image processing method includes: constructing a medical image segmentation model; using the medical image segmentation model to perform a first segmentation on a medical image to obtain multiple regions after the first segmentation of the medical image; the medical image being a knee joint cartilage functional nuclear magnetic resonance image; determining at least one region as a target region to determine multiple target regions; determining the coordinates of the corresponding boundary position according to the prediction value of each target region; using the coordinates of the boundary position as a segmentation criterion, and based on the segmentation criterion and the positional relationship between other target regions and the current target region among the multiple target regions, further segmenting other target regions among the multiple target regions to obtain multiple sub-regions; which can shorten the medical image segmentation time and save labor costs.

Description

Technical Field

[0001] This application relates to the technical field of medical image processing, and particularly relates to a medical image processing method, apparatus, and storage medium. Background Art

[0002] In the prior art, when segmenting and analyzing medical images, the main compartment analysis (taking the knee joint as an example) includes the following two types: 1. Rigorous registration of sagittal high-resolution CUBE images and T1ρ relaxation time map images for cartilage segmentation, as Figure 2 shown; 2. Using in-house software developed with Matlab (Mathworks, Natick, MA, USA) based on edge detection and Bezier splines to semi-automatically segment the medial femoral condyle (MF), medial tibia (MT), lateral femoral condyle (LF), and lateral tibia (LT) cartilage compartments, as Figure 1 shown. Next, the main compartments are divided into small compartments. For example, MF and LF are divided into central femoral condyles (cMF / cLF) and posterior femoral condyles (pMF / pLF) using the posterior boundary of the posterior horn of the meniscus; cMF and cLF are further divided into 3 load-bearing sub-compartments: anterior (cMF-a / cLF-a), central (cMF-c / cLF-c), and posterior (cMF-p / cLF-p), marked by the medial edge of the meniscal horn. Similarly, MT and LT are each divided into 3 sub-compartments: anterior (MT-a / LT-a), central (MT-c / LT-c), and posterior (MT-p / LT-p). Figure 2 Show small compartments (C and D) of load-bearing and non-load-bearing regions, LF: lateral femur; LT: lateral tibia; MF: medial femur; MT: medial tibia; cXF: the anterior (a), central (c), or posterior (p) part of the central part of the femoral condyle of X.

[0003] When segmenting medical images, it is necessary to obtain T1ρ and T2map relaxation time values. First, to account for small movements during acquisition, echoes 2-8 are respectively registered to the first echo of the T1ρ and T2 sequences; then, all echoes of the T2map sequence are registered to the first T1ρ echo. The relaxation time maps of T1ρ and T2 are constructed by pixel-by-pixel fitting all 8 T1ρ and T2-weighted images to an equation including the following 3 parameters using in-house developed software in matlab:

[0004]

[0005] In the above formula, S represents the image signal at a given time point; TSL represents the spin-lock time of the T1ρ map; TE represents the echo time of the T2map; A = M0 or the initial magnetization; B is a constant.

[0006] The cartilage regions of interest are overlaid on T1ρ and T2 maps. The cartilage splines are manually adjusted to avoid synovial fluid or surrounding anatomical structures. To eliminate artifacts caused by the partial volume effect of synovial fluid, voxels with T1ρ relaxation time ≥ 130 ms or T2 maps ≥ 100 ms were excluded. The average values of T1ρ and T2 of the defined cartilage regions were calculated.

[0007] Segmenting medical images in the prior art is laborious and time-consuming. The above steps need to be continuously repeated for each medical image segmentation. For example, 2 - 3 images are required for each lower limb of each subject to reduce errors, resulting in a large workload and being time-consuming and laborious. Also, researchers need to learn and proficiently use matlab for medical image post-processing and data calculation, etc., which requires high requirements for researchers. Summary of the Invention

[0008] Aiming at the technical problems of large workload and time-consuming and laborious in segmenting medical images in the prior art, this application proposes a medical image processing method, device, and storage medium.

[0009] According to one aspect of this application, a medical image processing method is provided, including:

[0010] Construct a medical image segmentation model;

[0011] Use the medical image segmentation model to perform a first segmentation on the medical image to obtain multiple regions after the first segmentation of the medical image; the medical image is a knee joint cartilage functional nuclear magnetic resonance image; each region has a corresponding predicted value, and the predicted values between different regions are different;

[0012] Determine at least one region as the target region to determine multiple target regions;

[0013] According to the predicted value of each target region, determine the coordinates of its corresponding boundary position;

[0014] Taking the coordinates of the boundary position as the segmentation criterion, based on the segmentation criterion and the positional relationship between other target regions and the current target region among the multiple target regions, further segment other target regions among the multiple target regions to obtain multiple sub-regions;

[0015] Assign values to and store each sub-region respectively.

[0016] In some embodiments, constructing the medical image segmentation model includes: constructing a trainable data set, where the trainable data set includes medical images and data for data annotation of at least partial regions of the medical images; constructing a deep learning segmentation network model; and training the deep learning segmentation network model to obtain the medical image segmentation model.

[0017] In some embodiments, constructing the deep learning segmentation network model includes: designing a U-shaped segmentation network including an encoding network and a decoding network; extracting features from the trainable data set by the encoding network through convolutional blocks; restoring the processed medical image data to the original size of the medical image by the decoding network and implementing the fusion of features from coarse to fine to extract deep feature information of the medical image; and obtaining a feature map consistent with the number of segmentation categories of the medical image and a predicted segmentation probability map based on the deep feature information of the medical image.

[0018] In some embodiments, training the deep learning segmentation network model includes: processing the data in the trainable data set to have the same resolution and size; performing online data augmentation on the first part of the processed data and inputting the data after online data augmentation into the deep learning segmentation network model; generating a predicted segmentation result through the deep learning segmentation network model, and calculating a loss function between the prediction result and the manually segmented medical image; using the second part of the processed data except the first part as a validation set to verify the segmentation accuracy of the deep learning segmentation network model; and performing repeated iteration using the data in the trainable data set to reduce and converge the loss function, and selecting the best deep learning segmentation network model according to the verification result of the validation set.

[0019] In some embodiments, constructing the medical image segmentation model further includes optimizing the medical image segmentation model, where optimizing the medical image segmentation model includes: using the best deep learning segmentation network model as an initial model; inputting the unlabeled trainable data set into the initial model to obtain a predicted segmented image; manually modifying the unlabeled trainable data set to reduce the data annotation time and obtain more labeled training data; and inputting the labeled training data into the initial model for training to obtain an optimized best deep learning segmentation network model.

[0020] In some embodiments, the first segmentation of the medical image by using the medical image segmentation model to obtain multiple regions after the first segmentation of the medical image includes: converting the medical image data to be processed into the same resolution and size as the data for training the deep learning segmentation network model; inputting the converted medical image data into the medical image segmentation model to obtain a predicted segmentation probability map; and converting the predicted segmentation probability map into predicted values, where the predicted values have multiple different values, and the multiple different values respectively represent different regions of the medical image.

[0021] In some embodiments, before determining at least one region as a target region to determine multiple target regions, the medical image processing method further includes: performing dilation and erosion and connected component calculation on the predicted segmentation result, selecting the region with the largest connected component in each segmentation category to eliminate the error discrete points caused by network misclassification; and generating an empty matrix to store the assignment of the sub-regions.

[0022] According to one aspect of the present application, there is provided a medical image processing apparatus, including a memory and a processor, the memory is used to store computer-executable programs, and the processor is used to call the computer-executable programs to implement the above-mentioned medical image processing method.

[0023] According to one aspect of the present application, there is provided a computer storage medium, on which a computer program is stored, and the computer program can be executed by a processor to implement the above-mentioned medical image processing method.

[0024] The medical image processing method of the present application can shorten the medical image segmentation time and save labor costs by constructing a medical image segmentation model; using the medical image segmentation model to perform a first segmentation on the medical image to obtain multiple regions after the first segmentation of the medical image; and performing a second segmentation on the multiple regions based on the boundary positions of at least one region in the multiple regions to obtain multiple sub-regions of the medical image. Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a schematic diagram of the human knee joint structure;

[0027] Figure 2 It is a sagittal plane view of the human knee joint and along its Line C and Line D;

[0028] Figure 3 is a flowchart of a medical image processing method according to an embodiment of the present application;

[0029] Figure 4 is a flowchart of a medical image processing method according to a specific embodiment of the present application;

[0030] Figure 5 is a schematic structural diagram of a U-shaped segmentation network according to an embodiment of the present application;

[0031] Figure 6 is an iterative schematic diagram of a deep learning segmentation network model according to an embodiment of the present application; and

[0032] Figure 7 is a schematic structural diagram of a medical image processing device according to an embodiment of the present application. Detailed implementation manners

[0033] In order to enable those skilled in the art of the present technology to better understand the solutions of the present application, and to fully understand how the present application uses technical means to solve technical problems and the implementation process of achieving corresponding technical effects for implementation, the technical solutions in the present application will be clearly and completely described below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Each feature in the present application and the embodiments can be combined with each other without conflict, and the formed technical solutions are all within the protection scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0035] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0036] In view of the problems of large workload, time-consuming and laborious, and high difficulty in segmenting medical images in the prior art, the present application proposes a medical image processing method. Figure 3 As a flowchart of the medical image processing method according to an embodiment of the present application, the following will refer to Figure 3 to describe the present application in detail.

[0037] As Figure 3 shown, the medical image processing method includes: S100, constructing a medical image segmentation model; S102, performing a first segmentation on the medical image by using the medical image segmentation model to obtain multiple regions after the first segmentation of the medical image; and S104, performing a second segmentation on the multiple regions based on the boundary positions of at least one region among the multiple regions to obtain multiple sub-regions of the medical image.

[0038] In step S100, constructing the medical image segmentation model includes: S1001, constructing a trainable data set; S1002, constructing a deep learning segmentation network model; and S1003, training the deep learning segmentation network model to obtain the medical image segmentation model.

[0039] In step S1001, the trainable data set includes data related to medical images and data annotations for at least part of the regions of the medical images. Taking human knee joint data as an example, a large number of existing MR medical images of knee joints T1 and T2 are collected, and professional doctors manually annotate the required regions of some data. The required regions include femoral cartilage (set as label 1), tibial cartilage (set as label 2), anterior horn of meniscus (set as label 3), and posterior horn of meniscus (set as label 4), thereby constructing a knee joint data set with four labels.

[0040] In step S1002, constructing the deep learning segmentation network model includes: designing a U-shaped segmentation network including an encoding network and a decoding network; extracting features from the trainable data set by the encoding network through convolutional blocks; the decoding network restores the processed medical image data to the original size of the medical image and realizes the fusion of features from coarse to fine to extract the deep feature information of the medical image; and obtaining a feature map consistent with the number of segmentation categories of the medical image and a predicted segmentation probability map based on the deep feature information of the medical image.

[0041] Specifically, design a U-shaped segmentation network, and this network is as Figure 5As shown, the U-shaped segmentation network includes an encoding network and a decoding network. The encoding network extracts features through convolutional blocks, and the decoding network restores the image data to the original image size while achieving the fusion of features from coarse to fine and extracting deep feature information.

[0042] In the encoding stage, 7 convolutional blocks are used. Each convolutional block consists of two repeated convolutional sub-blocks. Each convolutional sub-block contains a 2D convolutional layer, an intensity normalization layer, and a LeakyReLU activation function. When downsampling, the stride of the convolutional layer is set to 2, and the number of feature channels is controlled by the convolutional layer. The kernel size of the convolutional layer is always set to 3.

[0043] In the decoding stage, 6 completely symmetric convolutional blocks are adopted. Different from the encoding stage, in the decoding stage, when upsampling, a transposed convolution with a kernel size of 2 and a stride of 2 is used to restore the resolution before operating on the convolutional block. At the same time, before performing the convolutional block operation, the deep feature map after upsampling is combined with the shallow feature map of the same resolution in the encoding stage along the channels to avoid the loss of semantic information during the image decoding process. Finally, a 2D convolutional layer with 5 feature channels and a softmax layer are added. The 2D convolutional layer converts the feature map into a feature map consistent with the number of segmentation categories, and the softmax layer converts the feature map into a predicted segmentation probability map ranging from [0 - 1] (there are 4 label categories and 1 background, counted as 5 categories).

[0044] In step S1003, training the deep learning segmentation network model includes: processing the data in the trainable dataset to make it have the same resolution and size; performing online data augmentation on the first part of the processed data and inputting the data after online data augmentation into the deep learning segmentation network model; generating a predicted segmentation result through the deep learning segmentation network model and calculating the loss function between the predicted result and the manually segmented medical image; using the second part of the processed data except the first part as a validation set to verify the segmentation accuracy of the deep learning segmentation network model; and using the data in the trainable dataset for repeated iteration to reduce and converge the loss function, and selecting the best deep learning segmentation network model according to the verification result of the validation set.

[0045] Specifically, considering the differences in data resolution caused by different hospitals and imaging acquisition devices in the collected data samples, the spatial information of the data can be learned to unify all the data into the same resolution, and the data can be cropped to obtain the same size, such as 512*512. Subsequently, online data augmentation is performed on the first part (e.g., 80%) of the data in the dataset to increase data diversity. Online data augmentation will improve generalization and diversity more than pre-data augmentation. Data augmentation includes rotation, mirroring, downsampling, Gaussian noise, etc., to adapt to the data differences caused by factors such as patient position offset and machine scanning direction during actual clinical image data acquisition. The online augmented data is used as training data and fed into the constructed deep learning segmentation network model. The predicted segmentation result is generated by the model, and the loss function between the predicted result and the actual doctor's manual segmentation label is calculated. The U-shaped network adopts a deep supervision strategy, that is, the true label is downsampled to be the same size as the feature map of the decoder at each resolution. Subsequently, the output of the decoder at each resolution is jointly calculated with the true label at the same resolution for loss calculation. The loss function at each resolution is weighted by a coefficient of 1 / 2n, where n is the number of downsampling times. Subsequently, backpropagation is performed to update the network parameters. The remaining 20% of the data is used as the validation set to verify the classification accuracy under the current network. After repeated iterations, the loss function is reduced and converges, and the best network model M’ is selected according to the verification result of the validation set, as Figure 6 shown.

[0046] Furthermore, the medical image segmentation model can be further optimized, including: using the best deep learning segmentation network model as the initial model; inputting the unlabeled trainable dataset into the initial model to obtain the predicted segmentation image; manually modifying the unlabeled trainable dataset to reduce the data annotation time and obtain more labeled training data; and inputting the labeled training data into the initial model for training to obtain the optimized best deep learning segmentation network model.

[0047] Specifically, the collected unlabeled dataset is fed into the trained deep learning segmentation network model for testing to obtain the predicted segmentation result; this batch of data is returned to the doctor for manual modification to obtain more labeled training data while reducing the data annotation time. Subsequently, the increased data is trained as described above to optimize the model parameters and improve the model accuracy to obtain the optimal model M. In the model optimization process here, the network model is not initialized, but the above-mentioned best model M’ is used as the initial model for model optimization here. For the subsequently continuously collected datasets, the above model optimization method can continue to be used to continuously optimize the model and obtain a highly generalized segmentation model M, as Figure 6 shown.

[0048] In step S102, the medical image is first segmented using a medical image segmentation model to obtain multiple regions after the first segmentation of the medical image, including: converting the medical image data to be processed into the same resolution and size as the data for training the deep learning segmentation network model; inputting the converted medical image data into the medical image segmentation model to obtain a predicted segmentation probability map; and converting the predicted segmentation probability map into predicted values, where the predicted values have multiple different values, and the multiple different values respectively represent different regions of the medical image.

[0049] Specifically, taking the above knee joint dataset as an example, the resolution of the patient image data O to be segmented is converted to the same resolution as in the training process, and then the data is cropped or expanded to a size of 512*512 in a sliding window manner and input into the model, with a sliding window step size of 256. Subsequently, according to the trained model, a predicted segmentation probability map is obtained. Through the argmax operation, that is, the channel value where the maximum probability is located is selected in the corresponding channel, and the predicted probability map is converted into a predicted value P, where P ∈ {0, 1, 2, 3, 4}. A predicted value of 0 represents the background region, a predicted value of 1 represents the femoral cartilage region, a predicted value of 2 represents the tibial cartilage region, a predicted value of 3 represents the anterior horn of the meniscus region, and a predicted value of 4 represents the posterior horn of the meniscus region. After all the data blocks obtained by sliding window cropping or expansion are predicted, all the data blocks are stitched or cropped back to the original size and restored to the original resolution.

[0050] In step S104, the multiple regions are secondarily segmented based on the boundary positions of at least one of the multiple regions to obtain multiple sub-regions of the medical image, including:

[0051] Determining at least one region as a target region to identify multiple target regions;

[0052] According to the predicted value of each target region, determining the coordinates of its corresponding boundary position;

[0053] Using the coordinates of the boundary position as the segmentation criterion, and based on the segmentation criterion and the positional relationship between other target regions and the current target region among the multiple target regions, further segmenting other target regions among the multiple target regions to obtain multiple sub-regions;

[0054] Assigning values to and storing each of the sub-regions respectively.

[0055] Specifically, taking the above knee joint dataset as an example, this secondary segmentation includes the following steps:

[0056] A. For the predicted segmentation result P, first perform dilation and erosion and connected component calculation, and select the region with the largest connected component in each category to eliminate the error discrete points caused by network misclassification;

[0057] B. Generate an empty matrix new_P to store the data for subsequent sub-region division;

[0058] C. According to the label of the anterior horn of the meniscus (i.e., the part in P with a value of 3), detect the rightmost index position (x1, y1) of the anterior horn region of the meniscus, and use this index position y1 as the dividing line. Divide the region P1 of the femoral cartilage region (i.e., the part in P with a value of 1) that is less than the y1 index range into cLF-a, and assign the value 1 to this region and store it in the matrix new_P; Divide the region P5 of the tibial cartilage region (i.e., the part in P with a value of 2) that is less than the y1 index range into LT-a, and assign the value 5 to this region and store it in the matrix new_P;

[0059] D. According to the label of the posterior horn of the meniscus (i.e., the part in P with a value of 4), detect the leftmost index position (x2, y2) of the posterior horn region of the meniscus, and use this index position y2 as the dividing line. Divide the region P2 of the femoral cartilage region (i.e., the part in P with a value of 1) that is less than y2 and greater than the y1 index range into cLF-c, and assign the value 2 to the P2 region and store it in the matrix new_P; Divide the region P6 of the tibial cartilage region (i.e., the part in P with a value of 2) that is less than y2 and greater than the y1 index range into LT-c, and assign the value 6 to the P6 region and store it in the matrix new_P; Divide the region P7 of the tibial cartilage region (i.e., the part in P with a value of 2) that is greater than the y2 index range into LF-p, and assign the value 7 to the P7 region and store it in the matrix new_P;

[0060] E. According to the label of the posterior horn of the meniscus (i.e., the part with a P median value of 4), detect the uppermost index position (x3, y3) of the posterior horn region of the meniscus. Based on this index position point, traverse all the points in the femoral cartilage region (i.e., the part with a P median value of 1) that are within the index range greater than y2, and find the position (x4, y4) of the point with the closest distance to the point (x3, y3); calculate the direction vector v1 of the line connecting the point (x3, y3) and the point (x4, y4); traverse all the points in the femoral cartilage region (i.e., the part with a P median value of 1) that are within the index range greater than y2 again, and calculate the direction vector v2 of the line connecting each point to (x3, y3); perform a cross product of the two vectors. When the scalar value obtained from the calculation is positive, it indicates that the point (x4, y4) is in the clockwise direction of the vector v1. If it is negative, it indicates that the point (x4, y4) is in the counterclockwise direction of the vector v1. Thus, divide the points in the femoral cartilage region (i.e., the part with a P median value of 1) that are within the index range greater than y2 into two parts, P3 and P4; finally, calculate the minimum y-direction index values yp3-min and yp4-min of P3 and P4. If yp3-min < yp4-min, then the region P3 is divided into cLF-p, assign the value 3 to the P3 region and store it in the matrix new_P. The region P4 is divided into pLF, assign the value 4 to the P4 region and store it in the matrix new_P. If yp3-min > yp4-min, then the region P4 is divided into cLF-p, assign the value 3 to the P4 region and store it in the matrix new_P. The region P3 is divided into pLF, assign the value 4 to the P3 region and store it in the matrix new_P; F. Finally, automatically divide the overall cartilage region into each sub-region and store it in new_P, where new_P ∈ {0, 1, 2, 3, 4, 5, 6, 7}. A predicted value of 0 represents the background region, a predicted value of 1 represents the cLF-a region, a predicted value of 2 represents the cLF-c region, a predicted value of 3 represents the cLF-p region, a predicted value of 4 represents the pLF region, a predicted value of 5 represents the LT-a region, a predicted value of 6 represents the LT-c region, and a predicted value of 7 represents the LF-p region, as Figure 2 Sagittal plane view along line C.

[0061] The sub-region division of T2 data is the same as that of the TIρ method. A predicted value of 0 represents the background region, a predicted value of 1 represents the cMF-a region, a predicted value of 2 represents the cMF-c region, a predicted value of 3 represents the cMF-p region, and a predicted value of 4 represents the pMF region. A predicted value of 5 represents the LT-a region, a predicted value of 6 represents the LT-c region, and a predicted value of 7 represents the LF-p region, as Figure 2 Sagittal plane view along line D.

[0062] Through the above medical image processing method, the workload of the staff can be reduced and the processing difficulty can be lowered.

[0063] The operations of the above medical image processing method can all be implemented using Python code. Each functional area is encapsulated and the interfaces are connected to achieve one-key image analysis function. As Figure 4 shown, the medical images to be processed are input into the model, and with one-key operation, the integrated Python code is called to perform image segmentation, data extraction, calculate the average value and standard deviation of each sub-region, and finally export and transmit the original image and the final calculation results to the software page. In this way, an interactive software can be formed to enable the "one-key" analysis of the image processing steps, thereby realizing the batch import of medical images, batch segmentation of sub-regions, shortening the segmentation time, and saving labor costs.

[0064] Based on the above embodiments, the present application further provides a medical image processing device. As Figure 7 shown, the device includes a memory and a processor. The memory is used to store computer-executable programs, and the processor is used to call the computer-executable programs to implement the above medical image processing method.

[0065] The processor can be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the method in the above embodiments. For the content of the method, please refer to the foregoing embodiments, and details are not described herein again.

[0066] The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0067] Based on the above embodiments, the present application also provides a computer storage medium, on which a computer program is stored, and the computer program can be executed by a processor to implement the above-mentioned medical image processing method.

[0068] The storage medium can be an internal storage unit, such as a hard disk or a memory. The storage medium can also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0069] For the content of the method, please refer to the foregoing embodiments, and details are not described herein again.

[0070] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations. In the hardware implementation, the division between the functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be executed by several physical components in cooperation. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or be implemented as hardware, or be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that a communication medium typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0071] It should be understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. It should be noted that in this text, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article, or system. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article, or system comprising that element.

[0072] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments. As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A medical image processing method, comprising: Build a medical image segmentation model; Performing a first segmentation on the medical image using the medical image segmentation model to obtain a plurality of regions after the first segmentation of the medical image; The medical image is a functional magnetic resonance imaging image of knee cartilage; each of the regions has a corresponding prediction value, and the prediction values ​​of different regions are different; Prediction value P∈{0, 1, 2, 3, 4}, where the prediction value of 0 indicates the background area, the prediction value of 1 indicates the femoral cartilage area, the prediction value of 2 indicates the tibial cartilage area, the prediction value of 3 indicates the anterior horn area of ​​the meniscus, and the prediction value of 4 indicates the posterior horn area of ​​the meniscus; The predicted segmentation results are expanded and eroded and the connected domain is calculated, and the area with the largest connected domain in each segmentation category is selected to eliminate the erroneous discrete points caused by network misclassification; determining at least one area as a target area to determine a plurality of target areas; According to the predicted value of each target area, the coordinates of the corresponding boundary position are determined; Using the coordinates of the boundary position as a segmentation criterion, and based on the segmentation criterion and the positional relationship between the other target regions in the multiple target regions and the current target region, further segmenting the other target regions in the multiple target regions to obtain multiple sub-regions; Assigning and storing values ​​to each of the sub-regions respectively, and generating an empty matrix to store the assigned values ​​of the sub-regions; Determining the coordinates of the corresponding boundary position according to the predicted value of each target area; Using the coordinates of the boundary position as a segmentation criterion, and based on the segmentation criterion and the positional relationship between the other target regions in the multiple target regions and the current target region, further segmenting the other target regions in the multiple target regions to obtain multiple sub-regions; Assigning and storing values ​​to each of the sub-regions respectively, and generating an empty matrix to store the values ​​assigned to the sub-regions; including: According to the label of the anterior horn of the meniscus, the rightmost index position (x1, y1) of the anterior horn of the meniscus is detected, and the index position y1 is used as the dividing line to divide the area P1 of the femoral cartilage area smaller than the index range of y1 into cLF-a, and the area is assigned a value of 1 and stored in the matrix new_P; the area P5 of the tibial cartilage area smaller than the index range of y1 is divided into LT-a, and the area is assigned a value of 5 and stored in the matrix new_P; According to the label of the posterior horn of the meniscus, the leftmost index position (x2, y2) of the posterior horn of the meniscus is detected, and the index position y2 is used as the dividing line to divide the area P2 of the femoral cartilage area that is smaller than y2 and larger than the index range of y1 into cLF-c, and the P2 area is assigned a value of 2 and stored in the matrix new_P; the tibial cartilage area P6 that is smaller than y2 and larger than the index range of y1 is divided into LT-c, and the P6 area is assigned a value of 6 and stored in the matrix new_P; the tibial cartilage area P7 that is larger than the index range of y2 is divided into LF-p, and the P7 area is assigned a value of 7 and stored in the matrix new_P; According to the label of the posterior horn of the meniscus, detect the uppermost index position (x3, y3) in the posterior horn region of the meniscus, and based on this index position point, traverse all the points in the femoral cartilage region that are greater than the y2 index range to find the position (x4, y4) of the point with the closest distance to the point (x3, y3); calculate the direction vector v1 of the line connecting the point (x3, y3) and the point (x4, y4); traverse all the points in the femoral cartilage region that are greater than the y2 index range again, and calculate the direction vector v2 of the line connecting each point and (x3, y3); when the scalar value obtained by the cross product of the two vectors is positive, it means that the point (x4, y4) is in the clockwise direction of the vector v1, and if it is negative, it means that the point (x4, y4) is in the counterclockwise direction of the vector v1, thus dividing the points in the femoral cartilage region that are greater than the y2 index range into two parts P3 and P4; finally, calculate the minimum y-direction index values yp3-min and yp4-min of P3 and P4. If yp3-min < yp4-min, the region P3 is divided into cLF-p, the P3 region is assigned the value 3 and stored in the matrix new_P, the region P4 is divided into pLF, and the P4 region is assigned the value 4 and stored in the matrix new_P. If yp3-min > yp4-min, the region P4 is divided into cLF-p, the P4 region is assigned the value 3 and stored in the matrix new_P, the region P3 is divided into pLF, and the P3 region is assigned the value 4 and stored in the matrix new_P; Finally, the overall cartilage region is automatically divided into each sub-region and stored in the matrix new_P.

2. The medical image processing method according to claim 1, characterized in that: The construction of the medical image segmentation model includes: Constructing a trainable data set, where the trainable data set includes medical images and data for data annotation of at least part of the medical images; Constructing a deep learning segmentation network model; and Training the deep learning segmentation network model to obtain the medical image segmentation model.

3. The medical image processing method according to claim 2, characterized in that: The construction of the deep learning segmentation network model includes: Designing a U-shaped segmentation network including an encoding network and a decoding network; Extracting features from the trainable data set by the encoding network through convolutional blocks; Restoring the processed medical image data to the original size of the medical image by the decoding network and realizing the fusion of features from coarse to fine to extract the deep feature information of the medical image; and Obtaining a feature map and a predicted segmentation probability map of the medical image that are consistent with the number of segmentation categories based on the deep feature information of the medical image.

4. The medical image processing method according to claim 3, characterized in that: The training of the deep learning segmentation network model includes: Processing the data in the trainable data set to make it have the same resolution and size; Performing online data augmentation processing on the first part of the processed data and inputting the data after online data augmentation processing into the deep learning segmentation network model; Generating a predicted segmentation result through the deep learning segmentation network model, and calculating the loss function between the predicted result and the manually segmented medical image; Using the second part of the processed data except the first part as a validation set to verify the segmentation accuracy of the deep learning segmentation network model; and The data in the trainable data set is used to perform repeated iterations so that the loss function is reduced and converged, and the best deep learning segmentation network model is selected according to the verification result of the verification set.

5. The medical image processing method according to claim 4, characterized in that: The constructing of the medical image segmentation model further includes optimizing the medical image segmentation model, wherein the optimizing of the medical image segmentation model includes: Using the optimal deep learning segmentation network model as an initial model; Inputting an unlabeled trainable dataset into the initial model to obtain a predicted segmented image; Manually modifying the unlabeled trainable data set to reduce data labeling time and obtain more labeled training data; and The labeled training data is input into the initial model for training to obtain an optimized optimal deep learning segmentation network model.

6. The medical image processing method according to claim 4 or 5, characterized in that: The first segmentation of the medical image using the medical image segmentation model to obtain a plurality of regions after the first segmentation of the medical image includes: Converting the medical image data to be processed to the same resolution and size as the data for training the deep learning segmentation network model; inputting the converted medical image data into the medical image segmentation model to obtain a predicted segmentation probability map; and The predicted segmentation probability map is converted into a predicted value, wherein the predicted value has a plurality of different values, and the plurality of different values ​​respectively represent different regions of the medical image.

7. A medical image processing device, comprising a memory and a processor, wherein the memory is used to store a computer-executable program, and the processor is used to call the computer-executable program to implement the medical image processing method according to any one of claims 1 to 6.

8. A computer storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the medical image processing method according to any one of claims 1 to 6.

Images