Medical image segmentation method, device and electronic equipment

Abstract

The present invention discloses a medical image segmentation method, device and electronic device. It relates to the field of medical image segmentation. The method comprises: obtaining a target medical image; identifying the pixel type of each pixel in the target medical image through a target image segmentation model, and performing image segmentation on the target medical image according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, the labels include the pixel type of each pixel in the sample image, the labels are generated based on the real labels of N groups of orthogonal sections in the sample image, N is a positive integer greater than 1, and the image segmentation result is used to formulate a medical plan. The present invention solves the technical problem in the related art that each pixel in the training sample needs to be manually labeled, thereby making the training cost of the image segmentation model high, and further leading to the high cost of medical image segmentation.

Description

Technical Field

[0001] The present invention relates to the field of medical image segmentation, and in particular to a medical image segmentation method, device and electronic equipment. Background Art

[0002] Medical image segmentation is an important medical image processing technology that can separate structures or tissues of interest in medical images from the background. For example, it can achieve accurate segmentation and positioning of structures such as tumors, organs, blood vessels, and bones, thereby helping doctors formulate medical plans.

[0003] At present, there are various medical image segmentation technologies in the field of medical image segmentation, but these technologies usually require a large amount of refined annotated data, which is time-consuming and labor-intensive in the annotation process, resulting in high training costs for image segmentation models, which in turn leads to the problem of high cost of medical image segmentation.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a medical image segmentation method, device and electronic device to at least solve the technical problem in the related art that each pixel in the training sample needs to be manually labeled, thereby making the training cost of the image segmentation model high, and further leading to high medical image segmentation cost.

[0006] According to one aspect of an embodiment of the present invention, a medical image segmentation method is provided, comprising: acquiring a target medical image; identifying the pixel type of each pixel in the target medical image through a target image segmentation model, and performing image segmentation on the target medical image according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: a coronal plane dimension, a cross-sectional dimension, and an overall image dimension, and the label includes the pixel type of each pixel in the sample image, and the label is generated based on the real labels of N groups of orthogonal sections in the sample image, and the real label includes a preset pixel type for each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1, and the image segmentation result is used to formulate a medical plan.

[0007] Furthermore, the medical image segmentation method also includes: obtaining multiple sample images and true labels of N groups of orthogonal sections in each sample image, wherein each group of orthogonal sections includes a first section in the coronal direction and a second section in the transverse direction, and the sections in different groups of orthogonal sections are different; for each sample image, determining 2N initial labels of the sample image based on the N groups of orthogonal sections and the true labels of the sample image, wherein the initial label includes an initial pixel type of each pixel in the sample image, and an initial label has an association with a section in the N groups of orthogonal sections; determining a label of the sample image in the target dimension based on the 2N initial labels of the sample image and the true label of the sample image.

[0008] Furthermore, the true label includes a first true label corresponding to the first section and a second true label corresponding to the second section, wherein the medical image segmentation method also includes: for each group of orthogonal sections of the sample image, the sample image, the first section in the current group of orthogonal sections, and the first true label are input into the label generation model to obtain the first initial label of the sample image; the sample image, the second section in the current group of orthogonal sections, and the second true label are input into the label generation model to obtain the second initial label corresponding to the sample image; the first initial labels and second initial labels of N groups of orthogonal sections are determined as 2N initial labels.

[0009] Furthermore, the medical image segmentation method also includes: determining pixels whose pixel types do not exist in the true label from the sample image to obtain multiple target pixels; when the target dimension is the coronal plane dimension, according to the initial labels corresponding to the first section in the N groups of orthogonal sections in the 2N initial labels, determining the pixel types of the multiple target pixels in the coronal plane dimension, and determining the first label of the sample image in the coronal plane dimension according to the determined pixel types of the multiple target pixels and the true label; when the target dimension is the cross-sectional dimension, according to the initial labels corresponding to the second section in the N groups of orthogonal sections in the 2N initial labels, determining the pixel types of the multiple target pixels in the cross-sectional dimension, and determining the second label of the sample image in the cross-sectional dimension according to the determined pixel types of the multiple target pixels and the true label; when the target dimension is the overall dimension of the image, determining the third label of the sample image in the overall dimension of the image according to the first label and the second label.

[0010] Furthermore, the medical image segmentation method also includes: determining an initial label corresponding to the first slice in the N groups of orthogonal slices from 2N initial labels to obtain N initial labels; for each target pixel, determining the pixel type that matches the target pixel the most times in the N initial labels as the pixel type of the target pixel in the coronal plane dimension, or, for each target pixel, determining the initial label that matches the first slice that is closest to the target pixel to obtain the target initial label, and determining the pixel type that matches the target pixel in the target initial label as the pixel type of the target pixel in the coronal plane dimension.

[0011] Furthermore, the medical image segmentation method also includes: obtaining a first image segmentation model corresponding to the coronal plane dimension, a second image segmentation model corresponding to the cross-sectional dimension, and a third image segmentation model corresponding to the overall image dimension; obtaining multiple unlabeled sample images, and training the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the sample images and the unlabeled sample images, wherein, for each image segmentation model, the loss function value is calculated based on the information of the dimension corresponding to the image segmentation model, the recognition result of the sample image by the image segmentation model, and the recognition results of all image segmentation models for the unlabeled sample image, and the information of the dimension corresponding to the image segmentation model includes one of the following: a first label, a second label, and a third label; when the first image segmentation model, the second image segmentation model, and the third image segmentation model meet the preset iteration conditions, the target image segmentation model is determined from the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the model accuracy.

[0012] Furthermore, the information of the dimension corresponding to the image segmentation model also includes one of the following: a first weight of each pixel in the sample image in the coronal plane dimension, a second weight of each pixel in the sample image in the cross-sectional dimension, and a third weight of each pixel in the sample image in the overall dimension of the image, wherein the medical image segmentation method also includes: for each pixel of each sample image, determining the weight of the pixel in the first section dimension according to the distance between the pixel and the first section and the marking information of the pixel, wherein the marking information characterizes whether the pixel type exists in the true label; determining the weight of the pixel in the second section dimension according to the distance between the pixel and the second section and the marking information of the pixel; determining the first weight of each pixel in the coronal plane dimension according to the weights of all pixels in the sample image in each first section dimension; determining the second weight of each pixel in the sample image in the cross-sectional dimension according to the weights of all pixels in the sample image in each second section dimension; determining the third weight of each pixel in the sample image in the overall dimension of the image according to the first weight and the second weight.

[0013] Furthermore, the information of the dimension corresponding to the image segmentation model also includes one of the following: a first weight of each pixel in the sample image in the coronal dimension, a second weight of each pixel in the sample image in the cross-sectional dimension, and a third weight of each pixel in the sample image in the overall image dimension, wherein the medical image segmentation method also includes: calculating a first loss function value based on the prediction results of the image segmentation model for the sample image and the information of the dimension corresponding to the image segmentation model; calculating a second loss function value based on the prediction results of all image segmentation models for the unlabeled sample image; and determining the loss function value based on the first loss function value and the second loss function value.

[0014] According to another aspect of an embodiment of the present invention, a medical image segmentation device is also provided, including: a first acquisition module, used to acquire a target medical image; a processing module, used to identify the pixel type of each pixel in the target medical image through a target image segmentation model, and perform image segmentation on the target medical image according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: a coronal dimension, a cross-sectional dimension, and an overall image dimension, and the label includes the pixel type of each pixel in the sample image, and the label is generated based on the real labels of N groups of orthogonal sections in the sample image, and the real label includes a preset pixel type for each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1, and the image segmentation result is used to formulate a medical plan.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned medical image segmentation method when running.

[0016] According to another aspect of an embodiment of the present invention, an electronic device is also provided, which includes one or more processors; a memory for storing one or more programs, so that when the one or more programs are executed by the one or more processors, the one or more processors are implemented to run the programs, wherein the programs are configured to execute the above-mentioned medical image segmentation method when running.

[0017] In an embodiment of the present invention, a method is adopted in which labels of all pixels in a sample image are determined based on labels of orthogonal sections of the sample image, thereby training a target image segmentation model. A target medical image is acquired, and then the pixel type of each pixel in the target medical image is identified by the target image segmentation model. The target medical image is segmented according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: a coronal dimension, a cross-sectional dimension, and an overall image dimension. The label includes the pixel type of each pixel in the sample image, and the label is generated based on the true labels of N groups of orthogonal sections in the sample image. The true label includes a preset pixel type for each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1. The image segmentation result is used to formulate a medical plan.

[0018] In the above process, by generating the label of the sample image in the target dimension according to the real label of N groups of orthogonal sections in the sample image, it is avoided to manually label each pixel in the training sample, thereby reducing the labeling cost. Furthermore, by training the target image segmentation model using the sample image and the label of the sample image in the target dimension, and performing image segmentation according to the target image segmentation model, the training cost of the image segmentation model can be effectively reduced, thereby reducing the image segmentation cost. In addition, by generating the label of the sample image in the target dimension according to the real label of N groups of orthogonal sections, the problem of low accuracy of the determined pixel type when the pixel in the image is far away from a certain orthogonal section is avoided, thereby improving the accuracy of the label of the sample image in the target dimension, and then improving the accuracy of image segmentation.

[0019] It can be seen that the solution provided in the present application achieves the purpose of determining the labels of all pixels in the sample image based on the labels of the orthogonal sections of the sample image, thereby training the target image segmentation model, thereby achieving the technical effect of reducing the image segmentation cost, and further solving the technical problem in the related technology that it is necessary to manually label each pixel in the training sample, thereby making the training cost of the image segmentation model high, and thus leading to high medical image segmentation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 is a schematic diagram of an optional medical image segmentation method according to an embodiment of the present invention;

[0022] Figure 2is a schematic diagram of an optional set of orthogonal sections according to an embodiment of the present invention Figure 1 ;

[0023] Figure 3 is a schematic diagram of an optional set of orthogonal sections according to an embodiment of the present invention Figure 2 ;

[0024] Figure 4 is a schematic diagram of an optional set of orthogonal sections according to an embodiment of the present invention Figure 3 ;

[0025] Figure 5 is a schematic diagram of an optional method for determining a target image segmentation model according to an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of an optional distance between a pixel and a section according to an embodiment of the present invention;

[0027] Figure 7 is a schematic diagram of an optional medical image segmentation device according to an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] Example 1

[0033] According to an embodiment of the present invention, an embodiment of a medical image segmentation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Figure 1 is a schematic diagram of an optional medical image segmentation method according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0035] Step S101, acquiring a target medical image.

[0036] Optionally, electronic devices, application systems, servers and other devices can be used as the execution subject of the present application. In this embodiment, the target processing system is used as the execution subject to obtain the target medical image. The target medical image is a three-dimensional image, which can be CT (Computed Tomography)\MRI (Magnetic resonance imaging)\CBCT (Cone Beam CT)\PET (Positron Emission Tomography) and other images.

[0037] Step S102, identifying the pixel type of each pixel in the target medical image through the target image segmentation model, and performing image segmentation on the target medical image according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: coronal dimension, cross-sectional dimension, and overall image dimension, and the label includes the pixel type of each pixel in the sample image, and the label is generated based on the real label of N groups of orthogonal sections in the sample image, and the real label includes the preset pixel type of each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1, and the image segmentation result is used to formulate a medical plan.

[0038] The above-mentioned pixel type may include a first type and a second type, wherein the first type is a type that characterizes that the pixel belongs to the region of interest, and the second type is a type that characterizes that the pixel does not belong to the region of interest. For example, when the region where the pixel is located belongs to the bone, the pixel is determined to be of the first type, and when the region where the pixel is located does not belong to the bone, the pixel is determined to be of the second type. The aforementioned region of interest may also refer to a tumor, an organ, a blood vessel, etc., which is not specifically limited in this embodiment.

[0039] After determining the pixel type, the target image segmentation model can use the image region composed of pixels of the first type as the image segmentation result, and the target image segmentation model can also mark the image region composed of pixels of the first type in the target medical image, thereby using the marked target medical image as the image segmentation result. The image segmentation result can be used for making medical plans, medical image registration, medical image analysis, etc.

[0040] Optionally, the target image segmentation model can be trained based on multiple sample images and labels of each sample image in the target dimension. Wherein, the sample image is a three-dimensional medical image, which can be represented by X∈R L×W×H Represents, R represents a real number set, L, W, H represent the length, width and height of the image respectively, the sample image includes the pixel value of each pixel, and the label includes the pixel type of each pixel in the sample image.

[0041] In order to reduce the labeling cost, in this embodiment, for each sample image, the pixel types of the pixels in the N groups of orthogonal sections of the sample image are marked in advance to obtain the true label, and then the pixel types of all pixels in the sample image in the target dimension are determined based on the true label, so as to determine the label of the sample image in the target dimension based on the obtained result and the true label. Among them, the label of the sample image in a certain dimension can be determined according to the label corresponding to the dimension in the true label. For example, each group of orthogonal sections includes a first section in the coronal direction and a second section in the transverse direction. The label of the coronal dimension is determined according to the label corresponding to the first section in the true label, the label of the transverse dimension is determined according to the label corresponding to the second section in the true label, and the label of the overall dimension of the image is determined according to the overall true label.

[0042] Based on the scheme defined by the above steps S101 to S102, it can be known that in an embodiment of the present invention, the labels of all pixels in the sample image are determined according to the labels of the orthogonal sections of the sample image, so as to train a target image segmentation model. The target medical image is acquired, and then the pixel type of each pixel in the target medical image is identified by the target image segmentation model, and the target medical image is segmented according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained according to multiple sample images and the labels of each sample image in the target dimension, and the target dimension is one of the following: coronal dimension, cross-sectional dimension, and overall image dimension, the label includes the pixel type of each pixel in the sample image, and the label is generated according to the real labels of N groups of orthogonal sections in the sample image, and the real label includes the preset pixel type of each pixel in the N groups of orthogonal sections, N is a positive integer greater than 1, and the image segmentation result is used to formulate a medical plan.

[0043] In the above process, by generating the label of the sample image in the target dimension according to the real label of N groups of orthogonal sections in the sample image, it is avoided to manually label each pixel in the training sample, thereby reducing the labeling cost. Furthermore, by training the target image segmentation model using the sample image and the label of the sample image in the target dimension, and performing image segmentation according to the target image segmentation model, the training cost of the image segmentation model can be effectively reduced, thereby reducing the image segmentation cost. In addition, by generating the label of the sample image in the target dimension according to the real label of N groups of orthogonal sections, the problem of low accuracy of the determined pixel type when the pixel in the image is far away from a certain orthogonal section is avoided, thereby improving the accuracy of the label of the sample image in the target dimension, and then improving the accuracy of image segmentation.

[0044] It can be seen that the solution provided in the present application achieves the purpose of determining the labels of all pixels in the sample image based on the labels of the orthogonal sections of the sample image, thereby training the target image segmentation model, thereby achieving the technical effect of reducing the image segmentation cost, and further solving the technical problem in the related technology that it is necessary to manually label each pixel in the training sample, thereby making the training cost of the image segmentation model high, and thus leading to high medical image segmentation cost.

[0045] In an optional embodiment, the target processing system can generate a label of a sample image in a target dimension in the following manner: obtain multiple sample images and true labels of N groups of orthogonal slices in each sample image, wherein each group of orthogonal slices includes a first slice in a coronal direction and a second slice in a transverse direction, and the slices in different groups of orthogonal slices are different; for each sample image, determine 2N initial labels of the sample image based on the N groups of orthogonal slices and the true labels of the sample image, wherein the initial label includes an initial pixel type of each pixel in the sample image, and an initial label is associated with a slice in the N groups of orthogonal slices; determine the label of the sample image in the target dimension based on the 2N initial labels of the sample image and the true label of the sample image.

[0046] In this embodiment, N=3, each set of orthogonal slices includes a first slice in the coronal direction and a second slice in the transverse direction, and the position of each set of orthogonal slices is determined by the width and height of the image. For example, the positions of 1 / 4, 1 / 2, and 3 / 4 of the width and height are calculated respectively. Figure 2 is a schematic diagram of an optional set of orthogonal sections according to an embodiment of the present invention Figure 1 ,like Figure 2 As shown, the 1 / 4 position of the image width and the 1 / 4 position of the image height are defined as the positions of the first set of orthogonal sections. Figure 3 is a schematic diagram of an optional set of orthogonal sections according to an embodiment of the present invention Figure 2 ,like Figure 3 As shown, the 1 / 2 position of the image width and the 1 / 2 position of the image height are defined as the positions of the second set of orthogonal sections. Figure 4 is a schematic diagram of an optional set of orthogonal sections according to an embodiment of the present invention Figure 3 ,like Figure 4 As shown, the 3 / 4 position of the image width and the 3 / 4 position of the image height are defined as the positions of the third set of orthogonal sections, where: Figure 2 , Figure 3 as well as Figure 4 The L, W, and H in the image represent the length, width, and height of the image, respectively. The information of the three sets of orthogonal sections can be expressed as follows:

[0047] Pos_Slice1=(W 1 / 4 ,H 1 / 4 ) (1)

[0048] Pos_Slice2=(W 1 / 2 ,H 1 / 2 ) (2)

[0049] Pos_Slice3=(W 3 / 4 ,H 3 / 4 ) (3)

[0050] Among them, Pos_Slice i represents the position of the intersection of two sections in the i-th group of orthogonal sections, i∈[1,2,3], W j , H j They represent the j-proportional position of the aforementioned intersection line in the width and height of the image, j∈[1 / 4,1 / 2,3 / 4].

[0051] Optionally, the true label includes a first true label corresponding to the first slice and a second true label corresponding to the second slice. Since each sample image includes three sets of orthogonal slices, each sample image has a total of 6 slices (3 first slices and second slices), and the true label includes 6 labels (3 first true labels and 3 second true labels). The slices and true labels of the sample image can be expressed by formulas (4)-(5):

[0052]

[0053]

[0054] Among them, l represents the lth sample image, Indicates W in the lth sample image j The first section at Indicates W in the lth sample image j The first true label of the first slice at , express and A collection of Indicates H in the lth sample image j The second section at Indicates H in the lth sample image j The second true label of the second slice at , express and A collection of .

[0055] Figure 5 is a schematic diagram of an optional method for determining a target image segmentation model according to an embodiment of the present invention, such as Figure 5As shown, after obtaining multiple sample images and the true labels of N groups of orthogonal sections in each sample image, for each sample image, the target processing system can use the label generation model to determine the initial pixel type of all pixels in the sample image according to a section in the sample image and the true label corresponding to the section (the first true label or the second true label), and obtain an initial label, so that after processing all sections, 2N initial labels are obtained. The 2N initial labels include N initial labels in the coronal plane dimension and N initial labels in the cross-sectional dimension. The initial label in the coronal plane dimension is generated according to the first section, that is, it corresponds to the first section, and the initial label in the cross-sectional dimension is generated according to the second section, that is, it corresponds to the second section.

[0056] Afterwards, the label of the sample image in the target dimension is determined based on the 2N initial labels of the sample image and the true label of the sample image. Figure 5 As shown, the target processing system can determine the first label of the sample image in the coronal dimension based on N initial labels and true labels in the coronal dimension, determine the second label of the sample image in the cross-sectional dimension based on N initial labels and true labels in the cross-sectional dimension, and determine the third label of the sample image in the overall image dimension based on the first label, the second label and the true label.

[0057] It should be noted that by generating 2N initial labels based on 2N slices, and then determining the label of the sample image in the target dimension based on the 2N initial labels and the true label, it is possible to determine the final pixel type of the pixel by combining the information of multiple slice dimensions, thereby improving the accuracy of the determined label of the sample image in the target dimension.

[0058] In an optional embodiment, the true label includes a first true label corresponding to the first section and a second true label corresponding to the second section, wherein, in the process of determining 2N initial labels of the sample image based on N groups of orthogonal sections and true labels of the sample image, the target processing system can, for each group of orthogonal sections of the sample image, input the sample image, the first section in the current group of orthogonal sections, and the first true label into a label generation model to obtain the first initial label of the sample image, and input the sample image, the second section in the current group of orthogonal sections, and the second true label into the label generation model to obtain the second initial label corresponding to the sample image, thereby determining the first initial label and the second initial label of the N groups of orthogonal sections as 2N initial labels.

[0059] Optionally, the label generation model may be a SyN (Symmetric image normalization) model or other models. The target processing system may generate 2N initial labels through 2N slices and corresponding true labels. For example, the first initial label and the second initial label determined by formulas (6)-(7) are used as initial labels:

[0060]

[0061]

[0062] Indicates W in the lth sample image j The first initial label corresponding to the first slice at , SyNRA() represents the processing of the SyN model, X l represents the lth sample image, express and A collection of Indicates H in the lth sample image j The second initial label corresponding to the second section at express and A collection of .

[0063] Among them, the SyN model can align the first section of the sample image with other sections in the coronal direction of the sample image, so as to obtain the displacement field information of the other sections. After that, the first real label corresponding to the first section is processed using the displacement field information to obtain the label information corresponding to the other sections. Therefore, by using the SyN model to continuously align the first section with other sections in the coronal direction of the sample image, the first initial label of the sample image in the first section dimension can be determined. The aforementioned other sections refer to sections other than the first section. Optionally, the process of determining the second initial label is the same as the process of determining the first initial label, so it will not be repeated here.

[0064] It should be noted that by using the label generation model to determine the initial label, on the one hand, the work efficiency of determining the initial label is improved, and on the other hand, the accuracy of the initial label is improved.

[0065] In an optional embodiment, in the process of determining the label of the sample image in the target dimension according to the 2N initial labels of the sample image and the true label of the sample image, the target processing system can determine pixels of the pixel type that do not exist in the true label from the sample image to obtain multiple target pixels, so that when the target dimension is the coronal plane dimension, the pixel types of the multiple target pixels in the coronal plane dimension are determined according to the initial labels corresponding to the first section in the N groups of orthogonal sections in the 2N initial labels, and the first label of the sample image in the coronal plane dimension is determined according to the determined pixel types of the multiple target pixels and the true label; when the target dimension is the cross-sectional dimension, the pixel types of the multiple target pixels in the cross-sectional dimension are determined according to the initial labels corresponding to the second section in the N groups of orthogonal sections in the 2N initial labels, and the second label of the sample image in the cross-sectional dimension is determined according to the determined pixel types of the multiple target pixels and the true label; when the target dimension is the overall dimension of the image, the third label of the sample image in the overall dimension of the image is determined according to the first label and the second label.

[0066] In general, the target pixel is also an unlabeled pixel, which can be understood as a pixel in the sample image that does not belong to the N groups of orthogonal sections.

[0067] In the case where the target dimension is the coronal plane dimension, the target processing system can determine the initial label corresponding to the first section in the N groups of orthogonal sections from the 2N initial labels. In an optional embodiment, it is equivalent to determining the above-mentioned N first initial labels. Afterwards, the target processing system can determine the pixel types of multiple target pixels in the coronal plane dimension based on the obtained initial labels. For example, for each target pixel, the pixel type of the target pixel in the coronal plane dimension is determined according to any label among the N first initial labels. For another example, the pixel type of the target pixel in the coronal plane dimension is determined according to the label of the corresponding section closest to the target pixel among the N first initial labels. For another example, the pixel type of the target pixel with the most matches among the N first initial labels is determined as the pixel type of the target pixel. Afterwards, the target processing system can combine the pixel types of the multiple target pixels determined and the true labels to obtain the first label Y′ of the sample image in the coronal plane dimension. W .

[0068] In the case where the target dimension is a cross-sectional dimension, the target processing system can determine the initial label corresponding to the second section in the N groups of orthogonal sections from the 2N initial labels. In an optional embodiment, this is equivalent to determining the above-mentioned N second initial labels. Afterwards, the target processing system can determine the pixel types of multiple target pixels in the cross-sectional dimension based on the obtained initial labels. For example, for each target pixel, the pixel type of the target pixel in the cross-sectional dimension is determined according to any label among the N second initial labels. For another example, the pixel type of the target pixel in the cross-sectional dimension is determined according to the label of the N second initial labels whose corresponding section is closest to the target pixel. For another example, the pixel type of the target pixel with the most matches among the N second initial labels is determined as the pixel type of the target pixel. Afterwards, the target processing system can combine the pixel types of the multiple determined target pixels with the true labels to obtain the second label Y′ of the sample image in the cross-sectional dimension. H .

[0069] When the target dimension is the overall dimension of the image, the target processing system may first determine the first label Y′ W and the second label Y′ H , then for each pixel in the sample image, if the pixel is in the first label Y′ W and the second label Y′ H If the pixel type in the first label Y′ is consistent, the pixel type is determined as the pixel type of the pixel in the overall dimension of the image. W and the second label Y′ H If the pixel types in the sample image are different, the target pixel type is determined as the pixel type of the pixel in the overall dimension of the image. Thus, after determining the pixel types of all pixels in the sample image in the overall dimension of the image, the third label Y of the sample image in the overall dimension of the image is obtained. T '. The target pixel type may be one of a plurality of pixel types. In this embodiment, the target pixel type may be the second type.

[0070] It should be noted that by determining the first label of the sample image in the coronal dimension according to the initial label corresponding to the coronal dimension, determining the second label of the sample image in the cross-sectional dimension according to the initial label corresponding to the cross-sectional dimension, and determining the third label of the sample image in the overall image dimension according to the first label and the second label, accurate determination of the label of the sample image in the target dimension is achieved.

[0071] In an optional embodiment, in the process of determining the pixel types of multiple target pixels in the coronal plane dimension according to the initial labels corresponding to the first slice in the N groups of orthogonal slices among the 2N initial labels, the target processing system can determine the initial label corresponding to the first slice in the N groups of orthogonal slices from the 2N initial labels to obtain N initial labels, so that for each target pixel, the pixel type that matches the target pixel the most times among the N initial labels is determined as the pixel type of the target pixel in the coronal plane dimension, or, for each target pixel, the initial label that matches the first slice closest to the target pixel is determined to obtain the target initial label, and the pixel type that matches the target pixel in the target initial label is determined as the pixel type of the target pixel in the coronal plane dimension.

[0072] Optionally, the above-mentioned N initial labels are equivalent to the above-mentioned N first initial labels.

[0073] With respect to the first method of determining the pixel type of the target pixel in the coronal plane dimension mentioned above, for example, for each target pixel, assuming N=3, the pixel type of the target pixel in the first first initial label is the first type, the pixel type of the target pixel in the second first initial label is the first type, and the pixel type of the target pixel in the third first initial label is the second type, then the first type is the pixel type that matches the most times for the target pixel in the N initial labels, and the first type is determined as the pixel type of the target pixel in the coronal plane dimension.

[0074] With respect to the above-mentioned second method of determining the pixel type of the target pixel in the coronal plane dimension, for example, assuming that the pixel is at 1 / 5 of the width of the image, the first section at 1 / 4 of the width of the image can be determined as the first section closest to the target pixel, thereby determining the initial label matched by the first section as the target initial label, and determining the pixel type matched by the target pixel in the target initial label as the pixel type of the target pixel in the coronal plane dimension.

[0075] It should be noted that in the first method, since the pixel type with more matches is more likely to be the real pixel type of the target pixel, this method can improve the accuracy of the pixel type of the determined target pixel; in the second method, since the farther the target pixel is from a certain section, the smaller the correlation between the target pixel and the section, the lower the accuracy of the pixel type of the target pixel in the initial label determined according to the section, therefore, this method can also effectively improve the accuracy of the pixel type of the determined target pixel. The above two methods can be selected according to actual application requirements.

[0076] In an optional embodiment, if Figure 5As shown, the target processing system can determine the target image segmentation model in the following manner: obtain a first image segmentation model corresponding to the coronal plane dimension, a second image segmentation model corresponding to the cross-sectional dimension, and a third image segmentation model corresponding to the overall image dimension; obtain multiple unlabeled sample images, and train the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the sample images and the unlabeled sample images, wherein, for each image segmentation model, the loss function value is calculated based on the information of the dimension corresponding to the image segmentation model, the recognition result of the sample image by the image segmentation model, and the recognition result of the unlabeled sample image by all image segmentation models, and the information of the dimension corresponding to the image segmentation model includes one of the following: a first label, a second label, and a third label; when the first image segmentation model, the second image segmentation model, and the third image segmentation model meet the preset iteration conditions, the target image segmentation model is determined from the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the model accuracy.

[0077] Among them, the structures of the first image segmentation model, the second image segmentation model, and the third image segmentation model are the same, and the initial parameters of the first image segmentation model, the second image segmentation model, and the third image segmentation model can be the same or different. The above-mentioned image segmentation model is one of the first image segmentation model, the second image segmentation model, and the third image segmentation model, and the image segmentation model can be a V-Net image segmentation model.

[0078] The target processing system can train the above three image segmentation models in a semi-supervised manner to improve the generalization ability of the models. Therefore, the target processing system can obtain multiple unlabeled sample images, and then train the above three image segmentation models based on the sample images and the unlabeled sample images. Among them, the unlabeled sample images are three-dimensional medical images.

[0079] In each iteration of the training process, the target processing system can first perform supervised training on the three image segmentation models. Figure 5 As shown, for each image segmentation model, multiple sample images are input into the image segmentation model to obtain the recognition result of the pixel type of each pixel in the sample image by the image segmentation model. After that, the image segmentation model can calculate the first loss function value in the supervised training based on the information of the dimension corresponding to the image segmentation model and the recognition result of the sample image by the image segmentation model. For example, if a certain image segmentation model corresponds to the coronal plane dimension, the information of the dimension corresponding to the image segmentation model includes the first label.

[0080] Afterwards, the target processing system can perform unsupervised training on the three image segmentation models. Figure 5As shown, for each image segmentation model, the unlabeled sample image is input into the image segmentation model to obtain the recognition result of the pixel type of each pixel in the unlabeled sample image by the image segmentation model. After that, the image segmentation model can calculate the second loss function value in supervised training based on the recognition result of the unlabeled sample image by the image segmentation model and the recognition result of the unlabeled sample image by other image segmentation models (i.e., image segmentation models other than the current image segmentation model). Thus, the loss function value is determined based on the first loss function value and the second loss function value.

[0081] Furthermore, for each image segmentation model, the image segmentation model can optimize the model parameters according to the determined loss function value when the preset iteration condition is not reached, and perform a new round of iteration based on the image segmentation model after parameter optimization until the preset iteration condition is reached. The preset iteration condition may refer to the number of iterations of the image segmentation model reaching the preset number, or may refer to other iteration conditions.

[0082] Optionally, after the three image segmentation models are trained, Figure 5 As shown, the target processing system can determine the target image segmentation model from the first image segmentation model, the second image segmentation model and the third image segmentation model according to the model accuracy, for example, the image segmentation model with the highest model accuracy is determined as the target image segmentation model. The model accuracy may refer to the accuracy of the image segmentation model at the last iteration, or it may be that when the accuracy of the image segmentation model reaches a preset threshold during the training process, the accuracy of the image segmentation model in each subsequent iteration is recorded, so that the average value of the recorded model accuracy is determined as the final model accuracy of the model.

[0083] It should be noted that by training the first image segmentation model corresponding to the coronal dimension, the second image segmentation model corresponding to the cross-sectional dimension, and the third image segmentation model corresponding to the overall image dimension, not only can the image information be learned from the two perspectives of the coronal and cross-sectional directions, retaining the difference in orthogonal annotations, but also the overall information after the fusion of the coronal plane and the cross-sectional plane can be learned, thereby improving the model performance. By determining the target image segmentation model from the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the training results, the accuracy of image segmentation in practical applications can be further improved.

[0084] In an optional embodiment, the information of the dimension corresponding to the image segmentation model further includes one of the following: a first weight of each pixel in the sample image in the coronal dimension, a second weight of each pixel in the sample image in the cross-sectional dimension, and a third weight of each pixel in the sample image in the overall dimension of the image, wherein Figure 5As shown, the target processing system can determine the first weight, the second weight and the third weight in the following manner: for each pixel of each sample image, determine the weight of the pixel in the first section dimension according to the distance between the pixel and the first section and the labeling information of the pixel, wherein the labeling information characterizes whether the pixel type exists in the true label; determine the weight of the pixel in the second section dimension according to the distance between the pixel and the second section and the labeling information of the pixel; determine the first weight of each pixel in the sample image in the coronal dimension according to the weights of all pixels in the sample image in each first section dimension; determine the second weight of each pixel in the sample image in the cross-sectional dimension according to the weights of all pixels in the sample image in each second section dimension; determine the third weight of each pixel in the sample image in the overall image dimension according to the first weight and the second weight.

[0085] Optionally, for each pixel of each sample image, the target processing system may determine the weight of the pixel in the first slice dimension according to formula (8):

[0086]

[0087] Among them, j∈[1 / 4,1 / 2,3 / 4], k∈[1,L×W×H], Represents the kth pixel in the sample image in the sample image W j The weight of the first section dimension at , r represents the transformation rate, r can be a fixed value, or r can be iteratively updated according to the cosine descent method during the image segmentation model training process, and its initial value can be 0.95, that is, the weight value used for each calculation of the loss function value during the model training process is different, Represents the kth pixel in the sample image and the sample image W j For example, Figure 6 is a schematic diagram of an optional distance between a pixel and a cut plane according to an embodiment of the present invention, Figure 6 In the example, the first slice is at 3 / 4 of the width of the sample image. Figure 6 Medium Pixel

[0088] Optionally, for each pixel of each sample image, the target processing system may determine the weight of the pixel in the second slice dimension according to formula (9):

[0089]

[0090] Among them, j∈[1 / 4,1 / 2,3 / 4], k∈[1,L×W×H], Characterizes the kth pixel in the sample image in the sample image H j The weight of the second slice dimension at , Represents the kth pixel in the sample image and the sample image H j For example, at Figure 6 In the figure, the second slice is at the 3 / 4 ratio of the sample image height. Figure 6 Medium Pixel

[0091] After obtaining the weights of all pixels in the sample image in each first section dimension, such as Figure 5 As shown, the target processing system can add the weights of all pixels in each first section dimension on a pixel-by-pixel basis, that is, for each pixel, add the weights of the pixel in each first section dimension. Then, the result obtained by adding all pixels is normalized to obtain the first weight of each pixel in the coronal plane dimension in the sample image.

[0092] After obtaining the weights of all pixels in the sample image in each second slice dimension, such as Figure 5 As shown, the target processing system can add the weights of all pixels in each second section dimension on a pixel-by-pixel basis, that is, for each pixel, add the weights of the pixel in each second section dimension. Then, the result obtained by adding all pixels is normalized to obtain the second weight of each pixel in the cross-sectional dimension in the sample image.

[0093] After obtaining the first weight and the second weight of each pixel, for each pixel, Figure 5 As shown, the target processing system can add the first weight and the second weight of the pixel correspondingly on a pixel-by-pixel basis, and then normalize the result of adding all pixels to obtain the third weight of each pixel in the sample image in the cross-sectional dimension.

[0094] If a certain image segmentation model corresponds to the coronal plane dimension, the information of the dimension corresponding to the image segmentation model includes the first label and the first weight of each pixel in the sample image in the coronal plane dimension. The weight (first weight / second weight / third weight) is used to characterize the importance of the pixel, and can also be understood as the reliability of the pixel type in the label (first label / second label / third label).

[0095] It should be noted that by determining the weight of the pixel on the coronal plane and the weight of the cross-section according to the weight of the pixel on each section dimension, and determining the weight of the pixel in the overall dimension of the image according to the weight of the pixel on the coronal plane and the weight of the cross-section, it is achieved that the importance of the pixel in each dimension is determined based on the importance of the pixel to each section, thereby achieving the effect of more comprehensive calculation of the importance of the pixel, which is convenient for improving the accuracy of the model.

[0096] In an optional embodiment, the information of the dimension corresponding to the image segmentation model also includes one of the following: a first weight of each pixel in the sample image in the coronal dimension, a second weight of each pixel in the sample image in the cross-sectional dimension, and a third weight of each pixel in the sample image in the overall image dimension, wherein the target processing model can determine the loss function value of the image segmentation model in the following manner: calculating the first loss function value based on the prediction results of the image segmentation model for the sample image and the information of the dimension corresponding to the image segmentation model; calculating the second loss function value based on the prediction results of all image segmentation models for the unlabeled sample image; and determining the loss function value based on the first loss function value and the second loss function value.

[0097] The first loss function value includes the first sub-loss function value and the second sub-loss function value. Each image segmentation model can first calculate the first loss value of each sample image, and then calculate the first sub-loss function value according to the first loss values ​​of all sample images. The first loss value can be calculated by formula (10):

[0098]

[0099] in,* m Indicates the corresponding value in each dimension, A represents the coronal dimension, B represents the cross-sectional dimension, and C represents the overall dimension of the image. Represents the first loss value of the lth sample image in m dimension, represents the weight (first weight / second weight / third weight) of the k-th pixel in the sample image in the m-dimensional space. represents the pixel type in the label (first label / second label / third label) of the k-th pixel in the sample image under the m dimension (for example, 1 represents the first type and 0 represents the second type), It represents the recognition result of the image segmentation model in m dimension for the k-th pixel in the sample image, and the recognition result can represent the probability value that the k-th pixel is of the first type.

[0100] Optionally, each image segmentation model may first calculate the second loss value of each sample image, and then calculate the second sub-loss function value according to the second loss values ​​of all sample images. The second loss value may be calculated by formula (11):

[0101]

[0102] in, Represents the second loss value of the lth sample image in dimension m.

[0103] Optionally, each image segmentation model can first calculate the third loss value of each sample image, and then calculate the second loss function value based on the third loss values ​​of all sample images. The recognition result of the image segmentation model for the k-th pixel in the unlabeled sample image includes a first sub-recognition result and a second sub-recognition result. The first sub-recognition result is a one-hot prediction result. The one-hot prediction result is a vector composed of 0 and 1, with the same length as the number of categories, in which only one element is 1, indicating the pixel type determined by the model for the k-th pixel. The second sub-recognition result represents the probability value of the k-th pixel being of the first type. Figure 5 As shown, the image segmentation model can calculate the third loss value based on its own second sub-recognition result of the pixel in the unlabeled sample image and the first sub-recognition result of the pixel in the unlabeled sample image by other image segmentation models. The calculation method is shown in formula (12):

[0104]

[0105] in, represents the third loss value of the u-th unlabeled sample image in dimension m, represents the second sub-recognition result of the image segmentation model in m dimension for the k-th pixel in the unlabeled sample image, Represents the first sub-recognition result of the image segmentation model in n dimensions for the k-th pixel in the unlabeled sample image.

[0106] After determining the first loss function value and the second loss function value, the image segmentation model can add the first loss function value and the second loss function value to obtain the loss function value, or perform weighted summation of the first loss function value and the second loss function value to obtain the loss function value, or calculate the loss function value according to other calculation formulas.

[0107] It should be noted that during the training process, the loss function value of the model is calculated by combining the recognition results of the model itself with the recognition results of other models, so that the model can better learn image information, thereby further improving the accuracy of the model.

[0108] It can be seen that the solution provided in the present application achieves the purpose of determining the labels of all pixels in the sample image based on the labels of the orthogonal sections of the sample image, thereby training the target image segmentation model, thereby achieving the technical effect of reducing the image segmentation cost, and further solving the technical problem in the related technology that it is necessary to manually label each pixel in the training sample, thereby making the training cost of the image segmentation model high, and thus leading to high medical image segmentation cost.

[0109] Example 2

[0110] According to an embodiment of the present invention, an embodiment of a medical image segmentation device is provided, wherein: Figure 7 is a schematic diagram of an optional medical image segmentation device according to an embodiment of the present invention, such as Figure 7 As shown, the device comprises:

[0111] A first acquisition module 701, used to acquire a target medical image;

[0112] The processing module 702 is used to identify the pixel type of each pixel in the target medical image through the target image segmentation model, and perform image segmentation on the target medical image according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: coronal dimension, cross-sectional dimension, and overall image dimension. The label includes the pixel type of each pixel in the sample image, and the label is generated based on the real label of N groups of orthogonal sections in the sample image. The real label includes the preset pixel type of each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1. The image segmentation result is used to formulate a medical plan.

[0113] It should be noted that the above-mentioned first acquisition module 701 and processing module 702 correspond to steps S101 to S102 in the above-mentioned embodiment. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0114] Optionally, the medical image segmentation device also includes: a second acquisition module, used to acquire multiple sample images and the true labels of N groups of orthogonal sections in each sample image, wherein each group of orthogonal sections includes a first section in the coronal direction and a second section in the transverse direction, and the sections in different groups of orthogonal sections are different; a first determination module, used to determine, for each sample image, 2N initial labels of the sample image based on the N groups of orthogonal sections and the true labels of the sample image, wherein the initial label includes an initial pixel type of each pixel in the sample image, and an initial label is associated with a section in the N groups of orthogonal sections; a second determination module, used to determine the label of the sample image in the target dimension based on the 2N initial labels of the sample image and the true label of the sample image.

[0115] Optionally, the first determination module also includes: a first processing submodule, used for, for each group of orthogonal sections of the sample image, inputting the sample image, the first section in the current group of orthogonal sections, and the first true label into a label generation model to obtain a first initial label for the sample image; a second processing submodule, used for inputting the sample image, the second section in the current group of orthogonal sections, and the second true label into the label generation model to obtain a second initial label corresponding to the sample image; the first determination submodule, used to determine the first initial labels and second initial labels of N groups of orthogonal sections as 2N initial labels.

[0116] Optionally, the second determination module also includes: a second determination submodule, used to determine pixels of a pixel type that does not exist in the true label from the sample image to obtain multiple target pixels; a third determination submodule, used to determine the pixel types of multiple target pixels in the coronal plane dimension according to the initial label corresponding to the first section in the N groups of orthogonal sections in the 2N initial labels when the target dimension is the coronal plane dimension, and determine the first label of the sample image in the coronal plane dimension according to the determined pixel types of the multiple target pixels and the true label; a fourth determination submodule, used to determine the pixel types of multiple target pixels in the cross-sectional dimension according to the initial label corresponding to the second section in the N groups of orthogonal sections in the 2N initial labels when the target dimension is the cross-sectional dimension, and determine the second label of the sample image in the cross-sectional dimension according to the determined pixel types of the multiple target pixels and the true label; a fifth determination submodule, used to determine the third label of the sample image in the overall dimension of the image according to the first label and the second label when the target dimension is the overall dimension of the image.

[0117] Optionally, the third determination submodule also includes: a first determination unit, used to determine the initial label corresponding to the first slice in the N groups of orthogonal slices from 2N initial labels, to obtain N initial labels; a second determination unit, used to determine, for each target pixel, the pixel type that matches the target pixel the most times in the N initial labels as the pixel type of the target pixel in the coronal plane dimension, or, a third determination unit, used to determine, for each target pixel, the initial label that matches the first slice that is closest to the target pixel, to obtain the target initial label, and determine the pixel type that matches the target pixel in the target initial label as the pixel type of the target pixel in the coronal plane dimension.

[0118] Optionally, the medical image segmentation device also includes: a third acquisition module, used to acquire a first image segmentation model corresponding to the coronal plane dimension, a second image segmentation model corresponding to the cross-sectional dimension, and a third image segmentation model corresponding to the overall image dimension; a training module, used to acquire multiple unlabeled sample images, and train the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the sample images and the unlabeled sample images, wherein, for each image segmentation model, the loss function value is calculated based on the information of the dimension corresponding to the image segmentation model, the recognition result of the sample image by the image segmentation model, and the recognition results of all image segmentation models for the unlabeled sample image, and the information of the dimension corresponding to the image segmentation model includes one of the following: a first label, a second label, and a third label; a third determination module, used to determine the target image segmentation model from the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the model accuracy when the first image segmentation model, the second image segmentation model, and the third image segmentation model meet the preset iteration conditions.

[0119] Optionally, the medical image segmentation device also includes: a fourth determination module, which is used to determine the weight of the pixel in the first section dimension for each pixel of each sample image according to the distance between the pixel and the first section and the labeling information of the pixel, wherein the labeling information characterizes whether the pixel type exists in the real label; a fifth determination module, which is used to determine the weight of the pixel in the second section dimension according to the distance between the pixel and the second section and the labeling information of the pixel; a sixth determination module, which is used to determine the first weight of each pixel in the sample image in the coronal dimension according to the weights of all pixels in the sample image in each first section dimension; a seventh determination module, which is used to determine the second weight of each pixel in the sample image in the cross-sectional dimension according to the weights of all pixels in the sample image in each second section dimension; and an eighth determination module, which is used to determine the third weight of each pixel in the sample image in the overall image dimension according to the first weight and the second weight.

[0120] Optionally, the medical image segmentation device also includes: a first calculation module, used to calculate the first loss function value based on the prediction results of the image segmentation model for the sample image and the information of the dimension corresponding to the image segmentation model; a second calculation module, used to calculate the second loss function value based on the prediction results of all image segmentation models for the unlabeled sample image; and a ninth determination module, used to determine the loss function value based on the first loss function value and the second loss function value.

[0121] Example 3

[0122] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned medical image segmentation method when running.

[0123] Example 4

[0124] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, wherein: Figure 8 is a schematic diagram of an optional electronic device according to an embodiment of the present invention, such as Figure 8 As shown, the electronic device includes one or more processors; a memory for storing one or more programs, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to run the programs, wherein the programs are configured to execute the above-mentioned medical image segmentation method when running.

[0125] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0126] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0131] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A medical image segmentation method, characterized in that: include: Acquire a target medical image; The target image segmentation model is used to identify the pixel type of each pixel in the target medical image, and the target medical image is segmented according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: a coronal dimension, a cross-sectional dimension, and an overall image dimension. The label includes the pixel type of each pixel in the sample image, and the label is generated based on the real labels of N groups of orthogonal sections in the sample image. The real label includes the preset pixel type of each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1. The image segmentation result is used to formulate a medical plan; The labels of the sample images in the target dimension are generated in the following way: Acquire the multiple sample images and true labels of N groups of orthogonal slices in each sample image, wherein each group of orthogonal slices includes a first slice in a coronal direction and a second slice in a transverse direction, and slices in different groups of orthogonal slices are different; For each sample image, according to the N groups of orthogonal sections of the sample image and the true label, determine 2N initial labels of the sample image, wherein the initial label includes an initial pixel type of each pixel in the sample image, and an initial label is associated with one of the N groups of orthogonal sections; Determine from the sample image pixels of a type that does not exist in the true label, and obtain a plurality of target pixels; In the case where the target dimension is the coronal plane dimension, according to the initial labels among the 2N initial labels corresponding to the first section in the N groups of orthogonal sections, the pixel types of the multiple target pixels in the coronal plane dimension are determined, and according to the determined pixel types of the multiple target pixels and the true label, the first label of the sample image in the coronal plane dimension is determined; In the case where the target dimension is the cross-sectional dimension, determining the pixel types of the multiple target pixels in the cross-sectional dimension according to the initial labels corresponding to the second section in the N groups of orthogonal sections among the 2N initial labels, and determining the second label of the sample image in the cross-sectional dimension according to the determined pixel types of the multiple target pixels and the true label; When the target dimension is the overall dimension of the image, a third label of the sample image in the overall dimension of the image is determined according to the first label and the second label.

2. The method according to claim 1, characterized in that The real label includes a first real label corresponding to the first section and a second real label corresponding to the second section, wherein, according to the N groups of orthogonal sections of the sample image and the real labels, determining 2N initial labels of the sample image includes: For each group of orthogonal slices of the sample image, input the sample image, the first slice in the current group of orthogonal slices, and the first true label into a label generation model to obtain a first initial label of the sample image; Inputting the sample image, the second slice in the current group of orthogonal slices, and the second true label into the label generation model to obtain a second initial label corresponding to the sample image; The first initial labels and the second initial labels of the N groups of orthogonal sections are determined as the 2N initial labels.

3. The method according to claim 1, characterized in that Determining the pixel types of the plurality of target pixels in the coronal plane dimension according to the initial labels corresponding to the first section in the N groups of orthogonal sections among the 2N initial labels comprises: Determine an initial label corresponding to the first slice in the N groups of orthogonal slices from the 2N initial labels to obtain N initial labels; For each target pixel, the pixel type with the largest number of matches among the N initial labels is determined as the pixel type of the target pixel in the coronal plane dimension, or, For each target pixel, determine the initial label that matches the first slice closest to the target pixel to obtain the target initial label, and determine the pixel type that matches the target pixel in the target initial label as the pixel type of the target pixel in the coronal plane dimension.

4. The method according to claim 1, characterized in that: The target image segmentation model is obtained by: Acquire a first image segmentation model corresponding to the coronal dimension, a second image segmentation model corresponding to the cross-sectional dimension, and a third image segmentation model corresponding to the overall dimension of the image; Acquire multiple unlabeled sample images, and train the first image segmentation model, the second image segmentation model, and the third image segmentation model based on the sample images and the unlabeled sample images, wherein for each image segmentation model, calculate the loss function value based on the information of the dimension corresponding to the image segmentation model, the recognition result of the sample image by the image segmentation model, and the recognition results of the unlabeled sample images by all image segmentation models, and the information of the dimension corresponding to the image segmentation model includes one of the following: the first label, the second label, and the third label; When the first image segmentation model, the second image segmentation model and the third image segmentation model meet the preset iteration conditions, the target image segmentation model is determined from the first image segmentation model, the second image segmentation model and the third image segmentation model according to the model accuracy.

5. The method according to claim 4, characterized in that The information of the dimension corresponding to the image segmentation model also includes one of the following: a first weight of each pixel in the sample image in the coronal dimension, a second weight of each pixel in the sample image in the cross-sectional dimension, and a third weight of each pixel in the sample image in the overall dimension of the image, wherein the first weight, the second weight, and the third weight are determined in the following manner: For each pixel of each sample image, determine the weight of the pixel in the first section dimension according to the distance between the pixel and the first section and the label information of the pixel, wherein the label information represents whether the pixel type exists in the true label; Determining a weight of the pixel in a dimension of the second section according to a distance between the pixel and the second section and marking information of the pixel; Determine a first weight of each pixel in the sample image in the coronal plane dimension according to the weights of all pixels in the sample image in each first section dimension; Determine a second weight of each pixel in the sample image in the cross-sectional dimension according to the weights of all pixels in the sample image in each second section dimension; A third weight of each pixel in the sample image in the overall dimension of the image is determined according to the first weight and the second weight.

6. The method according to claim 4, characterized in that The information of the dimension corresponding to the image segmentation model also includes one of the following: a first weight of each pixel in the sample image in the coronal dimension, a second weight of each pixel in the sample image in the cross-sectional dimension, and a third weight of each pixel in the sample image in the overall dimension of the image, wherein the loss function value of the image segmentation model is determined by: Calculating a first loss function value according to a prediction result of the sample image by the image segmentation model and information of a dimension corresponding to the image segmentation model; Calculating a second loss function value according to the prediction results of all image segmentation models on the unlabeled sample image; The loss function value is determined according to the first loss function value and the second loss function value.

7. A medical image segmentation device, characterized in that: include: A first acquisition module, used for acquiring a target medical image; A processing module, used for identifying the pixel type of each pixel in the target medical image through a target image segmentation model, and performing image segmentation on the target medical image according to the pixel type of the pixel to obtain an image segmentation result, wherein the target image segmentation model is trained based on multiple sample images and labels of each sample image in a target dimension, and the target dimension is one of the following: a coronal plane dimension, a cross-sectional dimension, and an overall image dimension, and the label includes the pixel type of each pixel in the sample image, and the label is generated based on the real labels of N groups of orthogonal sections in the sample image, and the real label includes the preset pixel type of each pixel in the N groups of orthogonal sections, and N is a positive integer greater than 1, and the image segmentation result is used to formulate a medical plan; The medical image segmentation device also includes: A second acquisition module is used to acquire the multiple sample images and the true labels of N groups of orthogonal slices in each sample image, wherein each group of orthogonal slices includes a first slice in the coronal direction and a second slice in the transverse direction, and the slices in different groups of orthogonal slices are different; A first determination module is used to determine, for each sample image, 2N initial labels of the sample image according to the N groups of orthogonal sections of the sample image and the true label, wherein the initial label includes an initial pixel type of each pixel in the sample image, and an initial label is associated with one of the N groups of orthogonal sections; A second determination submodule is used to determine pixels of a type that does not exist in the true label from the sample image, to obtain a plurality of target pixels; A third determination submodule is used for determining, when the target dimension is the coronal plane dimension, pixel types of the multiple target pixels in the coronal plane dimension according to the initial labels corresponding to the first section in the N groups of orthogonal sections among the 2N initial labels, and determining a first label of the sample image in the coronal plane dimension according to the determined pixel types of the multiple target pixels and the true label; a fourth determination submodule, configured to determine, when the target dimension is the cross-sectional dimension, pixel types of the plurality of target pixels in the cross-sectional dimension according to an initial label in the 2N initial labels corresponding to a second section in the N groups of orthogonal sections, and determine a second label of the sample image in the cross-sectional dimension according to the determined pixel types of the plurality of target pixels and the true label; A fifth determination submodule is used to determine a third label of the sample image in the overall dimension of the image according to the first label and the second label when the target dimension is the overall dimension of the image.

8. An electronic device, characterized in that: The electronic device includes one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to run the programs, wherein the programs are configured to execute the medical image segmentation method described in any one of claims 1 to 6 when run.

Images