Training method of image segmentation model, image segmentation method and device

Abstract

The application relates to a training method, an image segmentation method and an image segmentation device of an image segmentation model. The method comprises the following steps: acquiring sample data; the sample data comprises a medical image of a sample object and an initial segmentation region of the sample object; wherein the initial segmentation region comprises an initial outermost segmentation region, and the initial segmentation region is used for representing a preset segmentation region of the sample object; generating a pseudo segmentation region according to the initial outermost segmentation region, as a target outermost segmentation region; taking the medical image as an input, taking the initial outermost segmentation region and the target outermost segmentation region as supervision information, training a to-be-trained image segmentation model, and obtaining a trained image segmentation model. The method can solve the problem that an actual segmentation region of an outermost image does not match a required segmentation region.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a training method for an image segmentation model, an image segmentation method, an apparatus, a computer device, a storage medium, and a computer program product. Background Technology

[0002] With the development of computer-aided technology in the field of medical imaging, more and more medical personnel are beginning to use automated computer algorithms to assist in the analysis of medical images. For example, image segmentation algorithms are used to segment medical images to solve complex and time-consuming medical image segmentation problems.

[0003] However, when using traditional image segmentation algorithms in the computer field to segment medical images, a mismatch occurs between the actual segmented region and the required segmented region in the top and bottom images of the cross-section of the segmented area. Summary of the Invention

[0004] Based on this, it is necessary to address the technical problem that after the above image segmentation algorithm performs segmentation, the actual segmented region and the required segmented region will not match in the top and bottom images of the cross-section of the segmented part. Therefore, it is necessary to provide a training method, image segmentation method, device, computer equipment, computer-readable storage medium and computer program product for an image segmentation model.

[0005] Firstly, this application provides a method for training an image segmentation model. The method includes:

[0006] Acquire sample data; the sample data includes a medical image of the sample object and an initial segmentation region of the sample object; wherein, the initial segmentation region includes an initial outermost segmentation region, which is used to represent a preset segmentation region for the sample object;

[0007] A pseudo-segmentation region is generated based on the initial outermost segmentation region, which serves as the target outermost segmentation region;

[0008] Using the medical image as input, and the initial outermost segmentation region and the target outermost segmentation region as supervision information, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0009] In one embodiment, the step of training the image segmentation model to be trained using the medical image as input and the initial outermost segmentation region and the target outermost segmentation region as supervision information to obtain the trained image segmentation model includes:

[0010] The medical image is input into the image segmentation model to be trained to obtain the predicted outermost segmentation region and the predicted second outermost segmentation region.

[0011] Obtain the first overlapping region between the predicted outermost segmentation region and the target outermost segmentation region, and the second overlapping region between the predicted second outermost segmentation region and the initial outermost segmentation region;

[0012] Obtain the total region consisting of the predicted outermost segmentation region, the target outermost segmentation region, the predicted second outermost segmentation region, and the initial outermost segmentation region;

[0013] Based on the first overlapping region, the second overlapping region, and the total region, a loss value is obtained. The image segmentation model to be trained is then trained based on the loss value to obtain a trained image segmentation model.

[0014] In one embodiment, obtaining the loss value based on the first overlapping region, the second overlapping region, and the total region includes:

[0015] The first overlapping region and the second overlapping region are added together to obtain the total overlapping region;

[0016] Obtain the ratio between the total overlapping area and the total area, and based on the ratio, obtain the loss value.

[0017] In one embodiment, generating a pseudo-segmentation region based on the initial outermost segmentation region as the target outermost segmentation region includes:

[0018] Determine the gradual trend from the initial sub-outermost segmentation region to the initial outermost segmentation region;

[0019] The initial outermost segmented region is adjusted according to the gradient trend to obtain the pseudo-segmented region, which serves as the target outermost segmented region.

[0020] In one embodiment, the initial segmentation region further includes an initial intermediate layer segmentation region; the method further includes:

[0021] Using the medical image as input, and the initial outermost segmentation region, the target outermost segmentation region, and the initial intermediate segmentation region as supervision information, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0022] Secondly, this application also provides an image segmentation method. The method includes:

[0023] Obtain the medical image of the object to be segmented;

[0024] The medical image is segmented using an image segmentation model to obtain a multi-layer image of the object to be segmented.

[0025] The outermost image is removed from the multi-layer image to obtain the image segmentation result of the object to be segmented;

[0026] The image segmentation model is trained using a medical image of the sample object as input and the initial outermost segmentation region and the target outermost segmentation region of the sample object as supervised information; the target outermost segmentation region is generated based on the initial outermost segmentation region.

[0027] Thirdly, this application also provides a training apparatus for an image segmentation model. The apparatus includes:

[0028] A sample acquisition module is used to acquire sample data; the sample data includes a medical image of the sample object and an initial segmentation region of the sample object; wherein, the initial segmentation region includes an initial outermost segmentation region, and the initial segmentation region is used to represent a preset segmentation region for the sample object;

[0029] The pseudo-region generation module is used to generate a pseudo-segmentation region based on the initial outermost segmentation region, which serves as the target outermost segmentation region.

[0030] The model training module is used to train the image segmentation model to be trained by taking the medical image as input and the initial outermost segmentation region and the target outermost segmentation region as supervision information, so as to obtain the trained image segmentation model.

[0031] Fourthly, this application also provides an image segmentation apparatus. The apparatus includes:

[0032] The image acquisition module is used to acquire medical images of the object to be segmented.

[0033] The image segmentation module is used to perform image segmentation processing on the medical image using an image segmentation model to obtain a multi-layer image of the object to be segmented; the image segmentation model is trained by taking the medical image of the sample object as input and using the initial outermost segmentation region and the target outermost segmentation region of the sample object as supervised information; the target outermost segmentation region is generated based on the initial outermost segmentation region;

[0034] The image removal module is used to remove the outermost image from the multi-layer image to obtain the image segmentation result of the object to be segmented.

[0035] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0036] Acquire sample data; the sample data includes a medical image of the sample object and an initial segmentation region of the sample object; wherein, the initial segmentation region includes an initial outermost segmentation region, which is used to represent a preset segmentation region for the sample object;

[0037] A pseudo-segmentation region is generated based on the initial outermost segmentation region, which serves as the target outermost segmentation region;

[0038] Using the medical image as input, and the initial outermost segmentation region and the target outermost segmentation region as supervision information, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0039] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0040] Acquire sample data; the sample data includes a medical image of the sample object and an initial segmentation region of the sample object; wherein, the initial segmentation region includes an initial outermost segmentation region, which is used to represent a preset segmentation region for the sample object;

[0041] A pseudo-segmentation region is generated based on the initial outermost segmentation region, which serves as the target outermost segmentation region;

[0042] Using the medical image as input, and the initial outermost segmentation region and the target outermost segmentation region as supervision information, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0043] Seventhly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0044] Acquire sample data; the sample data includes a medical image of the sample object and an initial segmentation region of the sample object; wherein, the initial segmentation region includes an initial outermost segmentation region, which is used to represent a preset segmentation region for the sample object;

[0045] A pseudo-segmentation region is generated based on the initial outermost segmentation region, which serves as the target outermost segmentation region;

[0046] Using the medical image as input, and the initial outermost segmentation region and the target outermost segmentation region as supervision information, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0047] The aforementioned image segmentation model training method, image segmentation method, device, computer equipment, storage medium, and computer program product generate a pseudo-segmentation region based on the initial outermost segmentation region of the sample object, which serves as the target outermost segmentation region. Using a medical image as input and the initial and target outermost segmentation regions as supervisory information, the image segmentation model to be trained is trained. Since the segmentation regions involved in the training include both actual and pseudo-segmentation regions, the trained image segmentation model can predict more image layers than the segmentable object. Consequently, during actual image segmentation, the outermost image in the segmented multi-layered images can be removed, thus solving the problem of mismatch between the actual segmentation region and the required segmentation region of the outermost image while ensuring that the number of image layers of the segmented object meets the requirements. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of a multi-layered segmented region obtained by a conventional image segmentation method in one embodiment.

[0049] Figure 2 This is a schematic diagram of the initial segmentation region and the actual segmentation region obtained for the tongue portion in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the training method of an image segmentation model in one embodiment;

[0051] Figure 4 This is a schematic diagram illustrating the correspondence between preset segmentation regions and predicted segmentation regions in one embodiment;

[0052] Figure 5 This is a flowchart illustrating an image segmentation method in one embodiment;

[0053] Figure 6 This is a structural block diagram of a training device for an image segmentation model in one embodiment;

[0054] Figure 7 This is a structural block diagram of an image segmentation device in one embodiment;

[0055] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0058] Currently, commonly used medical image segmentation algorithms often encounter a mismatch between the actual segmented region and the required shape at the top and bottom layers (the first and last layers of the predicted results along the Z-axis of the 3D medical image). Figure 1 As shown in the figure, this affects the algorithm's performance. Furthermore, image post-processing algorithms such as the maximum connected component method cannot solve this problem well because the segmented parts are connected in 3D space.

[0059] refer to Figure 2 This is a schematic diagram illustrating the initial segmentation region and the actual segmentation region of the tongue portion, as shown in an exemplary embodiment. Figure 2 Region 20 in the left image can represent the initial segmentation region, such as... Figure 2 As shown, the initial segmented region is a regular and complete region. However, the actual segmented region obtained may be as follows: Figure 2 The region 22 shown in the right-hand image has multiple irregular shapes and some fragments, which do not match the initial segmentation region 20. Moreover, this mismatch between the actual segmentation region and the initial segmentation region often occurs in the top and bottom images of the segmented object.

[0060] Therefore, this application proposes a training method for an image segmentation model to address the problem of mismatch between the actual segmentation regions and the initial segmentation regions of the top and bottom images. This method trains the image segmentation model by generating pseudo-regions, enabling the model to predict more image layers than the segmentable object. During actual image segmentation, the top and bottom images of the segmented multi-layered images can be removed. Since irregularly shaped fragments only appear at the top and bottom layers of the predicted mask images, the mask fragments output by the model after adding the generated pseudo-regions are actually outside the true segmentation region. Removing the extra predicted mask layers solves the fragment problem without significantly affecting the accurate prediction of the true segmentation region. This method solves the problem of mismatch between the actual segmentation regions and the initial segmentation regions of the top and bottom images while ensuring that the number of image layers of the segmented object meets the requirements.

[0061] In one embodiment, such as Figure 3 As shown, a training method for an image segmentation model is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0062] Step S310: Obtain sample data; the sample data includes a medical image of the sample object and the initial segmentation region of the sample object; wherein, the initial segmentation region includes the initial outermost segmentation region, which is used to represent the preset segmentation region for the sample object.

[0063] The sample objects can be organs or tissues, such as the tongue or heart.

[0064] Medical images refer to images of a sample object obtained through medical equipment. For example, medical images can be computed tomography (CT) images, magnetic resonance (MRI) images, etc.

[0065] The initial segmentation region can be understood as the segmentation region that medical personnel want to obtain, for example, reference. Figure 2 The image shown is of the tongue region. Figure 2 Region 20 in the left image can represent the initial segmentation region.

[0066] It is understandable that, since organs or tissues are 3D objects, when they are segmented, multiple layers of images can be obtained. Each layer of image can correspond to an initial segmentation region, that is, the initial segmentation region can include the initial outermost segmentation region and the initial intermediate segmentation region. The initial outermost segmentation region can include the initial top segmentation region and the initial bottom segmentation region.

[0067] In practice, medical images of the sample object can be obtained by scanning with a computer or by using a magnetic resonance imaging device. An initial segmentation region can be preset for each layer of the image that can be segmented from the sample object. The initial segmentation regions of each layer of the image can have a gradual trend. For example, from the bottom layer to the top layer, the initial segmentation region gradually increases, gradually decreases, increases first and then decreases, or decreases first and then increases.

[0068] Step S320: Generate a pseudo-segmentation region based on the initial outermost segmentation region, and use it as the target outermost segmentation region.

[0069] In this context, the pseudo-segmentation region refers to the region outside the true initial segmentation region.

[0070] In practice, since the organs and tissues used as sample objects may not be uniformly distributed, the segmentation regions required for different layers may not be the same. Therefore, before generating pseudo-segmentation regions based on the initial outermost segmentation region, the gradient trend from the initial outermost segmentation region to the pseudo-segmentation region can be determined first. The initial outermost segmentation region can be adjusted according to this gradient trend to obtain the pseudo-segmentation region.

[0071] Step S330: Using a medical image as input and the initial outermost segmentation region and the target outermost segmentation region as supervision information, train the image segmentation model to be trained to obtain the trained image segmentation model.

[0072] The image segmentation model can be a 3D Unet model.

[0073] In practice, after generating the target top-level segmentation region based on the initial outermost segmentation region, the medical image can be input into the image segmentation model to be trained to obtain the predicted outermost segmentation region and the predicted second outermost segmentation region. Using the initial outermost segmentation region and the target outermost segmentation region as supervision information, the loss value is calculated. Based on the loss value, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0074] In the training method of the above image segmentation model, a pseudo-segmentation region is generated based on the initial outermost segmentation region of the sample object, which serves as the target outermost segmentation region. Using medical images as input and the initial and target outermost segmentation regions as supervision information, the image segmentation model to be trained is trained. Since the segmentation regions involved in the training include both actual and pseudo-segmentation regions, the trained image segmentation model can predict more image layers than the segmentable object. Therefore, during actual image segmentation, the outermost image in the segmented multi-layered images can be removed, thus solving the problem of mismatch between the actual segmentation region and the required segmentation region of the outermost image while ensuring that the number of image layers of the segmented object meets the requirements.

[0075] In an exemplary embodiment, step S330 described above can be implemented through the following steps:

[0076] Step S3301: Input the medical image into the image segmentation model to be trained to obtain the predicted outermost segmentation region and the predicted second outermost segmentation region;

[0077] Step S3302: Obtain the first overlapping region between the predicted outermost segmentation region and the target outermost segmentation region, and the second overlapping region between the predicted second outermost segmentation region and the initial outermost segmentation region;

[0078] Step S3303: Obtain the total region consisting of the predicted outermost segmentation region, the target outermost segmentation region, the predicted second outermost segmentation region, and the initial outermost segmentation region.

[0079] Step S3304: Based on the first overlapping region, the second overlapping region, and the total region, obtain the loss value, and train the image segmentation model to be trained based on the loss value to obtain the trained image segmentation model.

[0080] Furthermore, in an exemplary embodiment, step S3304 further includes: adding the first overlapping region and the second overlapping region to obtain the total overlapping region; obtaining the ratio between the total overlapping region and the total region; and obtaining the loss value based on the ratio.

[0081] The loss value is used to measure the difference between the model's prediction and the actual region. The smaller the loss value, the better the model's prediction. The training process of the model is the process of optimizing the loss value.

[0082] In the specific implementation, the medical image is input into the image segmentation model to obtain the predicted segmentation regions of each layer, specifically including the predicted outermost segmentation region and the predicted second outermost segmentation region. The predicted segmentation regions of each layer and the preset segmentation regions (including the initial outermost segmentation region and the target outermost segmentation region) are substituted into the loss function to calculate the first overlap region between the predicted outermost segmentation region and the target outermost segmentation region, and the second overlap region between the predicted second outermost segmentation region and the initial outermost segmentation region. The total region composed of the predicted outermost segmentation region, the target outermost segmentation region, the predicted second outermost segmentation region, and the initial outermost segmentation region is obtained. Finally, the ratio of the total overlap region obtained by adding the first overlap region and the second overlap region to the total region is calculated to obtain the loss value. When the training loss value reaches the preset accuracy or the training times reach the preset number of times, training is stopped, and the trained image segmentation model is obtained.

[0083] The training loss function can be determined by combining BCE loss (Binary Cross Entropy) and Dice loss (a set similarity measure). The loss function relationship can be expressed as:

[0084]

[0085] Where DSC represents the loss value, X i Y represents the predicted segmentation region of the i-th layer. i X represents the preset segmentation region of the i-th layer (which can be understood as a segmentation label, including the initial segmentation region and the target outermost segmentation region). i ∩Y i X1 can represent the overlapping region between the predicted segmentation region of the i-th layer and the preset segmentation region. For example, when i = 1, X1 ∩ Y1 represents the overlapping region between the predicted segmentation region of the first layer and the preset segmentation region; when i = N, X1 ∩ Y1 represents the overlapping region between the predicted segmentation region of the first layer and the preset segmentation region. N ∩Y N This represents the overlapping region between the predicted segmentation region of layer N and the preset segmentation region. Since 1 and N are the outermost layers, X1∩Y1 and X N ∩Y N It can also represent the first overlapping region between the predicted outermost segmented region and the target outermost segmented region. Similarly, X2∩Y2 and X N-1 ∩Y N-1 Both can represent the second overlapping region between the predicted outermost segmentation region and the initial outermost segmentation region. |X i |+|Y i | can represent the size of the region formed by the predicted segmentation region of the i-th layer and the preset segmentation region.

[0086] In practice, the loss calculation considers not only the outermost and second-outermost layers but also the intermediate layers. Specifically, it requires calculating the first overlap between the predicted outermost segmentation region and the target outermost segmentation region, the second overlap between the predicted second-outermost segmentation region and the initial outermost segmentation region, and the third overlap between each predicted intermediate segmentation region and the initial intermediate segmentation region. These three overlaps are then summed to obtain the total overlap region, which includes the overlap between the predicted segmentation region and the preset segmentation region for each layer. Simultaneously, the predicted outermost segmentation region, the target outermost segmentation region, the predicted second-outermost segmentation region, the initial outermost segmentation region, the predicted intermediate segmentation region, and the initial intermediate segmentation region are summed—that is, the predicted segmentation regions of all layers are summed with the preset segmentation regions—to obtain the total region. The ratio between all overlapping regions and the total region is calculated. Since both the predicted and preset segmentation regions include overlapping areas, this ratio is multiplied by 2 to obtain the overall loss value for all layers. The model is then trained based on this loss value.

[0087] In the above embodiments, by treating all layers of the image as a whole, calculating the overall loss value based on the predicted segmentation region of each layer and the preset segmentation region, and training the image segmentation model, the accuracy of the trained image segmentation model can be improved.

[0088] In an exemplary embodiment, step S320 above, generating a pseudo-segmentation region based on the initial outermost segmentation region as the target outermost segmentation region, includes:

[0089] Step S3201: Determine the gradual trend from the initial sub-outermost segmentation region to the initial outermost segmentation region;

[0090] Step S3202: Adjust the initial outermost segmented region according to the gradient trend to obtain a pseudo-segmented region, which is used as the target outermost segmented region.

[0091] The gradient trend indicates the trend of size change between the two segmented regions.

[0092] The initial outermost segmentation region includes the initial top-level segmentation region and the initial bottom-level segmentation region.

[0093] The initial sub-outermost layer segmentation region refers to the segmentation region of the layer connected to the initial outermost layer. For example, if the number of layers of the sample object is 5-10, the initial outermost layer is the 5th and 10th layers, and the initial sub-outermost layer is the 4th and 9th layers.

[0094] Specifically, since the organs and tissues used as samples may not be uniformly distributed, the segmentation regions required for different layers may not be the same. Therefore, before generating pseudo-segmentation regions based on the initial outermost segmentation region, the gradient trend from the initial second outermost segmentation region to the initial outermost segmentation region can be determined first. The initial outermost segmentation region can be adjusted according to this gradient trend to obtain pseudo-segmentation regions, which can then be used as the target outermost segmentation region.

[0095] In this embodiment, since the initial outermost segmentation region includes the initial top-level segmentation region and the initial bottom-level segmentation region, the corresponding generated target outermost region also includes the target top-level segmentation region corresponding to the initial top-level segmentation region and the target bottom-level region corresponding to the initial bottom-level segmentation region.

[0096] The target top-level segmentation region can represent the segmentation region corresponding to an image layer with a higher number of top layers than the sample object. For example, if the top layer of the sample object is 15 layers, the target top-level segmentation region can represent the region of the 16th layer.

[0097] The second pseudo-segmentation region can represent the segmentation region corresponding to an image layer with a lower number of bottom layers than the sample object. For example, if the bottom layer of the sample object is 10 layers, the second pseudo-segmentation region can represent the region of the 9th layer.

[0098] In specific implementation, the process of determining the target top-level segmentation region may include: determining the first gradient trend from the initial second-level segmentation region to the initial top-level segmentation region, and performing dilation or erosion processing on the initial top-level segmentation region according to the first gradient trend to obtain the first pseudo-segmentation region, which is used as the target top-level segmentation region.

[0099] The process of determining the target bottom segmentation region may include: determining the second gradient trend from the initial penultimate segmentation region to the initial bottom segmentation region; performing dilation or erosion processing on the initial bottom segmentation region according to the second gradient trend to obtain the second pseudo segmentation region, which is used as the target bottom segmentation region.

[0100] The first and second gradient trends may be increasing or decreasing. The dilation process is an operation that finds a local maximum value, used for defect filling; in this embodiment, it is used to increase the segmented region. The erosion process, the opposite of the dilation process, is an operation that finds a local minimum value, used to eliminate burrs; in this embodiment, it is used to decrease the segmented region.

[0101] It is understandable that the segmentation trends of organs or tissues as sample objects in segmented images at each layer are mostly progressive. Within a certain number of layers (e.g., 3 layers), the trend generally does not change. Therefore, the gradient trend from the initial top-level segmented region to the target top-level segmented region can be equated to the size change trend from the initial second-level segmented region to the initial top-level segmented region. Thus, the size change trend from the initial second-level segmented region to the initial top-level segmented region can be determined as the first gradient trend. If the first gradient trend is determined to be an increasing trend, the initial top-level segmented region can be dilated to obtain the first pseudo-segmented region, which can be used as the target top-level segmented region. If the first gradient trend is determined to be a decreasing trend, the initial top-level segmented region can be eroded to obtain the first pseudo-segmented region, which can be used as the target top-level segmented region.

[0102] Similarly, the gradient trend from the initial bottom-level segmentation region to the target bottom-level segmentation region can be equated to the size change trend from the initial penultimate segmentation region to the initial bottom-level segmentation region. Therefore, the size change trend from the initial penultimate segmentation region to the initial bottom-level segmentation region can be determined as the second gradient trend. If the second gradient trend is an increasing trend, the initial bottom-level segmentation region can be expanded to obtain a second pseudo-segmentation region, which can be used as the target bottom-level segmentation region. If the second gradient trend is a decreasing trend, the initial bottom-level segmentation region can be eroded to obtain a second pseudo-segmentation region, which can be used as the target bottom-level segmentation region.

[0103] In this embodiment, by determining the gradient trend from the initial sub-outermost segmentation region to the initial outermost segmentation region, and performing dilation or erosion processing on the initial outermost segmentation region according to the gradient trend, the obtained pseudo-segmentation region can be made more accurate, thereby improving the prediction accuracy of the trained image segmentation model.

[0104] In one embodiment, the initial segmentation region further includes an initial intermediate layer segmentation region; step S330 further includes: using a medical image as input, and using the initial outermost layer segmentation region, the target outermost layer segmentation region, and the initial intermediate layer segmentation region as supervision information, training the image segmentation model to be trained to obtain the trained image segmentation model.

[0105] It should be noted that the intermediate layer in this application refers to the number of layers between the top layer and the bottom layer. The intermediate layer may include multiple layers. For example, if an organ is distributed in layers 3-7, then layer 3 is the bottom layer, layer 7 is the top layer, and each of layers 4-6 can represent an intermediate layer.

[0106] For example, suppose the initial segmentation regions of the sample object are distributed at layers 10-15 on the z-axis of 3D space. Layer 10 represents the bottom layer of the sample object, and the segmentation region of layer 10 is the initial bottom-layer segmentation region. Layer 15 represents the top layer of the sample object, and the segmentation region of layer 15 is the initial top-layer segmentation region. Based on the initial top-layer segmentation region of layer 15, a first pseudo-segmentation region of layer 16 (there is no corresponding region in the original image's layer 16) can be generated as the target top-layer segmentation region. Similarly, based on the initial bottom-layer segmentation region of layer 10, a second pseudo-segmentation region of layer 9 (there is no corresponding region in the original image's layer 9) can be generated as the target bottom-layer segmentation region.

[0107] Furthermore, the medical image of the sample object can be input into the image segmentation model to be trained to obtain the predicted segmentation regions of each layer from layer 9 to layer 16. Using the two pseudo-segmentation regions of layer 9 and layer 16, and the initial segmentation regions of each layer from layer 10 to layer 15 as supervision information, the loss value between the predicted segmentation regions and the actual segmentation regions is calculated. Based on this loss value, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

[0108] refer to Figure 4 The diagram illustrates the correspondence between the initial segmentation region and the predicted segmentation region as an exemplary embodiment. The first row in the diagram represents the initial segmentation regions of each layer preset for the sample object, and the second row represents the predicted segmentation regions of each layer obtained by the image segmentation model. Each column of segmentation regions corresponds to the predicted segmentation regions. Specifically, the predicted top-level segmentation region corresponds to the target top-level segmentation region (i.e., the first pseudo-segmentation region), the predicted second-level segmentation region corresponds to the initial top-level segmentation region, the predicted penultimate-level segmentation region corresponds to the initial bottom-level segmentation region, and the predicted bottom-level segmentation region corresponds to the target bottom-level segmentation region (i.e., the second pseudo-segmentation region).

[0109] In one exemplary embodiment, prior to step S330, the method further includes:

[0110] Step S330A: Determine the image type of the medical image;

[0111] Step S330B: Preprocess the medical image according to the image type to obtain the processed medical image;

[0112] The above step S330 further includes: using a medical image as input, and using the initial outermost segmentation region and the target outermost segmentation region as supervision information, training the image segmentation model to be trained to obtain the trained image segmentation model.

[0113] The image types may include computed tomography (CT) images, magnetic resonance imaging (MRI) images, etc.

[0114] In practice, different image types require different preprocessing methods. Therefore, before preprocessing medical images, it is necessary to determine the image type of the medical image and perform preprocessing according to the image type.

[0115] More specifically, for medical images of the type of computed tomography (CT) scan, the corresponding preprocessing methods may include determining the window width and window level, as well as normalization. The window width is the range of CT values ​​displayed on the CT image; tissues and lesions within this range are displayed with different simulated grayscale values. The window level is the center position of the window; for the same window width, different window levels will result in different CT values ​​within the included CT value range. For medical images of the type of magnetic resonance imaging (MRI), the corresponding preprocessing methods may include examination data processing, format conversion (e.g., from DICOM to NIFTI format), resampling, and registration.

[0116] Furthermore, after preprocessing the medical image according to the image type, the processed medical image is used as input, and the initial outermost segmentation region and the target outermost segmentation region are used as supervision information to train the image segmentation model to be trained, thus obtaining the trained image segmentation model.

[0117] In this embodiment, the medical image is preprocessed according to its image type, and then the preprocessed medical image is input into the image segmentation model to be trained for model training. This ensures that the preprocessed medical image meets the image requirements of the image segmentation model, which facilitates subsequent model training.

[0118] In one exemplary embodiment, such as Figure 5 As shown, an image segmentation method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0119] Step S510: Obtain the medical image of the object to be segmented;

[0120] Step S520: Perform image segmentation processing on the medical image using an image segmentation model to obtain a multi-layer image of the object to be segmented;

[0121] Step S530: Remove the outermost image from the multi-layer image to obtain the image segmentation result of the object to be segmented.

[0122] The image segmentation model is trained by taking a medical image of the sample object as input and using the initial outermost segmentation region and the target outermost segmentation region of the sample object as supervised information; the target outermost segmentation region is generated based on the initial outermost segmentation region.

[0123] The outermost image includes the top image and the bottom image.

[0124] The initial outermost segmentation region includes the initial top-level segmentation region and the initial bottom-level segmentation region, while the target outermost segmentation region includes the target top-level segmentation region and the target bottom-level segmentation region.

[0125] Among them, the image segmentation model is used to segment medical images into multi-layer images with a number of image layers greater than the number of separable layers of the object to be segmented.

[0126] In practice, after training the image segmentation model based on the medical image of the sample object, the initial segmentation region of the sample object, and the outermost segmentation region of the target, the trained image segmentation model can segment the image into more layers than the number of segments that the object to be segmented can have. Therefore, after performing image segmentation processing on the medical image of the object to be segmented using the image segmentation model, a multi-layered image with more layers than the number of segments that the object to be segmented can be obtained. Since the number of layers in the segmented image is greater than the number of segments that the object to be segmented can have, the top and bottom layers of the obtained multi-layered image can be discarded, and the remaining layers can be used as the image segmentation result of the object to be segmented.

[0127] The image segmentation method provided in this embodiment, after obtaining multi-layer images of the object to be segmented, removes the top and bottom images from the obtained multi-layer images, thereby ensuring that each segmented image matches its corresponding segmentation region. Since the number of layers of the segmented images will be greater than the number of segmentable layers of the object to be segmented, it can solve the problem of mismatch between the actual segmentation region of the top and bottom images and the initial segmentation region while ensuring that the number of image layers of the object to be segmented meets the requirements.

[0128] In an exemplary embodiment, before step S520, which performs image segmentation processing on the medical image using an image segmentation model, the method further includes:

[0129] Step S5101: Obtain the image format of the medical image;

[0130] Step S5102: If the image format of the medical image does not match the target image format corresponding to the image segmentation model, the image format of the medical image is converted to the target image format.

[0131] Among them, medical images can be in formats such as Nifti, Dicom, and Nrrd.

[0132] In practice, when training an image segmentation model using medical images of sample objects, a unified format is required for model training. Therefore, the trained image segmentation model can have a corresponding image format. Before performing image segmentation processing on medical images using the image segmentation model, it is necessary to obtain the image format of the medical image of the object to be segmented. The image format of the medical image of the object to be segmented is compared with the target image format corresponding to the image segmentation model. If the image format of the medical image does not match the target image format corresponding to the image segmentation model, i.e., they are different, the image format of the medical image needs to be converted to the target image format. Then, the converted medical image is input into the image segmentation model to obtain the multi-layer image of the object to be segmented.

[0133] In this embodiment, the image format of the medical image is obtained and compared with the target image format corresponding to the image segmentation model. If the two do not match, the image format of the medical image is converted into the target image format to ensure that the image segmentation model can successfully segment the medical image of the object to be segmented.

[0134] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0135] Based on the same inventive concept, this application also provides an image segmentation model training apparatus for implementing the image segmentation model training method described above, and an image segmentation apparatus for implementing the image segmentation method described above. The solutions provided by these two apparatuses are similar to those described in the above methods. Therefore, the specific limitations of the image segmentation model training apparatus and image segmentation apparatus embodiments provided below can be found in the limitations of the image segmentation model training method and image segmentation method described above, and will not be repeated here.

[0136] In one embodiment, such as Figure 6 As shown, a training device for an image segmentation model is provided, comprising: a sample acquisition module 610, a pseudo-region generation module 620, and a model training module 630, wherein:

[0137] The sample acquisition module 610 is used to acquire sample data; the sample data includes a medical image of the sample object and the initial segmentation region of the sample object; wherein, the initial segmentation region includes the initial outermost segmentation region, which is used to represent the preset segmentation region for the sample object;

[0138] The pseudo-region generation module 620 is used to generate a pseudo-segmentation region based on the initial outermost segmentation region, which serves as the target outermost segmentation region.

[0139] The model training module 630 is used to train the image segmentation model to be trained by taking medical images as input and the initial outermost segmentation region and the target outermost segmentation region as supervision information, so as to obtain the trained image segmentation model.

[0140] In one embodiment, the model training module 630 is specifically used to input a medical image into the image segmentation model to be trained, obtain a predicted outermost segmentation region and a predicted second outermost segmentation region; obtain a first overlapping region between the predicted outermost segmentation region and the target outermost segmentation region, and a second overlapping region between the predicted second outermost segmentation region and the initial outermost segmentation region; obtain a total region composed of the predicted outermost segmentation region, the target outermost segmentation region, the predicted second outermost segmentation region, and the initial outermost segmentation region; obtain a loss value based on the first overlapping region, the second overlapping region, and the total region; and train the image segmentation model to be trained based on the loss value to obtain a trained image segmentation model.

[0141] In one embodiment, the model training module 630 is further configured to add the first overlapping region and the second overlapping region to obtain the total overlapping region; obtain the ratio between the total overlapping region and the total region; and obtain the loss value based on the ratio.

[0142] In one embodiment, the pseudo-region generation module 620 is specifically used to determine the gradient trend from the initial sub-outermost segmentation region to the initial outermost segmentation region; adjust the initial outermost segmentation region according to the gradient trend to obtain a pseudo-segmentation region, which serves as the target outermost segmentation region.

[0143] In one embodiment, the initial segmentation region further includes an initial intermediate layer segmentation region; the model training module 630 is also used to train the image segmentation model to be trained by taking the medical image as input and the initial outermost layer segmentation region, the target outermost layer segmentation region and the initial intermediate layer segmentation region as supervision information, so as to obtain the trained image segmentation model.

[0144] The modules in the training device of the aforementioned image segmentation model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, such as Figure 7 As shown, an image segmentation apparatus is provided, including: an image acquisition module 710, an image segmentation module 720, and an image removal module 730, wherein:

[0146] Image acquisition module 710 is used to acquire medical images of the object to be segmented;

[0147] The image segmentation module 720 is used to perform image segmentation processing on medical images using an image segmentation model to obtain multi-layer images of the object to be segmented. The image segmentation model is trained by taking the medical image of the sample object as input and using the initial outermost segmentation region and the target outermost segmentation region of the sample object as supervised information. The target outermost segmentation region is generated based on the initial outermost segmentation region.

[0148] The image removal module 730 is used to remove the outermost image from multiple layers of images to obtain the image segmentation result of the object to be segmented.

[0149] In one embodiment, the image segmentation apparatus further includes a format conversion module for acquiring the image format of the medical image; and converting the image format of the medical image to the target image format when the image format of the medical image does not match the target image format corresponding to the image segmentation model.

[0150] Each module in the aforementioned image segmentation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0151] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a training method for an image segmentation model. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0152] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A training method for an image segmentation model, characterized in that, The method includes: Acquire sample data; the sample data includes a medical image of a sample object and the initial segmentation region of the sample object; wherein, the initial segmentation region is a segmentation label, including an initial outermost segmentation region, the sample object is a three-dimensional object, the segmentation of the sample object yields multi-layer images, each layer of the image has a corresponding initial segmentation region, and the initial outermost segmentation region represents the segmentation label of the outermost image in the multi-layer images; A pseudo-segmentation region is generated based on the initial outermost segmentation region, which serves as the target outermost segmentation region; the target outermost segmentation region represents the pseudo-segmentation label of the newly added layer outside the outermost image of the multi-layer image. The medical image is input into the image segmentation model to be trained to obtain the predicted outermost segmentation region and the predicted second outermost segmentation region; the predicted second outermost segmentation region represents the predicted segmentation region of the level connected to the predicted outermost layer. Obtain the first overlapping region between the predicted outermost segmentation region and the target outermost segmentation region, and the second overlapping region between the predicted second outermost segmentation region and the initial outermost segmentation region; Obtain the total region consisting of the predicted outermost segmentation region, the target outermost segmentation region, the predicted second outermost segmentation region, and the initial outermost segmentation region; The ratio between the total overlapping region (obtained by adding the first overlapping region and the second overlapping region) and the total region is calculated to obtain the loss value. Based on the loss value, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

2. The method according to claim 1, characterized in that, The step of generating a pseudo-segmentation region based on the initial outermost segmentation region as the target outermost segmentation region includes: Determine the gradient trend from the initial second outermost segmented region to the initial outermost segmented region; the initial second outermost segmented region represents the segmented region of the layer connected to the initial outermost layer; The initial outermost segmented region is adjusted according to the gradient trend to obtain the pseudo-segmented region, which serves as the target outermost segmented region.

3. The method according to claim 2, characterized in that, The initial outermost segmentation region includes an initial top segmentation region and an initial bottom segmentation region, and the target outermost segmentation region includes a target top segmentation region and a target bottom segmentation region; The method further includes: A first gradient trend is determined from the initial second-layer segmentation region to the initial top-layer segmentation region. The initial top-layer segmentation region is then subjected to dilation or erosion processing according to the first gradient trend to obtain a first pseudo-segmentation region, which is used as the target top-layer segmentation region. Determine a second gradient trend from the initial penultimate layer segmentation region to the initial bottom layer segmentation region; perform dilation or erosion processing on the initial bottom layer segmentation region according to the second gradient trend to obtain a second pseudo segmentation region, which serves as the target bottom layer segmentation region.

4. The method according to claim 1, characterized in that, The initial segmentation region further includes an initial intermediate layer segmentation region; the method further includes: Using the medical image as input, and the initial outermost segmentation region, the target outermost segmentation region, and the initial intermediate segmentation region as supervision information, the image segmentation model to be trained is trained to obtain the trained image segmentation model.

5. An image segmentation method, characterized in that, The method includes: Obtain the medical image of the object to be segmented; The medical image is segmented using an image segmentation model to obtain a multi-layer image of the object to be segmented. The outermost image is removed from the multi-layer image to obtain the image segmentation result of the object to be segmented; the outermost image includes the top and bottom images in the multi-layer image. The image segmentation model is trained using the image segmentation model training method described in any one of claims 1 to 4.

6. A training device for an image segmentation model, characterized in that, The device includes: A sample acquisition module is used to acquire sample data; the sample data includes a medical image of a sample object and an initial segmentation region of the sample object; wherein, the initial segmentation region is a segmentation label, including an initial outermost segmentation region, the sample object is a three-dimensional object, and the segmentation of the sample object yields multi-layer images, each layer of the image having a corresponding initial segmentation region, and the initial outermost segmentation region representing the segmentation label of the outermost image in the multi-layer images; The pseudo-region generation module is used to generate a pseudo-segmentation region based on the initial outermost segmentation region, which serves as the target outermost segmentation region; the target outermost segmentation region represents the pseudo-segmentation label of the newly added layer outside the outermost image of the multi-layer image. The model training module is used to input the medical image into the image segmentation model to be trained, and obtain the predicted outermost segmentation region and the predicted second outermost segmentation region; the predicted second outermost segmentation region represents the predicted segmentation region of the layer connected to the predicted outermost segmentation region; obtain the first overlapping region between the predicted outermost segmentation region and the target outermost segmentation region, and the second overlapping region between the predicted second outermost segmentation region and the initial outermost segmentation region; obtain the total region formed by the predicted outermost segmentation region, the target outermost segmentation region, the predicted second outermost segmentation region, and the initial outermost segmentation region; calculate the ratio between the total overlapping region obtained by adding the first overlapping region and the second overlapping region and the total region to obtain the loss value; and train the image segmentation model to be trained based on the loss value to obtain the trained image segmentation model.

7. An image segmentation apparatus, characterized in that, The device includes: The image acquisition module is used to acquire medical images of the object to be segmented. An image segmentation module is used to perform image segmentation processing on the medical image using an image segmentation model to obtain a multi-layer image of the object to be segmented; the image segmentation model is trained using the image segmentation model training method according to any one of claims 1 to 4; The image removal module is used to remove the outermost image from the multi-layer images to obtain the image segmentation result of the object to be segmented; the outermost image includes the top image and the bottom image in the multi-layer images.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images