Image segmentation method, model training method thereof and related device, equipment and medium

Abstract

The application discloses an image segmentation method and a model training method thereof, and related devices, equipment and media. The training method of the image segmentation model comprises the following steps: obtaining a sample medical image of a target organ; wherein a sample pixel point in the sample medical image is labeled with a sample label and a sample weight; based on the sample weight of the sample pixel point, the difference between the sample label and a predicted label is measured to obtain a model loss; and based on the model loss, the network parameters of the image segmentation model are adjusted. The above scheme can take into account the segmentation accuracy of different parts of the target organ during the image segmentation process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image segmentation method, a model training method thereof, and related devices, equipment and media. BACKGROUND

[0002] Medical images such as CT (Computed Tomography) images have extremely important significance in scenarios such as auxiliary diagnosis and surgical planning. For example, by segmenting medical images, segmentation results of related organs can be obtained to assist doctors in avoiding vital organs while resecting lesions, and so on.

[0003] However, in real scenarios, the segmentation objects are usually irregular, and when segmenting organs such as trachea and blood vessels, even multi-level branches with different thicknesses also need to be considered. In this case, manual correction relying on user interaction is usually needed after machine segmentation to improve the final segmentation accuracy, which is not only time-consuming and laborious, but also cannot guarantee the segmentation effect. Therefore, how to consider the segmentation accuracy of different parts of the target organ in the image segmentation process has become a problem to be solved. SUMMARY

[0004] The present application provides an image segmentation method, a model training method thereof, and related devices, equipment and media.

[0005] The first aspect of the present application provides a training method of an image segmentation model, comprising: obtaining a sample medical image of a target organ; wherein a sample pixel point in the sample medical image is labeled with a sample label and a sample weight, the sample label represents a target category to which the sample pixel point belongs, and the sample weight is set based on a sample distance from the sample pixel point to the surface of the target organ; performing target segmentation on the sample medical image by using an image segmentation model to obtain a predicted label of the sample pixel point; wherein the predicted label is used to at least represent the possibility of the sample pixel point being predicted to belong to the target category; based on the sample weight of the sample pixel point, the sample label and the predicted label are measured for difference to obtain a model loss; and based on the model loss, adjusting network parameters of the image segmentation model.

[0006] Therefore, a sample medical image of a target organ is obtained, and sample pixels in the sample medical image are labeled with sample labels and sample weights. The sample labels represent the target categories to which the sample pixels belong, and the sample weights are set based on sample distances from the sample pixels to the surface of the target organ, that is, not only are the sample weights of sample pixels of different categories different, but also sample pixels of the same category are different according to their distances to the surface of the target organ. Therefore, the sample weights can be used to differentiate the sample pixels, and based on this, an image segmentation model is used to perform target segmentation on the sample medical image to obtain predicted labels of the sample pixels. The predicted labels at least represent the likelihood of the sample pixels belonging to the target categories after prediction. The sample labels and the predicted labels are measured in terms of difference based on the sample pixels to obtain a model loss, and the network parameters of the image segmentation model are adjusted based on the model loss. Therefore, the sample weights are used to differentiate the sample pixels, and in the loss measurement process, the sample weights of the sample pixels are further used to differentiate the degree of bias of the predicted loss at different sample pixels, which helps the model to consider the learning effect of difficult-to-segment parts and easy-to-segment parts on the target organ, so that the segmentation accuracy of different parts of the target organ can be considered in the image segmentation process.

[0007] The difference between the sample labels and the predicted labels is measured based on the sample weights of the sample pixels to obtain a model loss, which includes: the difference between the sample labels and the predicted labels is measured based on the sample weights to obtain a first loss, and / or the difference between the sample labels and the predicted labels is measured based on the sample weights to obtain a second loss, and the model loss is obtained based on the first loss and / or the second loss; wherein the first loss is used to measure the area coincidence degree between the sample area and the predicted area of the target organ, and the sample area is the actually labeled area of the target organ, and the predicted area is the predicted area of the target organ model, and the second loss is used to measure the data distribution difference between the sample labels and the predicted labels.

[0008] Therefore, the difference between the sample labels and the predicted labels is measured based on the sample weights to obtain a first loss, and / or the difference between the sample labels and the predicted labels is measured based on the sample weights to obtain a second loss, and the model loss is obtained based on the first loss and / or the second loss, and the first loss is used to measure the area coincidence degree between the sample area and the predicted area of the target organ, and the sample area is the actually labeled area of the target organ, and the predicted area is the predicted area of the target organ model, and the second loss is used to measure the data distribution difference between the sample labels and the predicted labels. Therefore, it is beneficial to enrich the dimensions of loss measurement and improve the accuracy and comprehensiveness of the model loss.

[0009] The sample label is represented by a label value, the prediction label at least includes a probability value of a sample pixel point being predicted to belong to a target category, and a first loss is obtained by measuring a regional difference between the sample label and the prediction label based on a sample weight, including: for each sample pixel point, a first product of the label value and the sample weight of the sample pixel point is obtained, a second product of the probability value and the sample weight of the sample pixel point is obtained, and a third product of the label value, the probability value and the sample weight of the sample pixel point is obtained; a first sum value is obtained by summing the first products corresponding to each sample pixel point, a second sum value is obtained by summing the second products corresponding to each sample pixel point, and a third sum value is obtained by summing the third products corresponding to each sample pixel point; the first loss is obtained based on a ratio of the third sum value to a reference sum value; wherein the reference sum value is the sum of the first sum value and the second sum value, and the ratio is negatively correlated with the first loss.

[0010] Therefore, in the case that the model loss includes the first loss, for each sample pixel point, a first product of its label value and sample weight is obtained, a second product of its probability value and sample weight is obtained, and a third product of its label value, probability value and sample weight is obtained, and on this basis, a first sum value is obtained by summing the first products corresponding to each sample pixel point, a second sum value is obtained by summing the second products corresponding to each sample pixel point, and a third sum value is obtained by summing the third products corresponding to each sample pixel point, so that the first loss is obtained based on a ratio between the third sum value and a reference sum value, and the reference sum value is the sum of the first sum value and the second sum value, and the ratio is negatively correlated with the first loss. Since the above ratio can represent the coincidence degree of the sample region and the prediction region in the case of sample weight participating in weighting, and the higher the coincidence degree, the higher the model segmentation accuracy, and vice versa, by minimizing the first loss, the model segmentation accuracy can be improved from the optimization dimension of the regional coincidence degree.

[0011] The sample label is represented by a label value, the prediction label at least includes a probability value of a sample pixel point being predicted to belong to a target category, and a second loss is obtained by measuring a distribution difference between the sample label and the prediction label based on a sample weight, including: for each sample pixel point, a logarithmic value of the probability value of the sample pixel point is obtained, a fourth product of the label value and the sample weight of the sample pixel point is obtained, and a fifth product of the logarithmic value and the fourth product is obtained; the second loss is obtained based on a fourth sum value obtained by summing the fifth products corresponding to each sample pixel point; wherein the fourth sum value is negatively correlated with the second loss.

[0012] Therefore, in the case that the model loss comprises the second loss, for each sample pixel point, a logarithmic value of the probability value thereof is obtained, a fourth product of the label value and the sample weight thereof is obtained, and a fifth product of the logarithmic value and the fourth product is obtained, and on this basis, a second loss value is obtained based on a fourth sum value obtained by summing the fifth products corresponding to each sample pixel point, and the fourth sum value is negatively correlated with the second loss. Since the above fourth sum value can represent the data distribution difference between the sample label and the predicted label in the case that the sample weight is involved in weighting, the greater the fourth sum value, the greater the data distribution difference, and vice versa, the smaller the fourth sum value, the smaller the data distribution difference. Therefore, by minimizing the second loss, the segmentation accuracy of the model can be improved from the optimization dimension of the data distribution difference.

[0013] wherein the sample weight of the sample pixel point belonging to the target organ is higher than the sample weight of the sample pixel point not belonging to the target organ, and the sample weight of the sample pixel point belonging to the target organ is negatively correlated with the sample distance.

[0014] Therefore, the sample weight of the sample pixel point belonging to the target organ is set to be higher than the sample weight of the sample pixel point not belonging to the target organ, and the sample weight of the sample pixel point belonging to the target organ is set to be negatively correlated with the sample distance. Therefore, in the training process, the image segmentation model can focus on the segmentation accuracy of the sample pixel point belonging to the target organ, and among the sample pixel points belonging to the target organ, the sample pixel points closer to the surface of the target organ can be focused on, which is beneficial to improving the attention of the image segmentation model to the difficult segmentation part.

[0015] wherein the setting step of the sample weight comprises: for each sample pixel point belonging to the target organ, performing normalization processing on the distance to obtain a normalized value, and taking the sum of the difference between the first value and the normalized value and the second value as the sample weight of the sample pixel point belonging to the target organ; wherein the first value is not less than 1, and the sample weight of the sample pixel point not belonging to the target organ is the second value.

[0016] Therefore, for each sample pixel point belonging to the target organ, the distance corresponding thereto is normalized to obtain a normalized value, and the sum of the difference between the first value and the normalized value and the second value is taken as the sample weight of the sample pixel point belonging to the target organ, and the first value is not less than 1, and the sample weight of the sample pixel point not belonging to the target organ is directly set to the second value. Therefore, the sample weight of each sample pixel point can be obtained only by simple operations such as normalization, value addition and subtraction, which is beneficial to greatly reducing the calculation complexity of the sample weight.

[0017] The target organ includes a trunk segment and a plurality of branch segments extending from the trunk segment; in the process of training the image segmentation model, the frequency of selecting sample medical images located in the branch segments is higher than a preset threshold, and / or the frequency of selecting sample medical images located in the trunk segment is lower than the preset threshold.

[0018] Therefore, the target organ includes a trunk segment and a plurality of branch segments extending from the trunk segment, in which case, in the process of training the image segmentation model, the frequency of selecting sample medical images located in the branch segments is higher than a preset threshold, and the frequency of selecting sample medical images located in the trunk segment is lower than the preset threshold, so that over-sampling can be performed on the difficult-to-segment samples of the target organ in the training process, and under-sampling can be performed on the easy-to-segment samples of the target organ, which is beneficial to balancing the learning effects of different parts and enabling the image segmentation model to take into account the segmentation accuracy of different parts.

[0019] The image segmentation model includes an encoding network and a decoding network, the encoding network includes a plurality of encoding layers connected in sequence, and the decoding network includes a plurality of decoding layers connected in sequence; the image segmentation model is used to perform target segmentation on the sample medical image to obtain the predicted label of each sample pixel point, including: taking the feature map obtained by encoding the last encoding layer and the feature map obtained by decoding each decoding layer except the last decoding layer as a reference feature map; decoding based on each reference feature map to obtain a first decoding result; fusing the first decoding result and a second decoding result to obtain a sample decoding result; wherein the second decoding result is output by the last decoding layer, and the sample decoding result includes the predicted label of each sample pixel point.

[0020] Therefore, the image segmentation model includes an encoding network and a decoding network, and the encoding network includes a plurality of encoding layers connected in sequence, and the decoding network includes a plurality of decoding layers connected in sequence, on the basis of which, the feature map obtained by encoding the last encoding layer and the feature map obtained by decoding each decoding layer except the last decoding layer are taken as a reference feature map, then decoding is performed based on each reference feature map to obtain a first decoding result, and the first decoding result and a second decoding result are fused to obtain a sample decoding result, and the second decoding result is output by the last decoding layer, and the sample decoding result includes the predicted label of each sample pixel point, so that multi-level features can be densely connected in the target segmentation process, which is beneficial to reducing the probability of information loss in small parts caused by down-sampling and improving the segmentation accuracy.

[0021] The target organ includes at least one of a trachea and a blood vessel; and / or the sample distance is the nearest distance from the sample pixel point to the surface.

[0022] Therefore, in the case that the target organ is set as at least one of the trachea and the blood vessel, by setting the sample weight, the sampling times of different thick and thin pipelines can be balanced, so that the learning effects of different thick and thin pipelines are taken into account in the training process of the model, which is beneficial to improving the segmentation accuracy of different parts. By setting the sample distance as the nearest distance from the sample pixel point to the surface, the accuracy of the sample weight can be improved.

[0023] The second aspect of the present application provides an image segmentation method, comprising: obtaining a medical image of a target organ; performing target segmentation on the medical image by using an image segmentation model to obtain the target category to which each pixel point in the medical image belongs; wherein the image segmentation model is obtained by using the training method of the image segmentation model in the first aspect; and obtaining a segmentation result of the target organ based on the target category to which the pixel point belongs.

[0024] Therefore, by using the image segmentation model trained by the training method of the image segmentation model in the first aspect to perform target segmentation on the medical image of the target organ to obtain the target category to which each pixel point in the medical image belongs, and obtaining the segmentation result of the target organ based on the target category to which the pixel point belongs, the segmentation accuracy of different parts of the target organ can be taken into account in the image segmentation process.

[0025] In the case that the medical image is a three-dimensional image composed of a plurality of two-dimensional images stacked, the image segmentation model is used to perform target segmentation on the medical image to obtain the target category to which each pixel point in the medical image belongs, which comprises: taking each two-dimensional image as a current image, and combining the current image with a reference image of the current image to obtain a multi-channel image; wherein the number of image frames between the reference image and the current image is less than a preset number of frames; using the image segmentation model to process the multi-channel image to obtain the target category to which the pixel point in the current image belongs; and obtaining the target category to which each pixel point in the medical image belongs based on the target category to which the pixel point in each two-dimensional image belongs.

[0026] Therefore, in the case that the medical image is a three-dimensional image composed of a plurality of two-dimensional images stacked, each two-dimensional image is taken as a current image, and the current image is combined with a reference image of the current image to obtain a multi-channel image, and the number of image frames between the reference image and the current image is less than a preset number of frames, on this basis, the image segmentation model is used to process the multi-channel image to obtain the target category to which the pixel point in the current image belongs, and then the target category to which each pixel point in the medical image belongs is obtained based on the target category to which the pixel point in each two-dimensional image belongs, which can use context image merging as model input, and is beneficial to strengthening the connectivity of model output.

[0027] The target organ segmentation result is obtained based on the target category to which the pixel point belongs, including: acquiring a plurality of connected domains composed of pixel points belonging to the target organ; and obtaining the target organ segmentation result based on the largest connected domain.

[0028] Therefore, after obtaining the target category to which the pixel point belongs, the target organ segmentation result is obtained by acquiring a plurality of connected domains composed of pixel points belonging to the target organ and based on the largest connected domain, so that the false positive area outside the target organ can be removed, and the segmentation accuracy can be further improved.

[0029] The third aspect of the present application provides a training device of an image segmentation model, including: a sample acquisition module, a sample segmentation module, a loss measurement module, and a parameter adjustment module. The sample acquisition module is configured to acquire a sample medical image of a target organ. The sample medical image includes sample pixel points labeled with a sample label and a sample weight. The sample label indicates the target category to which the sample pixel point belongs, and the sample weight is set based on the sample distance from the sample pixel point to the surface of the target organ. The sample segmentation module is configured to perform target segmentation on the sample medical image using an image segmentation model to obtain a predicted label of the sample pixel point. The predicted label is used to indicate the possibility of the sample pixel point belonging to the target category. The loss measurement module is configured to measure the difference between the sample label and the predicted label based on the sample weight of the sample pixel point to obtain a model loss. The parameter adjustment module is configured to adjust the network parameters of the image segmentation model based on the model loss.

[0030] The fourth aspect of the present application provides an image segmentation device, including: an image acquisition module, a target segmentation module, and a result acquisition module. The image acquisition module is configured to acquire a medical image of a target organ. The target segmentation module is configured to perform target segmentation on the medical image using an image segmentation model to obtain the target category to which each pixel point in the medical image belongs. The image segmentation model is obtained by using the training device of the image segmentation model in the third aspect. The result acquisition module is configured to obtain a target organ segmentation result based on the target category to which the pixel point belongs.

[0031] The fifth aspect of the present application provides an electronic device, including a memory and a processor coupled to each other. The processor is configured to execute program instructions stored in the memory to implement the training method of the image segmentation model in the first aspect or the image segmentation method in the second aspect.

[0032] The sixth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon. The program instructions are executed by a processor to implement the training method of the image segmentation model in the first aspect or the image segmentation method in the second aspect.

[0033] The above scheme obtains a sample medical image of a target organ, and sample pixel points in the sample medical image are labeled with sample labels and sample weights. The sample labels represent target categories to which the sample pixel points belong, and the sample weights are set based on sample distances from the sample pixel points to a surface of the target organ, that is, sample weights of sample pixel points of different categories are different, and sample weights of sample pixel points of the same category are also different according to distances from the sample pixel points to the surface of the target organ. Therefore, the sample weights can be used to differentiate the sample pixel points, and on this basis, a target is segmented from the sample medical image by using an image segmentation model to obtain predicted labels of the sample pixel points. The predicted labels at least represent probabilities that the sample pixel points belong to the target categories after being predicted. The sample labels and the predicted labels are measured in terms of difference based on the sample pixel points to obtain a model loss, and network parameters of the image segmentation model are adjusted based on the model loss. Therefore, the sample weights are used to differentiate the sample pixel points, and in the loss measurement process, the sample weights of the sample pixel points are further used to differentiate the degree of bias of the predicted loss at different sample pixel points, which is beneficial to the model to consider learning effects on difficult-to-segment parts and easy-to-segment parts of the target organ. Therefore, in the image segmentation process, the segmentation accuracy of different parts of the target organ can be considered. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a flowchart of an embodiment of a training method of an image segmentation model of the present application;

[0035] Figure 2 is a framework diagram of an embodiment of an image segmentation model;

[0036] Figure 3 is a flowchart of an embodiment of an image segmentation method of the present application;

[0037] Figure 4 is a process diagram of an embodiment of an image segmentation method of the present application;

[0038] Figure 5 is a framework diagram of an embodiment of a training device of an image segmentation model of the present application;

[0039] Figure 6 is a framework diagram of an embodiment of an image segmentation device of the present application;

[0040] Figure 7 is a framework diagram of an embodiment of an electronic device of the present application;

[0041] Figure 8 is a framework diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0042] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0043] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, interfaces, techniques, in order to provide a thorough understanding of the present application.

[0044] The terms "system" and "network" are often used interchangeably herein. The term "and / or", herein, merely describes an associated relationship, which means that there can be three relationships, for example, A and / or B, which means that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" herein means two or more than two.

[0045] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of a training method of an image segmentation model of the present application. Specifically, it can include the following steps:

[0046] Step S11: Obtain a sample medical image of a target organ.

[0047] In the embodiments of the present disclosure, the sample pixel points in the sample medical image are labeled with sample labels, and the sample labels represent the target categories to which the sample pixel points belong. Specifically, the target categories can be included in a plurality of preset categories, and the plurality of preset categories include the target organ. For example, the plurality of preset categories can include the target organ and the image background, that is, the target categories to which the sample pixel points in the sample medical image belong are either the target organ or the image background, and other cases can be extended in this way, which will not be repeated here.

[0048] In one implementation scenario, the target organ can be set according to actual application needs. For example, in the case of needing to segment the trachea, the target organ can be set as the trachea; or in the case of needing to segment the blood vessels, the target organ can be set as the blood vessels, which is not limited here.

[0049] In one implementation scenario, the sample labels can be represented by specific label values, and the label values of the sample pixel points belonging to the target organ can be set as the first label value, and the label values of the sample pixel points not belonging to the target organ can be set as the second label value. For example, the first label value can be set as 1, and the second label value can be set as 0. For example, the target organ is set as the trachea, then the sample weight of the sample pixel points belonging to the trachea can be set as 1, and the sample weight of the sample pixel points not belonging to the trachea can be set as 0. Other cases can be extended in this way, which will not be repeated here.

[0050] In an implementation scenario, the sample medical image can include, but is not limited to, a CT image, and the like, without limitation.

[0051] In the embodiments of the present disclosure, the sample pixel points in the sample medical image can also be labeled with sample weights of the sample pixel points, and the sample weights can be set based on sample distances of the sample pixel points to the surface of the target organ. In order to further distinguish the sample weights of the sample pixel points belonging to the target organ and the sample pixel points not belonging to the target organ, the sample weights can be further set based on the sample labels and the aforementioned sample distances.

[0052] In an implementation scenario, the sample distance can be specifically the nearest distance of the sample pixel point to the surface of the target organ. Specifically, the surface of the target organ can be regarded as a surface formed by a series of pixel points, and for a certain sample pixel point, there is a distance D1 between a pixel point and the sample pixel point, which is not greater than the distance D2 between any other pixel point and the sample pixel point, and then the aforementioned distance D1 can be regarded as the nearest distance of the sample pixel point to the surface of the target organ. Taking the trachea as the target organ as an example, the sample distance can be the nearest distance of the sample pixel point to the surface of the trachea. Other cases can be similarly deduced, and will not be exemplified one by one here.

[0053] In an implementation scenario, as described above, the sample weights can be further set based on the sample labels and the sample distances. Specifically, according to the sample labels of the sample pixel points, it can be determined that the sample weights of the sample pixel points belonging to the target organ are higher than the sample weights of the sample pixel points not belonging to the target organ, and then the sample weights of the sample pixel points not belonging to the target organ can be uniformly set as a fixed value (such as 1), and then the sample weights of the sample pixel points belonging to the target organ can be further set (all higher than the aforementioned fixed value) based on the sample distances of the sample pixel points belonging to the target organ. Exemplarily, the sample weights of the sample pixel points not belonging to the target organ can be uniformly set as 1, and then the sample weights of the sample pixel points belonging to the target organ can be determined to be greater than 1, and other cases can be similarly deduced, and will not be exemplified one by one here. Further, according to the sample labels and the sample distances of the sample pixel points, the sample weights of the sample pixel points belonging to the target organ can be set to be negatively correlated with the sample distances, that is, for each sample pixel point belonging to the target organ, the closer to the surface of the target organ, the higher the sample weight, and vice versa, the farther away from the surface of the target organ, the lower the sample weight. Taking the trachea as the target organ as an example, for the coarse trachea, the sample weights of the edge pixel points are larger, and the sample weight of the center pixel point is smaller; and for the thin trachea, since the tracheal wall is also relatively thin, most of the sample pixel points belonging to the thin trachea have relatively large sample weights, so that the sampling times of the coarse and thin tracheas can be balanced. Other cases can be similarly deduced, and will not be exemplified one by one here.

[0054] In one specific implementation scenario, for each sample pixel point belonging to the target organ, the corresponding distance (e.g., the nearest distance from the sample pixel point to the surface of the target organ) can be normalized to obtain a normalized value, and the difference between the first value and the normalized value and the sum of the second value can be taken as the sample weight of the sample pixel point belonging to the target organ, and the first value can be set to be not less than 1, and the sample weight of the sample pixel point not belonging to the target organ can be directly set to the second value (i.e., the aforementioned fixed value). Exemplarily, the normalized value corresponding to the sample pixel point can be denoted as d, and the first value and the second value can both be set to 1, i.e., the sample weight of the sample pixel point not belonging to the target organ is uniformly set to 1, and the sample weight of the sample pixel point belonging to the target organ can be represented as 1-d+1 (i.e., 2-d). Other cases can be similarly deduced and will not be exemplified one by one here.

[0055] In one implementation scenario, in order to facilitate obtaining the sample weight of each sample pixel point in the training process, a distance probability map can be generated based on the sample weight of each sample pixel point. Taking a sample medical image as a three-dimensional image for example, the pixel value of the pixel point (i, j, k) in the distance probability map represents the sample weight of the sample pixel point (i, j, k) in the sample medical image. Other cases can be similarly deduced and will not be exemplified one by one here.

[0056] Step S12: performing target segmentation on the sample medical image by using the image segmentation model to obtain a predicted label of the sample pixel point.

[0057] In the embodiments of the present disclosure, the predicted label is used to at least represent the possibility that the sample pixel point is predicted to belong to the target category. In addition, the predicted label can further represent the possibility that the sample pixel point is predicted to belong to other categories, which is not limited here. Taking the case that the plurality of preset categories include the target organ and the image background for example, the predicted label can represent the possibility that the sample pixel point is predicted to belong to the target organ, and of course, the predicted label can also represent the possibility that the sample pixel point is predicted to belong to the target organ and the image background, respectively. Other cases can be similarly deduced and will not be exemplified one by one here.

[0058] In one implementation scenario, the sample medical image can be a three-dimensional image, i.e., the three-dimensional image can be regarded as being composed of a plurality of two-dimensional images arranged in layers, and then the collected CT image can be directly taken as the sample medical image. It should be noted that in this case, the image segmentation model is a three-dimensional segmentation model, i.e., the three-dimensional segmentation model can include a three-dimensional convolution kernel, and the specific meaning of the three-dimensional convolution kernel can be referred to related technical details, which will not be repeated here.

[0059] In one specific implementation scenario, the target organ can include a trunk segment and a plurality of branch segments extending from the trunk segment. Taking the target organ as the trachea as an example, the main trachea generates secondary organs through branching, each bronchus generates a plurality of segmental bronchus, from the main trachea to the terminal bronchiole, the tracheal tree is approximately composed of 6-8 generations of branches, and generally speaking, the more secondary the trachea is, the thinner the trachea is, and therefore the more difficult the segmentation is. In the case of the target organ being a blood vessel or other organs, the same can be applied, and examples are not repeated here. Based on this, in order to improve the segmentation accuracy of the image segmentation model at difficult-to-segment parts, in the process of training the image segmentation model, the frequency of selecting sample medical images located in the branch segment can be higher than a preset threshold, and the frequency of selecting sample medical images located in the trunk segment can be lower than the preset threshold. It should be noted that the frequency here refers to the percentage of the total number of times that the sample medical image is used in the training process. For example, the preset threshold can be set to 50%, that is, the frequency of selecting sample medical images located in the branch segment needs to be higher than the preset threshold, such as the frequency of selecting sample medical images located in the branch segment can be set to 70%, and the frequency of selecting sample medical images located in the trunk segment needs to be lower than the preset threshold, such as the frequency of selecting sample medical images located in the trunk segment can be set to 30%. Of course, in actual application, it can be set according to actual application needs, such as the segmentation difficulty of the target organ in the branch segment is much higher than the segmentation difficulty of the target organ in the trunk segment, then the difference between the frequency of selecting sample medical images located in the branch segment and the frequency of selecting sample medical images located in the trunk segment can be set to be larger, or, such as the segmentation difficulty of the target organ in the branch segment is slightly higher than the segmentation difficulty of the target organ in the trunk segment, then the difference between the frequency of selecting sample medical images located in the branch segment and the frequency of selecting sample medical images located in the trunk segment can be set to be smaller, and the specific value of the frequency is not limited here.

[0060] In one implementation scenario, considering directly performing target segmentation on a three-dimensional image, it is possible that some slices (e.g., coronal slices, or sagittal slices, or transverse slices) are mis-segmented due to similar shapes of target organs observed in the slices. In order to further improve the segmentation accuracy, a three-dimensional image segmentation model can be trained for each slice. For example, a three-dimensional image segmentation model for coronal slices can be trained, a three-dimensional image segmentation model for sagittal slices can be trained, and a three-dimensional image segmentation model for transverse slices can be trained. The segmentation results of the three three-dimensional image segmentation models can be fused to obtain the final segmentation result. Details can be referred to the related description of the image segmentation method embodiments below, and will not be described here in detail. In this case, during training of a three-dimensional image segmentation model for a slice, the sample medical image can be an image extracted from the three-dimensional image along the slice. For example, during training of a three-dimensional image segmentation model for coronal slices, the sample medical image can be an image extracted from the three-dimensional image along the coronal slice. During training of a three-dimensional image segmentation model for sagittal slices, the sample medical image can be an image extracted from the three-dimensional image along the sagittal slice. During training of a three-dimensional image segmentation model for transverse slices, the sample medical image can be an image extracted from the three-dimensional image along the transverse slice.

[0061] In one specific implementation scenario, as described above, a three-dimensional image can be regarded as being composed of a plurality of two-dimensional images stacked together. For example, a three-dimensional image with a resolution of 512*512*80 can be regarded as being composed of 512 two-dimensional images with a resolution of 512*80 stacked together in the coronal view. Similarly, the three-dimensional image can be regarded as being composed of 512 two-dimensional images with a resolution of 512*80 stacked together in the sagittal view. Similarly, the three-dimensional image can be regarded as being composed of 80 two-dimensional images with a resolution of 512*512 stacked together in the transverse view. Other cases can be similarly deduced.

[0062] In one specific implementation scenario, for any one view image segmentation model, the starting layer of the target organ from the view angle can be determined in combination with the aforementioned reference information, and the two-dimensional image in which the target organ first appears can be selected as the starting image. Meanwhile, the ending layer of the target organ from the view angle can also be determined, and the two-dimensional image in which the target organ last appears can be selected as the ending image. On this basis, a two-dimensional image is sequentially selected as a sample current image from the starting image to the ending image, and the sample current image and a sample reference image of the sample current image are combined to obtain a sample multi-channel image. The sample reference image is located between the starting image and the ending image, and the number of image frames between the sample reference image and the sample current image is less than a preset number of frames (e.g., 4 frames, 5 frames, 6 frames, etc.). Thus, the image segmentation model can be used to perform target segmentation on the sample multi-channel image to obtain the predicted label of the sample pixel point in the sample current image. Then, the above-mentioned step of sequentially selecting a two-dimensional image as a sample current image and the subsequent steps are re-executed until all the starting image to the ending image are selected. Based on this, the following loss measurement and parameter adjustment steps are re-executed to train the image segmentation model of the view until the training converges. In addition, in the case where the image segmentation models of the coronal view, the sagittal view and the transverse view need to be trained respectively, the above-mentioned three view image segmentation models can be trained in parallel by using the above-mentioned method until the training converges. Thus, the image segmentation models of the above-mentioned three views can be finally trained.

[0063] In one specific implementation scenario, as mentioned above, the target organ includes a trunk segment and a plurality of branch segments extending from the trunk segment. Similarly to the above-mentioned method, in the process of training the image segmentation model of any one view, the frequency of selecting the sample multi-channel image located in the branch segment can be set to be higher than a preset threshold, and the frequency of selecting the sample multi-channel image located in the trunk segment can be set to be lower than the preset threshold. For the specific meaning of the frequency and the preset threshold, please refer to the above-mentioned related description, which will not be repeated here.

[0064] In one implementation scenario, please refer to Figure 2 , Figure 2 is a schematic diagram of the framework of an embodiment of the image segmentation model. As shown in Figure 2 , the image segmentation model can include an encoding network and a decoding network. The encoding network can include a plurality of encoding layers connected in sequence, and the decoding network can include a plurality of decoding layers connected in sequence. The oblique downward arrow represents down-sampling, and the oblique upward arrow represents up-sampling. In addition, a skip connection (as shown by the dashed straight line in Figure 2 ) can also be provided between the encoding layer and the decoding layer. Alternatively, further, a decoding layer can also be provided between the encoding layer and the decoding layer. It should be noted that Figure 2The shown is only one implementation of the image segmentation model, and the specific structure is not limited here, such as can refer to U-Net, V-Net, U-Net++, etc., which will not be repeated here. On this basis, in the target segmentation process, the feature map obtained by encoding the last encoding layer and the feature map obtained by decoding each decoding layer except the last decoding layer can be used as a reference feature map, and then each reference feature map is decoded to obtain a first decoding result, so that the sample decoding result can be obtained by fusing the first decoding result and the second decoding result, and the second decoding result is output by the last decoding layer, and the sample decoding result includes the predicted label of each sample pixel point, so as to reduce the possibility of losing information of the bronchiole due to down-sampling through multi-level dense connection, which is beneficial to improve the decoding accuracy.

[0065] In a specific implementation scenario, the resolution of the reference feature map can be adjusted so that the resolutions of the reference feature maps are the same. For example, the resolutions of the reference feature maps before the last reference feature map can be adjusted, and the adjusted resolutions are the same as the resolution of the last reference feature map. On this basis, the reference feature maps with consistent resolutions can be fused (such as spliced) and then decoded to obtain the first decoding result.

[0066] In a specific implementation scenario, after obtaining the first decoding result, the first decoding result and the second decoding result can be spliced and then input into a 1*1 convolution to obtain the sample result.

[0067] In a specific implementation scenario, as described above, if the plurality of preset categories include the target organ and the image background, the above sample decoding result can be a two-channel image, and the pixel value of a pixel point in one channel image can represent the probability value that the corresponding sample pixel point in the sample medical image belongs to the target organ, and the pixel value of a pixel point in another channel image can represent the probability value that the corresponding sample pixel point in the sample medical image belongs to the image background.

[0068] Step S13: Based on the sample weight of the sample pixel point, the difference between the sample label and the predicted label is measured to obtain a model loss.

[0069] In an implementation scenario, the model loss can include a first loss, and the first loss can be specifically used to measure the area coincidence degree between the sample region of the target organ and the predicted region. Specifically, the area difference between the sample label and the predicted label can be measured based on the sample weight to obtain the first loss.

[0070] In one specific implementation scenario, as mentioned above, the label of the sample is represented by a label value. Further, the label value of the sample pixel point belonging to the target organ can be a first label value (e.g., 1), and the label value of the sample pixel point not belonging to the target organ can be a second label value (e.g., 0). In addition, the predicted label can at least include a probability value of the sample pixel point predicted to belong to the target category, and the upper limit of the probability value is the first label value (e.g., 1), and the lower limit of the probability value is the second label value (e.g., 0). For ease of description, for the i-th sample pixel point in the sample medical image, the sample weight of the sample pixel point can be denoted as D i , the label value of the sample pixel point can be denoted as l i , and the probability value of the sample pixel point predicted to belong to the target category can be denoted as p i .

[0071] In one specific implementation scenario, for each sample pixel point, the first product of the label value and the sample weight of the sample pixel point can be obtained, the second product of the probability value and the sample weight of the sample pixel point can be obtained, and the third product of the label value, the probability value and the sample weight of the sample pixel point can be obtained. On this basis, the first product corresponding to each sample pixel point can be summed to obtain a first sum value, the second product corresponding to each sample pixel point can be summed to obtain a second sum value, and the third product corresponding to each sample pixel point can be summed to obtain a third sum value. Based on this, the first loss can be obtained based on the ratio of twice the third sum value to a reference sum value, and the reference sum value is the sum of the first sum value and the second sum value, and the ratio is negatively correlated with the first loss. Specifically, the first loss L1 can be represented as:

[0072]

[0073] In the above formula (1), D i ·l i represents the first product, D i ·p i represents the second product, D i ·p i ·l i represents the third product, ∑i i∈V D i ·l i represents the first sum value, ∑i ∈V D i ·p i represents the second sum value, ∑i ∈V D i ·p i ·l i represents the third sum value, ∑i ∈V D i ·p i + ∑i ∈V D i·l i That is, V represents a set of sample pixels in a sample medical image. As can be seen from formula (1), the more accurate the prediction result of the image segmentation model is, the higher the area coincidence degree between the sample region and the predicted region is, and the smaller the first loss L1 is. Conversely, the less accurate the prediction result of the image segmentation model is, the lower the area coincidence degree between the sample region and the predicted region is, and the larger the first loss L1 is. Therefore, by minimizing the first loss L1 to optimize the parameters, the model accuracy of the image segmentation model can be improved.

[0074] In one implementation scenario, the model loss can include a second loss, and the second loss is specifically used to measure the data distribution difference between the sample label and the predicted label. Specifically, the sample label and the predicted label can be measured based on the sample weight to obtain the second loss.

[0075] In one specific implementation scenario, the specific setting manner of the sample label and the predicted label can be referred to the related description in the foregoing disclosed embodiments, which will not be described here.

[0076] In one specific implementation scenario, for each sample pixel, the logarithmic value of the probability value of the sample pixel can be obtained, and the fourth product of the label value of the sample pixel and the sample weight can be obtained, or the fifth product of the logarithmic value and the fourth product can be obtained. On this basis, the second loss is obtained based on the fourth sum value obtained by summing the fifth products corresponding to each sample pixel, and the fourth sum value is negatively correlated with the second loss. Specifically, the second loss L2 can be represented as:

[0077] L2=-∑ i∈V D i l i logp i ……(2)

[0078] In the above formula (2), logp i represents the logarithmic value of the probability value, D i l i represents the fourth product, D i l i logp i represents the fifth product, and ∑ i∈V D i l i logp iwherein V represents the fourth sum, and V represents a set of sample pixels in the sample medical image. As can be seen from equation (2), the more accurate the prediction result of the image segmentation model is, the smaller the difference in data distribution between the sample label and the predicted label is, and the smaller the first loss L2 is. Conversely, the less accurate the prediction result of the image segmentation model is, the greater the difference in data distribution between the sample label and the predicted label is, and the greater the first loss L2 is. Therefore, by minimizing the second loss L2 to optimize the parameters, the model accuracy of the image segmentation model can be improved.

[0079] In one implementation scenario, in the case where the model loss includes the first loss and the second loss, both the first loss and the second loss can be fused to obtain the model loss. It should be noted that the related meanings and calculation processes of the first loss and the second loss can be referred to the foregoing related description, which will not be repeated here. The model loss can be obtained by weighted fusion of the first loss and the second loss, which can be specifically represented as:

[0080] L = β1L1 + β2L2 …… (3)

[0081] In the above equation (3), L1 and L2 represent the first loss and the second loss respectively, and β1 and β2 represent the first weight of the first loss and the second weight of the second loss respectively. The first weight β1 and the second weight β2 can be set according to actual application needs. For example, in the loss measurement process, the area coincidence degree is considered as the focus, and then the first weight β1 can be set to be greater than the second weight β2, or in the loss measurement process, the data distribution difference is considered as the focus, and then the first weight β1 can be set to be less than the second weight β2, or in the loss measurement process, the area coincidence degree and the data distribution difference are equally important, and then the first weight β1 can be set to be equal to the second weight β2, which is not limited here.

[0082] Step S14: adjusting the network parameters of the image segmentation model based on the model loss.

[0083] Specifically, based on the model loss, an optimization method such as gradient descent can be used to adjust the network parameters of the image segmentation model. The specific adjustment process of the network parameters can be referred to the technical details of the optimization method such as gradient descent, which will not be repeated here.

[0084] The above scheme obtains a sample medical image of a target organ, and sample pixel points in the sample medical image are labeled with a sample label and a sample weight. The sample label represents a target category to which the sample pixel points belong, and the sample weight is set based on a sample distance from the sample pixel points to a surface of the target organ, that is, sample weights of sample pixel points of different categories are different, and sample pixel points of the same category also have differences according to distances from the sample pixel points to the surface of the target organ. Therefore, the sample weights can be used to differentiate the sample pixel points, and on this basis, an image segmentation model is used to perform target segmentation on the sample medical image to obtain a predicted label of the sample pixel points. The predicted label at least represents a possibility that the sample pixel points belong to the target category after prediction. The sample label and the predicted label are measured based on the sample pixel points to obtain a model loss, and network parameters of the image segmentation model are adjusted based on the model loss. Therefore, the sample weights are used to differentiate the sample pixel points, and in the loss measurement process, the sample weights of the sample pixel points are further used to differentiate the bias of the predicted loss at different sample pixel points, which is beneficial to the model to consider the learning effect of difficult-to-segment parts and easy-to-segment parts on the target organ. Therefore, in the image segmentation process, the segmentation accuracy of different parts of the target organ can be considered.

[0085] Please refer to Figure 3 , Figure 3 is a flowchart of an embodiment of the image segmentation method of the present application.

[0086] Specifically, the method can include the following steps:

[0087] Step S31: Obtain a medical image of a target organ.

[0088] Specifically, the medical image can include but is not limited to a CT image, and the target organ can include but is not limited to a trachea, a blood vessel, etc., without limitation.

[0089] Step S32: Perform target segmentation on the medical image by using an image segmentation model to obtain a target category to which each pixel point in the medical image belongs.

[0090] The image segmentation model in the embodiment of the present disclosure is obtained by using the training method of any of the image segmentation models described above. For the specific training process, please refer to the foregoing disclosure embodiments, which will not be described here. In addition, the target category can be included in a plurality of preset categories, and the plurality of preset categories include the target category. For the specific meaning of the target organ, the preset category, and the target category, please refer to the related description in the foregoing disclosure embodiments, which will not be described here.

[0091] In one implementation scenario, as described in the foregoing disclosed embodiments, the medical image is a three-dimensional image composed of a plurality of two-dimensional images arranged in layers. In order to improve the contextual connection during the target segmentation process, each two-dimensional image can be taken as a current image, and a multi-channel image can be obtained by combining the current image with a reference image of the current image, and the number of image frames between the reference image and the current image is not less than a preset number of frames (e.g., 4 frames, 5 frames, 6 frames, etc.). On this basis, the image segmentation model can be used to process the multi-channel image to obtain the target category to which the pixel point in the current image belongs. This process is repeated until the target segmentation of each two-dimensional image is completed. Then, the target category to which each pixel point in the medical image belongs can be obtained based on the target category to which the pixel point in each two-dimensional image belongs.

[0092] In one implementation scenario, as described in the foregoing disclosed embodiments, considering that directly performing target segmentation on a three-dimensional image may result in missegmentation due to the observation of similar target organ shapes in some bit planes (e.g., coronal plane, or sagittal plane, or transverse plane), in order to further improve the segmentation accuracy, different bit plane image segmentation models can be used to perform target segmentation on the medical image to obtain the target category to which each pixel point in the medical image belongs. On this basis, the target category to which each pixel point belongs can be determined based on the target category to which the pixel point belongs detected by the different bit plane image segmentation models. In addition, the specific training method of the different bit plane image segmentation models can be referred to the related description in the foregoing disclosed embodiments, which will not be described here. It should be noted that the different bit plane image segmentation models can have the same network structure or different network structures. For example, the different bit plane image segmentation models can all be set to U-Net++, or the transverse plane image segmentation model can be set to U-Net++, the sagittal plane image segmentation model can be set to U-Net, and the coronal plane image segmentation model can be set to V-Net, which is not limited here.

[0093] In one specific implementation scenario, in the process of performing target segmentation on the medical image by using different bit plane image segmentation models, in order to improve the contextual connection during the target segmentation process, the foregoing method of combining multi-channel images can also be used. Specifically, for an image segmentation model of a certain bit plane, during the target segmentation process, the three-dimensional image can be cut in the bit plane to obtain a plurality of two-dimensional images arranged in layers, and then the foregoing steps of taking each two-dimensional image as a current image and subsequent steps can be performed. Please refer to Figure 4 , Figure 4 is a process schematic diagram of an embodiment of the image segmentation method of the present application. As shown in Figure 4As shown, in the training process of the image segmentation model, the image segmentation model of the coronal plane, the image segmentation model of the sagittal plane and the image segmentation model of the transverse plane can be trained, and the three-dimensional image can be cut from the coronal plane, the sagittal plane and the transverse plane respectively, to obtain a plurality of two-dimensional images stacked along the vertical direction of the coronal plane, a plurality of two-dimensional images stacked along the vertical direction of the sagittal plane, and a plurality of two-dimensional images stacked along the vertical direction of the transverse plane. On this basis, the above steps of taking each two-dimensional image as a current image and subsequent steps are performed respectively to perform target segmentation by feeding the image segmentation model of the corresponding plane.

[0094] In a specific implementation scenario, taking a plurality of preset categories including a target organ and an image background as an example, after the target category to which each pixel point in the medical image belongs is detected by the image segmentation model of each plane, the category of each pixel point can be determined. If the pixel point is detected as belonging to the target organ in any plane of the image segmentation model, it can be determined that the pixel point belongs to the target organ, otherwise, if the pixel point is detected as belonging to the image background in each plane of the image segmentation model, it can be determined that the pixel point belongs to the image background, and thus the final target category of each pixel point can be obtained. Please continue to refer to Figure 4 Taking the target organ as the trachea as an example, the image segmentation model of each plane can detect the trachea mask, and the union of these trachea masks can be obtained to obtain the final trachea mask. It should be noted that the pixel points belonging to the trachea in the final trachea mask represent that they are finally determined to belong to the trachea, otherwise, the pixel points not belonging to the trachea in the final trachea mask represent that they are finally determined not to belong to the trachea. Other cases can be analogized in this way, which will not be illustrated one by one here.

[0095] In an implementation scenario, as described in the foregoing disclosed embodiments, the image segmentation model can include an encoding network and a decoding network, the encoding network can include a plurality of encoding layers connected in sequence, and the decoding network can include a plurality of decoding layers connected in sequence, in order to reduce the possibility of fine bronchus information loss due to down-sampling, the feature map obtained by encoding the last encoding layer, and the feature map obtained by decoding each decoding layer except the last decoding layer can be taken as reference feature maps, on this basis, the first decoding result can be obtained based on each reference feature map, and the final decoding result can be obtained by fusing the first decoding result and the second decoding result, and the second decoding result is output by the last decoding layer, the final decoding result includes the target category to which each pixel point belongs, and the specific process can be referred to the related description in the foregoing disclosed embodiments, which will not be described here. It should be noted that in the case where the target organ is a trachea, a blood vessel or the like containing a main stem segment and a plurality of branch segments extending from the main stem segment, the above multi-level feature dense connection can reduce the possibility of fine bronchus information loss due to down-sampling, thereby greatly improving the detection length.

[0096] Step S33: obtaining the segmentation result of the target organ based on the target category to which each pixel point belongs.

[0097] In an implementation scenario, the connected domain composed of the pixel points belonging to the target organ can be directly obtained as the segmentation result of the target organ in the medical image.

[0098] In an implementation scenario, in order to further improve the segmentation accuracy, a plurality of connected domains composed of the pixel points belonging to the target organ can be obtained first, and the segmentation result of the target organ can be obtained based on the largest connected domain, for example, the largest connected domain can be directly taken as the segmentation result of the target organ in the medical image, so as to remove the possible false positive area.

[0099] In an implementation scenario, taking the trachea as an example, after obtaining the segmentation result of the trachea, the lung lobes and lung segments in the medical image can be further segmented based on the prior information of the trachea segmentation result, and the position of the lesion (such as a lung nodule) in the medical image can be detected, so as to further assist the doctor.

[0100] The above scheme uses the image segmentation model trained in the steps of the training method embodiment of the image segmentation model to perform target segmentation on the medical image of the target organ, so as to obtain the target category to which each pixel point in the medical image belongs, and obtain the segmentation result of the target organ based on the target category to which each pixel point belongs, which is beneficial to taking into account the segmentation accuracy of different parts of the target organ in the image segmentation process.

[0101] Please refer to Figure 5 , Figure 5is a schematic diagram of a framework of an embodiment of the image segmentation model training apparatus 50 of the present application. The image segmentation model training apparatus 50 comprises a sample acquisition module 51, a sample segmentation module 52, a loss measurement module 53, and a parameter adjustment module 54. The sample acquisition module 51 is configured to acquire a sample medical image of a target organ. In the sample medical image, a sample pixel point is labeled with a sample label and a sample weight. The sample label indicates a target category to which the sample pixel point belongs, and the sample weight is set based on a sample distance from the sample pixel point to a surface of the target organ. The sample segmentation module 52 is configured to perform target segmentation on the sample medical image by using an image segmentation model to obtain a predicted label of the sample pixel point. The predicted label is used to indicate a possibility that the sample pixel point is predicted to belong to the target category. The loss measurement module 53 is configured to measure a difference between the sample label and the predicted label based on the sample weight of the sample pixel point to obtain a model loss. The parameter adjustment module 54 is configured to adjust network parameters of the image segmentation model based on the model loss.

[0102] The above scheme differentiates each sample pixel point by using the sample weight, and further differentiates a degree of bias of the predicted loss at different sample pixel points by using the sample weight of each sample pixel point in the loss measurement process. This is conducive to the model being able to take into account the learning effect on difficult-to-segment parts and easy-to-segment parts of the target organ, so that the segmentation accuracy of different parts of the target organ can be taken into account in the image segmentation process.

[0103] In some disclosed embodiments, the loss measurement module 53 comprises a first measurement sub-module configured to measure a regional difference between the sample label and the predicted label based on the sample weight to obtain a first loss. The loss measurement module 53 comprises a second measurement sub-module configured to measure a distribution difference between the sample label and the predicted label based on the sample weight to obtain a second loss. The loss measurement module 53 comprises a loss acquisition sub-module configured to obtain the model loss based on the first loss and / or the second loss. The first loss is used to measure a regional coincidence degree between a sample region of the target organ and a predicted region of the target organ, and the sample region is a region actually labeled for the target organ, and the predicted region is a region predicted by the model for the target organ. The second loss is used to measure a data distribution difference between the sample label and the predicted label.

[0104] In some disclosed embodiments, the first metric sub-module comprises a first weighting unit configured to obtain, for each sample pixel point, a first product of the label value of the sample pixel point and the sample weight, a second product of the probability value of the sample pixel point and the sample weight, and a third product of the label value, the probability value and the sample weight of the sample pixel point; the first metric sub-module comprises a first summing unit configured to sum the first products corresponding to the sample pixel points to obtain a first sum value, sum the second products corresponding to the sample pixel points to obtain a second sum value, and sum the third products corresponding to the sample pixel points to obtain a third sum value; the first metric sub-module comprises a ratio obtaining unit configured to obtain the first loss based on a ratio of twice the third sum value to a reference sum value; and the reference sum value is the sum of the first sum value and the second sum value, and the ratio is negatively correlated with the first loss.

[0105] In some disclosed embodiments, the second metric sub-module comprises a second weighting unit configured to obtain, for each sample pixel point, a logarithm value of the probability value of the sample pixel point, a fourth product of the label value of the sample pixel point and the sample weight, and a fifth product of the logarithm value and the fourth product; the second metric sub-module comprises a second summing unit configured to obtain the second loss based on a fourth sum value obtained by summing the fifth products corresponding to the sample pixel points; and the fourth sum value is negatively correlated with the second loss.

[0106] In some disclosed embodiments, the sample weight of the sample pixel point belonging to the target organ is higher than the sample weight of the sample pixel point not belonging to the target organ, and the sample weight of the sample pixel point belonging to the target organ is negatively correlated with the sample distance.

[0107] In some disclosed embodiments, the sample obtaining module 51 comprises a normalization sub-module configured to normalize the distance for each sample pixel point belonging to the target organ to obtain a normalized value; and the sample obtaining module 51 comprises a numerical operation sub-module configured to take the sum of the difference between the first numerical value and the normalized value and the second numerical value as the sample weight of the sample pixel point belonging to the target organ; wherein the first numerical value is not less than 1, and the sample weight of the sample pixel point not belonging to the target organ is the second numerical value.

[0108] In some disclosed embodiments, the target organ comprises a trunk segment and a plurality of branch segments extending from the trunk segment; and in the process of training the image segmentation model, the frequency of selecting the sample medical image located in the branch segment is higher than a preset threshold, and / or the frequency of selecting the sample medical image located in the trunk segment is lower than a preset threshold.

[0109] In some disclosed embodiments, the image segmentation model comprises an encoding network and a decoding network, the encoding network comprises a plurality of encoding layers connected in sequence, and the decoding network comprises a plurality of decoding layers connected in sequence; the sample segmentation module 52 comprises a feature map acquisition submodule, which uses the feature map obtained by encoding the last encoding layer and the feature map obtained by decoding each decoding layer except the last decoding layer as a reference feature map; the sample segmentation module 52 comprises a decoding submodule, which decodes based on each reference feature map to obtain a first decoding result; the sample segmentation module 52 comprises a fusion submodule, which fuses the first decoding result and a second decoding result to obtain a sample decoding result; wherein the second decoding result is output by the last decoding layer, and the sample decoding result comprises the predicted labels of each sample pixel point.

[0110] In some disclosed embodiments, the target organ comprises at least one of a trachea and a blood vessel; and / or, the sample distance is the nearest distance from the sample pixel point to the surface.

[0111] Please refer to Figure 6 , Figure 6 is a schematic diagram of an embodiment of the image segmentation device 60 of the present application. The image segmentation device 60 comprises an image acquisition module 61, a target segmentation module 62, and a result acquisition module 63. The image acquisition module is configured to acquire a medical image of a target organ. The target segmentation module is configured to perform target segmentation on the medical image using an image segmentation model to obtain a target category to which each pixel point in the medical image belongs. The image segmentation model is obtained using the training device of the image segmentation model of the third aspect. The result acquisition module is configured to obtain a segmentation result of the target organ based on the target category to which each pixel point belongs.

[0112] The above scheme uses the image segmentation model trained by the training device in the training device embodiment of the image segmentation model to perform target segmentation on the medical image of the target organ, to obtain the target category to which each pixel point in the medical image belongs, and to obtain the segmentation result of the target organ based on the target category to which each pixel point belongs, which is beneficial to taking into account the segmentation accuracy of different parts of the target organ during image segmentation.

[0113] In some disclosed embodiments, the medical image is a three-dimensional image, the three-dimensional image is composed of a plurality of two-dimensional images arranged in layers, the target segmentation module 62 includes an image combination sub-module, configured to take each two-dimensional image as a current image respectively, and combine the current image with a reference image of the current image to obtain a multi-channel image; wherein the number of image frames between the reference image and the current image is less than a preset number of frames; the target segmentation module 62 includes an image processing sub-module, configured to process the multi-channel image by using an image segmentation model to obtain a target category to which a pixel point in the current image belongs; the target segmentation module 62 includes a category determination sub-module, configured to obtain a target category to which each pixel point in the medical image belongs based on the target category to which the pixel point in each two-dimensional image belongs.

[0114] In some disclosed embodiments, the result obtaining module 63 includes a connected domain obtaining sub-module, configured to obtain a plurality of connected domains composed of pixel points belonging to the target organ; the result obtaining module 63 includes a connected domain screening sub-module, configured to obtain a segmentation result of the target organ based on the largest connected domain.

[0115] Please refer to Figure 7 , Figure 7 is a schematic diagram of a framework of an embodiment of the electronic device 70. The electronic device 70 includes a memory 71 and a processor 72 coupled with each other. The processor 72 is configured to execute program instructions stored in the memory 71 to implement the steps of any of the image segmentation model training method embodiments or implement the steps of any of the image segmentation method embodiments. In one specific implementation scenario, the electronic device 70 can include but is not limited to a microcomputer, a server, and in addition, the electronic device 70 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited herein.

[0116] Specifically, the processor 72 is configured to control itself and the memory 71 to implement the steps of any of the above-described image segmentation model training method embodiments, or implement the steps of any of the above-described image segmentation method embodiments. The processor 72 can also be referred to as a CPU (Central Processing Unit). The processor 72 can be an integrated circuit chip having a processing capability of signals. The processor 72 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 72 can be implemented by an integrated circuit chip together.

[0117] The above scheme differentiates each sample pixel point through the sample weight, and further differentiates the degree of bias of the prediction loss at different sample pixel points through the sample weight of each sample pixel point in the loss measurement process, which is beneficial to the model to consider the learning effect on the difficult-to-segment part and the easy-to-segment part of the target organ, and thus the segmentation accuracy of different parts of the target organ can be considered in the image segmentation process.

[0118] Please refer to Figure 8 , Figure 8 is a framework schematic diagram of an embodiment of the computer readable storage medium 80. The computer readable storage medium 80 stores program instructions 801 capable of being executed by the processor, and the program instructions 801 are used to implement the steps of any of the above-described image segmentation model training method embodiments, or implement the steps of any of the above-described image segmentation method embodiments.

[0119] The above scheme differentiates each sample pixel point through the sample weight, and further differentiates the degree of bias of the prediction loss at different sample pixel points through the sample weight of each sample pixel point in the loss measurement process, which is beneficial to the model to consider the learning effect on the difficult-to-segment part and the easy-to-segment part of the target organ, and thus the segmentation accuracy of different parts of the target organ can be considered in the image segmentation process.

[0120] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely an example, and the division of the units or components can be different, for example, some units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the display or discussion of the coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0121] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., they can be located in one place or distributed on network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0122] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0123] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0124] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has clearly informed the personal information processing rules before processing the personal information and obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that it has entered the personal information collection range and will collect personal information. If the individual voluntarily enters the collection range, it is considered to agree to collect personal information. Or on the device for processing personal information, through the pop-up information or by asking the individual to upload his / her personal information, the individual's authorization is obtained under the condition of using obvious mark / information to inform the personal information processing rules. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

Claims

1. A method for training an image segmentation model, characterized in that, The method comprises: obtaining a sample medical image of a target organ; wherein sample pixels in the sample medical image are marked with sample labels and sample weights, the sample labels representing target categories to which the sample pixels belong, and the sample weights being set based on sample distances from the sample pixels to a surface of the target organ, the sample weights of sample pixels belonging to the target organ being negatively correlated with the sample distances, and for each sample pixel belonging to the target organ: normalizing the sample distance to obtain a normalized value, and summing a difference between a first value and the normalized value and a second value to obtain the sample weight of the sample pixel belonging to the target organ, the first value being not less than 1, and the sample weight of a sample pixel not belonging to the target organ being fixedly set as the second value; performing target segmentation on the sample medical image by using an image segmentation model to obtain predicted labels of the sample pixels; wherein the predicted labels are used at least to represent probabilities that the sample pixels belong to the target categories, and the target organ comprises a main stem and a plurality of branch stems extending from the main stem, and in training the image segmentation model, a frequency of selecting the sample medical image located at the branch stems is higher than a frequency of selecting the sample medical image located at the main stem, and a difference between the frequencies is positively correlated with a difference between segmentation difficulties of the branch stems and the main stem; based on the sample weights of the sample pixels, measuring a difference between the sample labels and the predicted labels to obtain a model loss; based on the model loss, adjusting network parameters of the image segmentation model.

2. The method of claim 1, wherein, The method of measuring the difference between the sample labels and the predicted labels based on the sample weights of the sample pixels to obtain the model loss comprises: based on the sample weights, measuring a regional difference between the sample labels and the predicted labels to obtain a first loss, and / or measuring a distribution difference between the sample labels and the predicted labels to obtain a second loss; based on the first loss and / or the second loss, obtaining the model loss; wherein the first loss is used to measure a regional coincidence degree between a sample region of the target organ and a predicted region of the target organ, the sample region being a region actually marked for the target organ, and the predicted region being a region predicted by the model for the target organ, and the second loss is used to measure a data distribution difference between the sample labels and the predicted labels.

3. The method of claim 2, wherein, The sample labels are represented by label values, and the predicted labels comprise at least probability values that the sample pixels belong to the target categories; and the method of measuring a regional difference between the sample labels and the predicted labels based on the sample weights to obtain a first loss comprises: for each sample pixel, obtaining a first product of the label value and the sample weight of the sample pixel, obtaining a second product of the probability value and the sample weight of the sample pixel, and obtaining a third product of the label value, the probability value and the sample weight of the sample pixel. Summing the first products corresponding to the sample pixels to obtain a first sum value, summing the second products corresponding to the sample pixels to obtain a second sum value, and summing the third products corresponding to the sample pixels to obtain a third sum value; The first loss is obtained based on a ratio of twice the third sum value to a reference sum value, wherein the reference sum value is a sum of the first sum value and the second sum value, and the ratio is negatively correlated with the first loss.

4. The method of claim 2, wherein, The sample label is represented by a label value, and the predicted label at least includes a probability value of the sample pixel belonging to the target category; the second loss is obtained by measuring the distribution difference between the sample label and the predicted label based on the sample weight, including: For each sample pixel, a logarithmic value of the probability value of the sample pixel is obtained, a fourth product of the label value of the sample pixel and the sample weight is obtained, and a fifth product of the logarithmic value and the fourth product is obtained; The second loss is obtained based on a fourth sum value obtained by summing the fifth products corresponding to the sample pixels; wherein the fourth sum value is negatively correlated with the second loss.

5. The method of claim 1, wherein, The sample weight of the sample pixel belonging to the target organ is higher than the sample weight of the sample pixel not belonging to the target organ.

6. The method of claim 1, wherein, The image segmentation model includes an encoding network and a decoding network, the encoding network includes a plurality of encoding layers connected in sequence, and the decoding network includes a plurality of decoding layers connected in sequence; the target segmentation of the sample medical image by using the image segmentation model to obtain the predicted label of the sample pixel, including: The feature map encoded by the last encoding layer and the feature map decoded by each decoding layer except the last decoding layer are all used as reference feature maps; A first decoding result is obtained based on each reference feature map; A sample decoding result is obtained by fusing the first decoding result and a second decoding result; wherein the second decoding result is output by the last decoding layer, and the sample decoding result includes the predicted label of each sample pixel.

7. The method according to any one of claims 1 to 6, characterized in that, The target organ includes at least one of the trachea and the blood vessel; And / or, the sample distance is the nearest distance from the sample pixel to the surface.

8. An image segmentation method characterized by, Including: Obtaining a medical image of a target organ; Using an image segmentation model to perform target segmentation on the medical image to obtain a target category to which each pixel in the medical image belongs; wherein the image segmentation model is obtained by using the training method of the image segmentation model in any one of claims 1 to 7; Based on the target category to which the pixel belongs, a segmentation result of the target organ is obtained.

9. The method of claim 8, wherein, The medical image is a three-dimensional image composed of a plurality of two-dimensional images arranged in layers, and the target segmentation of the medical image by using the image segmentation model to obtain the target category to which each pixel in the medical image belongs, including: Each of the two-dimensional images is taken as a current image, and the current image is combined with a reference image of the current image to obtain a multi-channel image, wherein the number of image frames between the reference image and the current image is less than a preset number of frames; The image segmentation model is used to process the multi-channel image to obtain a target category to which the pixel point in the current image belongs; Based on the target category to which the pixel point in each of the two-dimensional images belongs, a target category to which each of the pixel points in the medical image belongs is obtained.

10. The method of claim 8, wherein, The target organ segmentation result is obtained based on the target category to which the pixel point belongs, comprising: Obtaining a plurality of connected domains composed of pixel points belonging to the target organ; Based on the largest connected domain, the segmentation result of the target organ is obtained.

11. A training device for an image segmentation model, characterized in that, Comprising: A sample acquisition module is configured to acquire a sample medical image of a target organ; wherein sample pixel points in the sample medical image are labeled with a sample label and a sample weight, the sample label indicates a target category to which the sample pixel point belongs, and the sample weight is based on a sample distance from the sample pixel point to a surface of the target organ, the sample weight of the sample pixel point belonging to the target organ is negatively correlated with the sample distance, and for each of the sample pixel points belonging to the target organ: the sample distance is normalized to obtain a normalized value, and the sum of a first value and the difference between the normalized value and a second value is taken as the sample weight of the sample pixel point belonging to the target organ, the first value is not less than 1, and the sample weight of the sample pixel point not belonging to the target organ is fixedly set as the second value; A sample segmentation module is configured to perform target segmentation on the sample medical image using an image segmentation model to obtain a predicted label of the sample pixel point; wherein the predicted label is used to indicate at least a possibility that the sample pixel point belongs to the target category, the target organ includes a main stem and a plurality of branch segments extending from the main stem, and in the process of training the image segmentation model, the frequency of selecting the sample medical image located at the branch segment is higher than the frequency of selecting the sample medical image located at the main stem, and the frequency difference between them is positively correlated with the segmentation difficulty difference between them; A loss measurement module is configured to measure the difference between the sample label and the predicted label based on the sample weight of the sample pixel point to obtain a model loss; A parameter adjustment module is configured to adjust network parameters of the image segmentation model based on the model loss.

12. An image segmentation apparatus characterized by comprising: Comprising: An image acquisition module is configured to acquire a medical image of a target organ; A target segmentation module is configured to perform target segmentation on the medical image using an image segmentation model to obtain a target category to which each pixel point in the medical image belongs; wherein the image segmentation model is obtained by the training device of the image segmentation model of claim 11; A result acquisition module is configured to obtain a segmentation result of the target organ based on the target category to which the pixel point belongs.

13. An electronic device, comprising: An image segmentation model training method according to any one of claims 1 to 7, or an image segmentation method according to any one of claims 8 to 9, is implemented by a processor executing program instructions stored in a memory coupled to the processor.

14. A computer-readable storage medium having stored thereon program instructions, wherein, The program instructions, when executed by a processor, implement an image segmentation model training method according to any one of claims 1 to 7, or an image segmentation method according to any one of claims 8 to 9.

Images