An Image Segmentation Method, System and Device Based on Re-calibrated Labels

Abstract

The present invention discloses an image segmentation method, system and device based on recalibrated labels. The method includes: obtaining a target data set of the same type as the source data set; selecting a decision maker and training the decision maker based on the target data set to obtain a trained decision maker; recalibrating the pictures of the source data set based on the trained decision maker to obtain new labels; and training an image segmentation model based on the source data set and the new labels to obtain a trained image segmentation model. The system includes: an obtaining module, a first training module, a recalibration module and a second training module. The device includes a memory and a processor for executing the above-mentioned image segmentation method based on recalibrated labels. By using the present invention, the problem that the existing outlier pixels interfere with the neural network training, resulting in a decrease in the network accuracy and a deterioration in the robustness, can be solved, thereby improving the annotation image segmentation accuracy. The present invention can be widely applied to the field of artificial intelligence image processing.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence image processing, and in particular, to an image segmentation method, system and device based on recalibrated labels. Background Art

[0002] At present, with the continuous improvement of the memory and computing power of hardware devices, deep learning algorithms have been widely implemented and have shown great promise in fields such as classification, segmentation, and object detection. For the task of deep learning image semantic segmentation, the most commonly used method is the supervised neural network method. In order to make the network have higher accuracy and stronger robustness, a large number of accurately labeled image tags are required for the network to learn. However, the annotation of medical images is mainly manual by doctors. For annotation objects with a relatively thin width that only occupy one or two pixels (such as blood vessels, hyphae, nerves, etc.), affected by the human eye resolution, the annotation of the center line will have deviations. This deviation is not very obvious to the naked eye, but it makes the accuracy of local annotation almost zero. When using these noisy tags to supervise the network learning, it will have a negative impact on the network recognition ability. Specifically, the existence of noise points forces the network to focus on outliers during learning, making the final result difficult to converge, and ultimately the accuracy of the network will be reduced due to the misguidance of the noisy tags. Summary of the Invention

[0003] The purpose of the present invention is to provide an image segmentation method, system and device based on recalibrated labels, aiming to solve the problems of reduced network accuracy and poor robustness caused by the interference of existing outlier pixels on neural network training, and improve the accuracy of annotated image segmentation.

[0004] The first technical solution adopted by the present invention is: an image segmentation method based on recalibrated labels, comprising the following steps:

[0005] Obtain a target dataset of the same type as the source dataset;

[0006] Select a decision maker and train the decision maker based on the target dataset to obtain a trained decision maker;

[0007] Recalibrate the images in the source dataset based on the trained decision maker to obtain new labels;

[0008] Train an image segmentation model based on the source dataset and the new labels to obtain a trained image segmentation model.

[0009] Further, the step of recalibrating the images in the source dataset based on the trained decision maker to obtain new labels specifically includes:

[0010] Recalibrate the images in the source dataset based on the trained decision maker to obtain recalibrated labels;

[0011] Based on the morphological closing operation, dilation and erosion operations are successively performed on the rescaled labels to obtain new labels.

[0012] Furthermore, the step of rescaling the images of the source dataset based on the trained decision maker to obtain the rescaled labels specifically includes:

[0013] Based on the trained decision maker, select a preset calibration method according to the recognition object to rescale the images of the source dataset to obtain the rescaled labels;

[0014] The preset calibration methods include continuous rescaling method, binarization rescaling method, and weighted rescaling method.

[0015] Furthermore, the formula of the continuous rescaling method is expressed as follows:

[0016]

[0017] In the above formula, n represents the number of decision makers, l i represents the decision output, l manual represents the biased label of manual annotation, binary(·) represents the binarization operation, and l final represents the rescaled label.

[0018] Furthermore, the formula of the binarization calibration method is expressed as follows:

[0019]

[0020] In the above formula, n represents the number of decision makers, l i represents the decision output, l manual represents the biased label of manual annotation, binary(·) represents the binarization operation, and l final represents the rescaled label.

[0021] Furthermore, the formula of the weighted rescaling method is expressed as follows:

[0022]

[0023] In the above formula, n represents the number of decision makers, l i represents the decision output, l manual represents the biased label of manual annotation, λ and μ represent preset model parameters, binary(·) represents the binarization operation, and l final represents the rescaled label.

[0024] The second technical solution adopted by the present invention is: An image segmentation system based on rescaled labels, including:

[0025] An acquisition module, configured to acquire a target data set of the same type as the source data set;

[0026] A first training module, configured to select a decision maker and train the decision maker based on the target data set to obtain a trained decision maker;

[0027] A recalibration module, which recalibrates the images in the source data set based on the trained decision maker to obtain new labels;

[0028] A second training module, which trains an image segmentation model based on the source data set and the new labels to obtain a trained image segmentation model.

[0029] The third technical solution adopted by the present invention is: An image segmentation device based on recalibrated labels, comprising:

[0030] At least one processor;

[0031] At least one memory, configured to store at least one program;

[0032] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned image segmentation method based on recalibrated labels.

[0033] The beneficial effects of the method, system and device of the present invention are: The present invention adopts the methods of contrast learning and majority voting, and corrects a single manually annotated label through multiple decision makers, which alleviates to a certain extent the problem of annotation deviation caused by factors such as resolution in manual annotation. At the same time, the decisions of other decision makers supplement the information learned from the standard data set in the same field, and this information is reliable, making the corrected annotation more consistent with the annotation style of the standard data set and more accurate, alleviating the problem caused by mislabeling in manual annotation, thereby improving the accuracy of labeled image segmentation. Description of the Drawings

[0034] Figure 1 is a flowchart of the steps of an image segmentation method based on recalibrated labels of the present invention;

[0035] Figure 2 is a structural block diagram of an image segmentation system based on recalibrated labels of the present invention. Detailed Embodiments

[0036] The following further elaborates the present invention in detail with reference to the drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of description and explanation, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0037] As Figure 1 shown, the present invention provides an image segmentation method based on recalibrated labels, and the method comprises the following steps:

[0038] S1. Obtain a target data set of the same type as the source data set;

[0039] Specifically, the target data is preferably homogeneous with the source biased data. Taking medical images as an example, if the source data set is a confocal image of the cornea, then the target data to be found is preferably also the same confocal image of the cornea, so that the prior knowledge of the two discriminators is consistent with the knowledge required for the discrimination of the source data set. At the same time, the annotation of the target data set must be accurate, so that the learned knowledge can be used to correct the labels of the source data set.

[0040] S2. Select discriminators and train the discriminators based on the target data set to obtain the trained discriminators;

[0041] Specifically, select two or more from different neural network models as the required discriminators, and train the discriminators with the selected target data set. The number of neural network discriminators is preferably an even number, and make a decision together with the discriminator of the manual annotation of the source data set. An odd number of discriminators is easier to determine the decision method. The model segmentation of the neural network can use common structures such as UNet, UNet++, laddernet or transformer.

[0042] S3. Recalibrate the pictures of the source data set based on the trained discriminators to obtain new labels;

[0043] S3.1. Recalibrate the pictures of the source data set based on the trained discriminators to obtain the recalibrated labels;

[0044] Specifically, based on the trained discriminators, select a preset calibration method according to the recognition object to recalibrate the pictures of the source data set to obtain the recalibrated labels; the preset calibration methods include a continuous recalibration method, a binarization recalibration method and a weighted calibration method. For the given n discriminators, the output set label = {l1, l2,..., l n} of the decisions at each pixel point, and the biased label l manual of the manual annotation, the voting schemes can be divided into the following three types:

[0045] The formula of the continuous recalibration method is expressed as follows:

[0046]

[0047] In the above formula, n represents the number of discriminators; l iIt represents the judgment output, which contains the information learned by the judge from the standard data set. This information represents the common features of similar data sets, and it will be used to correct the pixels that are obviously offset in the manual annotation. manual Indicates the manually annotated biased label, which is the basis of the dataset annotation and also participates in part of the input; binary(·) indicates the binarization operation, l final Represents the recalibrated label. The original label l i or manual It was replaced and no longer used. final It will be used as the label for subsequent training and testing.

[0048] The formula of the binary calibration method is as follows:

[0049]

[0050] In the above formula, n represents the number of decision makers; l i It represents the judgment output, which contains the information learned by the judge from the standard data set. This information represents the common features of similar data sets, and it will be used to correct the pixels that are obviously offset in the manual annotation. manual Indicates the manually annotated biased label, which is the basis of the dataset annotation and also participates in part of the input; binary(·) indicates the binarization operation, l final Represents the recalibrated label. The original label l i or manual It was replaced and no longer used. final It will be used as the label for subsequent training and testing.

[0051] The formula of the weighted calibration method is as follows:

[0052]

[0053] In the above formula, n represents the number of decision makers, λ and μ represent the preset model parameters, and l i It represents the judgment output, which contains the information learned by the judge from the standard data set. This information represents the common features of similar data sets, and it will be used to correct the pixels that are obviously offset in the manual annotation. manual Indicates the manually annotated biased label, which is the basis of the dataset annotation and also participates in part of the input; binary(·) indicates the binarization operation, l final Represents the recalibrated label. The original label l i or manual It was replaced and no longer used. final It will be used as the label for subsequent training and testing.

[0054] The continuous recalibration method will obtain a finer centerline than the binary recalibration, and the shape is more similar to the original manual annotation. However, it is also prone to breakage and difficult recognition due to being too thin. Specifically, which scheme to use can be determined according to the different recognition objects.

[0055] S3.2. Based on the morphological closing operation, perform dilation operation and erosion operation on the recalibrated label in sequence to obtain a new label.

[0056] Specifically, for the segmentation problem, due to the problem of the decision maker itself, it may cause the target nerve fibers in the final annotation output to break. At this time, the morphological closing operation can be used, that is, dilation first and then erosion, to achieve the purpose of stitching the broken parts. Whether to use this post-processing, as well as the determination of the type and size of the final kernel function of the closing operation, need to be determined according to the specific situation.

[0057] S4. Based on the source dataset and the new label, train an image segmentation model to obtain a trained image segmentation model.

[0058] Specifically, use the recalibrated label as the final label of the model for subsequent training and testing. In addition, it should be noted that this recalibration is only performed on the model training set, and a model is trained with it. When testing, the original label of the test set is still used, otherwise it will be difficult to compare with other algorithms.

[0059] The advantages of the present invention include: making use of both the accurately annotated samples in the source dataset and correcting the misannotated samples in the source dataset; using real images instead of homogeneous datasets generated by algorithms, and the samples used for training are closer to the real scenario; the result has strong interpretability and there is not too much algorithm process with subjective settings.

[0060] As Figure 2 shown, an image segmentation system based on recalibrated labels includes:

[0061] An acquisition module for acquiring a target dataset of the same type as the source dataset;

[0062] A first training module for selecting a decision maker and training the decision maker based on the target dataset to obtain a trained decision maker;

[0063] A recalibration module for recalibrating the pictures in the source dataset based on the trained decision maker to obtain a new label;

[0064] A second training module for training an image segmentation model based on the source dataset and the new label to obtain a trained image segmentation model.

[0065] The content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented in the system embodiments of the present invention are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0066] An image segmentation device based on recalibrated labels:

[0067] At least one processor;

[0068] At least one memory for storing at least one program;

[0069] When the at least one program is executed by the at least one processor, the at least one processor implements an image segmentation method based on recalibrated labels as described above.

[0070] The content in the above method embodiments is applicable to the device embodiments of the present invention. The functions specifically implemented in the device embodiments of the present invention are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0071] A storage medium storing instructions executable by a processor, characterized in that: the instructions executable by the processor are used to implement an image segmentation method based on recalibrated labels as described above when executed by the processor.

[0072] The content in the above method embodiments is applicable to the storage medium embodiments of the present invention. The functions specifically implemented in the storage medium embodiments of the present invention are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0073] The above is a specific description of the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. An image segmentation method based on recalibrated labels, characterized in that, Including the following steps: Obtain a target data set of the same type as the source data set; Select a decision maker and train the decision maker based on the target data set to obtain a trained decision maker; Re-calibrate the images in the source data set based on the trained decision maker to obtain new labels; Based on the source data set and the new labels, train an image segmentation model to obtain a trained image segmentation model; The step of re-calibrating the images in the source data set based on the trained decision maker to obtain new labels specifically includes: Re-calibrate the images in the source data set based on the trained decision maker to obtain re-calibrated labels; Based on morphological closing operation, perform dilation operation and erosion operation on the re-calibrated labels in sequence to obtain new labels; The step of re-calibrating the images in the source data set based on the trained decision maker to obtain re-calibrated labels specifically includes: Based on the trained decision maker, select a preset calibration method according to the recognition object to re-calibrate the images in the source data set to obtain re-calibrated labels; The preset calibration methods include a continuous re-calibration method, a binarization re-calibration method, and a weighted re-calibration method; The formula of the continuous re-calibration method is expressed as follows: In the above formula, n represents the number of decision-makers, and l i represents the decision output, and l manual represents the biased label marked manually, and binary(·) represents the binarization operation, and l final represents the recalibrated label; The formula of the binarization re-calibration method is expressed as follows: The formula of the weighted re-calibration method is expressed as follows: In the above formula, λ and μ represent preset model parameters.

2. An image segmentation system based on recalibrated labels, characterized in that, Used to execute the image segmentation method based on re-calibrated labels as described in claim 1, including: An acquisition module for acquiring a target data set of the same type as the source data set; A first training module for selecting a decision maker and training the decision maker based on the target data set to obtain a trained decision maker; A re-calibration module for re-calibrating the images in the source data set based on the trained decision maker to obtain new labels; A second training module for training an image segmentation model based on the source data set and the new labels to obtain a trained image segmentation model.

3. An image segmentation device based on recalibrated labels, characterized in that, Including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements an image segmentation method based on re-calibrated labels as described in claim 1.

Images