Multi-label classification method and multi-label classification system

Abstract

To provide a multiple label classification method for generating a marking label of a medical image.SOLUTION: A method comprises: a step for acquiring an initial data set including a medical image and a local input label for marking a marked part out of multiple abnormal characteristics in the medical image; a step for training a first multiple label classification model using the initial data set; a step for estimating a difficulty level of the medical image in the initial data set on the basis of prediction generated by the first multiple label classification model; a step for classifying the initial data set into different sub sets, according to the difficulty level of the medical image; a step for training a second multiple label classification model by a sub set in which the difficulty level gradually increases in multiple different course learning rounds; and a step for generating a prediction label for marking the plurality of abnormal characteristics in the medical image on the basis of the second multiple label classification model, for fully marking the all abnormal characteristics.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a method for classifying medical images, and more particularly to a classification method for generating multiple labeled marks for a single medical image. [Background technology]

[0002] Medical imaging tests, such as X-ray scans, magnetic resonance images, and computed tomography scans, are important tools for assessing patient conditions. Accurately identifying abnormalities in medical images typically requires experienced medical staff. Therefore, to improve the efficiency of performing medical imaging tests and reduce labor costs, it is desirable to build machine learning models that can detect and classify abnormal conditions in medical images. Summary of the Invention

[0003] One aspect of the present disclosure discloses a multi-label classification method including the steps of: obtaining an initial dataset including a plurality of medical images and a plurality of local input labels that mark marked portions of a plurality of abnormal features in the medical images; training a first multi-label classification model based on the initial dataset; estimating a plurality of difficulty levels of the medical images in the initial dataset based on predictions generated by the first multi-label classification model; dividing the initial dataset into at least a first subset and a second subset having a higher difficulty level than the first subset according to the difficulty levels of the medical images; training a second multi-label classification model based on the first subset in a first course learning round; training the second multi-label classification model based on the first subset and the second subset in a second course learning round; and generating a plurality of predicted labels that mark each of the abnormal features in the medical images using the second multi-label classification model.

[0004] In some embodiments, before training the first multi-label classification model, the multi-label classification method further includes performing image preprocessing on the plurality of medical images in the initial dataset, the preprocessing including at least one of image background removal, image window sampling, and sequence image stacking.

[0005] In some embodiments, there are M types of potential abnormal features in each of the medical images, and the local input labels include N types of potential abnormal features. Positive input labels or Negative Input Labels where M and N are positive integers and M>N, an unmarked portion of one of the abnormal features corresponding to the medical image in the initial dataset is unknown, and the first multi-label classification model includes a convolutional neural network, and the first multi-label classification model is trained according to the local input labels based on a masked binary cross-entropy loss function and does not consider the unmarked portion of the abnormal features.

[0006] In some embodiments, estimating a plurality of difficulty levels for the medical images in the initial dataset includes generating a plurality of probability values ​​for each of the abnormal features in the medical images using the first multi-label classification model; and estimating the difficulty levels according to a difficulty assessment function using the probability values ​​and the local input labels.

[0007] In some embodiments, the second multi-label classification model includes a convolutional neural network, and the second multi-label classification model is trained based on a masked binary cross-entropy loss function according to the local input labels.

[0008] In some embodiments, the medical images include a plurality of head computed tomography images, and the abnormal features include cerebral parenchymal hemorrhage, intraventricular hemorrhage, subarachnoid hemorrhage, subdural intracranial hemorrhage, and epidural hemorrhage.

[0009] In some embodiments, the second multi-label classification model is configured to classify, for a single medical image, cerebral parenchymal hemorrhage, intraventricular hemorrhage, subarachnoid hemorrhage, subdural intracranial hemorrhage, and epidural hemorrhage. Positive input labels or Negative Input Labels are used to generate five predicted labels.

[0010] In some embodiments, the multi-label classification method further includes generating a plurality of confidence values ​​corresponding to the predicted labels in the second multi-label classification model; calculating an absolute error based on the confidence values ​​and the local input labels; and displaying the predicted labels based on the order of the absolute error.

[0011] In some embodiments, the multi-label classification method further includes collecting modification instructions for modifying the predicted labels; obtaining a plurality of modified input labels according to the modification instructions; and training a third multi-label classification model with reference to the modified input labels in a plurality of course learning rounds.

[0012] Another aspect of the present disclosure is a multi-label classification system comprising: a storage unit for storing a plurality of computer-executable instructions; and a processing unit coupled to the storage unit for executing the computer-executable instructions to construct a first multi-label classification model and a second multi-label classification model, the processing unit comprising: obtaining an initial dataset including a plurality of medical images and a plurality of local input labels marking marked portions of a plurality of abnormal features in the medical images; training the first multi-label classification model based on the initial dataset; and calculating a prediction based on the predictions generated by the first multi-label classification model. and generating a plurality of predicted labels for marking each of the abnormal features in the medical images using the second multi-label classification model.

[0013] As a result, the predicted labels generated by the second multi-label classification model completely mark all abnormal features in each medical image, and there are no missing or unknown labels in the predicted labels. Furthermore, if the predicted labels generated by the second multi-label classification model are not applicable to the situation, medical staff can complete the verification and inspection with less time and effort. In this case, the multi-label classification method is used to efficiently obtain a multi-label classification model with high accuracy at a relatively low cost.

[0014] It should be noted that the above description and the following detailed description are provided to exemplify the present disclosure as examples, and are intended to aid in the interpretation and understanding of the content of the invention claimed by the present disclosure. [Brief explanation of the drawings]

[0015] To make the above and other objects, features and embodiments of the present disclosure more clear and understandable, the accompanying drawings are described below. [Figure 1] 1 is a flowchart illustrating a multi-label classification method according to some embodiments of the present disclosure. [Figure 2] FIG. 1 is a functional block diagram illustrating a multi-label classification system in accordance with some embodiments of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram illustrating an image background removal step performed on an original medical image to generate a preprocessed medical image. [Figure 4] FIG. 2 is a schematic diagram illustrating an image window sampling step performed on an original medical image to generate a preprocessed medical image. [Figure 5] FIG. 1 is a schematic diagram illustrating a sequence image stacking step performed on a series of original medical images to generate a preprocessed medical image. [Figure 6] FIG. 2 is a schematic diagram illustrating the multi-label classification method of FIG. 1. [Figure 7] 2 is a flowchart illustrating further steps involved in the multi-label classification method shown in FIG. 1 according to some embodiments of the present disclosure. [Figure 8] FIG. 10 is a schematic diagram illustrating the display of relevant information about a predicted label that does not apply to a display according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0016] Various embodiments or examples for implementing various features of the present disclosure are disclosed below. In the following description, parts and arrangements for simplifying the present disclosure in specific examples are used in the following discussion. Any examples discussed are for interpretation purposes only and are not intended to limit the scope and meaning of the present disclosure or examples in any way. Where appropriate, the same reference numerals are used to represent the same or similar parts among the drawings and in corresponding character descriptions.

[0017] Please refer to Figure 1, which is a flowchart illustrating a multi-label classification method 100 according to some embodiments of the present disclosure. The multi-label classification method 100 is used to generate, corresponding to each medical image, multiple labels for multiple abnormal features that may potentially appear in the image.

[0018] Please also refer to Figure 2, which shows a functional block diagram of a multi-label classification system 200 in some embodiments of the present disclosure, which may be used to perform the multi-label classification method 100 shown in Figure 1. As shown in Figure 2, the multi-label classification system 200 includes an input interface 220, a processing unit 240, a storage unit 260, and a display 280. In some embodiments, the multi-label classification system 200 may be a computer, a smartphone, a tablet computer, an image processing server, a data storage server, a tensor computing server, or other equivalent computing device.

[0019] The storage unit 260 is used to store computer-executable instructions. The processing unit 240 is coupled to the input interface 220, the storage unit 260, and the display 280. The processing unit 240 is used to execute the computer-executable instructions to implement the multi-label classification model discussed in subsequent examples.

[0020] 1 and 2, the input interface 220 of the multi-label classification system 200 performs step S110 to receive / acquire an initial dataset Dini from a data source (not shown). In some embodiments, the data source may be a data server that stores medical chart reports in a hospital. In some embodiments, the initial dataset Dini includes a plurality of medical images IMG and a plurality of local input labels PLB.

[0021] The input interface 220 is used to receive the initial dataset Dini, and may include a data transmission interface, a wireless communication circuit, a keyboard, a mouse, a microphone, or any equivalent input device. The processing unit 240 is connected to the input interface 220, the storage unit 260, and the display 280. The storage unit 260 is used to store program code. The program code stored in the storage unit 260 is used to drive the processing unit 240 to perform the multi-label classification method 100 shown in FIG. 1. In some embodiments, the processing unit 240 may be a processor, a graphics processor, an application specific integrated circuit, or any equivalent processing circuit. The display 280 may be a display panel, a monitoring screen, a projector, a touch panel, or any equivalent display.

[0022] In some embodiments, the plurality of medical images IMG includes head computed tomography images obtained by photographing a patient suffering from intracranial hemorrhage. In a practical situation, the plurality of medical images IMG may have different abnormal features, each corresponding to a different type of intracranial hemorrhage, such as cerebral parenchymal hemorrhage, intraventricular hemorrhage, subarachnoid hemorrhage, subdural intracranial hemorrhage, and epidural hemorrhage.

[0023] To accurately mark the different abnormal features of intracranial hemorrhage in medical images (e.g., computed tomography or magnetic angiography), the characteristics of each type of intracranial hemorrhage must be analyzed and understood in detail. An overview of the five types of intracranial hemorrhage is shown in Table 1 below. [Table 1]

[0024] Accurately marking intracranial hemorrhage in medical images is crucial for subsequent treatment and prognosis. That is, distinguishing between different types of intracranial hemorrhage, such as IPH, IVH, SAH, SDH, and EDH, is crucial for providing accurate subsequent treatment plans. Each type of hemorrhage requires a different treatment strategy, such as surgery, drainage, or conservative treatment, depending on its location and characteristics. Accurate diagnosis helps medical staff prevent complications such as increased intracranial pressure and brain herniation, monitors patient progression, and ensures prompt adjustment of treatment measures as needed. Essentially, accurate intracranial hemorrhage marking is crucial for customized and effective medical care and can significantly impact patient treatment outcomes. In some embodiments, the multi-label classification method 100 provides a method for efficiently training a multi-label classification model that can identify different types of intracranial hemorrhage and generate corresponding labels.

[0025] In some cases, a large amount of marking data is required to train a multi-label classification model. Manually marking each medical image in a training dataset with various abnormal features, such as IPH, IVH, SAH, SDH, and EDH, requires a large amount of medical manpower and time. To speed up the marking process, medical staff can search for relevant medical chart reports using keywords and retrieve relevant medical images from the medical chart reports that match the keywords. For example, to retrieve medical images with the feature "EDH," medical staff can enter keywords such as "EDH" and "epidural hemorrhage" to search for medical images with these keywords in the corresponding medical chart reports. Medical staff can directly mark the medical images as having the abnormal feature "EDH," or the medical staff can mark them after further verification. However, the rapid marking process achieved by the automated process described above may contain some data errors or omissions.

[0026] First, some labels are missing from the mark data. For example, a medical image of intracranial hemorrhage may be marked with a single abnormal feature label, such as "EDH." However, in reality, this medical image of intracranial hemorrhage may also contain other types of abnormal features (e.g., IPH, IVH, SAH, or SDH). If these other abnormal features are not recorded in the medical record report, the labels automatically generated by the medical record report may not accurately and completely reflect the actual mark information of these medical images. Second, the labels automatically generated by the medical record report may contain some incorrect interpretations. For example, if the medical record report states "test results show no EDH" or "patient does not have EDH," the correct label corresponding to the abnormal feature "EDH" should be "negative." In some cases, the keyword "EDH" may be detected without considering context interpretation, resulting in the automatically generated label being erroneously marked as "positive."

[0027] In some embodiments, the local input labels PLB in the initial dataset Dini can be automatically generated by medical reports (e.g., keyword comparison from medical chart reports). The local input labels PLB mark the marked portions of multiple abnormal features in the medical images IMG. See Table 2, which is a list of local input labels PLB, which in one illustrative example is used to display the marked portions of multiple medical images IMG1-IMGk in the initial dataset Dini. [Table 2]

[0028] As shown in Table 2, the local input label PLB corresponding to the first medical image IMG1 includes a "positive" input label for the first abnormal feature IPH on the first medical image IMG1 and a "negative" input label for the third abnormal feature SAH. In other words, the local input label PLB indicates that the first abnormal feature IPH is present in the first medical image IMG1 and that the third abnormal feature SAH is absent in the first medical image IMG1. Note that some other abnormal features (IVH, SDH, and EDH) in the local input label PLB for the first medical image IMG1 are currently unmarked. Whether these unmarked abnormal features are present in the first medical image IMG1 is unknown to the local input label PLB. In other words, the information in the initial dataset Dini has not yet determined whether the abnormal features IVH, SDH, and EDH are present in the first medical image IMG1. In this case, the local input label PLB for the first medical image IMG1 includes two confirmed labels.

[0029] Similarly, as shown in Table 2, the local input labels PLB corresponding to the second medical image IMG2 include a "positive" input label for the second abnormal feature IVH in the second medical image IMG2 and a "negative" input label for the fourth abnormal feature SDH. That is, the abnormal features for some of the local input labels PLB for the second medical image IMG2 (for IPH, SAH, and EDH) are not marked. In other words, based on the information of the initial dataset Dini, it is not determined whether the following abnormal features IPH, SAH, and EDH exist in the second medical image IMG2.

[0030] In other words, there is a possibility that M types of abnormal features are potentially present in each of the medical images IMG1 to IMGk, and the local input label PLB is a label related to N types of abnormal features for each of the medical images IMG1 to IMGk. Positive input labels or Negative Input Labels where M and N are positive integers and M>N.

[0031] 1 and 2, the processing unit 240 performs step S120 to perform image pre-processing on each medical image IMG in the initial data set Dini. In some embodiments, the image pre-processing includes at least one of image background removal, image window sampling, and sequential image stacking.

[0032] Please refer to Figure 3, which is a schematic diagram of performing an image background removal step S121 on an original medical image IMGa to generate a preprocessed medical image IMGp1. As shown in Figure 3, in the image background removal step S121, the brain region is cut out from the black background of the original medical image IMGa, and the brain region cut out in the preprocessing is enlarged to the size of the medical image IMGp1. In this case, after the image background removal step S121, the image information of the brain region can be retained as much as possible and enlarged and displayed in the medical image IMGp1, so that key information useful for model training can be extracted and retained in the preprocessed medical image IMGp1.

[0033] Please refer to Figure 4, which is a schematic diagram of performing an image window sampling step S122 on an original medical image IMGa to generate preprocessed medical images IMGp2 and IMGp3. As shown in Figure 4, in the image window sampling step S122, a contrast image adjustment is performed on the pixel values ​​of the original medical image IMGb. In this case, subdural window range sampling is performed on the original medical image IMGb to generate a preprocessed medical image IMGp2, and skeletal window value range sampling is performed on the original medical image IMGb to generate another preprocessed medical image IMGp3. In this case, subdural or skeletal image features are more clearly visible in the preprocessed medical images IMGp2 and IMGp3.

[0034] Please also refer to FIG. 5, which is a schematic diagram illustrating the sequential image stacking step S123 performed on a series of original medical images IMGc, IMGd, IMGe, IMGf, and IMGg to generate a preprocessed medical image IMGp4. As shown in FIG. 5, the original medical images IMGc, IMGd, IMGe, IMGf, and IMGg may be a series of sequential images captured during a patient's head computed tomography examination. In the sequential image stacking step S123, multiple adjacent original medical images may be integrated / stacked as a preprocessed medical image. For example, after preprocessing, three adjacent medical images IMGd, IMGe, and IMGf may be stacked as medical image IMGp4. Similarly, multiple adjacent medical images at other positions may be stacked as a similar preprocessed medical image. In this case, features in multiple medical images adjacent in scanning order may be stored / integrated into the same medical image IMGp4 after preprocessing.

[0035] In some embodiments, these preprocessed medical images can be used instead of the original medical images in subsequent steps S130-S170 of the multi-label classification method 100. In other embodiments, the image preprocessing step S120 can be skipped and the original medical images can be directly employed in subsequent steps S130-S170 of the multi-label classification method 100.

[0036] Please further refer to Figure 6, which is a schematic diagram illustrating the multi-label classification method 100 of Figure 1. As shown in Figures 1 and 5, after pre-processing the multiple medical images IMG in the initial dataset Dini by step S120, the initial dataset Dini includes multiple pre-processed medical images IMGp and multiple local input labels PLB.

[0037] In some embodiments, as shown in FIGS. 1, 2, and 6, the processing unit 240 performs step S130 to train a first multi-label classification model MD1 based on an initial dataset Dini. The initial dataset Dini includes preprocessed medical images IMGp and local input labels PLB. In some embodiments, the first multi-label classification model MD1 includes a convolutional neural network. The convolutional neural network may include a convolutional layer, an activation layer, a pooling layer, and / or a fully connected layer for classification. The first multi-label classification model MD1 can be trained using a backpropagation algorithm with a reward policy. The reward policy is defined by a loss function. In some embodiments, the first multi-label classification model MD1 is trained using a masked binary cross-entropy loss function with the local input labels PLB, where the loss function does not consider unmarked portions of abnormal features.

[0038] In step S130, the first multi-label classification model MD1 is used to generate predictions corresponding to various abnormal features in the medical image IMGp, and the predicted results of each abnormal feature are compared with the local input label PLB to calculate a loss value, thereby adjusting the weights / parameters in the convolutional neural network of the first multi-label classification model MD1.

[0039] In some embodiments, processing unit 240 calculates the masked binary cross-entropy loss function according to the following formula:

number

[0040] In the above formula (1), y i is the actual label of the medical image in the initial dataset Dini (based on the local input label PLB),

number

[0041] As shown in Figures 1, 2 and 6, the processing unit 240 performs step S140 to estimate the difficulty level of each medical image IMGp in the initial dataset based on the predictions generated by the first multi-label classification model MD1.

[0042] After the training of the first multi-label classification model MD1 is completed, the first multi-label classification model MD1 can generate a probability value corresponding to each abnormal feature in each medical image, as shown in Table 3. [Table 3]

[0043] In Table 3, the probability value corresponding to the abnormal feature "IPH" in the first medical image IMG1 is "0.82," which is close to 1, meaning that the first multi-label classification model MD1 predicts that the abnormal feature "IPH" is more likely to be present in the first medical image IMG1. The probability value corresponding to the abnormal feature "IVH" in the first medical image IMG1 is "0.22," which is closer to 0, meaning that the first multi-label classification model MD1 predicts that the abnormal feature "IVH" is less likely to be present in the first medical image IMG1. In some embodiments, the above probability values ​​can be generated by a convolutional neural network in the first multi-label classification model MD1.

[0044] In step S140, the processing unit 240 estimates the difficulty level of each medical image according to the following difficulty assessment function:

number

[0045] In equation (2), y is the actual label for one abnormal feature of a medical image (based on the local input label PLB) in the initial dataset Dini. If the label is "positive", y=1. If the label is "negative", y=0. In equation (2),

number

number

number

[0046] In some embodiments, in step S140, five abnormal features corresponding to each medical image IMGp can be estimated with five difficulty values, respectively. The maximum value among the five estimated difficulty values ​​is regarded as the difficulty level of this target medical image. Please refer to Table 4 for further details. It is a list of difficulty levels, which is used to indicate the difficulty level of each medical image IMG1-IMGk in the illustrative example. [Table 4]

[0047] The processing unit 240 executes step S150 to divide the (pre-processed) medical images IMG1 to IMGk in the initial data set Dini into different subsets G1 to G3 according to the difficulty level estimated from each of the medical images IMG1 to IMGk.

[0048] For example, medical images IMG1 and IMG2 with the lowest difficulty level may be divided into a first subset G1, medical images IMG3 and IMGk with the next lowest difficulty level may be divided into a second subset G2, and medical image IMG4 with the highest difficulty level may be divided into a third subset G3.

[0049] In some embodiments, the medical images IMG1-IMGk may be divided into 10 different subsets based on the difficulty level. In the present disclosure, the number of subsets is not limited to a specific number, and the total number of subsets after division may be adjusted according to the actual application and characteristics of the data.

[0050] As shown in FIGS. 1 and 2, the processing unit 240 executes step S160 to train a second multi-label classification model MD2 using subsets G1-G3 of increasing difficulty in different course learning rounds.

[0051] 6, in the first course learning round R1 in step S160, a second multi-label classification model MD2 is first trained based on a first subset G1 (including medical images IMG1 and IMG2 and their corresponding local input labels PLB) with the lowest difficulty level. In some embodiments, the time length of the first course learning round R1 may be set to one epoch calculation time.

[0052] Then, in step S160, in a second course learning round R2, a second multi-label classification model MD2 is again trained based on the first subset G1 and the second subset G2 (including the medical images IMG3 and IMGk and their corresponding local input labels PLB), where the difficulty level of the second subset G2 is higher than that of the first subset G1. In some embodiments, the time length of the second course learning round R2 may be set to one epoch calculation time.

[0053] Then, in step S160, in a third course learning round R3, a second multi-label classification model MD2 is again trained based on the first subset G1, the second subset G2, and the third subset G3 (including the medical image IMG4 and its corresponding local input label PLB), where the difficulty level of the third subset G3 is higher than that of the first subset G1 and the second subset G2. In some embodiments, the time length of the third course learning round R3 may be set to one epoch calculation time.

[0054] As shown in the above example, the second multi-label classification model MD2 is initially trained using the first subset G1 with the lowest difficulty level, allowing the second multi-label classification model MD2 to establish a certain level of prediction accuracy using training data with low difficulty. Next, in different course learning rounds, the second multi-label classification model MD2 is repeatedly trained using training data subsets G1-G3 with increasing levels of difficulty. In this case, the second multi-label classification model MD2 can sequentially acquire processing capabilities for training data with increasing levels of difficulty through multiple course learning rounds.

[0055] The three coarse training rounds R1-R3 shown in Figure 6 are described as examples, and the present disclosure is not limited thereto. In some embodiments, when the medical images in the initial dataset Dini are divided into two subsets, there are two coarse training rounds to sequentially train the second multi-label classification model MD2. In other embodiments, when the medical images in the initial dataset Dini are divided into ten subsets, there are ten coarse training rounds to sequentially train the second multi-label classification model MD2.

[0056] In some embodiments, the second multi-label classification model MD2 includes a convolutional neural network (CNN). The convolutional neural network may include a convolutional layer, an activation layer, a pooling layer, and / or a fully connected layer for classification. The second multi-label classification model MD2 can be trained with a reward policy using a backpropagation algorithm. The reward policy is defined by a loss function. In some embodiments, the second multi-label classification model MD2 is trained with sequentially selected subsets and corresponding local input labels PLB in each course learning round based on a masked binary cross-entropy loss function.

[0057] 1 and 2, the processing unit 240 performs step S170 to generate predicted labels FLB marked for each medical image based on the second multi-label classification model MD2, where the predicted labels FLB cover all abnormal features in the medical image.

[0058] In the illustrative example, the second multi-label classification model MD2 can generate five corresponding predicted labels for each medical image, which are positive or negative predicted labels for the five abnormal features IPH, IVH, SAH, SDH, and EDH, respectively. As shown in Table 5, it shows a list of predicted labels FLB that mark all abnormal features corresponding to each medical image in the illustrative example. [Table 5]

[0059] As shown in Table 5, the predicted labels FLB generated by the second multi-label classification model MD2 perfectly marks all abnormal features IPH, IVH, SAH, SDH, and EDH in each medical image. In this case, there are no missing / unknown labels in the predicted labels FLB.

[0060] The complete predicted label FLB generated by the second multi-label classification model MD2 can be used as an aid or auxiliary reference for doctors or medical staff in medical diagnosis, helping to provide effective treatment to patients. As shown in FIG. 2, the predicted label FLB (shown in Table 5) can be displayed on the display 280. In some cases, this allows medical staff (or patients) to quickly learn whether various abnormal features, such as IPH, IVH, SAH, SDH, and EDH, are present in medical images of the patient taken by head computed tomography (CT). This allows medical staff to quickly and accurately respond to the detected abnormal features (e.g., provide treatment or propose a treatment plan).

[0061] Because the second multi-label classification model MD2 may still make incorrect predictions in the process of generating the predicted labels FLB, in some embodiments the predicted labels FLB may still need to be reviewed and confirmed by a doctor, medical staff, or clinical scientist.

[0062] Please refer further to Figure 7, which is a flowchart illustrating further steps included in the multi-label classification method 100 shown in Figure 1 according to some embodiments of the present disclosure. As shown in Figure 7, after the multi-label classification method 100 completes steps S110-S170 described in the above embodiments, the multi-label classification method 100 further includes steps S181-S185 of inspecting and correcting the predicted labels.

[0063] As shown in FIGS. 2 and 7, the processing unit 240 performs step S181 to generate a plurality of confidence values ​​corresponding to the plurality of predicted labels FLB based on the second multi-label classification model MD2.

[0064] These confidence values ​​are the confidence of the second multi-label classification model MD2 for the predicted label FLB. In some embodiments, the confidence values ​​can be generated by a convolutional neural network in the second multi-label classification model MD2. If the second multi-label classification model MD2 is confident in the target predicted label, the confidence value for the target predicted label is closer to 1. On the other hand, if the second multi-label classification model MD2 is not confident in the target predicted label, the confidence value for the target predicted label is closer to 0. For example, if the confidence value for the abnormal feature "IPH" is "0.82" (closer to 1), this means that the second multi-label classification model MD2 predicts that the abnormal feature "IPH" is likely to be present in the medical image. On the other hand, if the confidence value calculated for the abnormal feature "IVH" in the medical image is "0.22" (closer to 0), this means that the second multi-label classification model MD2 predicts that the abnormal feature "IVH" is unlikely to be present in the medical image.

[0065] In this case, the processing unit 240 is used to generate a confidence value for each predicted label FLB. The processing unit 240 is used to compare the predicted labels FLB generated by the second multi-label classification model MD2 with the input labels (based on the local input labels PLB in the initial dataset Dini). Here, there is a chance that some predicted labels FLB differ from the input labels. If the predicted labels FLB do not correspond to the input labels, in step S182, the absolute errors of the incorresponding predicted labels FLB can be calculated based on both the confidence values ​​and the local input labels PLB, and the incorresponding predicted labels FLB can be displayed on the display 280 in order of absolute errors. In some embodiments, the absolute errors can be calculated in the same manner as the difficulty evaluation function shown in Equation (2), i.e., the absolute errors are calculated based on the confidence values ​​and the local input labels PLB. For example, the absolute errors can be calculated according to the following Equation (3):

number

[0066] As shown in equation (3), y is the actual label (based on the local input label PLB) for the abnormal feature of the medical image in the initial dataset Dini. If the input label is positive, y=1. If the input label is negative, y=0. In equation (3),

number

number

number

[0067] See also Figure 8, which is a schematic diagram of displaying associated information INFO about inapplicable predicted labels on a display 280 according to some embodiments.

[0068] Assume that the predicted label FLB for the second medical image IMG2 does not correspond to the local input label PLB, and the calculated absolute error for the second medical image IMG2 is 0.95. This means that the predicted label FLB for the second medical image IMG2 generated by the second multi-label classification model MD2 differs from the initial label in the local input label PLB, and at the same time, the second multi-label classification model MD2 itself has a high confidence value for its prediction. There are two potential causes for this situation: First, the local input label PLB for the second medical image IMG2 itself is incorrect, or the predicted label FLB is incorrect. In this case, medical staff should take the time to carefully verify whether the label marked for the second medical image IMG2 is correct. As shown in Figure 8, the displayed information INFO includes the complete predicted label FLB for the second medical image IMG2 and is displayed at the top of the list with the highest priority.

[0069] Assume that the predicted label FLB of the fourth medical image IMG4 does not match the local input label PLB, and the calculated absolute error for the fourth medical image IMG4 is 0.77. In this case, the medical staff should also take the time to carefully check whether the label marked on the fourth medical image IMG4 is correct. As shown in Figure 8, the displayed information INFO includes the complete predicted label FLB for the fourth medical image IMG4 and is displayed second in the order, below the complete predicted label FLB for the second medical image IMG2.

[0070] Similarly, if there are more medical images with predicted labels that do not fall under the local input label PLB, then those medical images can be displayed on the display 280 in order of their absolute errors.

[0071] In this case, medical staff (e.g., a doctor, medical staff, or clinical scientist) can efficiently check for inapplicable predicted labels according to the order of absolute error. If the medical staff finds that the predicted label FLB is incorrect, the medical staff can input a correction instruction via the input interface 220. The correction instruction may include whether the medical staff agrees or disagrees with the input of the predicted label FLB.

[0072] In step S183, the input interface 220 is used to collect a modification command CMD for modifying each predicted label FLB, where the modification command CMD includes agreeing or disagreeing with the predicted label FLB.

[0073] In step S184, the processing unit 240 can obtain the modified input label according to the modification command CMD, and combine both the local input label PLB and the modification command CMD to generate the modified input label. In some embodiments, in the above combination, the modification command CMD (collected based on the manual input of the medical staff) has a higher priority than the local input label PLB.

[0074] In step S185, the processing unit 240 is used to iteratively train the third multi-label classification model through multiple rounds of course learning using the modified input labels. In this case, the modified input labels have been inspected and verified by medical staff, and therefore are more reliable than the local input labels PLB automatically generated from the medical records. Therefore, by referring to the modified input labels, the third multi-label classification model can achieve a higher accuracy rate than the second multi-label classification model MD2 through multiple rounds of course learning. Here, the details of the execution of iteratively training the third multi-label classification model through multiple rounds of course learning using the modified input labels in step S185 are similar to those of iteratively training the second multi-label classification model MD2 through the local input labels PLB in step S160, and therefore will not be described again here.

[0075] Based on the above embodiment, if a situation occurs in which the predicted label generated by the second multi-label classification model MD2 does not match, medical staff can complete the verification and examination with less time and effort. Furthermore, the multi-label classification method 100 can automatically generate corrected input labels and train a third multi-label classification model, thereby achieving a higher accuracy rate. In this case, the multi-label classification method 100 can be used to efficiently obtain a highly accurate multi-label classification model at a relatively low cost.

[0076] Although specific embodiments of the present disclosure have been disclosed above, these embodiments are not intended to limit the present disclosure. Those skilled in the art can make various substitutions and improvements to the present disclosure without departing from the principle and spirit of the present disclosure. Therefore, the scope of protection of the present disclosure is determined by the appended claims. [Explanation of symbols]

[0077] 100 Multi-Label Classification Methods 200 Multi-label Classification Systems 220 Input Interface 240 processing units 260 Memory Unit 280 display S110, S120, S130, S140 steps S150, S160, S170 steps S181, S182, S183, S184, S185 steps Dini initial dataset IMG1, IMG2, IMG3, IMG4, IMGk Medical Images IMG, IMGa, IMGb, IMGc, IMGd, IMGe Medical Images IMGf, IMGg Medical Images IMGp, IMGp1, IMGp2, IMGp3, IMGp4 Medical Images MD1 First multi-label classification model MD2 Secondary multi-label classification model PLB Local Input Label FLB predicted labels CMD modification instructions G1, G2, G3 subsets R1 First Course Learning Round R2 Second Course Learning Round R3 Third Course Learning Round INFO Information

Claims

1. obtaining an initial dataset comprising a plurality of medical images and a plurality of local input labels marking marked portions of a plurality of abnormal features in the medical images; training a first multi-label classification model based on the initial dataset; estimating a plurality of difficulty levels for the medical images in the initial dataset based on the predictions generated by the first multi-label classification model; dividing the initial data set into at least a first subset and a second subset having a higher difficulty level than the first subset according to the difficulty level of the medical images; training a second multi-label classification model based on the first subset in a first course learning round; training the second multi-label classification model based on the first subset and the second subset in a second course learning round; generating a plurality of predicted labels marking each of the abnormal features in the medical image using the second multi-label classification model; A computer-implemented multi-label classification method comprising:

2. Before training the first multi-label classification model, the multi-label classification method includes:

2. The computer-implemented multi-label classification method of claim 1, further comprising: performing image preprocessing on the plurality of medical images in the initial dataset, the preprocessing including at least one of image background removal, image window sampling, and sequence image stacking.

3. 2. The computer-implemented multi-label classification method of claim 1, wherein the medical images each potentially contain M types of abnormal features, the local input labels indicate positive input labels or negative input labels for N types of abnormal features, where M and N are positive integers and M>N, an unmarked portion of one of the abnormal features corresponding to the medical images in the initial dataset is unknown, and the first multi-label classification model includes a convolutional neural network, and is trained using the local input labels based on a masked binary cross-entropy loss function and does not consider the unmarked portion of the abnormal features.

4. The step of estimating a plurality of difficulty levels of the medical images in the initial dataset comprises: generating a plurality of probability values ​​for each of the abnormal features in the medical image with the first multi-label classification model; and estimating the difficulty level according to a difficulty assessment function based on the probability value and the local input label.

5. 2. The computer-implemented method of claim 1, wherein the second multi-label classification model includes a convolutional neural network, and the second multi-label classification model is trained based on a masked binary cross-entropy loss function with the local input labels.

6. 2. The computer-implemented multi-label classification method of claim 1, wherein the medical images include a plurality of head computed tomography images, and the abnormal features include cerebral parenchymal hemorrhage, intraventricular hemorrhage, subarachnoid hemorrhage, subdural intracranial hemorrhage, and epidural hemorrhage.

7. 7. The computer-implemented multi-label classification method of claim 6, wherein the second multi-label classification model is used to generate five predicted labels for a single medical image, including positive input labels or negative input labels for cerebral parenchymal hemorrhage, intraventricular hemorrhage, subarachnoid space hemorrhage, subdural intracranial hemorrhage, and epidural hemorrhage.

8. generating a plurality of confidence values ​​corresponding to the predicted labels in the second multi-label classification model; calculating an absolute error based on the confidence values ​​and the local input labels; displaying the predicted labels based on the order of the absolute errors; The computer-implemented multi-label classification method of claim 1 further comprising:

9. collecting correction instructions regarding correction of the predicted labels; obtaining a plurality of modified input labels according to the modification instructions; training a third multi-label classification model with reference to the modified input labels over multiple course learning rounds; 10. The computer-implemented multi-label classification method of claim 8, further comprising:

10. a storage unit for storing a plurality of computer-executable instructions; a processing unit, coupled to the storage unit, for executing the computer-executable instructions to construct a first multi-label classification model and a second multi-label classification model; The processing unit, when executing the plurality of computer-executable instructions, obtaining an initial dataset comprising a plurality of medical images and a plurality of local input labels marking marked portions of a plurality of abnormal features in the medical images; training the first multi-label classification model based on the initial dataset; estimating a plurality of difficulty levels of the medical images in the initial dataset based on the predictions generated by the first multi-label classification model; Dividing the initial data set into at least a first subset and a second subset having a higher difficulty level than the first subset according to the difficulty level of the medical images; training a second multi-label classification model based on the first subset in a first course learning round; training the second multi-label classification model based on the first subset and the second subset in a second course learning round; A multi-label classification system that uses the second multi-label classification model to generate a plurality of predicted labels that mark each of the abnormal features in the medical image.

Images