Image classification model training method, classification method, device and computer equipment

Abstract

The embodiments of the present specification provide a training method, a classification method, an apparatus and a computer device of an image classification model. The method comprises: obtaining mild medical images and severe medical images respectively labeled with disease types; training an initial image classification model based on a severe medical image subgraph and a disease type corresponding to the severe medical image subgraph to obtain a primary image classification model; inputting a mild medical image subgraph into the primary image classification model to obtain a predicted disease type of the mild medical image subgraph; if the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the primary image classification model using the mild medical image subgraph to obtain an image classification model. By focusing on the influence of disease severity on lesion distribution density, without labeling the medical image subgraph, the accuracy of the image classification model in identifying the disease type in the medical image can be improved to a certain extent.

Description

TECHNICAL FIELD

[0001] The embodiments of the present specification relate to the field of image processing, and in particular, to a training method and a classification method of an image classification model, an apparatus and a computer device. BACKGROUND

[0002] At present, medical image examination plays an important role in assisting doctors to make medical diagnosis. Due to the large size of part of the medical images, the medical images are difficult to be directly input into the classification model for training due to the limitation of the image classification model on the input image size. The existing technical solutions mainly use a multi-instance learning weakly supervised learning method to cut the medical images into multiple medical sub-images, and take the label category of the medical image as the label category of the medical sub-image corresponding to the medical image, and then classify the medical image and the medical sub-image.

[0003] Due to the difference in lesion distribution density in the training samples of medical images in different disease stages, if the disease category of the mild medical image with low lesion distribution density is taken as the disease category of the mild medical image, more label noise will be introduced, which will limit the accuracy of the classification model for medical image classification to a certain extent. SUMMARY

[0004] The embodiments of the present specification provide a training method and a classification method of an image classification model, an apparatus and a computer device, which can improve the accuracy of medical image classification to a certain extent.

[0005] One embodiment of the present specification provides a training method of an image classification model, comprising: obtaining mild medical images and severe medical images respectively labeled with disease types; the mild medical images are medical images whose disease severity is marked as mild; the severe medical images are medical images whose disease severity is marked as severe; wherein the mild medical images include mild medical image sub-images representing part of the area; the severe medical images include severe medical image sub-images representing part of the area; training an initial image classification model based on the severe medical image sub-images and the disease types of the severe medical images to which the severe medical image sub-images belong, to obtain a primary image classification model; inputting the mild medical image sub-images into the primary image classification model to obtain the predicted disease types corresponding to the mild medical image sub-images; in the case that the predicted disease types of the mild medical image sub-images are the same as the disease types of the mild medical images to which the mild medical image sub-images belong, training the primary image classification model using the mild medical image sub-images to obtain an image classification model.

[0006] One embodiment of the present specification provides an image classification method, comprising: dividing an acquired medical image according to a preset number to obtain a plurality of local medical images; adjusting the local medical images to a specified resolution to obtain a plurality of medical image subgraphs; determining a disease type corresponding to the medical image subgraph using a training method of an image classification model as described in the above embodiment; and taking the disease type as a disease type of the medical image.

[0007] One embodiment of the present specification provides an image classification model training device, comprising: a training sample acquisition module configured to acquire a mild medical image and a severe medical image each labeled with a disease type; the mild medical image is a medical image whose disease severity is marked as mild; the severe medical image is a medical image whose disease severity is marked as severe; wherein the mild medical image comprises a mild medical image subgraph representing a partial region; the severe medical image comprises a severe medical image subgraph representing a partial region; a primary image classification model generation module configured to train an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, to obtain a primary image classification model; a mild medical image subgraph prediction module configured to input the mild medical image subgraph into the primary image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph; and an image classification model generation module configured to train the primary image classification model using the mild medical image subgraph in a case where the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, to obtain an image classification model.

[0008] One embodiment of the present specification provides an image classification device, comprising: a medical image division module configured to divide an acquired medical image according to a preset number to obtain a plurality of local medical images; a medical image subgraph generation module configured to adjust the local medical images to a specified resolution to obtain a plurality of medical image subgraphs; and a medical image classification module configured to determine a disease type corresponding to the medical image subgraph using a training method of an image classification model as described in the above embodiment; and taking the disease type as a disease type of the medical image.

[0009] One embodiment of the present specification provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method of the embodiment when executing the computer program.

[0010] One embodiment of the present specification provides a computer storage medium having a computer program stored thereon, the computer program is executed by a processor to implement the method of the embodiment.

[0011] The plurality of embodiments provided by the specification first pre-train the image classification model using the severe medical images to obtain a primary image classification model, then input the mild medical image subgraphs into the primary classification model to obtain a prediction result, select appropriate mild medical image subgraphs according to the prediction result for retraining, and thus obtain the image classification model, thereby improving the accuracy of the image classification model in classifying medical images to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 Fig. 1 shows a schematic diagram of a pathological image classification model in a scenario example provided by an embodiment of the specification.

[0013] Figure 2 Fig. 2 shows an interaction diagram of a pathological image classification system in a scenario example provided by an embodiment of the specification.

[0014] Figure 3 Fig. 3 shows a flowchart of a training method of an image classification model provided by an embodiment of the specification.

[0015] Figure 4 Fig. 4 shows a flowchart of an image classification method provided by an embodiment of the specification.

[0016] Figure 5 Fig. 5 shows a schematic diagram of a training device of an image classification model provided by an embodiment of the specification.

[0017] Figure 6 Fig. 6 shows a flowchart of an image classification device provided by an embodiment of the specification.

[0018] Figure 7 Fig. 7 shows a schematic diagram of a computer device provided by an embodiment of the specification. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the specification, the technical solutions in the embodiments of the specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only part of the embodiments of the specification, not all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the specification.

[0020] SUMMARY

[0021] In the related art, due to the large amount of data of the image of the pathological image, it is difficult to directly train the image classification model by the image compression method. Therefore, the pathological image needs to be divided into multiple pathological image subgraphs. In order to obtain the label of the pathological image subgraph, in addition to labeling the whole pathological image, the clinician also needs to label each pathological image subgraph. This image classification model training method based on full supervision learning needs the doctor to spend a lot of time for labeling, and due to the difference of the doctors, the quality of the training sample labeling is also different, thereby affecting the accuracy of the image classification model.

[0022] In order to reduce the labeling burden of the doctor, there are some attempts to train the image classification model by labeling each pathological image. Specifically, including: dividing the pathological image to obtain multiple pathological image subgraphs, and taking the category label of the pathological image as the category label of the pathological image subgraph, and then classifying the pathological image subgraph and the pathological image. But the lesion distribution density in pathological samples of different disease stages is different, so these weakly supervised learning methods based on multiple example learning introduce label noise in the training process, which limits the accuracy of the image classification model for pathological image classification.

[0023] Therefore, it is necessary to provide a method for improving the accuracy of the image classification model under the condition of labeling a small amount of training samples, by dividing the medical image into a light medical image with a mild disease severity and a heavy medical image with a severe disease severity, first using the heavy medical image to pre-train the image classification model to obtain a primary image classification model, then inputting the light medical image subgraph into the primary classification model to obtain a prediction result, selecting appropriate light medical image subgraphs for retraining according to the prediction result, thereby obtaining the image classification model, thereby improving the accuracy of the image classification model for medical image classification to a certain extent.

[0024] Scenario examples

[0025] Please refer to Figure 1An application scenario example of a pathological image classification model training method is provided in the specification. In order to obtain detailed information of a patient's body part, a doctor can remove part of the area from the patient's body part for pathological examination. Then, a 3DHistech slice scanner can be used to scan the pathological section to obtain a digital pathological image sent to the client for the doctor to view. The doctor can label the type of the digital pathological image and label the disease severity of the digital pathological image as severe, moderate or mild on the client. The server can be used to obtain the digital pathological image labeled by the doctor in the client as a training sample set. Among them, the severe medical image with the disease severity marked as severe, the moderate medical image with the disease severity marked as moderate, the mild medical image with the disease severity marked as mild, and the asymptomatic medical image without corresponding disease type. Among them, the asymptomatic medical image can be divided into a first medical image, a second medical image and a third medical image.

[0026] Firstly, each severe medical image and first medical image can be respectively divided into 256 local severe medical images or 256 local first medical images. 100 local severe medical images or local first medical images with lower ratio of background area to total pixel number are selected from the local severe medical images or local first medical images respectively and compressed into 256*256 severe medical image subgraphs or first medical image subgraphs. Moreover, the disease type marked by the severe medical image can be taken as the disease type corresponding to the severe medical image subgraph. The severe medical image subgraph and the first medical image subgraph are input into an initial convolutional neural network model for training to obtain a primary image classification model.

[0027] Then, each moderate medical image and second medical image can be respectively divided into 256 local moderate medical images or 256 local second medical images. 100 local moderate medical images or local second medical images with lower ratio of background area to total pixel number are selected from the local moderate medical images or local second medical images respectively and compressed into 256*256 moderate medical image subgraphs or second medical image subgraphs. The moderate medical image subgraph and the second medical image subgraph are input into the primary image classification model respectively to obtain the probability of the disease category corresponding to the moderate medical image subgraph and the second medical image subgraph. The disease category corresponding to the probability value greater than the preset threshold 0.5 probability is taken as the disease category predicted by the moderate medical image subgraph or the second medical image subgraph. The moderate medical image subgraph with the same disease category as the moderate medical image marked by the primary image classification model and the second medical image subgraph predicted to exist the disease type are set to not exist the corresponding disease type to train the primary image classification model to obtain a secondary image classification model.

[0028] Finally, each of the mild medical image and the third medical image can be respectively cut into 256 partial mild medical images or 256 partial third medical images. 100 partial mild medical images or partial third medical images with lower ratio of background area to total pixel are selected from the partial mild medical images or the partial third medical images, and compressed into a 256*256 mild medical image subgraph or a third medical image subgraph. The mild medical image subgraph and the third medical image subgraph are respectively input into the intermediate image classification model to obtain the similarity of the disease category corresponding to the mild medical image subgraph and the third medical image subgraph. The disease category corresponding to the similarity greater than the preset threshold 0.5 is taken as the disease category predicted by the mild medical image subgraph or the third medical image subgraph. The intermediate image classification model is trained using the disease category predicted by the mild medical image subgraph, the mild medical image subgraph with the same disease category as the labeled category of the mild medical image, and the third medical image subgraph predicted to exist the disease type, and the third medical image subgraph is set to not exist the corresponding disease type, to obtain the image classification model.

[0029] After the medical image is cut, a plurality of partial medical images are obtained, and then the partial medical images are adjusted to a specified resolution of 256*256 to obtain a plurality of medical image subgraphs. Then, the medical image subgraph is input into the image classification model to obtain the disease type corresponding to the medical image subgraph. The disease type corresponding to the medical image subgraph can be taken as the disease type of the medical image corresponding to the medical image subgraph. In addition, the position of the partial medical image corresponding to the medical image subgraph in the medical image can be taken as the area where the lesion is located in the medical image.

[0030] The above only provides one scene example for the present specification, and does not limit the present application. Any modification, equivalent replacement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0031] System architecture

[0032] Please refer to Figure 2The embodiments of the present specification provide a medical image classification system, and the medical image classification method provided by the present specification can be applied to the medical image classification system. The medical image classification system can include a hardware environment formed by a medical imaging device 110, a client 120, and a server 130. The medical imaging device 110 is connected with the client 120, and the server 130 is connected with the client 120 through a communication network. The communication network can be a wired network or a wireless network. The medical imaging device 110 performs examination and imaging on a body part of a patient to obtain a medical image. The medical imaging device 110 transmits the medical image to the client 120 through communication. The client 120 sends the medical image to the server 130, and the server 130 receives the medical image. The medical imaging device 110 can be, but is not limited to, at least one of a slice scanner, an ultrasonic medical device, a CT medical examination device, and an MRI medical examination device. The client 120 can be an electronic device with network access capability. Specifically, for example, the client can be a desktop computer, a tablet computer, a notebook computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a television, a smart speaker, a microphone, etc. The smart wearable device includes, but is not limited to, a smart bracelet, a smart watch, smart glasses, a smart helmet, a smart necklace, etc. Alternatively, the client can also be software capable of running in the electronic device. Those skilled in the art can know that the number of the above-mentioned clients 120 can be one or more, and the types thereof can be the same or different. For example, the above-mentioned clients 120 can be one, or the above-mentioned clients 120 can be dozens or hundreds, or more. The number and type of the clients 120 are not limited in the embodiments of the present application. The server 130 can be an electronic device with certain computing and processing capabilities. It can have a network communication module, a processor, a memory, etc. Of course, the server can also refer to software running in the electronic device. The server can also be a distributed server, which can be a system with multiple processors, memories, network communication modules, etc. working cooperatively. Alternatively, the server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server can also be a new technical means capable of realizing the corresponding functions of the embodiments of the present specification. For example, it can be a new form of "server" based on quantum computing.

[0033] Example method

[0034] Please refer to Figure 3 One embodiment of the present specification provides a training method of an image classification model. The training method of the image classification model can be applied to an electronic device. The training method of the image classification model can include the following steps.

[0035] Step S210: Obtain a mild medical image and a severe medical image respectively labeled with a disease type; the mild medical image is a medical image whose disease severity is marked as mild; the severe medical image is a medical image whose disease severity is marked as severe; wherein the mild medical image includes a mild medical image subgraph representing a partial region; the severe medical image includes a severe medical image subgraph representing a partial region.

[0036] In some cases, due to the large amount of data of the training sample, the training sample cannot be directly input into the image classification model for training, and the training sample needs to be divided into multiple medical image subgraphs. However, the categories of the medical image subgraphs are not labeled, and the categories of the medical images can be used as the categories of the medical image subgraphs. Since the medical images contain lesions of different densities at different stages of the disease course. Therefore, the medical images can be divided into mild medical images and severe medical images according to the disease severity of the patients represented by the medical images.

[0037] The mild medical image can be a medical image in which the lesion region accounts for a small proportion of the entire medical image according to the doctor after the patient undergoes medical image examination. Specifically, for example, the obtained medical image is a medical image obtained by scanning the pathological section of the patient after slicing the body part of the patient, and the data amount of the medical image obtained after scanning ranges from 18039*27000 to 54290*41820. The 3DHISTECH section scanner can send the medical image to the client, and the doctor can label the medical image as severe or mild on the client. Of course, the medical image can also be obtained by scanning the pathological section using other scanners. The medical image can also be obtained by checking the body part of the patient using an ultrasonic medical device, a CT medical examination device, and an MRI medical examination device.

[0038] The severe medical image can be a medical image in which the lesion region accounts for a large proportion of the entire medical image according to the doctor after the patient undergoes medical image examination.

[0039] The disease type is the type of the body part of the patient represented by the medical image. Specifically, for example, the pathological image of the muscle biopsy of the patient is scanned using the 3DHISTECH section scanner, and then dyed, and the disease type corresponding to the pathological image can include necrotizing myopathy, muscular dystrophy, etc. For another example, the lung of the patient is examined by medical image, and the disease type represented by the medical image can include lung nodules, pulmonary fibrosis, etc. The disease type in the training sample used by the embodiments of the present specification is not limited, and the above-mentioned disease types can be combined arbitrarily.

[0040] The severe medical image subgraph is obtained by cutting the severe medical image. Specifically, for example, the medical image can be cut according to a fixed size to obtain a plurality of medical image subgraphs. Among them, the size of the medical image subgraph can be set to 256*256 data amount, and the coincidence degree of adjacent medical images can be set to 0, or the coincidence degree of adjacent medical images can be set according to a certain threshold. For example, two adjacent images can be divided according to the coincidence degree of 20% along the horizontal axis and the coincidence degree of 25% along the vertical axis. Of course, the cutting method of the severe medical image subgraph can also be a combination form in which part of the adjacent severe medical image subgraphs have overlapping regions and part of the adjacent severe medical image subgraphs do not have overlapping regions.

[0041] The mild medical image subgraph can be obtained by cutting the mild medical image. The generation method of the mild medical image subgraph can be the same as the generation method of the severe medical image subgraph described above, and will not be repeated here.

[0042] Step S220: training an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, to obtain a primary image classification model.

[0043] In some cases, since the range of the region representing the lesion in the severe medical image is large and the density is high, the category corresponding to the severe medical image can be used as the category of the severe medical image subgraph. Then, the image classification model is trained based on the severe medical image subgraph and the category corresponding to the severe medical image subgraph, thereby reducing the annotation amount of the training samples.

[0044] The method of obtaining the primary image classification model can be inputting the medical image subgraph into a classic convolutional neural network LeNet as a classification model of the medical image subgraph. For example, in the training process, the learning rate adaptive method used is the Adam method, the initial learning rate is set to exp(-5), the batch size is set to 64, the number of iterations is set to 50 times, and the cross-entropy loss function is used to calculate the loss function of the classic convolutional neural network LeNet. The formula of the loss function is as follows:

[0045]

[0046] wherein, wherein y i represents the category label of the i-th severe pathological image sample, x ij is the j-th subgraph obtained by cutting the i-th severe pathological image sample, the function f1 is the primary image classification model trained by the severe medical image, and θ is the initialization parameter of the function f1. Among them, there are 30 severe medical images in this training, and each medical image is cut into 100 severe medical image subgraphs.

[0047] The initial image classification model can also be a convolutional neural network (CNN), a deep neural network (DNN), or a recurrent neural network (RNN), etc. The specific type of the initial image classification model is not limited in the present embodiment. The initial image classification model can include an input layer, a convolutional layer, a pooling layer, a connection layer, and other neural network layers, which are not specifically limited in the present embodiment. In addition, the number of each type of neural network layer is not limited in the present embodiment.

[0048] Step S230: inputting the mild medical image subgraph into the initial image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph.

[0049] In some cases, the above-mentioned initial image classification model has the ability to identify specific categories of medical images. Therefore, the mild medical image subgraph can be input into the initial image classification model to predict the mild medical image subgraph.

[0050] The predicted disease type is obtained by inputting the mild medical image subgraph into the initial image classification model. Specifically, for example, the disease types represented by the medical images of a certain body part can include five types A, B, C, D, and E, and the initial image classification model outputs a subgraph of the medical images of a certain body part and the similarity of the five types A, B, C, D, and E is 0.8, 0.6, 0.54, 0.47, and 0.21, respectively. According to the set threshold value of 0.7, the disease type corresponding to the subgraph is A. In some cases, multiple subgraphs of medical images of a certain body part can correspond to different disease types, which can indicate that the initial medical image classification model has made a mistake in classifying the medical images, or the patient's medical images represent that the patient has multiple diseases. In this case, we can take the disease category corresponding to the mild medical image subgraph with a similarity greater than 0.7 as the disease category of the mild medical image.

[0051] Step S240: in the case where the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the initial image classification model using the mild medical image subgraph to obtain an image classification model.

[0052] In some cases, the training samples (i.e., severe medical image subgraphs) used by the primary image classification model may not have a corresponding disease type, but when used as training samples, they are considered to be the disease type corresponding to the severe medical image, and the image classification model needs to be further corrected. Therefore, the mild medical image with the same predicted disease type as the mild medical image subgraph using the mild medical image subgraph and the disease type corresponding to the mild medical image can be used for further learning training.

[0053] The method of obtaining the image classification model can be further training the primary image classification model using the mild medical image subgraph whose predicted disease type by the primary image classification model is the same as the disease type labeled by the mild medical image, thereby obtaining the image classification model. Specifically, for example, the disease type labeled by the mild medical image is A, and the disease type predicted by the primary image classification model for the mild medical image subgraph is also A, then the mild medical image subgraph can be used as a training sample for correcting the primary image classification model. For another example, in the case that the disease type predicted by the primary image classification model for the mild medical image subgraph is B, the mild medical image subgraph is not used as a training sample for correcting the primary image classification model. Of course, in the case that the mild medical image is labeled with disease types A and B, the mild medical image subgraph predicted to be disease type A and / or B can be used as a training sample for correcting the primary image classification model. In the training process, the learning rate adaptive method used is the Adam method, and the initial learning rate is set to exp(-5). The batch size is set to 64, the number of iterations is set to 1, and the cross-entropy loss function used is as follows:

[0054]

[0055] wherein K i represents the number of mild medical image subgraphs whose predicted disease type by the primary image classification model is the same as the disease type of the mild medical image.

[0056] By dividing the medical images into mild medical images with mild disease severity and severe medical images with severe disease severity, the image classification model is first pre-trained using the severe medical images to obtain a primary image classification model, then the mild medical image subgraph is input into the primary classification model to obtain a prediction result, and the appropriate mild medical image subgraph is selected for further training according to the prediction result, thereby obtaining the image classification model, thereby improving the accuracy of the image classification model for medical image classification to a certain extent.

[0057] In some embodiments, the training of the primary image classification model using the mild medical image subgraph to obtain an image classification model can include training the primary image classification model using the mild medical image subgraph and the severe medical image subgraph to obtain an image classification model.

[0058] In some cases, in order to enable the image classification model to not discard some image features learned by the primary image classification model during the training process, the primary image classification model can also be trained based on the severe medical image subgraph and the predicted disease type being the same as the disease type of the mild medical image to obtain an image classification model.

[0059] In some embodiments, the training method of the image model can further include: obtaining a moderate medical image labeled with a disease type; wherein the moderate medical image is a medical image with a disease severity marked as moderate; inputting the moderate medical image into the primary image classification model to obtain a predicted disease type corresponding to the moderate medical image subgraph; and correspondingly, in the case that the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the primary image classification model using the mild medical image subgraph to obtain an image classification model can include: in the case that the predicted disease type of the moderate medical image subgraph is the same as the disease type of the moderate medical image to which the moderate medical image subgraph belongs, training the primary image classification model using the moderate medical image subgraph to obtain a middle-level image classification model; inputting the mild medical image subgraph into the middle-level image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph; and in the case that the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the middle-level image classification model using the mild medical image subgraph to obtain an image classification model.

[0060] In some cases, since only one transition from severe medical images to mild medical images using two layers cannot well reflect the hierarchical relationship of medical images at different disease stages, a medical image with a disease category marked as moderate can be introduced to construct a three-layer differential medical image subgraph noise from easy to difficult, thereby further improving the accuracy of the image classification model.

[0061] In the embodiment, after the primary image classification model is trained by the severe medical image subgraph, the moderate medical image subgraph can be input into the primary image classification model to obtain the predicted disease type of the moderate medical image, and the primary image classification model is updated and trained by the medical image whose predicted disease type is the same as the disease type labeled by the moderate medical image, to obtain the intermediate image classification model. Finally, the mild medical image subgraph is input into the intermediate image classification model to obtain the predicted disease type of the mild medical image, and the intermediate image classification model is updated and trained by the medical image whose predicted disease type is the same as the disease type labeled by the mild medical image, to obtain the image classification model. Of course, more levels of training processes from easy to difficult can also be constructed according to the needs of business scenarios, and only the medical image with greater lesion density or wider range is needed to be trained first, and finally the mild medical image labeled as the lightest disease severity is trained, so as to obtain the image classification model trained by multiple layers.

[0062] In some embodiments, the training of the intermediate image classification model using the mild medical image subgraph to obtain the image classification model can include training the intermediate image classification model using the mild medical image subgraph, the predicted disease type, and the moderate medical image subgraph with the same disease type as the moderate medical image, and the severe medical image subgraph to obtain the image classification model.

[0063] In some cases, in order to make the image classification model not discard some image features learned by the intermediate image classification model in the training process, the primary image classification model can also be trained based on the severe medical image subgraph, the predicted disease type, and the moderate medical image subgraph with the same disease type as the moderate medical image, and the predicted disease type and the mild medical image subgraph with the same disease type as the mild medical image, to obtain the image classification model. Of course, in the case of multiple levels of disease severity, the correct classification result in the previous model stage training process can be used as the training sample for the current model training, so that the image training model can avoid discarding the features of the remaining training samples in the training process of the current stage, and the purpose of further improving the classification accuracy of the image classification model for medical images can be achieved.

[0064] In some embodiments, the training method of the image classification model can further include: obtaining medical images labeled as having no disease; wherein the medical images include first medical images used to participate in generating a primary image classification model and second medical images used to participate in generating an image classification model; accordingly, in the step of training an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, to obtain a primary image classification model, can include: training an initial image classification model based on the severe medical image subgraph, the disease type of the severe medical image to which the severe medical image subgraph belongs, and a first medical image subgraph to obtain a primary image classification model; wherein the first medical image subgraph is a partial region of the first medical image; accordingly, in the step of training the primary image classification model using the mild medical image subgraph to obtain an image classification model, can include: training the primary image classification model using the mild medical image subgraph and a second medical image subgraph to obtain an image classification model; wherein the second medical image subgraph is a partial region of the second medical image.

[0065] In some cases, some medical images without disease can also be selected as negative sample labels input to the image classification model during the training process of the image classification model, so that the image classification model can learn that in some cases, the medical image subgraph does not exist corresponding disease category.

[0066] In the present embodiment, first, the medical images without disease can be divided into first medical images used to participate in generating a primary image classification model in the first stage and second medical images used to participate in generating an image classification model in the second stage. Then, an initial image classification model is trained based on the severe medical image subgraph, the disease type corresponding to the severe medical image subgraph, and the first medical image subgraph to obtain a primary image classification model. Finally, the primary image classification model is trained using the predicted disease type, the mild medical image subgraph with the same disease type as the mild medical image, and the second medical image subgraph to obtain an image classification model. It should be noted that the training stage corresponding to the first medical image and the second medical image in the present embodiment is not limited. In addition, the number of first medical images and second medical images in the present embodiment is not specifically limited; wherein the first medical images and the second medical images can be partially repeated.

[0067] In some embodiments, the step of training the primary image classification model using the mild medical image subgraph and the second medical image subgraph to obtain the image classification model can include: inputting the second medical image subgraph into the primary classification model to obtain a first prediction result of the second medical image; and correspondingly, training the primary image classification model using the mild medical image subgraph, the first prediction result of the second medical image subgraph, and a second medical image subgraph of a second medical image of a different disease type from the second medical image subgraph to obtain the image classification model.

[0068] In some cases, in the process of generating the image classification model in the second stage of training of the image classification model, in order to reduce the number of training times of the image classification model and improve the performance of the image classification model, the second medical image subgraph can be first input into the primary image classification model to obtain a first predicted disease category of the second medical image subgraph. In the case where the disease category corresponding to the second medical image subgraph is predicted incorrectly, the disease category corresponding to the second medical image subgraph is corrected to be non-existent, and then the first prediction result of the second medical image subgraph is used for the second medical image subgraph to participate in the training process of the image classification model.

[0069] In some embodiments, the training method of the image classification model can further include: dividing the severe medical image into a plurality of local severe medical images according to a preset number; wherein the number of local severe medical images is the same as the preset number; determining a local severe medical image for participating in the training of the image classification model according to the number of pixels representing the target region in the local severe medical image and the number of pixels in the local severe medical image; and adjusting the local severe medical image for participating in the training of the image classification model to a target resolution to obtain a severe medical image subgraph.

[0070] In some cases, in the process of dividing the medical image, due to the large proportion of non-target region pixels in some medical image subgraphs, these medical image subgraphs with a large proportion of non-target region pixels are not suitable for participating in the training process of the image classification model. Therefore, these medical image subgraphs with a large proportion of non-target region pixels can be excluded from the training samples.

[0071] The target region can be a region representing a body part in the medical image. Specifically, for example, in a CT image, the lung CT image of a patient needs to be obtained, but in the actual scanning process, the image also includes some information around the patient's lungs, which may affect the classification of the lung image. Therefore, in the process of selecting training samples, medical image subgraphs that affect the classification of the lung image need to be excluded.

[0072] In the embodiment, the heavy medical image can be first divided into a plurality of heavy medical image subgraphs with the same data amount. The ratio of the pixels of the non-target region corresponding to the plurality of heavy medical image subgraphs to the pixels in the heavy medical image subgraph is calculated, and the heavy medical image subgraphs are sorted from low to high according to the ratio. Then, according to the set threshold number, the heavy medical image subgraphs with high ranking are selected to participate in the training process of generating the primary image classification model. Specifically, for example, in the process of dividing the heavy medical image, a sliding window with a data amount of 1 / 16 of the heavy medical image in the horizontal and vertical directions is selected for sliding window, and no overlapping region is set between adjacent windows. The ratio of the non-target region pixels in the window is calculated during the sliding window process, and the first 100 local regions with a low ratio of non-target region pixels to sliding window are selected for downsampling processing, which is compressed into a 256*256 data amount of heavy medical image subgraph, so that each heavy medical image can be divided into 100 256*256 data amount of heavy medical image subgraphs. Of course, the moderate medical image subgraph, the light medical image subgraph, the first medical image subgraph, the second medical image subgraph and the like described in the specification can be obtained by the method described in the embodiment, and will not be described here.

[0073] In some embodiments, the step of inputting the light medical image subgraph into the primary image classification model to obtain the predicted disease type corresponding to the light medical image subgraph can include: inputting the light medical image subgraph into the primary image classification model to obtain the similarity of the light medical image subgraph and different disease types; in the case where the similarity is greater than a preset threshold, the disease type corresponding to the similarity is taken as the predicted disease type corresponding to the light medical image subgraph.

[0074] In some cases, the image classification model can calculate the similarity of the medical image subgraph and the disease type respectively, and return the disease category corresponding to the medical image subgraph according to the set threshold. Therefore, the predicted disease type of the image classification model can be determined by calculating the similarity of the light medical image subgraph and the corresponding disease type.

[0075] In this embodiment, the disease category corresponding to the mild medical image can be determined according to the similarity of the mild medical image and different disease categories. Specifically, for example, the disease categories of the mild medical image subgraphs are A, B, C, D, and E, and after calculation, the similarities of A, B, C, D, and E and the mild medical image subgraph are 0.9, 0.75, 0.64, 0.48, and 0.12 respectively. The threshold value is set to 0.7, so the disease types corresponding to the mild medical image subgraph are A and B. Of course, the disease type corresponding to the maximum similarity can also be determined as the disease type corresponding to the mild medical image subgraph according to the threshold value. Of course, if the similarity between A, B, C, D, and E and the mild medical image subgraph is less than 0.7, the disease category corresponding to the mild medical image subgraph is determined as non-existent.

[0076] In this embodiment, the mild medical image includes five mild medical image subgraphs containing disease types. The similarities between the five mild medical image subgraphs containing disease types and their corresponding disease types are 0.84, 0.72, 0.91, 0.85, and 0.76 respectively. Therefore, the probability of the mild medical image subgraph corresponding to the disease type can be set to 0.91.

[0077] Referring to Figure 4 One embodiment of the present specification provides an image classification method. The image classification method can be applied to an electronic device. The image classification method can include the following steps.

[0078] Step S310: The acquired medical image is divided according to a preset number to obtain a plurality of local medical images.

[0079] Step S320: Adjust the local medical image to a specified resolution to obtain a plurality of medical image subgraphs.

[0080] Step S330: The training method of the image classification model as described in the above embodiment is used to determine the disease type corresponding to the medical image subgraph; wherein the disease type is the disease type of the medical image.

[0081] In some cases, due to the large amount of data of part of the medical image, it is difficult to directly compress the medical image and input it into the image classification model for judgment. Therefore, the medical image can be divided into a plurality of medical image subgraphs, and then the medical image subgraph is input into the image classification model for judgment to obtain the disease type corresponding to the medical image subgraph, and the disease type corresponding to the medical image subgraph can be used as the disease type corresponding to the medical image.

[0082] The image classification method in the embodiment and the image classification model obtained by the training method of the image classification model in the above embodiment are the same, and will not be described here. It should be noted that in the embodiment, each medical image subgraph can be judged, and in order to ensure the accuracy of the medical image classification, the medical image subgraph with a large ratio of pixels representing non-target regions to the pixels of the medical image subgraph does not need to be removed.

[0083] In some embodiments, the image classification method can further include: taking the position of the local medical image corresponding to the medical image subgraph including the disease type in the medical image as the region of the lesion in the medical image.

[0084] In some cases, by identifying the region representing the lesion in the medical image, the time of the doctor reading the film can be reduced. Specifically, for example, the number of pixels of a pathological medical image is 18000*27000. It is divided in the horizontal and vertical directions respectively, so as to obtain 600 medical image subgraphs of 900*900. In such a case, if the doctor needs to view the whole medical image, it will waste a lot of time of the doctor. By labeling the medical image subgraph that may exist in the lesion region, the doctor only needs to view the labeled region in the medical image, so as to save a lot of time of the doctor, and also can avoid the visual fatigue of the doctor caused by long time reading the film, so as to misjudge the medical image.

[0085] Example apparatus, electronic device, storage medium, and software

[0086] Please refer to Figure 5 One embodiment of the present specification also provides a training device of an image classification model. The training device of the image classification model can include: a training sample acquisition module, a primary image classification model generation module, a mild medical image subgraph prediction module and an image classification model generation module.

[0087] The training sample acquisition module is configured to acquire mild medical images and severe medical images respectively labeled with disease types; the mild medical images are medical images with disease severity marked as mild; the severe medical images are medical images with disease severity marked as severe; wherein the mild medical images include mild medical image subgraphs representing part of the region; the severe medical images include severe medical image subgraphs representing part of the region.

[0088] The primary image classification model generation module is configured to train an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, to obtain a primary image classification model.

[0089] The mild medical image subgraph prediction module is configured to input the mild medical image subgraph into the primary image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph.

[0090] The image classification model generation module is configured to train the primary image classification model using the mild medical image subgraph in a case where the predicted disease type of the mild medical image subgraph is the same as a disease type of a mild medical image to which the mild medical image subgraph belongs, to obtain an image classification model.

[0091] Referring to Figure 6 One embodiment of the present specification also provides an image classification device. The image classification device can include a medical image segmentation module, a medical image subgraph generation module, and a medical image classification module.

[0092] The medical image segmentation module is configured to segment the obtained medical image according to a preset number to obtain a plurality of local medical images.

[0093] The medical image subgraph generation module is configured to adjust the local medical images to a specified resolution to obtain a plurality of medical image subgraphs.

[0094] The medical image classification module is configured to determine a disease type corresponding to the medical image subgraph using the method according to any one of claims 1-8, wherein the disease type is taken as a disease type of the medical image.

[0095] The modules involved in the image classification device provided by the present embodiment can be explained with reference to the foregoing embodiments, and will not be described again.

[0096] The specific functions and effects of the training device of the image classification model and / or the image classification device can be explained with reference to other embodiments of the present specification, and will not be described again. The modules in the training device of the image classification model can be realized by software, hardware, and combinations thereof. The modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0097] Referring to Figure 7 SUMMARY Scenario examples Figure 1 System architecture Figure 2 Example method Figure 3 Figure 4 Example apparatus, electronic device, storage medium, and software Figure 5 Figure 6 Figure 7 SUMMARY Scenario examples Figure 1 System architecture Figure 2 Example One embodiment of the present specification also provides a computer device, which can include a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the training method of the image classification model and / or the image classification method in any of the foregoing embodiments.

[0098] The embodiments of the present specification also provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a computer, causes the computer to perform the training method of the image classification model and / or the image classification method in any of the above embodiments.

[0099] The embodiments of the present specification also provide a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the training method of the image classification model and / or the image classification method in any of the above embodiments.

[0100] It can be understood that the specific examples herein are only to help those skilled in the art better understand the embodiments of the present specification, and not to limit the scope of the present application.

[0101] It can be understood that in various embodiments of the present specification, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present specification.

[0102] It can be understood that the various embodiments described in the present specification can be implemented alone or in combination, and the embodiments of the present specification do not limit this.

[0103] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present specification have the same meanings as those commonly understood by those skilled in the art of the present specification. The terms used in the present specification are only for the purpose of describing the specific embodiments and are not intended to limit the scope of the present specification. The term "and / or" used in the present specification includes any and all combinations of one or more of the related listed terms. The singular forms "a", "an" and "the" used in the embodiments of the present specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0104] It can be understood that the processor of the embodiments of the present specification can be an integrated circuit chip with processing capability. In the implementation process, each step of the method embodiments described above can be completed by integrated logic circuits in hardware or instructions in software form in the processor. The processor described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiments of the present specification can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present specification can be directly embodied as a hardware coding processor for execution, or a combination of hardware and software modules in the coding processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the storage, and the processor reads the information in the storage, and combines the hardware to complete the steps of the above method.

[0105] It can be understood that the memory in the embodiments of the present specification can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM). It should be noted that the memory of the system and method described herein is intended to include but not limited to these and any other suitable type of memory.

[0106] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present specification.

[0107] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0108] In several embodiments provided in the specification, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, and the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0109] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0110] In addition, each functional unit in each embodiment of the specification can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0111] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the specification or the essential part of the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the specification. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0112] The above is only a specific embodiment of the specification, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the specification, which should be covered within the protection scope of the specification. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A method for training an image classification model, the method comprising: The method comprises: obtaining a mild medical image and a severe medical image respectively marked with a disease type; the mild medical image is a medical image whose disease severity is marked as mild; the severe medical image is a medical image whose disease severity is marked as severe; wherein the mild medical image comprises a mild medical image subgraph representing a partial region; the severe medical image comprises a severe medical image subgraph representing a partial region; training an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, to obtain a primary image classification model; inputting the mild medical image subgraph into the primary image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph; in the case where the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the primary image classification model using the mild medical image subgraph to obtain an image classification model; wherein the mild medical image and the severe medical image are both taken from a training sample set, and the images contained in the training sample set are annotated by doctors in the client.

2. The method of claim 1, wherein, The training of the primary image classification model using the mild medical image subgraph to obtain an image classification model comprises: training the primary image classification model using the mild medical image subgraph and the severe medical image subgraph to obtain an image classification model.

3. The method of claim 1, wherein, The method further comprises: obtaining a moderate medical image marked with a disease type; wherein the moderate medical image is a medical image whose disease severity is marked as moderate; inputting the moderate medical image into the primary image classification model to obtain a predicted disease type corresponding to the moderate medical image subgraph; Correspondingly, in the case where the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the primary image classification model using the mild medical image subgraph to obtain an image classification model comprises: in the case where the predicted disease type of the moderate medical image subgraph is the same as the disease type of the moderate medical image to which the moderate medical image subgraph belongs, training the primary image classification model using the moderate medical image subgraph to obtain a middle-level image classification model; inputting the mild medical image subgraph into the middle-level image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph; in the case where the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, training the middle-level image classification model using the mild medical image subgraph to obtain an image classification model.

4. The method of claim 3, wherein, The training of the middle-level image classification model using the mild medical image subgraph to obtain an image classification model comprises: training the middle-level image classification model using the mild medical image subgraph, the moderate medical image subgraph, and the severe medical image subgraph to obtain an image classification model.

5. The method of claim 1, wherein, The method further comprises: acquire medical images labeled as no disease; wherein the medical images include first medical images for participating in generating a primary image classification model and second medical images for participating in generating an image classification model; Correspondingly, in the step of training an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, and obtaining a primary image classification model, comprises: training an initial image classification model based on the severe medical image subgraph, the disease type of the severe medical image to which the severe medical image subgraph belongs, and the first medical image subgraph, and obtaining a primary image classification model; wherein the first medical image subgraph is a partial region of the first medical image; Correspondingly, in the step of training the primary image classification model using the mild medical image subgraph to obtain an image classification model, comprises: training the primary image classification model using the mild medical image subgraph and the second medical image subgraph to obtain an image classification model; wherein the second medical image subgraph is a partial region of the second medical image.

6. The method of claim 5, wherein, The step of training the primary image classification model using the mild medical image subgraph and the second medical image subgraph to obtain an image classification model comprises: inputting the second medical image subgraph into the primary image classification model to obtain a first prediction result of the second medical image; Correspondingly, training the primary image classification model using the mild medical image subgraph, the first prediction result of the second medical image subgraph, and the second medical image subgraph of the second medical image subgraph with a different disease type, and obtaining an image classification model.

7. The method of claim 1, wherein, The method further comprises: splitting the severe medical image according to a preset number to obtain a plurality of local severe medical images; wherein the number of local severe medical images is the same as the preset number; determining a local severe medical image for participating in image classification model training according to the number of pixels representing the target region in the local severe medical image and the number of pixels in the local severe medical image; adjusting the local severe medical image for participating in image classification model training to a target resolution to obtain a severe medical image subgraph.

8. The method of claim 1, wherein, The step of inputting the mild medical image subgraph into the primary image classification model to obtain the predicted disease type corresponding to the mild medical image subgraph comprises: inputting the mild medical image subgraph into the primary image classification model to obtain the similarity of the mild medical image subgraph and different disease types; in the case where the similarity is greater than a preset threshold, the disease type corresponding to the similarity is taken as the predicted disease type corresponding to the mild medical image subgraph.

9. An image classification method characterized by, The method comprises: splitting the acquired medical images according to a preset number to obtain a plurality of local medical images; adjusting the local medical images to a specified resolution to obtain a plurality of medical image subgraphs; determining the disease type corresponding to the medical image subgraph using the method of any one of claims 1-8; wherein the disease type is taken as the disease type of the medical image.

10. The method of claim 9, wherein, The method further comprises: locating a local medical image corresponding to a medical image subgraph comprising a disease type in the medical image as a region of a lesion in the medical image.

11. A training device for an image classification model, characterized in that, Comprise: a training sample acquisition module, configured to acquire a mild medical image and a severe medical image respectively labeled with a disease type; the mild medical image is a medical image whose disease severity is marked as mild; the severe medical image is a medical image whose disease severity is marked as severe; wherein the mild medical image comprises a mild medical image subgraph representing a partial region; the severe medical image comprises a severe medical image subgraph representing a partial region; a primary image classification model generation module, configured to train an initial image classification model based on the severe medical image subgraph and the disease type of the severe medical image to which the severe medical image subgraph belongs, to obtain a primary image classification model; a mild medical image subgraph prediction module, configured to input the mild medical image subgraph into the primary image classification model to obtain a predicted disease type corresponding to the mild medical image subgraph; an image classification model generation module, configured to train the primary image classification model using the mild medical image subgraph in a case where the predicted disease type of the mild medical image subgraph is the same as the disease type of the mild medical image to which the mild medical image subgraph belongs, to obtain an image classification model; wherein the mild medical image and the severe medical image are both taken from a training sample set, and the training sample set contains images annotated by doctors in the client.

12. An image classification apparatus characterized by comprising: Comprise: a medical image segmentation module, configured to segment the acquired medical image according to a preset number to obtain a plurality of local medical images; a medical image subgraph generation module, configured to adjust the local medical images to a specified resolution to obtain a plurality of medical image subgraphs; a medical image classification module, configured to determine the disease type corresponding to the medical image subgraph using the method of any one of claims 1-8; wherein the disease type is the disease type of the medical image.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to realize the method of any one of claims 1-8. The processor executes the computer program to realize the method of any one of claims 1-8.

Images