Medical image recognition model training method, recognition method, device and equipment

Abstract

The embodiments of the present application disclose a medical image recognition model training method, recognition method, device and equipment, which extracts a target area image composed of M slice images from a three-dimensional medical image. Select N consecutive slice images from the target area image as training samples and input them into the deep learning model to be trained to obtain recognition prediction results, where M and N are positive integers. The training samples correspond to the central slice image recognition labels. Substitute the recognition prediction results and the recognition labels into the weighted cross entropy loss function to obtain the cross entropy loss value. The deep learning model to be trained is trained according to the cross entropy loss value. The weighted cross entropy loss function has weights corresponding to the recognition label categories, and the weights are determined according to the number of training samples with the recognition label category, the total number of training samples and the number of recognition label categories. The sum of the weights multiplied by the correction coefficient is the number of recognition label categories. The setting of the weights can alleviate the low model recognition accuracy caused by the small number of positive samples.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a medical image recognition model training method, recognition method, device and equipment. Background Art

[0002] In technologies such as CT (Computed Tomography), patients are scanned using X-rays, etc. to obtain the patient's original data, and three-dimensional medical images of the patient can be reconstructed based on the original data.

[0003] With the development of computer vision and artificial intelligence, it is possible to use artificial intelligence technology to identify target objects in 3D medical images. Whether the target object is included in the 3D medical image helps doctors analyze the patient's condition and can serve as a reference for doctors to analyze the patient's condition.

[0004] At present, deep learning models can be used to identify target objects in three-dimensional medical images, but there are usually problems with low recognition accuracy and poor recognition effect. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a medical image recognition model training method, recognition method, device and equipment for training a medical image recognition model with high recognition accuracy, and the target object in the medical image identified based on the medical image recognition model is more accurate.

[0006] To solve the above problems, the technical solutions provided in the embodiments of the present application are as follows:

[0007] A training method for a medical image recognition model, the method comprising:

[0008] Extracting a target area image from a three-dimensional medical image, wherein the target area image is composed of M slice images, where M is a positive integer;

[0009] Inputting a training sample into a deep learning model to be trained to obtain a recognition prediction result of the training sample, wherein the training sample is composed of N consecutive slice images selected from the target area image, and the recognition label of the training sample is the recognition label of the central slice image of the training sample, where N is a positive integer and is less than or equal to M;

[0010] Substituting the recognition prediction result and the recognition label of the training sample into the weighted cross entropy loss function to obtain a cross entropy loss value, wherein the weighted cross entropy loss function sets a corresponding weight for each recognition label category, wherein the weight is determined according to the number of training samples with the recognition label category, the total number of training samples, and the number of the recognition label categories, and the sum of the weights multiplied by the correction coefficient is the number of the recognition label categories;

[0011] The deep learning model to be trained is trained according to the cross entropy loss value, and the step of inputting the training samples into the deep learning model to be trained is re-executed until a preset condition is met to obtain a medical image recognition model.

[0012] In a possible implementation, when the three-dimensional medical image is a lung CT image, extracting the target area image from the three-dimensional medical image includes:

[0013] Identify the lung area in the lung CT image and obtain a lung binary image;

[0014] Filling the holes or gaps in the binary lung image to obtain a filled binary lung image;

[0015] Performing a difference operation on the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area;

[0016] Determine a binary image of the middle mediastinum region according to the binary image of the lungs;

[0017] The binary image of the intrapulmonary tissue region and the binary image of the mid-mediastinum region are combined to obtain a combined binary image, and a circumscribed cubic region of the combined binary image is cut out from the lung CT image as a target region image.

[0018] In a possible implementation, identifying the lung region in the lung CT image to obtain a lung binary image includes:

[0019] Determine an area in a lung CT image that meets a preset CT value range, set the area in the preset CT value range as a first pixel value, and set other areas in the lung CT image as a second pixel value, to generate a preprocessed binary image;

[0020] Identifying a connected domain region from the preprocessed binary image, and extracting a maximum connected domain region from the preprocessed binary image as a candidate lung binary image;

[0021] If the starting coordinates of the candidate lung binary image meet the preset conditions, the candidate lung binary image is determined as a lung binary image, and the region of the lung binary image having the first pixel value is a lung region;

[0022] If the starting coordinates of the candidate lung binary image do not meet the preset conditions, the preset CT value range is reset, and the steps of determining the area that meets the preset CT value range from the lung image and subsequent steps are re-executed.

[0023] In a possible implementation, determining the binary image of the mediastinum region according to the binary image of the lungs includes:

[0024] Indenting the circumscribed cubic area of ​​the lung binary image inward by a preset distance to obtain an indented cubic area;

[0025] The indented cubic region is set to a first pixel value, and the lung region within the indented cubic region is set to a second pixel value, so as to obtain a binary image of the middle mediastinum region.

[0026] In a possible implementation, the method further includes:

[0027] Randomly select at least one random transformation method;

[0028] The slice image is randomly transformed according to a target random transformation method within a transformation range corresponding to the target random transformation method to regenerate the slice image, and the target random transformation method is each of the at least one selected random transformation method.

[0029] A medical image target recognition method, the method comprising:

[0030] Extracting a target area image from the medical image to be identified, wherein the target area image is composed of M slice images, where M is a positive integer;

[0031] Inputting N consecutive slice images selected from the target area image into a medical image recognition model to obtain a recognition result of a central slice image of the N consecutive slice images; the N consecutive slice images are the i-th to the i+N-1-th slice images, i is each integer in the range of 1 to M-N+1, i and N are positive integers, and N is less than or equal to M; the medical image recognition model is trained according to the training method for the medical image recognition model according to any one of claims 1 to 5;

[0032] When the number of target recognition results of the central slice image recognition results is greater than the number threshold, it is determined that the medical image to be recognized contains the target recognition object.

[0033] In a possible implementation, when the medical image to be identified is a lung CT image, extracting the target area image from the medical image to be identified includes:

[0034] Identify the lung area in the lung CT image and obtain a lung binary image;

[0035] Filling the holes or gaps in the binary lung image to obtain a filled binary lung image;

[0036] Performing a difference operation on the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area;

[0037] Determine a binary image of the middle mediastinum region according to the binary image of the lungs;

[0038] The binary image of the lung tissue region and the binary image of the middle mediastinum region are combined to obtain a combined binary image, and a circumscribed cubic region of the combined binary image is cut out from the lung CT image as a target region image.

[0039] A training device for a medical image recognition model, the device comprising:

[0040] An extraction unit, used for extracting a target area image from a three-dimensional medical image, wherein the target area image is composed of M slice images, where M is a positive integer;

[0041] An input unit, used to input a training sample into a deep learning model to be trained to obtain a recognition prediction result of the training sample, wherein the training sample is composed of N consecutive slice images selected from the target area image, and the recognition label of the training sample is the recognition label of the central slice image of the training sample, and N is a positive integer, which is less than or equal to M;

[0042] An acquisition unit is used to substitute the recognition prediction result and the recognition label of the training sample into a weighted cross entropy loss function to obtain a cross entropy loss value, wherein the weighted cross entropy loss function sets a corresponding weight for each recognition label category, wherein the weight is determined according to the number of training samples with the recognition label category, the total number of training samples, and the number of the recognition label categories, and the sum of the weights multiplied by the correction coefficient is the number of the recognition label categories;

[0043] A training unit is used to train the deep learning model to be trained according to the cross entropy loss value, and re-execute the input of training samples into the deep learning model to be trained until a preset condition is met to obtain a medical image recognition model.

[0044] A medical image target recognition device, the device comprising:

[0045] An extraction unit, used for extracting a target area image from the medical image to be identified, wherein the target area image is composed of M slice images, where M is a positive integer;

[0046] An input unit is used to input N consecutive slice images selected from the target area image into a medical image recognition model to obtain a recognition result of a central slice image of the N consecutive slice images; the N consecutive slice images are the i-th to the i+N-1-th slice images, i is each integer in the range of 1 to M-N+1, i and N are positive integers, and N is less than or equal to M; the medical image recognition model is trained according to the above-mentioned training method for the medical image recognition model;

[0047] A determination unit is used to determine that the medical image to be identified contains a target identification object when the number of target identification results of the identification results of the central slice image is greater than a number threshold.

[0048] A training device for a medical image recognition model comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the training method for the medical image recognition model as described above is implemented.

[0049] A medical image target recognition device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the medical image target recognition method as described above is implemented.

[0050] A computer-readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to execute the training method for a medical image recognition model as described above, or to execute the medical image target recognition method as described above.

[0051] It can be seen that the embodiments of the present application have the following beneficial effects:

[0052] The embodiment of the present application provides a medical image recognition model training method, recognition method, device and equipment, and the medical image recognition model training method includes: extracting a target area image from a three-dimensional medical image, the target area image is composed of M slice images, M is a positive integer. Selecting N consecutive slice images from the target area image as training samples, N is a positive integer, and N is less than or equal to M. Inputting the training sample into the deep learning model to be trained, and obtaining the recognition prediction result of the training sample. The training sample corresponds to an identification label, specifically the identification label of the central slice image of the training sample. The identification label can be divided into different categories according to the application scenario. Constructing a weighted cross entropy loss function, substituting the identification prediction result and the identification label of the training sample into the weighted cross entropy loss function to obtain a cross entropy loss value. The deep learning model to be trained is trained according to the cross entropy loss value, and the model training is stopped when the preset condition is reached to obtain a medical image recognition model. Among them, the weighted cross entropy loss function sets a corresponding weight for each identification label category, and the weight is determined according to the number of training samples with the identification label category, the total number of training samples, and the number of identification label categories, and satisfies the sum of each weight multiplied by the correction coefficient to be the number of identification labels. Based on this, since the weight is related to the number of training samples with the identification label category, when the number of training samples of a certain category in the training samples is small, a larger weight value can be set for the training samples of this category. Since the weight value is used to calculate the value of the weighted cross entropy loss function, it is equivalent to increasing the penalty coefficient of the loss amount corresponding to the training samples of this category, which can alleviate the low model recognition accuracy caused by the small number of training samples of a certain category to a certain extent, and can improve the recognition accuracy of the medical image recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of an exemplary scenario provided for an embodiment of the present application;

[0054] Figure 2 A flowchart of a medical image recognition model training method provided in an embodiment of the present application;

[0055] Figure 3 A flowchart of extracting a target area image from a three-dimensional medical image provided in an embodiment of the present application;

[0056] Figure 4 A flowchart of a medical image target recognition method provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the structure of a medical image recognition model training device provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the structure of a medical image target recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0060] In order to facilitate the understanding and explanation of the technical solutions provided by the embodiments of the present application, the background technology of the present application is first introduced below.

[0061] In technologies such as CT, the target object is scanned using X-rays, etc. to obtain the original data of the target object, and the three-dimensional medical image of the patient can be reconstructed based on the original data.

[0062] With the development of computer vision and artificial intelligence, it is possible to identify target objects in 3D medical images with the help of artificial intelligence technology. Whether the 3D medical images include target objects can help doctors analyze the patient's condition and can serve as a reference for doctors to analyze the patient's condition. The target object is, for example, a lesion.

[0063] At present, deep learning models can be used to identify target objects in three-dimensional medical images, but there are usually problems with low accuracy of recognition results and poor recognition effects. When training deep learning models, the amount of sample data is crucial. When the amount of training sample data of different categories is balanced and the amount of sample data is large, the accuracy of the trained deep learning model is high. When the amount of training sample data of different categories is unbalanced, especially when the number of training samples of a certain category is much less than the number of training samples of other categories, the recognition accuracy of the deep learning model will be low and the recognition effect will be poor. For example, when a deep learning model is trained using three-dimensional medical images with target objects (which can be regarded as positive samples) and three-dimensional medical images without target objects (which can be regarded as negative samples). The deep learning model is used to identify whether there are target objects in three-dimensional medical images. If the number of three-dimensional medical images with target objects is too small compared to the number of three-dimensional medical images without target objects, the accuracy of the deep learning model will be low.

[0064] Based on this, the present application embodiment provides a medical image recognition model training method, recognition method, device and equipment. In order to facilitate understanding of the medical image recognition model training method provided in the present application embodiment, combined with Figure 1 The exemplary application scenarios are described below. Figure 1 A schematic diagram of an exemplary scenario provided for an embodiment of the present application.

[0065] like Figure 1As shown, a target region image is extracted from a three-dimensional medical image, and the target region image is composed of M slice images, where M is a positive integer. The target region image is an image region that may contain a target object. For example, when the target object is a lesion, the target region image is an image region that may contain a lesion.

[0066] Select N consecutive slice images from the target area image as training samples, where N is a positive integer and N is less than or equal to M. Input the training samples into the deep learning model to be trained to obtain the recognition prediction results of the training samples. The training samples correspond to recognition labels, specifically the recognition labels of the central slice images of the training samples. The recognition labels of the training samples can be regarded as the expected results of the recognition prediction results of the training samples. The recognition labels can be divided into different categories according to the application scenarios. For example, when the application scenario is to use a medical image recognition model to identify the presence or absence of lesions, the recognition labels are divided into two recognition label categories: lesion-presence and lesion-free. Correspondingly, the training samples are images with lesions (which can be regarded as positive samples) or images without lesions (which can be regarded as negative samples).

[0067] Construct a weighted cross entropy loss function, substitute the recognition prediction results and recognition labels of the training samples into the weighted cross entropy loss function to obtain the cross entropy loss value. Train the deep learning model to be trained according to the cross entropy loss value, stop the model training when the preset conditions are met, and obtain the medical image recognition model. The medical image recognition model is the deep learning model to be trained that has been trained. Among them, the weighted cross entropy loss function sets the corresponding weight for each recognition label category, and the weight is determined according to the number of training samples with the recognition label category, the total number of training samples, and the number of recognition label categories, and satisfies the sum of each weight multiplied by the correction coefficient to be the recognition label.

[0068] It should be noted that Figure 1 The block diagram shown is only used as an example for explanation, and the medical image recognition model training method provided in the embodiment of the present application is not limited to the above example.

[0069] Based on the above description, the embodiment of the present application provides a specific implementation method of a medical image recognition model training method.

[0070] See also Figure 2 , Figure 2 A flowchart of a medical image recognition model training method provided in an embodiment of the present application. Figure 2 As shown, the method includes S201-S204:

[0071] S201: extracting a target area image from a three-dimensional medical image, where the target area image is composed of M slice images, where M is a positive integer.

[0072] As an example, in CT technology, by scanning a patient using X-rays or the like, the original data of the patient can be obtained, and a three-dimensional medical image of the patient can be reconstructed based on the original data. The three-dimensional medical image obtained by CT technology can be called a three-dimensional CT image. For example, a three-dimensional lung CT image, a three-dimensional brain CT image. It can be understood that a three-dimensional CT image is a slice image, which is composed of multiple slice images. The target area image in the embodiment of the present application is composed of M slice images.

[0073] The target area image is an image area that may contain the target object, and the target area image is smaller than the three-dimensional medical image and is a partial image in the three-dimensional medical image. For example, when the target object is a lesion, the target area image is an image area in the three-dimensional medical image where the lesion may exist. Specifically, if the lesion is pulmonary embolism and the target object is pulmonary embolism, the medical image recognition model can be a recognition model for pulmonary embolism. Because pulmonary embolism exists in pulmonary blood vessels, and the pulmonary blood vessels and surrounding soft tissues are close and tightly adhered together. In addition, pulmonary embolism and the mediastinum domain of the lung are also closely related. If the medical image recognition model can be a recognition model for pulmonary embolism, the optional target area image in the embodiment of the present application is an image including pulmonary blood vessels, surrounding soft tissues and the mediastinum domain of the lung.

[0074] Based on this, compared to directly using the deep learning model to identify the target object of the entire three-dimensional medical image, extracting the target area image from the three-dimensional medical image and then using the deep learning model to identify the target area image can reduce the difficulty of deep learning feature recognition, facilitate the deep learning model to identify the target object, and improve the recognition efficiency and accuracy of the deep learning model.

[0075] In specific implementation, image processing technology can be used to extract the target area image from the three-dimensional medical image. When the three-dimensional medical image is a lung CT image, the embodiment of the present application provides a specific implementation method for extracting the target area image from the three-dimensional medical image, please see S301-S305 below for details. It can be understood that the medical image recognition model training method provided in the embodiment of the present application is not limited to being used to recognize lung CT images, but can also be used to recognize other types of medical images such as brain CT images.

[0076] It should be noted that after acquiring the target area image, in order to increase the data volume of the training sample and improve the generalization ability and robustness of the deep learning model to be trained, the acquired target area image can also be subjected to data enhancement processing to obtain the enhanced target area image. The target area image (i.e., M slice images) of the embodiment of the present application may be the target area image after data enhancement processing of the initially extracted target area image. In a possible implementation method, the embodiment of the present application provides a specific implementation method of data enhancement, please refer to A1-A2 below for details.

[0077] S202: Input the training sample into the deep learning model to be trained to obtain the recognition prediction result of the training sample, where the training sample is composed of N consecutive slice images selected from the target area image, and the recognition label of the training sample is the recognition label of the central slice image of the training sample, where N is a positive integer and N is less than or equal to M.

[0078] After obtaining the target area image composed of M slice images, select N consecutive slice images from the target area image as training samples. Input the training samples into the deep learning model to be trained, and the recognition prediction results corresponding to the training samples can be obtained.

[0079] Specifically, the deep learning model to be trained is used to identify the category of the central slice image of the training sample (for example, to identify whether the central slice image of the training sample includes the target object). The training sample corresponds to an identification label, and accordingly, the identification label of the training sample is the identification label of the central slice image of the training sample (for example, including the target object or not including the target object). Wherein, N is a positive integer, and N is less than or equal to M. When N is an odd number, the central slice image of the training sample is the slice image at the center position of the training sample. As an optional example, when N is an even number, the central slice image of the training sample is any one of the two slice images at the center position of the training sample.

[0080] For example, if the input layer of the deep learning model to be trained is 3 channels, 3 consecutive slice images need to be input at the same time. The 3 consecutive slice images are used as a training sample, and the corresponding identification label is the identification label corresponding to the central slice image. In specific implementation, all training samples can be selected from the target area image (i.e., M slice images). Some training samples are randomly selected from all the training samples and input into the deep learning model to be trained at the same time to train the deep learning model to be trained. It should be noted that before the training samples are input into the deep learning model to be trained, the size of the slice images in the training samples should be uniformly transformed into the input image size required by the deep learning model to be trained, for example, 512*512.

[0081] It can be understood that by selecting N continuous slice images, the deep learning model to be trained can extract the image features of N continuous slice images, making the extracted image features richer, and the extracted image features are all used for the deep recognition model to be trained to identify the central slice image, making the recognition result of the deep learning model to be trained for the central slice image more accurate.

[0082] As an optional example, the deep learning model to be trained can adopt the EfficientNet-b7 model. Before training the deep learning model to be trained, the model parameters need to be initialized. Optionally, the parameters trained on ImageNet can be transferred and used as the initialization model parameters of the deep learning model to be trained in the embodiment of the present application. Among them, ImageNet is a large visualization database for visual object recognition software research.

[0083] As another optional example, in order to improve the generalization ability of the deep learning model to be trained and prevent overfitting of the deep learning model to be trained, an adaptive average pooling layer can be inserted between the convolution layer and the fully connected layer of the EfficientNet-b7 model, and then a dropout layer can be added after the adaptive average pooling layer. Among them, the activation function of the fully connected layer output is the sigmoid function for binary classification and the softmax function for multi-classification.

[0084] S203: Substitute the recognition prediction results and recognition labels of the training samples into the weighted cross entropy loss function to obtain the cross entropy loss value. The weighted cross entropy loss function sets a corresponding weight for each recognition label category. The weight is determined according to the number of training samples with the recognition label category, the total number of training samples and the number of recognition label categories. The sum of the weights multiplied by the correction coefficient is the number of recognition label categories.

[0085] After obtaining the recognition prediction results and corresponding recognition labels of the training samples, the recognition prediction results and corresponding recognition labels of the training samples are input into the weighted cross entropy loss function to obtain the cross entropy loss value. The cross entropy loss value is used to train the deep learning model to be trained.

[0086] Among them, the identification label category corresponding to the training sample is determined according to the actual application scenario. As an optional example, when the target object is a lesion, the deep learning model to be trained is used to identify the type of training samples, and the types of training samples are images with lesions and images without lesions (i.e., identifying whether there are lesions in the training samples), there are two types of identification labels, namely, lesions and no lesions. At this time, the weighted cross entropy loss function is a weighted binary cross entropy loss function.

[0087] As another optional example, when the deep learning model to be trained is used to identify the types of training samples, and the types of training samples are target object A, target object B, and target object C, there are three types of identification labels, namely target object A, target object B, and target object C. The weighted cross entropy loss function is adaptively modified.

[0088] It should be noted that, taking into account the imbalance of training samples, for example, the number of positive samples (such as images with lesions) is far less than that of negative samples. The embodiment of the present application uses a weighted cross entropy loss function to train the deep learning model to be trained. The weighted cross entropy loss function sets a corresponding weight for each recognition label category. The weight is determined based on the number of training samples with the recognition label category, the total number of training samples, and the number of recognition label categories. The sum of the weights multiplied by the correction coefficient is the number of recognition label categories. Specifically, the calculation formula for the corresponding weight set for each recognition label category is as follows:

[0089] w=n_samples / (n_classes * n_sample)

[0090] Among them, w is the weight corresponding to any training sample category (or any identification label category), n_samples is the total number of training samples, n_classes is the number of identification label categories, and n_sample is the number of training samples with this identification label category.

[0091] Based on this, since the weight is related to the number of training samples with the identification label category, when the number of training samples of a certain category in the training samples is small, a larger weight value can be set for the training samples of this category. Since the weight value is used to calculate the value of the weighted cross entropy loss function, it is equivalent to increasing the penalty coefficient of the loss amount corresponding to the training samples of this category, which can alleviate the low model recognition accuracy caused by the small number of training samples of a certain category to a certain extent, and can improve the recognition accuracy of the medical image recognition model.

[0092] In addition, when the training sample categories are extremely unbalanced, the weight value corresponding to the training sample category with a small number of samples determined by the weighted cross entropy loss function will be too large. Excessive or small weights will lead to a decrease in the recognition accuracy of the model. In order to improve the recognition accuracy of the model, the embodiment of the present application introduces a limiting condition:

[0093] (w1+w2+…+wk)*α = k

[0094] Among them, w1…wk are the weights corresponding to each identification label category, a total of k. k is n_classes, that is, the number of identification label categories. α is the correction coefficient, which is used to correct w1 to wk. This restriction ensures that the sum of each weight multiplied by the correction coefficient is the number of identification label categories.

[0095] On this basis, the actual weight corresponding to each recognition label category is set to the product of the weight and the correction coefficient, such as w1*α is the actual weight corresponding to one of the recognition label categories. The weights set in this way or the weights obtained by fine-tuning based on the actual weights can improve the recognition accuracy of the deep learning model to be trained and avoid the generation of huge weight values.

[0096] S204: Train the deep learning model to be trained according to the cross entropy loss value, and re-enter the training samples into the deep learning model to be trained until the preset conditions are met to obtain a medical image recognition model.

[0097] The deep learning model to be trained is trained according to the cross entropy loss value, and when the preset conditions are met, a medical image recognition model is obtained. The medical image recognition model is the deep learning model to be trained after the training is completed. When the preset conditions are not met, the model parameters are updated, and the training samples are input into the deep learning model to be trained again until the preset conditions are met.

[0098] As an optional example, the preset condition is that the cross entropy loss value reaches a preset threshold, or reaches a preset number of training times, or satisfies both the cross entropy loss value reaching a preset threshold and reaching a preset number of training times. Among them, the embodiment of the present application does not limit the preset threshold and the preset number of training times, and can be set according to the needs of the actual application scenario.

[0099] It should be noted that in practical applications, multiple sets of training samples can be used to train the deep learning model to be trained multiple times to obtain multiple corresponding medical image recognition models. Then, the multiple medical image recognition models are evaluated, and the medical image recognition model with the highest recognition accuracy among the multiple medical image recognition models is selected as the final medical image recognition model.

[0100] As an optional example, since the F1 score is an evaluation indicator that comprehensively considers multiple factors and is a good model evaluation benchmark, the F1 score can be used to evaluate the quality of the medical image recognition model.

[0101] In specific implementation, the F1 score is calculated as follows:

[0102] P=TP / (TP+FP) R=TP / (TP+FN) F1=2PR / (P+R)

[0103] Among them, TP means that the prediction category is true, the identification label category is true, and the prediction is correct. FP means that the prediction category is true, the identification label category is false, and the prediction is wrong. FN means that the prediction category is false, the identification label category is true, and the prediction is wrong. P is the prediction accuracy, and R is the recall rate. For example, when the deep learning model to be trained is used to identify the types of training samples, and the types of training samples are images with lesions and images without lesions, the identification label type is divided into lesion or no lesion. In this scenario, the identification label category of true means the presence of lesions. The identification label category of false means the absence of lesions.

[0104] Prediction accuracy refers to the prediction results, which means the probability of identifying the label category as true among all training samples where the predicted category is true. Recall rate refers to the probability of predicting the category as true among training samples where the identified label category is true. F1 score considers both prediction accuracy and recall rate, making them both reach the highest value at the same time and achieving a balance.

[0105] The embodiment of the present application provides a medical image recognition model training method, including: extracting a target area image from a three-dimensional medical image, the target area image consisting of M slice images, M is a positive integer. Selecting N consecutive slice images from the target area image as training samples, N is a positive integer. Inputting the training sample into the deep learning model to be trained, and obtaining the recognition prediction result of the training sample. The training sample corresponds to an identification label, specifically the identification label of the central slice image of the training sample. The identification label can be divided into different categories according to the application scenario. Constructing a weighted cross entropy loss function, substituting the identification prediction result and the identification label of the training sample into the weighted cross entropy loss function to obtain a cross entropy loss value. The deep learning model to be trained is trained according to the cross entropy loss value, and the model training is stopped when the preset condition is reached to obtain a medical image recognition model. Among them, the weighted cross entropy loss function sets a corresponding weight for each identification label category, and the weight is determined according to the number of training samples with the identification label category, the total number of training samples, and the number of identification label categories, and satisfies the sum of each weight multiplied by the correction coefficient to be the number of identification labels. Based on this, since the weight is related to the number of training samples with the identification label category, when the number of training samples of a certain category in the training samples is small, a larger weight value can be set for the training samples of this category. Since the weight value is used to calculate the value of the weighted cross entropy loss function, it is equivalent to increasing the penalty coefficient of the loss amount corresponding to the training samples of this category, which can alleviate the low model recognition accuracy caused by the small number of training samples of a certain category to a certain extent, and can improve the recognition accuracy of the medical image recognition model.

[0106] In a possible implementation, the present application embodiment further provides another medical image recognition model training method, which, in addition to the above S201-S204, further includes:

[0107] A1: Randomly select at least one random transformation method.

[0108] Data enhancement methods are used for data enhancement, such as the RandAugment method. Its main idea is to enhance data based on random transformation.

[0109] The RandAugment method has two input parameters, namely the number of random transformation methods and the transformation amplitude. The number of random transformation methods is the total number of at least one random transformation method in A1. Each random transformation method determines a transformation range, and the image is transformed according to the transformation range. The transformation amplitude is the upper limit of the right boundary value in the transformation range, for example, it can be 10.

[0110] Random transformation methods include, for example, random image rotation, image contrast enhancement, filled rectangle enhancement, and image zoom in / out enhancement. Each random transformation method corresponds to a transformation index. For example, the transformation index in the random image rotation method is the rotation angle, and the minimum value of the rotation angle is 0 and the maximum value is 20. The transformation index in the image contrast enhancement method is the saturation factor, and the minimum value of the saturation factor is 0.1 and the maximum value is 1.9. The transformation index in the filled rectangle enhancement method is the ratio coefficient of the width and height of the rectangle to the width and height of the image, and its minimum value is 0 and the maximum value is 0.4. The transformation index in the image zoom in / out enhancement method is the zoom factor, and the minimum value of the zoom factor is 0.8 and the maximum value is 1.2.

[0111] A2: randomly transforming the slice image according to a target random transformation method within a transformation range corresponding to the target random transformation method to regenerate the slice image, wherein the target random transformation method is each of the at least one selected random transformation method.

[0112] As an optional example, the transformation range is a range between 0 and the transformation value V. The transformation amplitude is the maximum value of the transformation value V. The transformation value V in each random transformation mode is determined according to the maximum and minimum values ​​of the transformation index in the random transformation mode. The specific calculation method is as follows:

[0113] V=min+(K / 30)*(max-min)

[0114] Wherein, K is a random number, which can be generated by Poisson distribution. When K>30, K=30. min is the minimum value of the transformation index in each random transformation method, and max is the maximum value of the transformation index in each random transformation method.

[0115] After determining the transformation range corresponding to the target random transformation mode, the slice image is randomly transformed in the transformation range corresponding to the target random transformation mode according to the target random transformation mode to regenerate the slice image. The target random transformation mode is each of the at least one selected random transformation mode.

[0116] Based on A1-A2, the slice images were data enhanced. The large amount of training samples can make the generalization ability of the deep learning model to be trained stronger.

[0117] In practical applications, when the three-dimensional medical image is a lung CT image, the lung CT image can be used to identify pulmonary embolism. Specifically, the lung CT image can be used to obtain a target area image, and then the pulmonary embolism can be identified based on the target area image.

[0118] Therefore, based on the above method embodiments, the embodiments of the present application are combined with Figure 3 A specific implementation method for extracting a target area image from a three-dimensional medical image is provided when the three-dimensional medical image is a lung CT image. Figure 3 The flowchart of extracting the target area image from the three-dimensional medical image provided by the embodiment of the present application. Figure 3 As shown, steps S301-S305 are included:

[0119] S301: Identify the lung region in the lung CT image to obtain a lung binary image.

[0120] It should be noted that the lung CT image includes areas such as the lung region, the lung blood vessels, and the soft tissues around the lung blood vessels. Pulmonary embolism is located on the lung blood vessels. The target area image of the lung CT image is an image including the lung blood vessels, the soft tissues around the lung blood vessels, and the mediastinum in the lung.

[0121] In order to extract the target area image from the three-dimensional medical image, the lung area in the lung CT image is first identified to obtain a lung binary image. The lung binary image is an image obtained by segmenting the lung CT image into the lung area and the non-lung area. For example, the lung area is represented by 1 and the non-lung area is represented by 0.

[0122] In addition, in order to ensure that the analyzed image has consistent features and facilitate subsequent processing and recognition, the lung CT image is first resampled so that the interval between the resampled slice images of the lung CT image is 1 mm. Then, the lung CT image is window transformed so that the CT value of the lung CT image is between -1024 and 1024. Based on this, the lung area in the lung CT image is identified to obtain a lung binary image.

[0123] In a possible implementation, the embodiment of the present application provides a specific implementation method for identifying the lung area in a lung CT image and obtaining a lung binary image. Please refer to B1-B4 below for details.

[0124] S302: Filling the holes or gaps in the lung binary image to obtain a filled lung binary image.

[0125] After obtaining the lung binary image, the holes or gaps in the lung binary image are filled to obtain a filled lung binary image. In practical applications, a morphological closing operation can be used for filling.

[0126] It can be understood that the cavities or gaps in the binary lung image can usually be regarded as pulmonary blood vessels and surrounding soft tissues.

[0127] It should be noted that when filling the holes or gaps in the binary lung image, the holes or gaps in the binary lung image should be filled as much as possible.

[0128] S303: performing a difference operation on the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue region.

[0129] By performing a difference operation between the filled lung domain binary image and the lung domain binary image before filling, we can obtain a tissue image of the cavities or gaps in the lung binary image, that is, a binary image of the lung tissue area. The binary image of the lung tissue area mainly includes pulmonary blood vessels and soft tissue around pulmonary blood vessels. Since pulmonary embolism is located on the pulmonary blood vessels, the binary image of the lung tissue area may also contain a complete pulmonary embolism.

[0130] In a specific implementation, a difference operation is performed between each corresponding pixel value in the lung domain binary image after filling and the lung domain binary image before filling, and a binary image of the lung tissue area is obtained according to the result of the pixel difference operation.

[0131] S304: Determine a binary image of the middle mediastinum region according to the binary image of the lungs.

[0132] The middle mediastinum region of the lung represents the area near the heart. Since pulmonary embolism is strongly correlated with the area near the heart, in order to identify pulmonary embolism more accurately, it is also necessary to obtain images corresponding to the middle mediastinum region of the lung.

[0133] In specific implementation, since the middle mediastinum region of the lung is located between the two lungs, the binary image of the middle mediastinum region can be determined according to the binary image of the lung. The binary image of the middle mediastinum region corresponds to the middle mediastinum region of the lung.

[0134] In a possible implementation, the embodiment of the present application provides a specific implementation method for determining a binary image of the mediastinum region based on a binary image of the lungs. For details, please refer to C1-C2 below.

[0135] S305: Taking the union of the binary image of the lung tissue region and the binary image of the middle mediastinum region to obtain a combined binary image, and cutting out a circumscribed cubic region of the combined binary image from the lung CT image as a target region image.

[0136] In specific implementation, the vertex coordinates of the circumscribed cubic region of the combined binary image may be obtained, and the circumscribed cubic region of the combined binary image may be cut out from the lung CT image according to the vertex coordinates, and used as the target region image.

[0137] The target area image removes the areas irrelevant to pulmonary embolism identification to the maximum extent, avoiding the interference of other lung areas on the identification process when using the deep learning model to be trained for identification. Finally, the target area image is saved as a slice sequence image using image processing tools.

[0138] Based on the contents of S301-S305, when the three-dimensional medical image is a lung CT image, the lung CT image is used to first obtain the target area image in the lung CT image, and the pulmonary embolism is identified based on the target area image. Compared with directly identifying pulmonary embolism based on the lung CT image, identifying pulmonary embolism based on the target area image can alleviate the recognition difficulty of the medical image recognition model and improve the recognition accuracy of the medical image recognition model.

[0139] It should be noted that when the three-dimensional medical image is a lung CT image, after obtaining the target area image, the target area image is used to train the deep learning method to be trained. The training samples obtained from the target area image include two types of images with pulmonary embolism and images without pulmonary embolism. Correspondingly, the identification labels are two identification label categories: with lesions and without lesions.

[0140] Obviously, the above embodiments are only part of the embodiments of the present application, not all of the embodiments. The training method of the medical image recognition model provided in the embodiments of the present application can also be applied to the training of other recognition models, such as a model for recognizing lesions in brain CT images.

[0141] In a possible implementation, the present application embodiment provides a specific implementation of identifying the lung region in the lung CT image in S301 to obtain a lung binary image, including:

[0142] B1: Determine an area in the lung CT image that meets a preset CT value range, set the area in the preset CT value range as a first pixel value, and set other areas in the lung CT image as a second pixel value to generate a preprocessed binary image.

[0143] The CT value reflects the degree of X-ray absorption by human tissue. It is understandable that most of the areas corresponding to the preset CT value range are lung areas. For example, the preset CT value range is [-900, -600], the minimum value is -900, and the maximum value is -600, which means that most of the areas within [-900, -600] are lung areas. The lung CT image is subjected to a multi-threshold segmentation operation using the preset CT value range, and the areas that meet the preset CT value range are set to the first pixel value, and the other areas in the lung CT image are set to the second pixel value, to obtain a segmented preprocessed binary image.

[0144] As an optional example, the first pixel value is 1 and the second pixel value is 0.

[0145] B2: Identify the connected domain area from the preprocessed binary image, and extract the largest connected domain area from the preprocessed binary image as the candidate lung binary image.

[0146] The area corresponding to the first pixel value in the preprocessed binary image is mainly the lung area. At this time, in order to make the lung area segmentation in the lung CT image more accurate, after obtaining the preprocessed binary image, the connected domain area is identified from the preprocessed binary image. The connected domain area generally refers to the image area composed of foreground pixels with the same pixel value and adjacent positions in the image.

[0147] In the lung CT image, the lung area is the largest connected domain area. In order to make the lung area segmentation in the lung CT image more accurate, the largest connected domain area is extracted from the preprocessed binary image as a candidate lung binary image.

[0148] In a specific implementation, the maximum connected domain area can be determined by the number of voxel points, and the area with the same voxels and the largest number of voxel points is the maximum connected domain area.

[0149] B3: If the starting coordinates of the candidate lung binary image meet the preset conditions, the candidate lung binary image is determined as the lung binary image, and the region with the first pixel value in the lung binary image is the lung region.

[0150] In order to ensure the correctness of the segmentation, it is also necessary to judge the boundary conditions of the candidate lung binary image. Usually, the lung area is in the middle of the lung CT image, and the boundary of the lung area is at a distance from the boundary of the lung CT image. If the starting coordinates of the candidate lung binary image meet the preset conditions, the candidate lung binary image is determined as a lung binary image. For example, if the initial X-axis coordinates of the candidate lung binary image are greater than 10, it indicates that the candidate lung binary image obtained by B1-B2 is a lung binary image.

[0151] Obviously, the lung region is the region with the first pixel value in the lung binary image.

[0152] B4: If the starting coordinates of the candidate lung binary image do not meet the preset conditions, reset the preset CT value range, and re-execute the steps of determining the area that meets the preset CT value range from the lung image and subsequent steps.

[0153] When the starting coordinates of the candidate lung binary image do not meet the preset conditions, it indicates that there is a difference between the candidate lung binary image and the real lung binary image, and it is necessary to reset the preset CT value range, for example, reset it to [-900, -350], and re-execute B1 and subsequent steps.

[0154] Based on the contents of B1-B4, it can be seen that the purpose of identifying the lung area in the lung CT image is achieved, so that the obtained lung binary image can more accurately represent the lung area.

[0155] In a possible implementation, the present application embodiment provides a specific implementation of determining a binary image of the mediastinum region according to a binary image of the lungs in S304, including:

[0156] C1: Indent the circumscribed cubic area of ​​the lung binary image inward by a preset distance to obtain an indented cubic area.

[0157] Since the lung binary image is a stereo image, the circumscribed cubic region of the lung binary image can be obtained. The circumscribed cubic region of the lung binary image represents the lung region.

[0158] As an optional example, relying on the binary lung image, the initial vertex coordinates and three side lengths of the circumscribed cube of the binary lung image are calculated. Among them, the initial vertex coordinates are the vertex coordinates with the smallest vertex coordinate value on the circumscribed cube of the binary lung image. The three side lengths are the length, width and height of the circumscribed cube. Then, each side of the circumscribed cube of the binary lung image is indented toward the center of the circumscribed cube by a preset distance to obtain the indented cube area and the vertex coordinates of the indented cube area.

[0159] The preset distance may be determined according to the image effect obtained in the actual application scenario and is not limited here.

[0160] C2: The indented cube area is set to the first pixel value, and the lung area within the indented cube area is set to the second pixel value, to obtain a binary image of the middle mediastinum area.

[0161] The indented cube region is set to a first pixel value, for example, 1. In a specific implementation, the indented cube region may be set to the first pixel value according to the vertex coordinates of the indented cube region.

[0162] The indented cubic region also contains a lung region, and the lung binary image needs to be reused to remove the lung region portion in the indented cubic region, that is, the lung region portion in the indented cubic region is set to a second pixel value, such as 0, to obtain a binary image of the middle mediastinum region.

[0163] Based on the contents of C1-C2 above, it can be seen that the binary image of the middle mediastinum region is determined by the binary image of the lung. If the target area image contains the binary image of the middle mediastinum region, the recognition accuracy of the deep learning model to be trained for pulmonary embolism can be improved.

[0164] Based on the above-mentioned embodiment of the training method of the medical image recognition model, the present application embodiment also provides a medical image target recognition method. Figure 4 , Figure 4 This is a flow chart of a medical image target recognition method provided in an embodiment of the present application. Figure 4 As shown, the method includes S401-S403:

[0165] S401: extracting a target region image from the medical image to be identified, where the target region image is composed of M slice images, where M is a positive integer.

[0166] The medical image to be identified may be a lung CT image or a brain CT image. The medical image recognition model identifies the target object based on the target area image extracted from the medical image to be identified.

[0167] In a possible implementation, the embodiment of the present application provides a specific implementation method for extracting a target area image from a medical image to be identified, please refer to D1-D5 below for details.

[0168] S402: Input N consecutive slice images selected from the target area image into the medical image recognition model to obtain a recognition result of the central slice image of the N consecutive slice images; the N consecutive slice images are the i-th to the i+N-1-th slice images, i is each integer in the range of 1 to M-N+1, i and N are positive integers, and N is less than or equal to M; the medical image recognition model is trained according to the training method of the medical image recognition model of any of the above embodiments.

[0169] The medical image recognition model is a model with the best F1 score among multiple models trained by the medical image recognition model training method provided by any of the above embodiments.

[0170] Select N consecutive slice images from M slice images, and each of the N consecutive slice images is taken as a sample. Specifically, the N consecutive slice images are the i-th to the i+N-1-th slice images. Wherein, i is each integer in the range of 1 to M-N+1. For example, when M=100 and N=3, the three consecutive slice images in the first group of images are the 1st to the 3rd of the 100 slice images, and the three consecutive slice images in the last group of images are the 98th to the 100th of the 100 slice images.

[0171] It should be noted that after determining M slice images, or determining N consecutive slice images, the size of the slice images is uniformly transformed into the image input size of the medical image recognition model, for example, 512*512. The consecutive N slice images are input into the medical image recognition model in the slice order for recognition, and the recognition result of the central slice image of the consecutive N slice images is obtained. The recognition result of the central slice image is used as the recognition result of the consecutive N slice images. The recognition result includes the recognition category of the central slice image and the probability value corresponding to the recognition category, for example, there is a lesion or there is no lesion, the probability value of the lesion is 0.7, and the probability value of the lesion is 0.3. Usually, the sum of the probability values ​​corresponding to each recognition category is 1. Determine that the recognition type corresponding to the central slice image is a recognition category whose probability exceeds the probability threshold. For example, if the probability threshold is 0.5, the probability of having a lesion exceeds the probability threshold, and the recognition category of the central slice image is determined to be a lesion, that is, there is a lesion in the central slice image.

[0172] It is understandable that when there are two identification categories, the probability threshold is usually 0.5. When there are more than two identification categories, the probability threshold can be determined according to the actual needs of the actual application scenario.

[0173] S403: When the number of target recognition results of the central slice image recognition results is greater than the number threshold, it is determined that the medical image to be recognized contains the target recognition object.

[0174] The medical image recognition model is used to identify the recognition results of the central slice image. When the number of the recognition results of the central slice image for a certain target recognition result is greater than the number threshold, it is determined that the medical image to be recognized contains the target recognition object. For example, when the recognition result of the central slice image for the recognition result containing the lesion is greater than the number threshold, it is determined that the entire medical image to be recognized contains the lesion. In this way, the recognition accuracy and reliability of the medical image recognition model are improved to a certain extent, avoiding the inaccurate situation of determining that the medical image to be recognized contains the target recognition object as long as there is a target recognition result.

[0175] It is understandable that the quantity threshold is set according to the actual needs of the actual application scenario and is not limited here.

[0176] Based on the contents of S401-S403 above, it can be known that the trained medical image recognition model can be used to identify whether there is a target object in the medical image to be recognized. Since the medical image recognition model can alleviate the low model recognition accuracy caused by the small number of training samples during the training process to a certain extent, the recognition accuracy of the medical image recognition model is improved. The recognition accuracy of the target object based on the medical image recognition model is relatively high.

[0177] In a possible implementation, the present application embodiment provides a specific implementation of extracting a target area image from a medical image to be identified in S401, including:

[0178] D1: Identify the lung area in the lung CT image and obtain a lung binary image.

[0179] D2: Fill the holes or gaps in the lung binary image to obtain a filled lung binary image.

[0180] D3: Perform a difference operation between the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area.

[0181] D4: Determine the binary image of the middle mediastinum region based on the binary image of the lungs.

[0182] D5: The binary image of the lung tissue region and the binary image of the middle mediastinum region are combined to obtain a combined binary image, and the circumscribed cube region of the combined binary image is cut out from the lung CT image as the target region image.

[0183] It should be noted that the technical details of D1-D5 can be found in the embodiments described in the above S301-S305, which will not be repeated here.

[0184] See also Figure 5 , Figure 5 A schematic diagram of the structure of a medical image recognition model training device provided in an embodiment of the present application. Figure 5 As shown, the medical image recognition model training device includes:

[0185] An extraction unit 501 is used to extract a target area image from a three-dimensional medical image, wherein the target area image is composed of M slice images, where M is a positive integer;

[0186] An input unit 502 is used to input a training sample into a deep learning model to be trained to obtain a recognition prediction result of the training sample, wherein the training sample is composed of N consecutive slice images selected from the target area image, and the recognition label of the training sample is the recognition label of the central slice image of the training sample, where N is a positive integer and is less than or equal to M;

[0187] An acquisition unit 503 is used to substitute the recognition prediction result of the training sample and the recognition label into a weighted cross entropy loss function to obtain a cross entropy loss value, wherein the weighted cross entropy loss function sets a corresponding weight for each recognition label category, wherein the weight is determined according to the number of training samples with the recognition label category, the total number of training samples, and the number of the recognition label categories, and the sum of the weights multiplied by the correction coefficient is the number of the recognition label categories;

[0188] The training unit 504 is used to train the deep learning model to be trained according to the cross entropy loss value, and re-execute the input of the training samples into the deep learning model to be trained until the preset conditions are met to obtain the medical image recognition model.

[0189] In a possible implementation, when the three-dimensional medical image is a lung CT image, the extraction unit 501 includes:

[0190] The first recognition subunit is used to recognize the lung area in the lung CT image to obtain a lung binary image;

[0191] A filling subunit, used for filling the holes or gaps in the lung binary image to obtain a filled lung binary image;

[0192] An operator unit, used for performing a difference operation between the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area;

[0193] A first determining subunit is used to determine a binary image of the middle mediastinum region according to the binary image of the lung;

[0194] The cutting subunit is used to obtain a combined binary image by taking the union of the binary image of the lung tissue area and the binary image of the middle mediastinum area, and to cut out a circumscribed cube area of ​​the combined binary image from the lung CT image as a target area image.

[0195] In a possible implementation, the first identification subunit includes:

[0196] A second determination subunit is used to determine an area in the lung CT image that meets a preset CT value range, set the area in the preset CT value range as a first pixel value, and set other areas in the lung CT image as a second pixel value, to generate a preprocessed binary image;

[0197] A second identification subunit is used to identify a connected domain region from the preprocessed binary image, and extract a maximum connected domain region from the preprocessed binary image as a candidate lung binary image;

[0198] a third determination subunit, configured to determine the candidate lung binary image as a lung binary image if the starting coordinates of the candidate lung binary image meet a preset condition, and the region of the lung binary image having the first pixel value is a lung region;

[0199] The execution subunit is used for resetting the preset CT value range if the starting coordinates of the candidate lung binary image do not meet the preset conditions, and re-executing the steps of determining the area that meets the preset CT value range from the lung image and subsequent steps.

[0200] In a possible implementation manner, the first determining subunit includes:

[0201] An indentation subunit, used to indent the circumscribed cubic area of ​​the lung binary image inward by a preset distance to obtain an indented cubic area;

[0202] The setting subunit is used to set the indented cube area to a first pixel value and the lung area within the indented cube area to a second pixel value to obtain a binary image of the middle mediastinum area.

[0203] In a possible implementation manner, the device further includes:

[0204] A selection unit, used for randomly selecting at least one random transformation mode;

[0205] The random transformation unit is used to randomly transform the slice image according to a target random transformation method within a transformation range corresponding to the target random transformation method to regenerate the slice image, and the target random transformation method is each of the at least one selected random transformation method.

[0206] It should be noted that the technical details of the medical image recognition model training device provided in the embodiments of the present application can be found in the relevant embodiments of the above-mentioned medical image recognition model training method, which will not be repeated here.

[0207] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a medical image target recognition device provided in an embodiment of the present application. Figure 6 As shown, the medical image target recognition device includes:

[0208] An extraction unit 601 is used to extract a target area image from the medical image to be identified, wherein the target area image is composed of M slice images, where M is a positive integer;

[0209] An input unit 602 is used to input N consecutive slice images selected from the target area image into a medical image recognition model to obtain a recognition result of a central slice image of the N consecutive slice images; the N consecutive slice images are the i-th to the i+N-1-th slice images, i is each integer in the range of 1 to M-N+1, i and N are positive integers, and N is less than or equal to M; the medical image recognition model is trained according to the training method of the medical image recognition model described in any of the above embodiments;

[0210] The determination unit 603 is configured to determine that the medical image to be identified contains a target identification object when the number of target identification results of the identification results of the central slice image is greater than a number threshold.

[0211] In a possible implementation, when the medical image to be identified is a lung CT image, the extraction unit 601 includes:

[0212] The recognition subunit is used to recognize the lung area in the lung CT image and obtain a lung binary image;

[0213] A filling subunit, used for filling the holes or gaps in the lung binary image to obtain a filled lung binary image;

[0214] An operator unit, used for performing a difference operation between the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area;

[0215] A determination subunit, configured to determine a binary image of a middle mediastinum region according to the binary image of the lungs;

[0216] The cutting subunit is used to obtain a combined binary image by taking the union of the binary image of the lung tissue area and the binary image of the middle mediastinum area, and to cut out a circumscribed cubic area of ​​the combined binary image from the lung CT image as a target area image.

[0217] It should be noted that the technical details of the medical image target recognition device provided in the embodiments of the present application can be found in the relevant embodiments of the above-mentioned medical image target recognition method, which will not be repeated here.

[0218] In addition, an embodiment of the present application also provides a training device for a medical image recognition model, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the training method for the medical image recognition model as described in any of the above embodiments.

[0219] In addition, an embodiment of the present application also provides a medical image target recognition device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the medical image target recognition method described in any of the above embodiments is implemented.

[0220] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes the training method of the medical image recognition model described in any of the above embodiments, or executes the medical image target recognition method described in the above embodiments.

[0221] The embodiment of the present application provides a medical image recognition model training device, a recognition device and an apparatus, and the medical image recognition model training process includes: extracting a target area image from a three-dimensional medical image, the target area image is composed of M slice images, M is a positive integer. Selecting N consecutive slice images from the target area image as training samples, N is a positive integer. Inputting the training sample into the deep learning model to be trained, and obtaining the recognition prediction result of the training sample. The training sample corresponds to an identification label, specifically the identification label of the central slice image of the training sample. The identification label can be divided into different categories according to the application scenario. Constructing a weighted cross entropy loss function, substituting the recognition prediction result and the identification label of the training sample into the weighted cross entropy loss function to obtain a cross entropy loss value. The deep learning model to be trained is trained according to the cross entropy loss value, and the model training is stopped when the preset condition is met to obtain a medical image recognition model. Among them, the weighted cross entropy loss function sets a corresponding weight for each identification label category, and the weight is determined according to the number of training samples with the identification label category, the total number of training samples and the number of identification label categories, and satisfies the sum of each weight multiplied by the correction coefficient to be the number of identification labels. Based on this, since the weight is related to the number of training samples with the identification label category, when the number of training samples of a certain category in the training samples is small, a larger weight value can be set for the training samples of this category. Since the weight value is used to calculate the value of the weighted cross entropy loss function, it is equivalent to increasing the penalty coefficient of the loss amount corresponding to the training samples of this category, which can alleviate the low model recognition accuracy caused by the small number of training samples of a certain category to a certain extent, and can improve the recognition accuracy of the medical image recognition model.

[0222] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0223] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0224] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0225] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0226] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for a medical image recognition model, characterized in that: The method comprises: Extracting a target area image from a three-dimensional medical image, wherein the target area image is composed of M slice images, where M is a positive integer; Inputting a training sample into a deep learning model to be trained to obtain a recognition prediction result of the training sample, wherein the training sample is composed of N consecutive slice images selected from the target area image, and the recognition label of the training sample is the recognition label of the central slice image of the training sample, where N is a positive integer and is less than or equal to M; Substituting the recognition prediction result and the recognition label of the training sample into the weighted cross entropy loss function to obtain a cross entropy loss value, wherein the weighted cross entropy loss function sets a corresponding weight for each recognition label category, wherein the weight is determined according to the number of training samples with the recognition label category, the total number of training samples, and the number of the recognition label categories, and the sum of the weights multiplied by the correction coefficient is the number of the recognition label categories; Training the deep learning model to be trained according to the cross entropy loss value, and re-implementing the step of inputting the training samples into the deep learning model to be trained until a preset condition is met, thereby obtaining a medical image recognition model; When the three-dimensional medical image is a lung CT image, extracting the target area image from the three-dimensional medical image includes: Identify the lung area in the lung CT image and obtain a lung binary image; Filling the holes or gaps in the binary lung image to obtain a filled binary lung image; Performing a difference operation on the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area; Determine a binary image of the middle mediastinum region according to the binary image of the lungs; The binary image of the intrapulmonary tissue region and the binary image of the mid-mediastinum region are combined to obtain a combined binary image, and a circumscribed cubic region of the combined binary image is cut out from the lung CT image as a target region image.

2. The method according to claim 1, characterized in that The identifying of the lung region in the lung CT image to obtain a lung binary image includes: Determine an area in a lung CT image that meets a preset CT value range, set the area in the preset CT value range as a first pixel value, and set other areas in the lung CT image as a second pixel value, to generate a preprocessed binary image; Identifying a connected domain region from the preprocessed binary image, and extracting a maximum connected domain region from the preprocessed binary image as a candidate lung binary image; If the starting coordinates of the candidate lung binary image meet the preset conditions, the candidate lung binary image is determined as a lung binary image, and the region of the lung binary image having the first pixel value is a lung region; If the starting coordinates of the candidate lung binary image do not meet the preset conditions, the preset CT value range is reset, and the steps of determining the area that meets the preset CT value range from the lung CT image and subsequent steps are re-executed.

3. The method according to claim 1 or 2, characterized in that: Determining the binary image of the mediastinum region according to the binary image of the lungs includes: Indenting the circumscribed cubic area of ​​the lung binary image inward by a preset distance to obtain an indented cubic area; The indented cubic region is set to a first pixel value, and the lung region within the indented cubic region is set to a second pixel value, so as to obtain a binary image of the middle mediastinum region.

4. The method according to claim 1, characterized in that: The method further comprises: Randomly select at least one random transformation method; The slice image is randomly transformed according to a target random transformation method within a transformation range corresponding to the target random transformation method to regenerate the slice image, and the target random transformation method is each of the at least one selected random transformation method.

5. A medical image target recognition method, characterized in that: The method comprises: Extracting a target area image from the medical image to be identified, wherein the target area image is composed of M slice images, where M is a positive integer; Inputting N consecutive slice images selected from the target area image into a medical image recognition model to obtain a recognition result of a central slice image of the N consecutive slice images; the N consecutive slice images are the i-th to the i+N-1-th slice images, i is each integer in the range of 1 to M-N+1, i and N are positive integers, and N is less than or equal to M; the medical image recognition model is trained according to the training method for the medical image recognition model according to any one of claims 1 to 4; When the number of target recognition results of the central slice image is greater than a number threshold, it is determined that the medical image to be recognized contains a target recognition object; When the medical image to be identified is a lung CT image, extracting the target area image from the medical image to be identified includes: Identify the lung area in the lung CT image and obtain a lung binary image; Filling the holes or gaps in the binary lung image to obtain a filled binary lung image; Performing a difference operation on the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area; Determine a binary image of the middle mediastinum region according to the binary image of the lungs; The binary image of the lung tissue region and the binary image of the middle mediastinum region are combined to obtain a combined binary image, and a circumscribed cubic region of the combined binary image is cut out from the lung CT image as a target region image.

6. A training device for a medical image recognition model, characterized in that: The device comprises: An extraction unit, used for extracting a target area image from a three-dimensional medical image, wherein the target area image is composed of M slice images, where M is a positive integer; An input unit, used to input a training sample into a deep learning model to be trained to obtain a recognition prediction result of the training sample, wherein the training sample is composed of N consecutive slice images selected from the target area image, and the recognition label of the training sample is the recognition label of the central slice image of the training sample, and N is a positive integer, which is less than or equal to M; An acquisition unit is used to substitute the recognition prediction result and the recognition label of the training sample into a weighted cross entropy loss function to obtain a cross entropy loss value, wherein the weighted cross entropy loss function sets a corresponding weight for each recognition label category, wherein the weight is determined according to the number of training samples with the recognition label category, the total number of training samples, and the number of the recognition label categories, and the sum of the weights multiplied by the correction coefficient is the number of the recognition label categories; A training unit, used for training the deep learning model to be trained according to the cross entropy loss value, and re-implementing the step of inputting the training samples into the deep learning model to be trained until a preset condition is met to obtain a medical image recognition model; When the three-dimensional medical image is a lung CT image, the extraction unit includes: The first recognition subunit is used to recognize the lung area in the lung CT image to obtain a lung binary image; A filling subunit, used for filling the holes or gaps in the lung binary image to obtain a filled lung binary image; An operator unit, used for performing a difference operation between the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area; A first determining subunit is used to determine a binary image of the middle mediastinum region according to the binary image of the lung; The cutting subunit is used to obtain a combined binary image by taking the union of the binary image of the lung tissue area and the binary image of the middle mediastinum area, and to cut out a circumscribed cube area of ​​the combined binary image from the lung CT image as a target area image.

7. A medical image target recognition device, characterized in that: The device comprises: An extraction unit, used for extracting a target area image from the medical image to be identified, wherein the target area image is composed of M slice images, where M is a positive integer; An input unit, used for inputting N consecutive slice images selected from the target area image into a medical image recognition model to obtain a recognition result of a central slice image of the N consecutive slice images; the N consecutive slice images are the i-th to the i+N-1-th slice images, i is each integer in the range of 1 to M-N+1, i and N are positive integers, and N is less than or equal to M; the medical image recognition model is trained according to the training method for the medical image recognition model according to any one of claims 1 to 4; a determination unit, configured to determine that the medical image to be identified contains a target identification object when the number of target identification results of the identification results of the central slice image is greater than a number threshold; When the medical image to be identified is a lung CT image, the extraction unit includes: The recognition subunit is used to recognize the lung area in the lung CT image and obtain a lung binary image; A filling subunit, used for filling the holes or gaps in the lung binary image to obtain a filled lung binary image; An operator unit, used for performing a difference operation between the filled lung binary image and the lung binary image to obtain a binary image of the lung tissue area; A determination subunit, configured to determine a binary image of a middle mediastinum region according to the binary image of the lungs; The cutting subunit is used to obtain a combined binary image by taking the union of the binary image of the lung tissue area and the binary image of the middle mediastinum area, and to cut out a circumscribed cubic area of ​​the combined binary image from the lung CT image as a target area image.

8. A training device for a medical image recognition model, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the medical image recognition model as described in any one of claims 1 to 4 is implemented.

9. A medical image target recognition device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the medical image target recognition method as described in claim 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the training method for a medical image recognition model as described in any one of claims 1 to 4, or executes the medical image target recognition method as described in claim 5.

Images