Model training method and related device

Abstract

The invention provides a model training method and a related device. The method comprises the steps of obtaining a training sample set; inputting the training sample set into the initial model, and determining a classification prediction result, a first image quality score and a second image quality score corresponding to each medical image; determining a total loss function based on the classification label result, the classification prediction result, the first image quality score and the second image quality score; and adjusting parameters of the initial model based on the total loss function until the training is completed to obtain an image evaluation model. The first image quality score is used as a quality label, for each medical image, the quality label can dynamically learn a quality evaluation standard from a training sample set, the quality label does not depend on traditional manual labeling, the first image quality score is calculated through the relative classifiability of the medical images, and the quality evaluation accuracy of the medical images is improved. The accuracy of the quality label can be greatly improved by considering the similarity, and the image evaluation model can accurately and quickly perform classification and quality evaluation on the medical image.

Description

Technical Field

[0001] The present application relates to the field of model training, and in particular to a model training method and related devices. Background Art

[0002] The quality of medical images, such as clarity, is a key factor in evaluating image usability, which directly affects the effectiveness and reliability of subsequent image processing steps, such as lesion detection, image segmentation, image registration and fusion. In related technologies, the quality evaluation of medical images mainly relies on the observer's perception and judgment, but this method is time-consuming and easily affected by many factors, such as the observer's experience, emotions, and viewing environment. Therefore, how to quickly and accurately judge the quality of medical images has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the purpose of this application is to provide a model training method and related devices, which can greatly improve the accuracy of quality labels by considering similarity, so that the trained image evaluation model can accurately and quickly classify and evaluate the quality of medical images. The specific scheme is as follows:

[0004] On the one hand, the present application provides a model training method, comprising:

[0005] Acquire a training sample set; the training sample set includes a plurality of medical images and classification label results corresponding to each of the medical images; the classification label results include categories under at least one dimension;

[0006] Inputting the training sample set into an initial model, determining a classification prediction result, a first image quality score, and a second image quality score corresponding to each of the medical images; the first image quality score is used to identify the similarity between the medical image and a first image set, the first image set including at least one medical image in the training sample set that has the same category as the medical image;

[0007] Determining a total loss function based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score by the initial model;

[0008] The parameters of the initial model are adjusted based on the total loss function until the training is completed to obtain an image evaluation model.

[0009] On the other hand, the embodiment of the present application also provides a model training device, including:

[0010] An acquisition unit, configured to acquire a training sample set; the training sample set includes a plurality of medical images and classification label results corresponding to each of the medical images; the classification label results include categories under at least one dimension;

[0011] a first determining unit, configured to input the training sample set into an initial model, and determine a classification prediction result, a first image quality score, and a second image quality score corresponding to each of the medical images; the first image quality score is used to identify a similarity between the medical image and a first image set, the first image set including at least one medical image in the training sample set that has the same category as the medical image;

[0012] A second determining unit, configured to determine a total loss function based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score through the initial model;

[0013] A training unit is used to adjust the parameters of the initial model based on the total loss function until the training is completed to obtain an image evaluation model.

[0014] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0015] The memory is used to store program code and transmit the program code to the processor;

[0016] The processor is configured to execute the method described above according to the instructions in the program code.

[0017] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the above aspects.

[0018] The embodiment of the present application provides a model training method and related devices, obtaining a training sample set; the training sample set includes multiple medical images and classification label results corresponding to each medical image; the classification label results include categories under at least one dimension; the training sample set is input into the initial model to determine the classification prediction results, the first image quality score and the second image quality score corresponding to each medical image; the first image quality score is used to identify the similarity between the medical image and the first image set, and the first image set includes at least one medical image with the same category as the medical image in the training sample set; the total loss function is determined by the initial model based on the classification label results, the classification prediction results, the first image quality score and the second image quality score; the parameters of the initial model are adjusted based on the total loss function until the training is completed to obtain the image evaluation model. That is, the first image quality score is used as a quality label, and for each medical image, its quality label is determined based on the similarity between the medical image and a large number of medical images that are consistent with the category of the medical image. If the similarity is greater, it means that the medical image is more in line with the characteristics of the medical image of this category, and the medical image can be more accurately classified into this category, that is, the medical image can be more accurately classified, and further indicates that the higher the quality of the medical image, the more accurate the quality label will be. In short, the quality labels in this application can dynamically learn quality assessment standards from the training sample set, so that the determined quality labels are more accurate. The quality labels do not rely on traditional manual labeling. This application establishes a correlation between accurate image classification and high image quality. The first image quality score is calculated through the relative classifiability of the medical image. By considering the similarity, the accuracy of the quality label can be greatly improved, so that the trained image evaluation model can accurately and quickly classify and evaluate the quality of medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flow chart of a model training method provided in an embodiment of the present application is shown;

[0021] Figure 2 A structural block diagram of a model training device provided in an embodiment of the present application;

[0022] Figure 3 A structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0025] As described in the background art, in the related art, the quality evaluation of medical images mainly relies on the observer's perception and judgment, but this method is time-consuming and easily affected by many factors, such as the observer's experience, emotions, and viewing environment.

[0026] Based on the above technical problems, the embodiments of the present application provide a model training method and related devices, establish a correlation between accurate image classification and high image quality, calculate the first image quality score through the relative classifiability of the medical image, and greatly improve the accuracy of the quality label by considering the similarity, so that the trained image evaluation model can accurately and quickly classify and evaluate the quality of medical images.

[0027] The model training method provided in the embodiment of the present application can be implemented by a computer device, which can be a terminal device or a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.

[0028] To facilitate understanding, a model training method and related devices provided in an embodiment of the present application are described in detail below in conjunction with the accompanying drawings.

[0029] refer to Figure 1 As shown, it is a flow chart of a model training method provided in an embodiment of the present application, and the method may include the following steps.

[0030] S1, obtain the training sample set.

[0031] In an embodiment of the present application, a training sample set may be obtained, and the training sample set is used to train the initial model. The training sample set may include multiple medical images and classification label results corresponding to each medical image. The medical image may be a medical image obtained by examining a patient, and the classification label result is a category result to which the medical image belongs, and the classification label result is used as a category label of the medical image.

[0032] The classification label result includes categories under at least one dimension. Different dimensions are different aspects of evaluation, that is, a medical image can be evaluated in multiple aspects, so as to obtain multiple categories to which the medical image belongs. Dimensions can be, for example, examination items, lesion locations, image features, etc. There can be multiple categories under one dimension. For example, there are three categories under the examination item dimension, namely, chest posteroanterior film, abdominal upright film, and lumbar spine AP film. There are two categories under the lesion location dimension, namely, left upper lung field and mediastinum. There are two categories under the image feature dimension, namely, mass and cavity shadow.

[0033] As an example, the classification label result of a medical image may include three categories, which belong to different dimensions. The classification label result is abdominal upright film: abdominal cavity: mass. As an example, the classification label result of another medical image may include five categories, which belong to three dimensions, namely, examination items, lesion location, and image characteristics. The classification label result is chest frontal film: left upper lung field: mass, cavity; right lower lung field: cavity shadow.

[0034] In a possible implementation, in S1, a training sample set is obtained, which may specifically be S11-S12.

[0035] S11, performing format conversion and cropping processing on the initial medical image to obtain a medical image, and performing label parsing on the medical image knowledge graph to obtain classification label results corresponding to each medical image.

[0036] Specifically, the initial medical image can be read from a Digital Imaging and Communications in Medicine (DICOM) file. Since the size and format of the acquired initial medical image are various, it is necessary to perform format conversion and cropping on it. That is, the initial medical image is format converted so that it can be converted into a unified format that can be processed by the model, such as converted into a JPG image format. In addition, the initial medical image can also be cropped to obtain a unified image size, such as cropping it to an image size of 448*448, and converting it into a tensor recognizable by the model to obtain a medical image.

[0037] The medical image knowledge graph can be understood as a knowledge graph constructed for medical images. During the construction process, medical image data can be obtained, which includes examination mode, examination part, examination findings, examination impressions and medical image lists, as well as the label information of the corresponding standardized examination items. Based on medical image data and medical images, the disease information in the medical images can be classified to obtain the medical image knowledge graph.

[0038] More specifically, in the process of constructing the knowledge graph, the entity categories in the knowledge graph can be obtained based on medical imaging data, such as lesion location: left upper lung field, mediastinum, image features: lung mass, cavity shadow, examination items: chest posteroanterior film, abdominal upright film, lumbar spine AP film. Then, the disease categories existing in the medical image are annotated to obtain the medical image knowledge graph. For example, chest AP film: left upper lung field: mass, cavity shadow; right lower lung field: cavity shadow, and for example, abdominal upright film: abdominal cavity: mass.

[0039] By performing label parsing on the medical image knowledge graph and parsing it according to different dimensions of the knowledge graph, such as lesion location, image features, and examination items, a label matrix list is generated through this multi-label classification method, so that the classification label results corresponding to each medical image can be obtained.

[0040] In actual operation, the total number of categories C in all dimensions can be counted and a label matrix list of length C can be formed. For the classification label result of a medical image, it can be recorded in the following way: a list of all 0s of length C is constructed according to the label matrix list, and for the label j of the medical image, the value of the corresponding position in the list is changed to 1.

[0041] S12, constructing a training sample set based on the multiple medical images and the classification label results corresponding to each medical image.

[0042] In this way, after obtaining the medical images and the corresponding classification label results, a training sample set can be constructed, so that the size and format of the medical images in the training sample set are more uniform, thereby simplifying the model's processing of medical images, and the classification label results of each medical image are more accurate.

[0043] S2, inputting the training sample set into the initial model to determine the classification prediction result, the first image quality score and the second image quality score corresponding to each medical image.

[0044] Specifically, the training sample set can be input into the initial model so that the initial model can determine the classification prediction result, the first image quality score and the second image quality score corresponding to each medical image. The classification prediction result is the classification result predicted by the initial model for the medical image, that is, the classification prediction value. The second image quality score can be the quality result predicted by the initial model for the medical image, that is, the quality prediction value.

[0045] The first image quality score is used to identify the similarity between the medical image and the first image set, and the first image set includes at least one medical image in the training sample set that has the same category as the medical image. That is, for a medical image, all medical images in the training sample set that have the same category as the medical image are added to the first image set, and the medical image in the first image set may be one or more.

[0046] As an example, if the classification label result of the medical image is abdominal upright film, all medical images of abdominal upright film in the training sample set are summarized to form a first image set. If the classification label result of the medical image is abdominal upright film: abdominal cavity: mass, all medical images representing abdominal upright film in the training sample set are combined into a first image set, all medical images representing the abdominal cavity are combined into a first image set, and all medical images with masses are combined into a first image set.

[0047] For a medical image, the similarity between the medical image and its corresponding first image set is calculated. The greater the similarity, the more the medical image conforms to the first image set, that is, the more the medical image conforms to the characteristics of this category of medical images, the more accurately the initial model can classify the medical image into this category when classifying, that is, the more accurately the medical image can be classified, and the higher the quality of the medical image, that is, the quality of the medical image can be indicated based on the size of the similarity, that is, the first image quality score of the medical image can be determined based on the similarity. In addition, the first image quality score can be used as a quality label during model training to adjust model parameters.

[0048] In practical applications, the initial model may include a feature extraction layer, a classification task layer, and a hierarchical regression task layer. The feature extraction layer is used to extract the features of medical images and obtain the corresponding image feature vectors. As an example, the feature extraction layer may be Resnet49, DenseNet120, etc., that is, the last layer, i.e., the classification layer, of image classification models such as Resnet50, DenseNet121, and Vision Transformer (VIT) is deleted.

[0049] The classification task layer can be used to determine the classification prediction results of the medical image. Specifically, the classification task layer can be a fully connected layer. A fully connected layer is set to map the image feature vector to the vector space of each category to obtain the feature vector related to the category, thereby determining the classification prediction results of the medical image. In other words, the image feature vector and the classification label result can be input into the classification task layer to obtain the classification prediction result.

[0050] The hierarchical regression task layer can be used to determine the second image quality score of the medical image. Specifically, the classification task layer can also be a fully connected layer, which can map the image feature vector to a vector of size 1 to obtain a feature vector related to quality, thereby determining the second image quality score of the medical image. In other words, the image feature vector and the first image quality score can be input into the hierarchical regression task layer to obtain a predicted second image quality score.

[0051] S3, determining a total loss function based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score through the initial model.

[0052] Specifically, the classification label result can be used as the class label value of the medical image, the classification prediction result can be used as the class prediction value of the medical image, the first image quality score can be used as the quality label value of the medical image, and the second image quality score can be used as the quality prediction value of the medical image. In this way, based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score, the total loss function of the initial model can be determined.

[0053] S4, adjusting the parameters of the initial model based on the total loss function until the training is completed to obtain the image evaluation model.

[0054] Specifically, back propagation can be performed based on the total loss function to calculate the gradient of the model parameters, and the parameters of the initial model can be adjusted based on the gradient and the preset learning rate, and the above steps are repeated until the model training is completed to obtain the image evaluation model. The model training is completed when the performance of the initial model on the training sample set reaches a satisfactory level, or when the number of iterations reaches a preset number of iterations.

[0055] In addition, the data set can be made and divided before model training, that is, not only the training sample set can be used, but also the validation set and the test set can be used. For example, the labeled data set can be divided into the data of the training sample set, the validation set and the test set in a ratio of 7:1:2. Among them, the validation set is a data set used to verify the performance of the model. During the model training process, the validation set can be used to adjust the model parameters and hyperparameters to optimize the model performance and avoid overfitting. That is, the validation set is used for model selection and does not participate in the determination of learning parameters, but is used to select model parameters and hyperparameters with smaller model errors, that is, to determine a better quality image evaluation model. The test set is a data set used to evaluate the performance of the model. After the model training is completed, the test set is used to evaluate the generalization ability of the image evaluation model.

[0056] In summary, the present application uses the first image quality score as a quality label. For each medical image, its quality label is determined based on the similarity between the medical image and a large number of medical images that are consistent with the category of the medical image. If the similarity is greater, it means that the medical image is more consistent with the characteristics of medical images of this category, and the medical image can be more accurately classified into this category, that is, the more accurately the medical image can be classified, and further indicates that the higher the quality of the medical image, the more accurate the quality label. In short, the quality label in the present application can dynamically learn the quality evaluation standard from the training sample set, so that the determined quality label is more accurate. The quality label does not rely on traditional manual annotation. The present application establishes a correlation between accurate image classification and high image quality. The first image quality score is calculated by the relative classifiability of the medical image. By considering the similarity, the accuracy of the quality label can be greatly improved, so that the trained image evaluation model can accurately and quickly classify and evaluate the quality of medical images.

[0057] In a possible implementation, in S2, the training sample set is input into the initial model to determine the first image quality score corresponding to each medical image, which may specifically include S21-S22.

[0058] S21, for each medical image, feature extraction is performed through the initial model to obtain an image feature vector.

[0059] Specifically, the medical images may be input into the feature extraction layer of the initial model in batches. A batch may include multiple medical images. Feature extraction operations are performed through the feature extraction layer to obtain an image feature vector corresponding to each medical image.

[0060] S22: For each medical image, calculate the similarity between the image feature vector and the class center of the first image set through the initial model to determine a first image quality score of the medical image.

[0061] Feature extraction can also be performed on each medical image in the first image set to obtain image feature vectors corresponding to each medical image. For these image feature vectors, the class center can be used to represent their average value, that is, the class center of the first image set is used to identify the average value of the image feature vectors of the medical images in the first image set, so that the class center of the first image set can reflect the characteristics of the images in the first image set, and can then represent the first image set to participate in the calculation.

[0062] In other words, the class center can be understood as the center point or average representation of the image feature vector of a class of samples. In classification tasks, the class center is the average value of the image feature vectors of all samples of the class. For example, when the class is abdominal upright radiographs, the class center is the average value of the image feature vectors corresponding to all medical images of abdominal upright radiographs. In this way, the class center, as a representative of a class of samples, can define the position of each class in the feature space. During training, the model will try to bring the image feature vectors belonging to the same class closer to their class center to enhance intra-class compactness.

[0063] That is to say, when calculating the similarity between a medical image and its corresponding first image set, the first image quality score of the medical image can be calculated by calculating the similarity between the image feature vector of the medical image and the class center of the first image set.

[0064] As an example, the first image quality score may be represented by Class Center Similarity (CCS), which can measure the similarity between an image feature vector and its class center, and is usually achieved by calculating the cosine value of the angle between the two. The calculation formula is as follows:

[0065] CCS xi =cos(θ yi )

[0066] Among them, CCS xi For medical images x i The class center angle similarity, θ yi For medical images x i The image feature vector and its class center w yi The angle between.

[0067] In summary, by calculating the similarity between the image feature vector of the medical image and the class center of the category to which the medical image belongs, the first image quality score as the quality label value is determined. The calculation method is more rigorous, which greatly increases the accuracy of the quality label value, and the calculation process is relatively simple.

[0068] In a possible implementation, when determining the first image quality score, not only the similarity between the image feature vector and its class center may be considered, but also the similarity between the image feature vector and its negative class center (Negative Class Center) may be considered.

[0069] In S22, for each medical image, the similarity between the image feature vector and the class center of the first image set is calculated by using the initial model to determine the first image quality score of the medical image, which may specifically include S221-S223.

[0070] S221, for each medical image, calculate the similarity between the image feature vector and the class center of the first image set through the initial model to determine a first similarity result.

[0071] Specifically, the similarity result between the image feature vector calculated by the initial model and the class center can be recorded as the first similarity result, that is, the class center angle similarity is taken as the first similarity result.

[0072] S222: For each medical image, calculate the similarity between the image feature vector and the negative class center of the second image set through the initial model to determine a second similarity result.

[0073] The negative class center can be understood as the class center of other classes that are different from the class to which a given sample (ie, medical image) belongs. For example, if the class to which a medical image belongs is the left upper lung field, the class center of the mediastinum is the negative class center of the medical image.

[0074] The second image set includes at least one medical image having a different category from the medical image in the training sample set, that is, if the category of a medical image is category A, all medical images of category B in the training sample set can form a second image set. In short, the category of the medical image in the second image set is completely different from the category of the medical image.

[0075] The negative class center of the second image set is used to identify the average value of the image feature vectors of the medical images in the second image set, that is, the features of the medical images in the second image set can be extracted to obtain the corresponding image feature vectors, and the average value of these image feature vectors can be used as the negative class center of the second image set. Therefore, the negative class center of a medical image can help the initial model learn how to distinguish the image feature vector of the medical image from other categories that do not belong to its category. By maximizing the distance between the image feature vector and its negative class center, the model can enhance the separation between classes.

[0076] Specifically, the similarity between the image feature vector and the negative class center of the second image set can be determined as the second similarity result. The second similarity result can reflect the degree of similarity between a medical image and a medical image in a category to which it does not belong. The less similar they are, the easier it is to distinguish the medical image from other categories of medical images. The more accurate the model's classification result for the medical image is, the higher the image quality of the medical image is, for example, the higher the image clarity is.

[0077] As an example, the negative class center similarity (NCCS) can be used to represent the second similarity result. NCCS can measure the similarity between an image feature vector and its negative class center, and can be obtained by calculating the cosine value of the two.

[0078] S223: Determine a first image quality score of the medical image based on the first similarity result and the second similarity result by using an initial model.

[0079] Specifically, since the first similarity result can reflect the similarity between a medical image and its class center, it can judge the similarity between it and the category to which it belongs, and the second similarity result can reflect the similarity between the medical image and its negative class center, it can judge the similarity between it and the category to which it does not belong, the first image quality score determined based on the first similarity result and the second similarity result comprehensively considers the similarity between the medical image and the category to which it belongs, as well as the similarity between it and other categories, thereby improving the accuracy of the first image quality score and being able to more accurately score the quality of the medical image.

[0080] In a possible implementation, when the training sample set has at least three categories in the same dimension, for example, in the examination item dimension, there are three categories, namely, chest posteroanterior radiographs, abdominal upright radiographs, and lumbar anteroposterior radiographs. Then, in S222, the similarity between the image feature vector and the negative class center of the second image set is calculated by the initial model to determine the second similarity result, which can be specifically S2221-S2222.

[0081] S2221, calculating the similarity between the image feature vector and the negative class center of each third image set through the initial model, and determining multiple third similarity results.

[0082] Specifically, the third image set includes at least one medical image in the training sample set that is in the same dimension as the category of the medical image and in another category different from the category of the medical image. That is, if the classification label result of the medical image is a category in a certain dimension, the medical images in the training sample set that belong to another category in the same dimension can be combined into a third image set. For example, if the classification label result of the medical image is a posterior-anterior chest radiograph, all medical images belonging to an upright abdominal radiograph form a third image set, and all medical images belonging to an AP lumbar radiograph form a third image set.

[0083] For each medical image, if there is more than one corresponding third image set, the similarity between the image feature vector of the medical image and the negative class center of each third image set can be calculated to determine multiple third similarity results. A third similarity result can be understood as the similarity between the image feature vector and its corresponding negative class center.

[0084] S2222, determine a second similar result from multiple third similar results.

[0085] Specifically, all third similarity results may be compared, and the third similarity result corresponding to the image feature vector and the negative class center closest thereto may be used as the second similarity result, that is, the negative class center corresponding to the second similarity result is closest to the image feature vector.

[0086] The nearest negative class center similarity (NNCCS) is used as the second similarity result. NNCCS can represent the similarity between the image feature vector and the nearest negative class center. The cosine value closest to the image feature vector angle is found among all negative class centers. The calculation formula can be:

[0087] Among them, NNCCS xi For medical images x i The nearest negative class center angle similarity, θ j For medical images x i The image feature vector and its class center u yi The angle between them, C is the total number of categories in the training sample set, and yi is the medical image x i The category to which it belongs.

[0088] Furthermore, when determining the first image quality score of the medical image based on the first similarity result and the second similarity result, the calculation formula can be expressed as:

[0089]

[0090] Among them, CR xi For medical images x i The first image quality score is ε, which is a constant to ensure that the denominator is not 0. The CR value reflects the relative position of the sample in the feature space. A high CR value means that the sample is easy to be correctly classified, thus indicating that the quality of the medical image is high.

[0091] In summary, in the present application, when calculating the first image quality score of a medical image, the similarity between the medical image and the negative class centers of other categories closest to the medical image is taken into account. There is no need to make all negative class centers participate in the calculation of the first image quality score, which simplifies the calculation process and ensures the accuracy of the calculation results.

[0092] In a possible implementation, in S3, a total loss function is determined based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score through the initial model, which may include S31-S33.

[0093] S31, determining a first loss function based on the classification label result and the classification prediction result through the initial model.

[0094] Specifically, the corresponding loss function can be calculated in terms of the classification of medical images, and the loss function is recorded as the first loss function. The first loss function can be calculated in the classification task layer, that is, it is determined by the classification label result and the classification prediction result.

[0095] As an example, the first loss function L cls It can be expressed as:

[0096]

[0097] Where N is the number of batches, m is the angle gap, and s is the scale parameter set by the function.

[0098] S32, determining a second loss function based on the first image quality score and the second image quality score through the initial model.

[0099] Specifically, the loss function in terms of the quality of the medical image can be recorded as the second loss function, and the difference between the first image quality score as the quality label value and the second image quality score as the quality prediction value is calculated to obtain the second loss function. In practical applications, the second loss function can be calculated in the hierarchical task regression layer.

[0100] As an example, Smooth L1-Loss can be used to calculate the difference between the quality label value and the quality prediction value, then the second loss function L CR It can be expressed as:

[0101]

[0102] S33: Determine the total loss function based on the first loss function and the second loss function.

[0103] Specifically, the first loss function and the second loss function can be summarized to obtain the total loss function of the initial model. The total loss function L can be expressed as:

[0104]

[0105] Among them, λ is the setting parameter.

[0106] In practical applications, the initial model can also include two optimizers, one as an optimizer for classification training, such as RMS prop (Root Mean Square Prop), and the other as an optimizer for quality training, such as Stochastic Gradient Descent (SGD). In the two optimizers, model optimization can be performed using two loss functions respectively.

[0107] In summary, by calculating the loss functions of image classification and image quality separately, each loss function will be more accurate. That is, by combining the classification center loss and regression loss, the model can simultaneously optimize the image classification performance and image quality assessment performance. The design of this loss function allows the model to learn two tasks at the same time in a unified training process, making the model training results better.

[0108] In a possible implementation, after the training is completed to obtain the image evaluation model, in the model application stage, the target medical image can be input into the image evaluation model to obtain the target classification result and the target image quality score of the target medical image. In this way, the classification of the target medical image and the quality score of the image can be directly obtained through the image evaluation model. Only one model is needed to evaluate the image in two aspects, which greatly improves the image processing efficiency and does not require manual judgment.

[0109] Based on the above model training method, the present application embodiment also provides a model training device, referring to Figure 2 As shown, it is a structural block diagram of a model training device provided in an embodiment of the present application, and the device may include:

[0110] An acquisition unit 201 is used to acquire a training sample set; the training sample set includes a plurality of medical images and classification label results corresponding to each of the medical images; the classification label results include categories under at least one dimension;

[0111] A first determining unit 202 is configured to input the training sample set into an initial model, and determine a classification prediction result, a first image quality score, and a second image quality score corresponding to each of the medical images; the first image quality score is used to identify the similarity between the medical image and a first image set, wherein the first image set includes at least one medical image in the training sample set that has the same category as the medical image;

[0112] A second determining unit 203 is used to determine a total loss function based on the classification label result, the classification prediction result, the first image quality score and the second image quality score through the initial model;

[0113] The training unit 204 is used to adjust the parameters of the initial model based on the total loss function until the training is completed to obtain the image evaluation model.

[0114] Optionally, the first determining unit is used to:

[0115] For each of the medical images, extracting features through the initial model to obtain an image feature vector;

[0116] For each of the medical images, the similarity between the image feature vector and the class center of the first image set is calculated through the initial model to determine the first image quality score of the medical image; the class center of the first image set is used to identify the average value of the image feature vectors of the medical images in the first image set.

[0117] Optionally, the first determining unit is used to:

[0118] For each of the medical images, calculating the similarity between the image feature vector and the class center of the first image set by using the initial model to determine a first similarity result;

[0119] For each of the medical images, the similarity between the image feature vector and the negative class center of a second image set is calculated by the initial model to determine a second similarity result; the second image set includes at least one medical image in the training sample set that has a different class from the medical image; the negative class center of the second image set is used to identify the average value of the image feature vectors of the medical images in the second image set;

[0120] The first image quality score of the medical image is determined by the initial model based on the first similarity result and the second similarity result.

[0121] Optionally, when the training sample set has at least three categories under the same dimension, the first determining unit is used to:

[0122] Calculating the similarity between the image feature vector and the negative class center of each third image set through the initial model, and determining a plurality of third similarity results; the third image set includes at least one medical image in the training sample set that is in the same dimension as the class of the medical image and in another category that is different from the class of the medical image;

[0123] The second similarity result is determined from the plurality of third similarity results; the distance between the negative class center corresponding to the second similarity result and the image feature vector is the shortest.

[0124] Optionally, the second determining unit is used to:

[0125] Determining a first loss function based on the classification label result and the classification prediction result through the initial model;

[0126] Determining a second loss function based on the first image quality score and the second image quality score by the initial model;

[0127] Based on the first loss function and the second loss function, the total loss function is determined.

[0128] Optionally, the acquiring unit is used to:

[0129] Performing format conversion and cropping processing on the initial medical image to obtain a medical image, and performing label parsing on the medical image knowledge graph to obtain classification label results corresponding to each of the medical images;

[0130] The training sample set is constructed based on the multiple medical images and the classification label results corresponding to each of the medical images.

[0131] Optionally, the device further comprises:

[0132] The output unit is used to input the target medical image into the image evaluation model to obtain the target classification result and the target image quality score of the target medical image.

[0133] In another aspect, the present application provides a computer device, referring to Figure 3 , which is a structural diagram of a computer device provided in an embodiment of the present application, wherein the computer device includes a processor 310 and a memory 320:

[0134] The memory 320 is used to store program codes and transmit the program codes to the processor 310;

[0135] The processor 310 is used to execute the method provided in the above embodiment according to the instructions in the program code.

[0136] The computer device may include a terminal device or a server, and the aforementioned apparatus may be configured in the computer device.

[0137] On the other hand, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is used to execute the method provided by the above embodiment.

[0138] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by program instruction hardware, and the above program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the above storage medium can be at least one of the following media: read-only memory (English: Read-only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store program codes.

[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0140] The above is only a preferred implementation of the present application. Although the present application has been disclosed as a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a training sample set; the training sample set includes a plurality of medical images and classification label results corresponding to each of the medical images; the classification label results include categories under at least one dimension; Inputting the training sample set into an initial model, determining a classification prediction result, a first image quality score, and a second image quality score corresponding to each of the medical images; the first image quality score is used to identify the similarity between the medical image and a first image set, the first image set including at least one medical image in the training sample set that has the same category as the medical image; Determining a total loss function based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score by the initial model; The parameters of the initial model are adjusted based on the total loss function until the training is completed to obtain an image evaluation model.

2. The method according to claim 1, characterized in that Inputting the training sample set into the initial model to determine the first image quality score corresponding to each of the medical images includes: For each of the medical images, extracting features through the initial model to obtain an image feature vector; For each of the medical images, the similarity between the image feature vector and the class center of the first image set is calculated through the initial model to determine the first image quality score of the medical image; the class center of the first image set is used to identify the average value of the image feature vectors of the medical images in the first image set.

3. The method according to claim 2, characterized in that The step of calculating, for each of the medical images, the similarity between the image feature vector and the class center of the first image set by using the initial model to determine a first image quality score of the medical image comprises: For each of the medical images, calculating the similarity between the image feature vector and the class center of the first image set by using the initial model to determine a first similarity result; For each of the medical images, the similarity between the image feature vector and the negative class center of a second image set is calculated by the initial model to determine a second similarity result; the second image set includes at least one medical image in the training sample set that has a different class from the medical image; the negative class center of the second image set is used to identify the average value of the image feature vectors of the medical images in the second image set; The first image quality score of the medical image is determined by the initial model based on the first similarity result and the second similarity result.

4. The method according to claim 3, characterized in that When the training sample set has at least three categories under the same dimension, calculating the similarity between the image feature vector and the negative class center of the second image set by the initial model to determine the second similarity result includes: Calculating the similarity between the image feature vector and the negative class center of each third image set through the initial model, and determining a plurality of third similarity results; the third image set includes at least one medical image in the training sample set that is in the same dimension as the class of the medical image and in another category that is different from the class of the medical image; The second similarity result is determined from the plurality of third similarity results; the distance between the negative class center corresponding to the second similarity result and the image feature vector is the shortest.

5. The method according to any one of claims 1 to 4, characterized in that: The determining of a total loss function based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score by the initial model includes: Determining a first loss function based on the classification label result and the classification prediction result through the initial model; Determining a second loss function based on the first image quality score and the second image quality score by the initial model; Based on the first loss function and the second loss function, the total loss function is determined.

6. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining a training sample set includes: Performing format conversion and cropping processing on the initial medical image to obtain a medical image, and performing label parsing on the medical image knowledge graph to obtain classification label results corresponding to each of the medical images; The training sample set is constructed based on the multiple medical images and the classification label results corresponding to each of the medical images.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: A target medical image is input into the image evaluation model to obtain a target classification result and a target image quality score of the target medical image.

8. A model training device, characterized in that: include: An acquisition unit, configured to acquire a training sample set; the training sample set includes a plurality of medical images and classification label results corresponding to each of the medical images; the classification label results include categories under at least one dimension; A first determination unit is configured to input the training sample set into an initial model to determine a classification prediction result, a first image quality score, and a second image quality score corresponding to each of the medical images; the first image quality score is used to identify a similarity between the medical image and a first image set, the first image set including at least one medical image in the training sample set that has the same category as the medical image; A second determining unit, configured to determine a total loss function based on the classification label result, the classification prediction result, the first image quality score, and the second image quality score through the initial model; A training unit is used to adjust the parameters of the initial model based on the total loss function until the training is completed to obtain an image evaluation model.

9. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

Images