Image classification model construction and image classification method, device and computer equipment

Abstract

The application relates to the technical field of image processing, and discloses an image classification model construction and image classification method and device and computer equipment. In the application, the images of a sample set are input into a feature extractor preinitialized with first parameters in a preset neural network, a feature vector corresponding to each image is obtained, the feature vector corresponding to the image is selected as a first node center of each neuron in a hidden layer in the preset neural network, a first width is determined, a first loss function is calculated, and the first parameters, the first node center and the first width are optimized so that the first loss function value is minimum; based on the optimized first parameters, the first node center, the first width and a first weight value of an output layer in the preset neural network preinitialized, the probability that an output feature vector belongs to each category is calculated, a second loss function is calculated, and the second node center, the second width and the first weight value are optimized so that the second loss function value is minimum, thereby improving the accuracy and reliability of the image classification model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to image classification model construction, image classification methods, apparatus, and computer equipment. Background Technology

[0002] Medical image classification has broad applications in disease diagnosis, treatment selection, and prognostic assessment. Highly reliable medical image classification methods can provide accurate predicted categories and prediction confidence levels. Accurate predicted categories reduce the risks of misdiagnosis and incorrect treatment selection; prediction confidence quantifies the reliability of the predicted category, and accurate prediction confidence helps doctors correctly assess the reliability of the predicted category. Based on this, doctors can combine their professional knowledge to make more precise diagnostic decisions, thereby effectively reducing safety risks and further improving the quality of diagnosis and treatment.

[0003] While deep learning-based medical image classification methods have demonstrated superior accuracy in predicting categories compared to traditional methods, two major problems remain. Firstly, due to the complexity and diversity of tissues and organs, medical images generally exhibit significant intra-class differences and inter-class similarities, limiting the accuracy of image category prediction. Secondly, the excessive number of parameters in networks used for medical image classification—often exceeding the number of training images—leads to a large number of redundant parameters. This causes the network's prediction confidence for most samples to approach 1, making it prone to overconfident and erroneous predictions. Such high confidence can mislead doctors into adopting incorrect predictions, thus compromising the reliability of medical image classification. Summary of the Invention

[0004] In view of this, the present invention provides an image classification model construction and image classification method, apparatus and computer equipment to solve the problem of low accuracy and reliability of image classification in related technologies.

[0005] In a first aspect, the present invention provides an image classification model construction method, the method comprising: inputting images in a training sample set into a feature extractor with pre-initialized first parameters in a preset neural network to obtain a feature vector corresponding to each image, wherein the image has a category label; selecting feature vectors corresponding to multiple images in the training sample set as the first node center of each neuron in the hidden layer of the preset neural network based on preset requirements, and determining a first width corresponding to the first node center; optimizing the first parameters, the first node center, and the first width by calculating a first loss function between the probability that the current feature vector corresponding to any image in the training sample set belongs to each category and the actual category corresponding to the current feature vector, so as to minimize the value of the first loss function, thereby obtaining optimal parameters, a second node center, and a second width; calculating the category to which the current feature vector belongs and the confidence score corresponding to each category output by the preset neural network based on the first weight of the output layer in the preset neural network; optimizing the second node center, the second width, and the first weight by calculating a second loss function between the confidence score corresponding to each category and the classification accuracy, so as to minimize the value of the second loss function, thereby obtaining a third node center, a third width, and a second weight, thereby obtaining an image classification model.

[0006] The image classification model construction method provided by this invention involves inputting images from a training sample set into a feature extractor with pre-initialized first parameters in a preset neural network to obtain feature vectors corresponding to each image. Based on preset requirements, feature vectors corresponding to multiple images in the training sample set are selected as the first node centers of each neuron in the hidden layer of the preset neural network. The first width corresponding to the first node center is determined, a first loss function is calculated, and the first node center, first parameter, and first width are optimized to minimize the value of the first loss function. Based on the first weights of the output layer in the preset neural network that are pre-initialized, the probability of the current feature vector belonging to each category is calculated, a second loss function is calculated, and the second node center, second width, and first weights are optimized to minimize the value of the second loss function, thus obtaining an image classification model. This method effectively promotes intra-class consistency and inter-class discriminative representation learning in neural networks, improves the accuracy of image category prediction, and solves the problem of traditional neural network models easily outputting incorrectly predicted categories with excessively high confidence by incorporating prediction confidence error as supervision information into the training process of the neural network, greatly improving the reliability of image classification.

[0007] In one alternative implementation, the first parameter is obtained by acquiring the parameters of the feature extractor in the pre-trained RadImageNet model.

[0008] The feature extractor in RadImageNet has good feature extraction capabilities for different types of images. Using the parameters of the feature extractor in the RadImageNet model as the initial values ​​of the first parameter in the feature extractor of the preset neural network allows the preset neural network to obtain good initial parameters, which is beneficial to improving the representation ability of the feature vector corresponding to the image, thereby improving the accuracy of the image classification model in predicting the category.

[0009] In one optional implementation, determining the first width corresponding to the first node center includes: calculating the distance between the first node center and the node centers corresponding to all other categories except the category to which the first node center belongs; and selecting the minimum distance as the first width corresponding to the first node center.

[0010] This method calculates the distance between the center of the first node and the center of all other nodes in all categories except the category to which the first node belongs, and selects the minimum distance as the first width corresponding to the first node. This can improve the training efficiency of the preset neural network and enhance the accuracy of the image classification model in predicting image categories.

[0011] In one optional implementation, the preset neural network is a deep RBF (Radial Basis Function) neural network model. The calculation of the probability that the current feature vector belongs to each category includes: calculating the RBF score of the current feature vector and each first node center; calculating the probability that the current feature vector belongs to each first node center based on the RBF score of the current feature vector and each first node center; and adding the calculated probabilities that the current feature vector belongs to the node centers corresponding to each category to obtain the probability that the current feature vector belongs to each category.

[0012] RBF neural networks are locally approximating neural networks. This invention selects RBF neural networks as the preset neural network, which can greatly accelerate the learning speed and improve the training efficiency of image classification models. Furthermore, by calculating the RBF scores of the current feature vector and each first node center, the probability of the current feature vector belonging to each first node center is calculated. The probabilities of the node centers corresponding to each category are added together to obtain the probability of the feature vector belonging to each category. Then, the first loss function is calculated, and each parameter is optimized to make it more accurate.

[0013] In one optional implementation, the step of calculating the category to which the current feature vector belongs and the confidence score corresponding to each category output by the preset neural network includes: obtaining the category component values ​​output by the current feature vector through the preset neural network; normalizing the category component values ​​to obtain the confidence score corresponding to each category to which the current feature vector belongs, and the sum of the confidence scores corresponding to each category is 1.

[0014] This invention, after obtaining the output values ​​of each category component, normalizes these values ​​to obtain the confidence score for each category. Then, it calculates the loss function between the confidence score and the classification accuracy. This solves the problem of traditional neural network models easily outputting incorrectly predicted categories with excessively high confidence, greatly improving the reliability of image classification.

[0015] In an optional implementation, the first loss function is the cross-entropy loss function, and the second loss function is a hybrid loss function consisting of focus loss, a difference term between confidence scores and classification accuracy for each category, and an L1 sparse regularization term.

[0016] This invention uses the cross-entropy loss function as the first loss function and a hybrid loss function consisting of focus loss, the difference between confidence scores and classification accuracy for each category, and an L1 sparse regularization term as the second loss function. By training a preset neural network using the L1 sparse regularization term, redundant output weights can be removed and confidence (i.e., probability) can be calibrated. This not only improves the accuracy of image category prediction by the image category model, but also solves the problem that traditional neural network models are prone to outputting incorrectly predicted categories with excessively high confidence, thus greatly improving the reliability of image classification.

[0017] Secondly, the present invention provides an image classification method, the method comprising: inputting an image to be classified into an image classification model to obtain a predicted category and a corresponding prediction confidence, wherein the image classification model is constructed using the image classification model construction method described in the first aspect or any optional embodiment of the first aspect; selecting the category corresponding to the highest probability as the predicted category of the image to be classified, and outputting the highest probability as the prediction confidence.

[0018] The image classification method provided by this invention improves the accuracy of image classification by inputting the image to be classified into an image classification model to obtain each category and the probability corresponding to each category, and selecting the category with the highest probability as the predicted category of the image to be classified.

[0019] Thirdly, the present invention provides an image classification model construction apparatus, the apparatus comprising: an image feature extraction module, configured to input images from a training sample set into a feature extractor with pre-initialized first parameters in a preset neural network to obtain a feature vector corresponding to each image, wherein the image carries a category label; a first node center determination module, configured to select feature vectors corresponding to multiple images in the training sample set as the first node centers of neurons in the hidden layer of the preset neural network based on preset requirements, and determine a first width corresponding to the first node centers; and a first parameter optimization module, configured to calculate a first loss between the probability that the current feature vector corresponding to any image in the training sample set belongs to a category and the actual category corresponding to the current feature vector. The system employs a first loss function to optimize the first parameter, first node center, and first width, minimizing the first loss function value to obtain the optimal parameters, second node center, and second width. An output category acquisition module is used to calculate the category to which the current feature vector belongs and the corresponding confidence score for each category, based on the first weight of the output layer in a pre-initialized preset neural network. A second parameter optimization module is used to optimize the second node center, second width, and first weight by calculating the second loss function between the confidence score corresponding to each category and the classification accuracy, minimizing the second loss function value to obtain the third node center, third width, and second weight, thus obtaining the image classification model.

[0020] Fourthly, embodiments of the present invention also provide an image classification apparatus, the apparatus comprising: an image input module, configured to input an image to be classified into the image classification model to obtain a predicted category and a corresponding prediction confidence, wherein the image classification model is constructed using the image classification model construction apparatus described in the third aspect; and a prediction result output module, configured to select the category corresponding to the highest probability as the predicted category of the image to be classified, and output the highest probability as the prediction confidence.

[0021] Fifthly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the image classification model construction method of the first aspect or any corresponding embodiment described above, or to perform the image classification method of the second aspect described above.

[0022] In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the image classification model construction method of the first aspect or any corresponding embodiment thereof, or to perform the image classification method of the second aspect. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the image classification model construction method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating another image classification model construction method according to an embodiment of the present invention;

[0026] Figure 3 This is a flowchart illustrating another image classification model construction method according to an embodiment of the present invention;

[0027] Figure 4 This is a flowchart illustrating an image classification method according to an embodiment of the present invention;

[0028] Figure 5 This is a structural block diagram of an image classification model construction apparatus according to an embodiment of the present invention;

[0029] Figure 6 This is a structural block diagram of an image classification device according to an embodiment of the present invention;

[0030] Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Medical image classification has broad applications in disease diagnosis, treatment selection, and prognostic assessment. Highly reliable medical image classification methods can provide accurate predicted categories and prediction confidence levels. Accurate predicted categories reduce the risks of misdiagnosis and incorrect treatment selection; prediction confidence quantifies the reliability of the predicted category, and accurate prediction confidence helps doctors correctly assess the reliability of the predicted category. Based on this, doctors can combine their professional knowledge to make more precise diagnostic decisions, thereby effectively reducing safety risks and further improving the quality of diagnosis and treatment.

[0033] While deep learning-based medical image classification methods have demonstrated superior accuracy in predicting categories compared to traditional methods, two major problems remain. Firstly, due to the complexity and diversity of tissues and organs, medical images generally exhibit significant intra-class differences and inter-class similarities, limiting the accuracy of image category prediction. Secondly, the excessive number of parameters in networks used for medical image classification—often exceeding the number of training images—leads to a large number of redundant parameters. This causes the network's prediction confidence for most samples to approach 1, making it prone to overconfident and erroneous predictions. Such high confidence can mislead doctors into adopting incorrect predictions, thus compromising the reliability of medical image classification.

[0034] According to an embodiment of the present invention, an embodiment of an image classification model construction method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] This embodiment provides a method for constructing an image classification model, which can be used with the aforementioned computer equipment. Figure 1 This is a flowchart of an image category model construction method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0036] Step S101: Input the images in the training sample set into the feature extractor with the first parameter pre-initialized in the preset neural network to obtain the feature vector corresponding to each image. The images are labeled with category labels.

[0037] The initial value of the first parameter of the feature extractor is not limited in this embodiment and can be determined by human experience. The preset neural network is also not limited and can be determined according to the actual situation. The preset neural network includes an input layer, a hidden layer, and an output layer; for example, RBF (Radial Basis Function) is only used as an example. Images from the training sample set are input into the feature extractor in the preset neural network whose first parameter is pre-initialized to obtain the feature vector corresponding to each image.

[0038] Step S102: Based on preset requirements, select feature vectors corresponding to multiple images in the training sample set as the first node center of each neuron in the hidden layer of the preset neural network, and determine the first width corresponding to the first node center.

[0039] In this embodiment of the application, the images in the training sample set can be selected by medical experts from the training dataset, representing representative images for each category, which can be represented as I = {i yj |y = 1,...,Y;j = 1,...,N y}, where y represents the y-th class, Y represents the number of classes, and i yj Represents the j-th image of class y, N y The number of representative images of class y can be represented as C = {c_i}. The feature vectors corresponding to the selected images can be used as the node centers of each neuron in the hidden layer of the pre-defined neural network. yj |y = 1,...,Y;j = 1,...,N y}, where c yj This represents the center of the j-th node in class y. After determining the node center, there are no restrictions on the method for determining the width corresponding to the node center. For example, the average distance between each node center and the other node centers can be used as the width of the node center. This is just an example.

[0040] Step S103: By calculating the first loss function between the probability of the current feature vector corresponding to any image in the training sample set belonging to each category and the actual category corresponding to the current feature vector, optimize the first parameter, the first node center, and the first width to minimize the value of the first loss function, and obtain the optimal parameter, the second node center, and the second width.

[0041] In this embodiment, any image from the training sample set is selected for training a pre-defined neural network. The actual category corresponding to the feature vector is known from the training dataset. The feature vector obtained after the feature vector passes through the feature extractor can be represented as f(x; θ), where x represents the selected image for training. The probability of the feature vector belonging to each category is calculated. The category of the first loss function is not limited; for example, it can be the mean squared error loss function, which is only used as an example. After obtaining the probability of the feature vector belonging to each category, the mean of the sum of squares of the errors between the predicted category probability and the actual corresponding point is calculated. Gradient descent can be used to update the first node center, the first parameter, and the first width, so that the distance between other categories different from the actual category and the actual category increases, thereby reducing the distance between the predicted category and the actual category, and thus obtaining the second node center. Optimal parameter θ * Second width This promotes intra-class consistency and inter-class discriminative representation learning, as an example only.

[0042] Step S104: Based on the first weight of the output layer in the pre-initialized preset neural network, calculate the category to which the current feature vector belongs and the confidence score corresponding to each category through the output of the preset neural network.

[0043] This invention can initialize the weights ω between the hidden layer and the output layer in a preset neural network based on the least squares method. y =[ω 1y ,…,ω Ny ] represents the connection weight vector between the centers of N nodes and the y-th output node. The selected feature vector is input into the preset neural network to obtain the output vector, where the output vector is each category and the corresponding component value of each category. The obtained component values ​​can be standardized so that the result falls in the interval [0,1], thus obtaining the category to which the feature vector belongs and the confidence score corresponding to each category. This is just an example.

[0044] Step S105: By calculating the second loss function between the confidence score and classification accuracy corresponding to each category of the current feature vector, optimize the second node center, second width and first weight to minimize the value of the second loss function, obtain the third node center, third width and second weight, and obtain the image classification model.

[0045] In this embodiment of the invention, the classification accuracy can be obtained by inputting all feature vectors with category labels into a preset neural network with pre-initialized first weights, obtaining each category corresponding to each feature vector, comparing it with the actual category corresponding to each feature vector, and calculating the ratio between the number of predicted categories that match the actual categories and the number of all feature vectors. After obtaining the confidence scores corresponding to each category of the feature vectors, the loss function between the confidence scores corresponding to each category and the classification accuracy can also be calculated according to the method of the above embodiment, which will not be elaborated here. Alternatively, the gradient descent method can be used to update the weights, the second node center, and the second width.

[0046] The image classification model construction method provided by this invention involves inputting images from a training sample set into a feature extractor with pre-initialized first parameters in a preset neural network to obtain feature vectors corresponding to each image. Based on preset requirements, feature vectors corresponding to multiple images in the training sample set are selected as the first node centers of neurons in the hidden layer of the preset neural network. The first width corresponding to the first node center is determined, a first loss function is calculated, and the first node center, first parameter, and first width are optimized to minimize the value of the first loss function. Based on the first weights of the output layer in the preset neural network that are pre-initialized, the probability of the current feature vector belonging to each category is calculated, a second loss function is calculated, and the second node center, second width, and first weights are optimized to minimize the value of the second loss function, thus obtaining an image classification model. This method effectively promotes intra-class consistency and inter-class discriminative representation learning in neural networks, improves the accuracy of image category prediction, and solves the problem of traditional neural network models easily outputting incorrectly predicted categories with excessively high confidence by incorporating prediction confidence error as supervision information into the training process of the neural network, greatly improving the reliability of image classification.

[0047] This embodiment provides a method for constructing an image classification model, which can be used with the aforementioned computer equipment. Figure 2 This is a flowchart of an image classification model construction method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0048] Step S201: Input the images in the training sample set into the feature extractor with the first parameter pre-initialized in the preset neural network to obtain the feature vector corresponding to each image.

[0049] The images are labeled with category tags; please refer to the detailed process. Figure 1 Step S101 shown will not be repeated here.

[0050] Step S202: Based on preset requirements, select feature vectors corresponding to multiple images in the training sample set as the first node center of each neuron in the hidden layer of the preset neural network, and determine the first width corresponding to the first node center.

[0051] Specifically, step S202 includes:

[0052] Step S2021: Calculate the distance between the center of the first node and the center of all other nodes in all categories except the category to which the first node belongs.

[0053] The embodiments of this application can calculate the Euclidean distance between the center of the first node and all other nodes corresponding to all other categories except the category of the center of the first node, and can also calculate the cosine distance of the included angle, etc., which are only examples.

[0054] Step S2022: Select the minimum distance as the first width corresponding to the center of the first node.

[0055] Specifically, after calculating the distances between the center of each node and the centers of all other nodes except the center of the first node, the minimum distance can be selected as the initial width of that node, i.e., the first width, which can be expressed as δ={δ yj |y = 1,...,Y;j = 1,...,N y}, where δ yj This represents the width of the j-th RBF neuron of class y.

[0056] This method calculates the distances between the center of the first node and the center of all other nodes in all categories except the category to which the first node belongs, and selects the minimum distance as the first width corresponding to the first node. This can improve the training efficiency of the preset neural network, not only improving the accuracy of the image classification model in predicting image categories, but also improving the training efficiency of the image classification model.

[0057] Step S203: By calculating the first loss function between the probability of the current feature vector corresponding to any image in the training sample set belonging to each category and the actual category corresponding to the current feature vector, optimize the first parameter, the first node center, and the first width to minimize the value of the first loss function, and obtain the optimal parameter, the second node center, and the second width.

[0058] Specifically, the preset neural network can be an RBF neural network model, and step S203 above includes...

[0059] Step S2031: Calculate the RBF score between the current feature vector and the center of each first node.

[0060] In this embodiment, the distance between the feature vector extracted by the feature extractor and the center of each node is transformed into a similarity between (0,1], and the RBF score between the current feature vector and the center of each first node is calculated by the following formula:

[0061]

[0062] Where K(f(x;θ),c yj ,δ yj ) represents the RBF score between the current feature vector and the center of the j-th node of the y-th class.

[0063] Step S2032: Based on the current feature vector and the RBF scores of each first node center, calculate the probability that the current feature vector belongs to each first node center.

[0064] The probability that the current feature vector belongs to the center of each first node is calculated using the following formula:

[0065]

[0066] Where p(x∈c) ij ) represents the probability that the current feature vector belongs to the center of the j-th node in the y-th class; μ represents the scale parameter; n represents 1, 2, 3, ..., N. y N y c represents the number of node centers in class y; Y represents the number of classes; yn This represents the center of the nth node in the y-th class.

[0067] Step S2033: Add the calculated probabilities of the current feature vector belonging to the node center of each category to obtain the probability of the current feature vector belonging to each category.

[0068] In this embodiment of the application, after obtaining the probability that the current feature vector belongs to each node center, the probabilities of all node centers corresponding to a certain category can be added together to obtain the probability that the feature vector belongs to each category.

[0069] Specifically, the probability of the current feature vector belonging to each category is calculated using the following formula:

[0070]

[0071] Where, p RBF (y|x) represents the probability that image x belongs to category y.

[0072] RBF neural networks are locally approximating neural networks. This invention selects RBF neural networks as the preset neural network, which can greatly accelerate the learning speed and improve the training efficiency of image classification models. Furthermore, by calculating the RBF scores of the current feature vector and each first node center, calculating the probability that the current feature vector belongs to each first node center, and adding the probabilities of the node centers corresponding to each category, the probability of the feature vector belonging to each category is obtained. Then, the first loss function is calculated, and each parameter is optimized to make it more accurate.

[0073] Step S204: Based on the first weight of the output layer in the pre-initialized preset neural network, calculate the category to which the current feature vector belongs and the confidence score corresponding to each category output by the preset neural network.

[0074] Specifically, step S204 includes:

[0075] Step S2041: Obtain the values ​​of each category component output by the current feature vector through the preset neural network; for the y-th component of the logits vector output by the preset neural network for image x, it is...

[0076] Step S2042: Normalize the values ​​of each category component to obtain the confidence score of the current feature vector belonging to each category. The sum of the confidence scores of each category is 1.

[0077] The values ​​of each category are normalized using the following formula:

[0078]

[0079] The sum of the probabilities for each category is 1.

[0080] This invention, after obtaining the output category component values, normalizes each category component value to obtain the confidence score corresponding to each category. Then, it calculates the loss function between the confidence score and the classification accuracy, solving the problem that traditional neural network models easily output incorrectly predicted categories with excessively high confidence, and greatly improving the reliability of image classification.

[0081] Step S205 involves calculating the second loss function between the confidence score and classification accuracy corresponding to each category of the current feature vector, optimizing the second node center, second width, and first weight to minimize the second loss function value, and obtaining the third node center, third width, and second weight, thus obtaining the image classification model. For details, please refer to [link to details]. Figure 1 Step S105 of the illustrated embodiment will not be described again here.

[0082] This embodiment provides a method for constructing an image classification model, which can be used with the aforementioned computer equipment. Figure 3 This is a flowchart of an image classification model construction method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0083] Step S301: Input the images in the training sample set into the feature extractor with the first parameter pre-initialized in the preset neural network to obtain the feature vector corresponding to each image.

[0084] The images are labeled with category tags. See details below. Figure 2 Step S201 shown will not be repeated here.

[0085] Step S302: The first parameter is obtained by acquiring the parameters of the feature extractor in the pre-trained RadImageNet model.

[0086] In this embodiment, the parameters of the feature extractor in the pre-trained RadImageNet model are used as the first parameters of the feature extractor in the preset neural network. The feature extractor in RadImageNet has good feature extraction capabilities for different types of images. Using the parameters of the feature extractor in the RadImageNet model as the initial values ​​of the first parameters in the feature extractor of the preset neural network enables the preset neural network to obtain good initial parameters, which is beneficial to improving the representation ability of the feature vector corresponding to the image, thereby improving the accuracy of the image classification model in predicting the image category.

[0087] Step S303: Based on preset requirements, select feature vectors corresponding to multiple images in the training sample set as the first node centers of each neuron in the hidden layer of the preset neural network, and determine the first width corresponding to the first node center. For details, please refer to... Figure 2 Step S202 shown will not be repeated here.

[0088] Step S304: By calculating the first loss function between the probability of the current feature vector corresponding to any image in the training sample set belonging to each category and the actual category corresponding to the current feature vector, the first parameter, the first node center, and the first width are optimized to minimize the value of the first loss function, thus obtaining the optimal parameter, the second node center, and the second width. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 shown will not be repeated here.

[0089] Step S305: Based on the first weights of the output layer in the pre-initialized preset neural network, calculate the category to which the current feature vector belongs and the confidence score corresponding to each category, output by the preset neural network. For details, please refer to [link to relevant documentation]. Figure 2 Step S204 shown will not be repeated here.

[0090] Step S306: By calculating the second loss function between the confidence score and classification accuracy corresponding to the current feature vector belonging to each category, the second node center, second width, and first weight are optimized to minimize the value of the second loss function, thus obtaining the third node center, third width, and second weight, and finally the image classification model. For details, please refer to [link to details]. Figure 1 Step S105 of the illustrated embodiment will not be described again here. For details, please refer to [link to relevant documentation]. Figure 2 Step S205 shown will not be repeated here.

[0091] Step S307: The first loss function is the cross-entropy loss function, and the second loss function is a hybrid loss function consisting of focus loss, the difference between confidence scores and classification accuracy for each category, and L1 sparse regularization.

[0092] Based on the RBF score, the cross-entropy loss function between the probability of the current feature vector belonging to each category and the actual category is as follows:

[0093]

[0094] in, t represents the cross-entropy loss function value between category y and the actual category. y The value t can be either 0 or 1. When the current feature vector belongs to a certain category that is the same as the actual category, t... y The value is 1 if it is not 0 otherwise.

[0095] The cross-entropy loss function value can be minimized using stochastic gradient descent, while optimizing the first parameter, the first node center, and the first width. This reduces the distance between the input image and the RBF node center of its own class in the feature space, while increasing the distance between the RBF node centers of other classes. This promotes intra-class consistency and inter-class discriminative representation learning, thereby improving the accuracy of class prediction.

[0096] The relationship between the confidence scores and classification accuracy for each category can be calculated using a hybrid loss function consisting of focus loss, a difference term between the confidence scores and classification accuracy for each category, and an L1 sparse regularization term.

[0097]

[0098] in, The value of the loss function is represented; S represents the number of training samples in each batch; β≥1 is a hyperparameter of FL. This represents the coefficient of the auxiliary loss term in DCA (Decision Curve Analysis); p(j|x s ) indicates that x s The posterior probability of being predicted as class j; x s When the actual category is j, then l s [j] = 1, otherwise l s [j] = 0; ζ > 0 is the L1 sparse regularization coefficient.

[0099] Stochastic proximal gradient descent can be used to minimize the loss function value. Redundant output weights can be removed and confidence levels calibrated.

[0100] This invention uses the cross-entropy loss function as the first loss function and a hybrid loss function consisting of focus loss, the difference between confidence scores and classification accuracy for each category, and an L1 sparse regularization term as the second loss function. By training a preset neural network using the L1 sparse regularization term, redundant output weights can be removed and confidence (i.e., probability) can be calibrated. This not only improves the accuracy of image category prediction by the image category model, but also solves the problem that traditional neural network models are prone to outputting incorrectly predicted categories with excessively high confidence, thus greatly improving the reliability of image classification.

[0101] This embodiment provides an image classification method that can be used with the aforementioned computer equipment. Figure 4 This is a flowchart of an image classification method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0102] Step S401: Input the image to be classified into the image classification model to obtain the predicted category and the corresponding prediction confidence. The image classification model is constructed using the image classification model construction method described in the above embodiment.

[0103] In this embodiment, the image to be classified is directly input into the image classification model to obtain the predicted category and the probability corresponding to each category.

[0104] Step S402: Select the category corresponding to the highest probability as the predicted category of the image to be classified, and output the highest probability as the prediction confidence.

[0105] In this embodiment of the application, after obtaining the predicted category and the confidence score corresponding to each category, the category with the highest probability can be selected as the predicted category of the image to be classified, and the highest probability can be output as the prediction confidence score.

[0106] The image classification method provided by this invention improves the accuracy of image classification by inputting the image to be classified into an image classification model to obtain each category and the confidence score corresponding to the category, and selecting the category with the highest probability as the predicted category of the image to be classified.

[0107] This embodiment also provides an image classification model construction apparatus, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0108] This embodiment provides an image classification model construction device, such as... Figure 5As shown, the system includes an image feature extraction module 501, used to input images from the training sample set into a feature extractor with pre-initialized first parameters in a preset neural network to obtain a feature vector corresponding to each image, with each image carrying a category label; a first node center determination module 502, used to select feature vectors corresponding to multiple images in the training sample set as the first node centers of neurons in the hidden layer of the preset neural network based on preset requirements, and determine the first width corresponding to the first node center; and a first parameter optimization module 503, used to optimize the first parameters by calculating a first loss function between the probability that the current feature vector corresponding to any image in the training sample set belongs to a category and the actual category corresponding to the current feature vector. The first node center and first width are used to minimize the first loss function value, thus obtaining the optimal parameters, second node center, and second width. The output category acquisition module 504 is used to calculate the category to which the current feature vector belongs and the confidence score corresponding to each category by outputting the current feature vector through the preset neural network based on the first weight of the output layer in the preset neural network. The second parameter optimization module 505 is used to optimize the second node center, second width, and first weight by calculating the second loss function between the confidence score corresponding to each category of the current feature vector and the classification accuracy, so as to minimize the second loss function value, thus obtaining the third node center, third width, and second weight, and obtaining the image classification model.

[0109] In some alternative implementations, the first parameter is obtained by acquiring the parameters of the feature extractor in the pre-trained RadImageNet model.

[0110] In some optional implementations, the first node center determination module includes:

[0111] The distance calculation unit is used to calculate the distance between the center of the first node and the center of all other nodes in all other categories except the category to which the center of the first node belongs.

[0112] The first width determination module is used to select the minimum distance as the first width corresponding to the center of the first node.

[0113] In some optional implementations, the first parameter optimization module includes:

[0114] The RBF score calculation unit is used to calculate the RBF score between the current feature vector and the center of each first node.

[0115] The probability calculation unit is used to calculate the probability that the current feature vector belongs to each of the first node centers based on the RBF scores of the current feature vector and each first node center.

[0116] The category probability calculation module is used to add the probabilities of the current feature vector belonging to the node centers of each category to obtain the probability of the current feature vector belonging to each category.

[0117] In some optional implementations, the output category acquisition module includes: a category component value acquisition unit, used to acquire the category component values ​​output by the current feature vector through a preset neural network; and a normalization processing unit, used to normalize the category component values ​​to obtain the confidence scores corresponding to the current feature vector belonging to each category, wherein the sum of the confidence scores corresponding to each category is 1.

[0118] In some alternative implementations, the first loss function is the cross-entropy loss function, and the second loss function is a hybrid loss function consisting of focus loss, a difference term between confidence scores and classification accuracy for each category, and an L1 sparse regularization term.

[0119] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0120] This embodiment provides an image classification device, such as... Figure 6 As shown, it includes an image input module 601, which is used to input the image to be classified into an image classification model to obtain the predicted category and the corresponding prediction confidence. The image classification model is constructed using the image classification model construction device of the above embodiment. The prediction result output module 602 is used to select the category corresponding to the highest probability as the predicted category of the image to be classified, and output the highest probability as the prediction confidence.

[0121] In this embodiment, the image classification model building device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0122] This invention also provides a computer device having the above-described features. Figure 5 or Figure 6 The image classification model or image classification device shown is constructed.

[0123] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0124] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0125] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0126] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0127] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0128] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0129] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0130] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0131] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined herein.

Claims

1. A method for constructing an image classification model, characterized in that, The method includes: The images in the training sample set are input into the feature extractor with the first parameter pre-initialized in the preset neural network to obtain the feature vector corresponding to each image, and the image has a category label; the preset neural network is a deep RBF neural network. Based on preset requirements, feature vectors corresponding to multiple images in the training sample set are selected as the first node centers of each neuron in the hidden layer of the preset neural network, and the first width corresponding to the first node center is determined. Determining the first width corresponding to the center of the first node includes: calculating the distance between the center of the first node and the center of nodes corresponding to all other categories except the category to which the center of the first node belongs; and selecting the minimum distance as the first width corresponding to the center of the first node. By calculating the first loss function between the probability of the current feature vector corresponding to any image in the training sample set belonging to each category and the actual category corresponding to the current feature vector, the first parameter, the first node center, and the first width are optimized to minimize the value of the first loss function, thereby obtaining the optimal parameter, the second node center, and the second width. Based on the first weight of the output layer in the pre-initialized preset neural network, the current feature vector is calculated and the category to which the current feature vector belongs and the confidence score corresponding to each category are output through the preset neural network; By calculating the second loss function between the confidence score and classification accuracy corresponding to each category of the current feature vector, the second node center, second width, and first weight are optimized to minimize the value of the second loss function, thereby obtaining the third node center, third width, and second weight, and thus obtaining the image classification model.

2. The image classification model construction method according to claim 1, characterized in that, The first parameter is obtained by acquiring the parameters of the feature extractor in the pre-trained RadImageNet model.

3. The image classification model construction method according to claim 1, characterized in that, The calculation of the probability that the current feature vector belongs to each category includes: Calculate the RBF score between the current feature vector and the center of each first node; Based on the current feature vector and the RBF scores of each first node center, calculate the probability that the current feature vector belongs to each first node center; The probabilities of the current feature vector belonging to the node center corresponding to each category are added together to obtain the probability of the current feature vector belonging to each category.

4. The image classification model construction method according to claim 1, characterized in that, The step of calculating the category to which the current feature vector belongs and the confidence score corresponding to each category, through a preset neural network, includes: Obtain the values ​​of each category component of the current feature vector output by a preset neural network; The values ​​of each category component are normalized to obtain the confidence scores of the current feature vector belonging to each category. The sum of the confidence scores of each category is 1.

5. The image classification model construction method according to claim 1, characterized in that, The first loss function is the cross-entropy loss function, and the second loss function is a combination of focus loss, the difference between the confidence scores of each category and the classification accuracy, and... L 1. A hybrid loss function composed of sparse regularization terms.

6. An image classification method, characterized in that, The method includes: The image to be classified is input into the image classification model to obtain the predicted category and the corresponding prediction confidence. The image classification model is constructed using the image classification model construction method as described in any one of claims 1-5. The category with the highest probability is selected as the predicted category of the image to be classified, and the highest probability is output as the prediction confidence.

7. An image classification model construction device, characterized in that, The device includes: The image feature extraction module is used to input images from the training sample set into a feature extractor with pre-initialized first parameters in a preset neural network to obtain a feature vector corresponding to each image, wherein the image has a category label; the preset neural network is a deep RBF neural network. The first node center determination module is used to select feature vectors corresponding to multiple images in the training sample set as the first node centers of each neuron in the hidden layer of the preset neural network based on preset requirements, and determine the first width corresponding to the first node center; the determination of the first width corresponding to the first node center includes: calculating the distance between the first node center and the node centers corresponding to all other categories except the category to which the first node center belongs; and selecting the minimum distance as the first width corresponding to the first node center. The first parameter optimization module is used to optimize the first parameter, the first node center, and the first width by calculating the first loss function between the probability of the current feature vector corresponding to any image in the training sample set belonging to each category and the actual category corresponding to the current feature vector, so as to minimize the value of the first loss function and obtain the optimal parameter, the second node center, and the second width. The output category acquisition module is used to calculate the category to which the current feature vector belongs and the confidence score corresponding to each category, based on the first weight of the output layer in the pre-initialized preset neural network. The second parameter optimization module is used to optimize the second node center, second width, and first weight by calculating the second loss function between the confidence score and classification accuracy corresponding to each category of the current feature vector, so as to minimize the value of the second loss function, and obtain the third node center, third width, and second weight, thus obtaining the image classification model.

8. An image classification device, characterized in that, The device includes: An image input module is used to input the image to be classified into the image classification model to obtain the predicted category and the corresponding prediction confidence. The image classification model is constructed using the image classification model construction device as described in any one of claims 1-5. The prediction result output module selects the category corresponding to the highest probability as the predicted category of the image to be classified, and outputs the highest probability as the prediction confidence.

9. A computer device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the image classification model construction method of any one of claims 1 to 5, or the image classification method of claim 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the image classification model construction method of any one of claims 1 to 5, or to perform the image classification method of claim 6.

Images