Depression rating method based on interpretability
By constructing a convolutional neural network image classification model and generating a heat map, the problem of the lack of explanatory nature of deep learning models in depression assessment is solved, and the explanatory nature and computational efficiency of the model are improved.
Patent Information
- Application Number
- CN202411766478.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-25
AI Technical Summary
The lack of explanatory deep learning models in the assessment of depression makes it difficult for doctors and patients to understand the basis for the evaluation results, and at the same time, the calculation costs are high and time-consuming and resource-consuming.
Using an interpretable depression rating method, a convolutional neural network image classification model is constructed, a heat map is generated to show the key features of model decisions, and a gradient is calculated through backpropagation to obtain weights, and an average gradient value is calculated to explain the model's decision-making process.
The explanatory model is realized, the key features are visually displayed, hidden patterns are revealed, and the education and training are assisted, which improves the trust of doctors and patients in the evaluation results, and reduces the computational cost.
Smart Images
Figure CN119694538A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning interpretable technology, and in particular to an interpretable depression rating method. Background Art
[0002] Deep learning detection has advantages over manual evaluation in terms of objectivity, efficiency, and standardization, but in intelligent evaluation, due to the characteristics of black box models, its internal decision-making process is difficult to explain. In the depression assessment scenario, doctors and patients usually need to understand the basis of the evaluation results. Therefore, the low interpretability of the model may undermine people's trust in the evaluation results. And the computational cost is usually high, especially for large-scale data and complex models. The sampling and training of local models required to generate explanations may consume a lot of time and computing resources. Therefore, we propose an interpretable depression rating method to solve the above problems. Summary of the invention
[0003] 1. Technical issues to be solved
[0004] In view of the deficiencies of the prior art, the present invention provides an explainable depression rating method, which solves the problems raised in the above background technology.
[0005] (II) Technical solution
[0006] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:
[0007] A depression rating method based on interpretability includes the following steps:
[0008] S1: Data preparation, which includes collecting facial images of patients with depression, image preprocessing and data annotation;
[0009] S2: Build, train, evaluate and optimize the convolutional neural network (F) image classification model;
[0010] S3: Select the fully connected layer of the network, obtain the model weights, ensure that the model has been loaded with the trained weights, and select a convolutional layer in the middle of the model as the target layer for generating the heat map;
[0011] S4: Input the image to be explained into the model and forward propagate through the model until the target layer;
[0012] S5: Starting from the target layer, back propagate to calculate the gradient, obtain the weight, and for each filter in the target layer, calculate the corresponding average gradient value;
[0013] S6: Compare the image to be predicted with the visualized image and display the predicted depression level score.
[0014] Furthermore, the method of collecting facial images of patients with depression in S1 is: shooting by mobile phone, camera or Internet search; the specific content of image preprocessing is: adjusting image size, cropping and normalization to meet the input requirements of the model; the specific content of data annotation is: using image annotation tools to label each image with a category label and generate a corresponding annotation file.
[0015] Furthermore, during the construction of the convolutional neural network (F) image classification model in S2, the model structure, including the number of layers and the number of neurons, should be adjusted according to the input image.
[0016] Furthermore, the specific content of the model training in S2 is: inputting the preprocessed image data and the corresponding labels into the model for training; and continuously optimizing the model parameters through multiple iterative training so that the model can classify images more accurately; and during the training process, using the loss function to measure the difference between the model prediction and the true label, and updating the model parameters through the back propagation algorithm.
[0017] Furthermore, the specific content of the model evaluation in S2 is: using a validation set or a test set to evaluate the performance of the model, measured by indicators such as accuracy, recall, and F1 score.
[0018] Furthermore, the specific content of the model optimization in S2 is: adjusting the model structure or hyperparameters, such as learning rate and batch size, according to the evaluation results. Different optimization algorithms, such as gradient descent and Adam, can be used to accelerate the training process and improve model performance.
[0019] Furthermore, the specific content of calculating the average gradient value in S5 is: inputting the facial image (x) of the depression patient into a convolutional neural network (F), which includes multiple convolutional layers, pooling layers and fully connected layers; the parameters of the model are represented by θ;
[0020] The function of the convolutional neural network (F) is to extract the features of the image, transform these features, and finally output the logits scores of patients with different depression levels. Logits refers to the original scores before the softmax function is applied. They represent the unnormalized values of the probability that the input image belongs to the category of patients with different depression levels. The mathematical formula is:
[0021] F(x;θ)=(y 1 ,…,y c ,…y N )
[0022] According to the characteristics of neural networks, the convolutional neural network (F) extracts the highest-level feature map (A1, A2, ..., Ak ) has achieved a balance between the convolutional layer and the fully connected layer, and has good category discrimination, so the highest layer feature map can be used to locate the target of interest, which can be formally described as:
[0023]
[0024] Among them, A k represents the kth channel of the highest level feature map, Represents the weight of the channel, and its calculation formula is
[0025]
[0026] Among them, A k,i,j represents the element of the kth channel at (i, j), and Z represents the normalization factor. Since the channel weight is obtained from the derivative of category c, the channel weight contains relevant information about category c, which is why the explanation image can be used to explain different decision results.
[0027] (III) Beneficial effects
[0028] Compared with the prior art, the present invention provides an interpretable depression rating method, which has the following beneficial effects:
[0029] The present invention, by judging depression levels based on interpretable depression scores and through interpretable images, can achieve: 1. Intuitive display of key features: Heat maps can intuitively show which areas play a key role when the model makes a specific prediction, which makes it easier for doctors and other non-professionals to understand the decision-making basis of the model.
[0030] 2. Reveal hidden patterns: By observing the heat map, subtle patterns and associations in the image can be discovered, which may be difficult for experts to identify with the naked eye, which helps to reveal new clues and features in disease diagnosis.
[0031] 3. Assisting education and training: Heat maps can be used as teaching tools to help novice doctors learn and understand complex diagnostic processes. They can also be used to verify the correctness of expert judgments and provide additional insights. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 is a flow chart of the method of the present invention;
[0033] Figure 2 This is a schematic diagram of the gradient acquisition structure of the present invention. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] Example
[0036] like Figure 1-2 As shown, an explainable depression rating method proposed in one embodiment of the present invention includes the following steps:
[0037] S1: Data preparation, which includes collecting facial images of patients with depression, image preprocessing and data annotation;
[0038] The facial images of patients with depression are collected by taking photos with a mobile phone or camera or searching on the Internet. The specific contents of image preprocessing are: adjusting the image size, cropping and normalization to meet the input requirements of the model. The specific contents of data labeling are: using image labeling tools to label each image with a category label and generate a corresponding labeling file.
[0039] S2: Build, train, evaluate and optimize the convolutional neural network (F) image classification model;
[0040] In the process of building the convolutional neural network (F) image classification model, the model structure, including the number of layers and neurons, should be adjusted according to the input image;
[0041] The specific content of model training is: input the preprocessed image data and the corresponding labels into the model for training; and continuously optimize the model parameters through multiple iterations of training so that the model can classify images more accurately; and during the training process, use the loss function to measure the difference between the model prediction and the true label, and update the model parameters through the back propagation algorithm;
[0042] The specific contents of model evaluation are: using the validation set or test set to evaluate the performance of the model, measured by the indicators of accuracy, recall rate, and F1 score;
[0043] The specific content of model optimization is: adjust the model structure or hyperparameters, such as learning rate and batch size, according to the evaluation results. Different optimization algorithms, such as gradient descent and Adam, can be used to accelerate the training process and improve model performance.
[0044] S3: Select the fully connected layer of the network, obtain the model weights, ensure that the model has been loaded with the trained weights, and select a convolutional layer in the middle of the model as the target layer for generating the heat map;
[0045] S4: Input the image to be explained into the model and forward propagate through the model until the target layer;
[0046] S5: Starting from the target layer, back propagate to calculate the gradient, obtain the weight, and for each filter in the target layer, calculate the corresponding average gradient value;
[0047] The specific content of calculating the average gradient value is as follows: the facial image (x) of a depressed patient is input into a convolutional neural network (F), which contains multiple convolutional layers, pooling layers, and fully connected layers; the parameters of the model are represented by θ;
[0048] The function of the convolutional neural network (F) is to extract the features of the image, transform these features, and finally output the logits scores of patients with different depression levels. Logits refers to the original scores before the softmax function is applied. They represent the unnormalized values of the probability that the input image belongs to the category of patients with different depression levels. The mathematical formula is:
[0049] F(x;θ)=(y 1 ,…,y c ,…y N )
[0050] According to the characteristics of neural networks, the convolutional neural network (F) extracts the highest-level feature map (A1, A2, ..., A k ) has achieved a balance between the convolutional layer and the fully connected layer, and has good category discrimination, so the highest layer feature map can be used to locate the target of interest, which can be formally described as:
[0051]
[0052] Among them, A k represents the kth channel of the highest level feature map, Represents the weight of the channel, and its calculation formula is
[0053]
[0054] Among them, A k,i,j represents the element of the kth channel at (i, j), and Z represents the normalization factor. Since the channel weight is obtained from the derivative of category c, the channel weight contains relevant information about category c, which is why the explanation image can be used to explain different decision results.
[0055] S6: Compare the image to be predicted with the visualized image and display the predicted depression level score.
[0056] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A depression rating method based on interpretability, characterized by: The steps include: S1: Data preparation, which includes collecting facial images of patients with depression, image preprocessing and data annotation; S2: Build, train, evaluate and optimize the convolutional neural network (F) image classification model; S3: Select the fully connected layer of the network, obtain the model weights, ensure that the model has been loaded with the trained weights, and select a convolutional layer in the middle of the model as the target layer for generating the heat map; S4: Input the image to be explained into the model and forward propagate through the model until the target layer; S5: Starting from the target layer, back propagate to calculate the gradient, obtain the weight, and for each filter in the target layer, calculate the corresponding average gradient value; S6: Compare the image to be predicted with the visualized image and display the predicted depression level score.
2. The explainable depression rating method according to claim 1, characterized in that: The method of collecting facial images of patients with depression in S1 is: shooting by mobile phone, camera or Internet search; the specific content of image preprocessing is: adjusting image size, cropping and normalization to meet the input requirements of the model; the specific content of data annotation is: using image annotation tools to label each image with a category label and generate a corresponding annotation file.
3. The explainable depression rating method according to claim 1, characterized in that: In the process of constructing the convolutional neural network (F) image classification model in S2, the model structure, including the number of layers and the number of neurons, should be adjusted according to the input image.
4. The explainable depression rating method according to claim 1, characterized in that: The specific content of the model training in S2 is: inputting the preprocessed image data and the corresponding labels into the model for training; and continuously optimizing the model parameters through multiple iterations of training so that the model can classify images more accurately; and during the training process, using the loss function to measure the difference between the model prediction and the true label, and updating the model parameters through the back propagation algorithm.
5. The explainable depression rating method according to claim 1, characterized in that: The specific content of the model evaluation in S2 is: using a validation set or a test set to evaluate the performance of the model, measured by indicators such as accuracy, recall, and F1 score.
6. The explainable depression rating method according to claim 1, characterized in that: The specific content of the model optimization in S2 is: adjusting the model structure or hyperparameters, such as learning rate and batch size, according to the evaluation results. Different optimization algorithms, such as gradient descent and Adam, can be used to accelerate the training process and improve model performance.
7. The explainable depression rating method according to claim 1, characterized in that: The specific content of calculating the average gradient value in S5 is: inputting the facial image (x) of the depression patient into a convolutional neural network (F), which includes multiple convolutional layers, pooling layers and fully connected layers; the parameters of the model are represented by θ; The function of the convolutional neural network (F) is to extract the features of the image, transform these features, and finally output the logits scores of patients with different depression levels. Logits refers to the original scores before the softmax function is applied. They represent the unnormalized values of the probability that the input image belongs to the category of patients with different depression levels. The mathematical formula is: F(x;θ)=(y 1 ,…,and c ,…and N ) According to the characteristics of neural networks, the convolutional neural network (F) extracts the highest-level feature map (A1, A2, ..., A k ) has achieved a balance between the convolutional layer and the fully connected layer, and has good category discrimination, so the highest layer feature map can be used to locate the target of interest, which can be formally described as: Among them, A k represents the kth channel of the highest level feature map, Represents the weight of the channel, and its calculation formula is Among them, A k,i,j represents the element of the kth channel at (i, j), and Z represents the normalization factor. Since the channel weight is obtained from the derivative of category c, the channel weight contains relevant information about category c, which is why the explanation image can be used to explain different decision results.