Disease image recognition system based on Shapley value guidance

Abstract

The invention provides a disease image recognition system based on Shapley value guidance, and relates to the field of medical image processing. The method comprises the following steps: acquiring a first image through an image acquisition module, carrying out data preprocessing, inputting a second image into an established diagnosis model establishment module, carrying out classified diagnosis, feature extraction and RA classification on an X-ray film image by adopting a trained diagnosis model, carrying out Shapley value analysis and visualization processing on an RA classification result, and obtaining a diagnosis result of the X-ray film; and obtaining an image recognition result based on a result of the Shapley value analysis module in combination with the current clinical data. According to the method, image interference is reduced, data features are effectively extracted, the image recognition rate and accuracy are improved, and the transparency of the model is enhanced, so that the misdiagnosis rate and missed diagnosis rate are reduced, the accuracy and reliability of rheumatoid arthritis diagnosis are improved, the user experience is improved, and the working efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular, to a disease image recognition system guided by Shapley value. Background Art

[0002] Currently, with the rapid development of computer vision and deep learning technologies, convolutional neural networks (CNNs), as a powerful image analysis tool, have shown great potential in medical image diagnosis; in particular, the VGG model, with its deep convolutional structure and powerful feature extraction ability, has high application value in the diagnosis of patient diseases. For example: Rheumatoid Arthritis (RA) is an autoimmune disease with synovitis as the pathological basis, which can lead to joint deformities. RA is mainly manifested by pain, swelling, and morning stiffness in small joints with systemic symmetry, among which the proximal interphalangeal joints, metacarpophalangeal joints, wrist joints, and knee joints are most easily affected. The synovial inflammation, cartilage, and bone destruction of RA can be evaluated by various imaging techniques, but compared with various devices and examinations such as ultrasound, magnetic resonance imaging (MRI), and conventional X-ray photography (CR), X-ray is still an essential imaging device for RA.

[0003] In the prior art, the accuracy of pathological feature recognition of disease images depends on doctors' clinical experience, patients' symptom descriptions, medical images, and laboratory test results, etc., and there are the following problems: 1. It is necessary to comprehensively judge multiple pieces of information, which is time-consuming and cumbersome, affected by personal experience and knowledge level, and there may be misidentification of images due to incomplete information integration or misjudgment, thus increasing the risk of misdiagnosis and missed diagnosis and endangering personal safety; 2. The experience of relevant personnel is difficult to quantify and systematize, traditional methods are usually less efficient, and it is difficult to process a large number of complex images in a short time; 3. There is a lack of effective tools or methods to help relevant personnel extract useful images from a large amount of clinical data, and there is a lack of comparative verification. Therefore, it is particularly important to find a solution to improve the efficiency of disease image recognition. Summary of the Invention

[0004] The main object of the present invention is to provide a disease image recognition system guided by Shapley value, so as to solve the technical problems of long time consumption, insufficient judgment accuracy, and difficulty in general applicability in the process of disease graph recognition in the prior art.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: A disease image recognition system guided by Shapley value, comprising the following modules: An image acquisition module, configured to acquire a first image of the diseased part; An image preprocessing module, which is used to preprocess the first image collected by the image acquisition module to obtain a second image; A diagnostic model building module, based on the VGG16 network structure, uses the VGG feature extraction and RA classification module to build a diagnostic model, which is used to input the second image, extract the pathological features of the second image for deep learning and perform RA classification; A Shapley value analysis module, which obtains the RA classification result of the diagnostic model building module and performs Shapley value analysis and visualization processing; A verification module, which obtains the image recognition result based on the result of the Shapley value analysis module combined with the current clinical data.

[0006] In the preferred solution, the first image of the diseased part is collected. For the pathological features of rheumatoid arthritis, according to the principle of left-right symmetry, X-ray films of the corresponding joints on both sides are obtained. Specifically: For the proximal interphalangeal joints, X-ray films are taken of the corresponding proximal interphalangeal joints of the left and right hands respectively; For the metacarpophalangeal joints, X-ray films are collected for each metacarpophalangeal joint with left-right symmetry; For the wrist and knee joints, anteroposterior and lateral X-ray films of the left and right joints are taken.

[0007] In the preferred solution, the image acquisition module collects images of the diseased part by using X-ray scanning. The voltage is set to 60 kV, the exposure dose is 3 ms, the film focal length is 100 cm, and the images are taken at two angles of anteroposterior and lateral; The pathological features include: joint structure-related features, soft tissue-related features, and symmetry features, where: The joint structure-related features include joint space change, joint surface erosion, and bone hyperplasia; The soft tissue-related features include synovial thickening and soft tissue swelling, The symmetry features include the symmetry of joint involvement, which is identified by comparing the image features of the corresponding joints on both sides.

[0008] In the preferred solution, the image preprocessing module preprocesses the first image, including: First, perform picture scaling, central cropping, vectorization, use the symn8 wavelet transform, and global normalization; Then, extract the shape features related to the pathological features to obtain valid data: perform a similarity calculation based on deep learning, compare the shape similarities of the two left-right symmetric images extracted, and when the waveform similarity between the two is within the preset shape similarity threshold range, it is valid data; Finally, the valid data is used for end-to-end second image feature representation through the VGG module, and classification is performed using a fully connected layer.

[0009] In a preferred solution, before the image preprocessing module preprocesses the first image, it further includes: according to the data type of the first image, performing cropping and brightening for the parts with excessive unfilled areas, abnormal angles, and blackened picture colors in the first image. Specifically: S201: Resize all images to (256, 256) pixels, and crop a central area of (224, 224) pixels from them; S202: Subsequently, convert the images into tensors of PyTorch and normalize the pixel values to [0, 1]; S203: And perform standardization for each channel using the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]; S204: Convert the picture data into a unified preset format.

[0010] In a preferred solution, it further includes: A1: Adopt the method of data augmentation to randomly change the brightness, contrast, and rotation angle of the pictures. Specifically: The brightness adjustment formula is: ; In the formula, α is the contrast adjustment coefficient, and β is the brightness adjustment value; The rotation angle adjustment formula in data augmentation is: ; In the formula, (x, y) are the pixel coordinates of the original image, (x′, y′) are the pixel coordinates after rotation, (xc, yc) is the rotation center point (usually the center of the image), and θ is the rotation angle; A2: Then adopt Gaussian filtering to remove Gaussian noise in the images, and adopt histogram equalization to increase the contrast of the pictures, where: The two-dimensional Gaussian function formula is: ; In the formula, G(x, y) is the value of the Gaussian function at the point (x, y), σ is the standard deviation, is the normalization coefficient; The Gaussian filtering formula is: ; In the formula, I(x, y) is the pixel value of the original image at the point (x, y)(x, y), I′(x, y) is the pixel value of the convolved image at the point (x, y), G(i, j) is the value of the Gaussian kernel at the position (i, j), and k is the radius of the Gaussian kernel; The histogram equalization formula is as follows: ; In the formula, rk is the gray level of the original image (k = 0, 1, 2, …, L−1); sk is the gray level after equalization; L is the total number of gray levels; nj is the number of pixels with gray level rj; N is the total number of pixels in the image; T(rk) is the transformation function that maps the original gray level rk to the new gray level sk.

[0011] In the preferred solution, the diagnostic model building module is specifically as follows: Based on the VGG16 network structure, 5 convolutional blocks are adopted in the convolutional layer, including 8 convolutional layers. The size of each convolutional kernel is mainly 3×3, the stride is 1, the padding is 1, the activation function is ReLU, and the stacking times of the convolutional blocks are adjusted; The pooling layer adopts max pooling, the size of the pooling kernel is 2×2, and the stride is 2; The number of output nodes of the last fully connected layer is set to 2; The Softmax function is used as the activation function for classification.

[0012] In the preferred solution, after the diagnostic model building module is built, model training is carried out to obtain the optimal diagnostic model, specifically as follows: The model adopts the Sequential structure, including 5 convolutional blocks and 3 fully connected layers. Each convolutional block consists of a convolutional layer, ReLU activation, and a max pooling layer. The number of channels gradually increases from 3 to 512. The fully connected layers include an input layer with 25088 dimensions, two hidden layers with 512 dimensions. After each hidden layer, ReLU activation and Dropout(0.5) are connected, and the final output is 2 classes; The training parameters include a learning rate of 0.05, a batch size of 32, and 100 training epochs, and the SGD optimizer is selected.

[0013] In the preferred solution, the Shapley value analysis module is specifically as follows: S401: Use the KernelSHAP method in the Shap library to calculate the Shapley value. The formula is as follows: GradientExplainer uses the method of expected gradient, and the calculation formula is as follows: ; In the formula, is the SHAP value of feature i, is the input sample to be explained, is the reference input sample sampled from the background dataset; is the probability distribution of the background data; is the gradient of the model f(x) with respect to the input x; is the increment of the input feature i; S402: The visualization process includes using the matplotlib library tool to overlay the Shapley values on the original hand image in the form of a heatmap: First, convert the image to the RGB format, then perform color mapping on each pixel of the image according to the magnitude of the Shapley values, set an appropriate color mapping table, and display the areas of key concern. The color mapping formula for the heatmap is: ; In the formula, v is the Shapley value and c is the color mapping function.

[0014] In a preferred solution, the verification module is a Web-app application system built using Python and the Streamlit framework, including a prediction result acquisition module, an operation page, an example image display page, a data storage and SHAP analysis page.

[0015] The present invention provides a disease image recognition system guided by Shapley values. The first image is collected by an image acquisition module and undergoes data preprocessing. Then, the second image is input into the built diagnostic model building module for feature extraction and RA classification. Next, the RA classification result is subjected to Shapley value analysis and visualization processing. Finally, in combination with current clinical data, a verification mechanism is provided to obtain a prediction result. Through standardized image acquisition, comprehensive image preprocessing, and optimized diagnostic model training, the pathological features of rheumatoid arthritis can be more accurately extracted. The Shapley value analysis module can explain the decision-making process of the diagnostic model in detail, improving the interpretability and credibility of the model. At the same time, the Web-app application system enhances the user experience, increases the credibility of the system, effectively extracts data features, improves the recognition rate and accuracy of the image, thereby helping to reduce the misdiagnosis rate and missed diagnosis rate, improving the accuracy and reliability of the diagnosis of rheumatoid arthritis, enhancing the user experience, and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below with reference to the drawings and embodiments: Figure 1 is the structural diagram of the prediction system of the present invention; Figure 2 is the schematic diagram of the data preprocessing process of the present invention; Figure 3 is the processing of the image by data augmentation in the present invention; Figure 4 is the performance display diagram of the VGG model of the present invention on the training set; Figure 5It is a performance display diagram of the VGG model of the present invention on the validation set; Figure 6 It is a performance display diagram of the VGG model of the present invention on another validation set; Figure 7 It is the VGG model of the present invention on Figure 6 Performance display diagram on the corresponding test set; Figure 8 It is a heat map display diagram of the abnormal parts of the hand X-ray of RA patients processed by Shapley of the present invention; Figure 9 It is a webapp application diagram in the embodiment of the present invention; Figure 10 It is a system structure diagram of model training in the embodiment of the present invention. Detailed implementation manners

[0017] Embodiment 1 This embodiment is described with RA pictures. In actual applications, it is applicable to other image processing situations.

[0018] Such as Figures 1-10 As shown, a disease image recognition system guided by Shapley value includes the following modules: An image acquisition module for acquiring a first image of the diseased part.

[0019] An image preprocessing module for preprocessing the first image acquired by the image acquisition module to obtain a second image.

[0020] A diagnostic model building module, based on the VGG16 network structure, builds a diagnostic model using VGG feature extraction and RA classification modules, for inputting the second image, extracting the pathological features of deep learning of the second image and performing RA classification.

[0021] A Shapley value analysis module for obtaining the RA classification result of the diagnostic model building module and performing Shapley value analysis and visualization processing.

[0022] A verification module for obtaining an image recognition result based on the result of the Shapley value analysis module combined with the current clinical data.

[0023] Such as Figure 1As shown in the figure, in this embodiment, the first image is collected by the image acquisition module and undergoes data preprocessing. Then, the second image is input into the constructed diagnosis model building module for feature extraction and RA classification. Next, the RA classification results are subjected to Shapley value analysis and visualization processing. Finally, combined with the current clinical data, a verification mechanism is provided to obtain the prediction results. Through standardized image acquisition, comprehensive image preprocessing, and optimized diagnosis model training, the pathological features of rheumatoid arthritis can be extracted more accurately, which not only improves the efficiency of medical image analysis but also enhances the transparency of the model. The Shapley value analysis module can explain the decision-making process of the diagnosis model in detail, improving the interpretability and credibility of the model. At the same time, the Web-app application system enhances the user experience. Based on the results of the Shapley value analysis module and combined with the current clinical data, the image recognition results are obtained; it reduces picture interference, effectively extracts data features, improves the recognition rate and accuracy of the images, thereby helping to reduce the misdiagnosis rate and missed diagnosis rate, improving the accuracy and reliability of the diagnosis of rheumatoid arthritis, enhancing the user experience, and improving work efficiency.

[0024] Furthermore, in this embodiment, data mining technology is used to mine the diagnostic rules of RA, providing indicators and rules for potential diagnoses. By combining clinical data and the diagnostic results of deep learning, more reliable medical image analysis results are provided, which helps doctors make more accurate decisions in actual work and enhances the practical application value of the system.

[0025] In this embodiment, the pathological features focus on the pixel values in areas such as joint spaces, deformities, and joint ankylosis.

[0026] In the preferred solution, the first image of the diseased part is collected. For the pathological features of rheumatoid arthritis, according to the principle of left-right symmetry, X-ray films of the corresponding joints on both sides are obtained. Specifically: For the proximal interphalangeal joints, X-ray films are taken of the corresponding proximal interphalangeal joints of the left and right hands respectively.

[0027] For the metacarpophalangeal joints, X-ray films are collected for each metacarpophalangeal joint that is left-right symmetric.

[0028] For the wrist and knee joints, anteroposterior and lateral X-ray films of the left and right joints are taken.

[0029] In this embodiment, by collecting X-ray films of the proximal interphalangeal joints, metacarpophalangeal joints, wrist joints, and knee joints according to the left-right symmetry principle, the image data of the joints most easily affected by rheumatoid arthritis can be accurately obtained. It comprehensively covers the key joint parts, improves the integrity and accuracy of the data, enhances the accuracy and reliability of the model for diagnosing rheumatoid arthritis, and reduces the probability of misdiagnosis and missed diagnosis.

[0030] In the preferred solution, the image acquisition module acquires images of the diseased part by X-ray scanning, with the voltage set at 60 kV, the exposure dose at 3 ms, the film focal length at 100 cm, and the images are taken from two angles, the anterior-posterior and lateral views.

[0031] The pathological features include: joint structure-related features, soft tissue-related features, and symmetry features, where: The joint structure-related features include joint space changes, joint surface erosion, and bone hyperplasia; The soft tissue-related features include synovial thickening and soft tissue swelling, The symmetry features include the symmetry of joint involvement, which is identified by comparing the image features of the corresponding joints on the left and right sides.

[0032] In this embodiment, the acquisition of hand images is taken as an example for illustration.

[0033] According to the actual clinical needs, a suitable medical imaging device is selected to acquire hand images. During the acquisition process, the operation is carried out strictly in accordance with the device operation procedures to ensure the quality and consistency of the images. In this embodiment, the hand X-ray photographs are obtained by professional X-ray technicians using the Wandong VX3733 X-ray scanner and saved in the jpg format. The voltage is set at 60 kV, the exposure dose is 3 ms, and the film focal length is 100 cm, ensuring that the pictures are taken from two angles, the anterior-posterior and lateral views.

[0034] The precise setting of the parameters can be adaptively changed to ensure that X-ray images with high clarity and appropriate contrast are obtained. The details of the joint structure and soft tissues can be clearly shown, providing a good image basis for subsequent analysis of pathological features. For example, when observing subtle features such as joint surface erosion and synovial thickening, the lesion features in high-quality images are more easily identified, thus improving the accuracy of the data.

[0035] Regarding the key pathological feature of the symmetry of joint involvement in rheumatoid arthritis, the image acquisition of the bilateral corresponding joints combined with the anterior-posterior and lateral views enables a detailed comparison of the features such as the joint space, joint surface condition, and soft tissue condition of the corresponding joints on the left and right sides from multiple angles. Through comprehensive comparison, the subtle asymmetrical differences are highlighted, thereby improving the diagnostic sensitivity for early lesions.

[0036] In the preferred solution, the image preprocessing module preprocesses the first image, including: First, perform image scaling, central cropping, vectorization, use the symn8 wavelet transform, and global normalization.

[0037] Then, extract the shape features related to pathological features to obtain valid data: perform similarity calculation based on deep learning, compare the shape similarity of the two symmetric left and right images extracted, and when the waveform similarity between the two is within the preset shape similarity threshold range, it is valid data.

[0038] Finally, the valid data is subjected to end-to-end second image feature representation through the VGG module, and classification is performed using a fully connected layer.

[0039] As Figure 2 shown, in this embodiment, through data preprocessing and screening of valid data, the data accuracy and reliability are improved.

[0040] Furthermore, the shape similarity comparison can be achieved through the following steps: Before performing the shape similarity calculation, first apply an edge detection algorithm, such as the Canny algorithm, to the two symmetric left and right images respectively to accurately extract the contours of target regions such as joints in the images.

[0041] Convert the extracted contour curves into complex sequences and transform them to the frequency domain through discrete Fourier transform.

[0042] Finally, use the Fourier description operators of the left and right images extracted to measure the shape similarity by calculating the Euclidean distance between the two. The formula is: ; In the formula, and are the coefficients of the corresponding Fourier description operators of the left and right images, and n is the number of Fourier coefficients selected. The smaller the distance value, the higher the shape similarity of the left and right images.

[0043] Through the shape similarity comparison, image pairs with overly large shape differences caused by various factors (such as patient body position changes, slight equipment deviations, etc.) during the acquisition process are screened out, improving the data quality.

[0044] In the preferred solution, before the image preprocessing module preprocesses the first image, it further includes: according to the data type of the first image, perform cropping and brightening for the parts with too much unfilled area, abnormal angles, and blackened picture colors in the first image to improve the picture quality. Specifically: S201: Resize all images to (256, 256) pixels and crop a central region of (224, 224) pixels from them.

[0045] S202: Subsequently, convert the image into a Tensor of PyTorch and normalize the pixel values to [0, 1].

[0046] S203: Normalize each channel using the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225].

[0047] S204: Convert the image data into a unified preset format.

[0048] As Figure 1 shown, for the captured image data type, after calculating the global mean and standard deviation to obtain mean and std, normalize the image. Then perform grayscale processing, size adjustment, and scaling.

[0049] In the preferred solution, it further includes: A1: Adopt data augmentation to randomly change the brightness, contrast, and rotation angle of the picture. Specifically: The brightness adjustment formula is: ; In the formula, α is the contrast adjustment coefficient, and β is the brightness adjustment value.

[0050] The rotation angle adjustment formula in data augmentation is: ; In the formula, (x, y) are the pixel coordinates of the original image, (x′, y′) are the pixel coordinates after rotation, (xc, yc) is the rotation center point (usually the center of the image), and θ is the rotation angle.

[0051] A2: Then use Gaussian filtering to remove Gaussian noise in the image and use histogram equalization to increase the contrast of the picture. Among them: The two-dimensional Gaussian function formula is: ; In the formula, G(x, y) is the value of the Gaussian function at the point (x, y), σ is the standard deviation, is the normalization coefficient.

[0052] The Gaussian filtering formula is: ; In the formula, I(x, y) is the pixel value of the original image at the point (x, y)(x, y), I′(x, y) is the pixel value of the convolved image at the point (x, y), G(i, j) is the value of the Gaussian kernel at the position (i, j), and k is the radius of the Gaussian kernel.

[0053] The histogram equalization formula is: ; Wherein, rk is the gray level of the original image (k = 0, 1, 2, …, L−1); sk is the gray level after equalization; L is the total number of gray levels; nj is the number of pixels with gray level rj; N is the total number of pixels in the image; T(rk) is the transformation function that maps the original gray level rk to the new gray level sk.

[0054] As Figure 3 shown, in this embodiment, data augmentation is adopted to randomly change the brightness, contrast, and rotation angle of the picture to increase the generalization ability of the model. Gaussian filtering is used to remove Gaussian noise in the image, and histogram equalization is used to increase the contrast of the picture to improve the recognition ability of the image.

[0055] In the preferred solution, the diagnostic model building module is specifically: Based on the VGG16 network structure, 5 convolutional blocks are adopted in the convolutional layer, including 8 convolutional layers. The size of each convolutional kernel is mainly 3×3, the stride is 1, the activation function is ReLU, and the stacking times of the convolutional blocks are adjusted.

[0056] The pooling layer uses max pooling, the size of the pooling kernel is 2×2, and the stride is 2.

[0057] The number of output nodes of the last fully connected layer is set to 2.

[0058] The Softmax function is used as the activation function for classification.

[0059] In this embodiment, taking the VGG16 network as an example, the network structure includes 13 convolutional layers, 5 pooling layers, and 3 fully connected layers.

[0060] Table 1 Comparison of this solution with other VGG models

[0061] In the present invention, the number of convolutional layers is changed to 5 and the stacking times of the convolutional layers are appropriately adjusted, effectively reducing the calculation cost and the number of parameters, while also ensuring the accuracy of the model; the number of output nodes of the last fully connected layer is set to 2 (representing RA disease and non-disease respectively), and the Softmax function is used as the activation function for classification. The learning rate is set to 0.05, the number of iterations is 100 to help the model better fit the training data, and the SGD optimizer is selected as the optimizer. Its architecture is as Figure 10 shown.

[0062] In the preferred solution, after the diagnostic model building module is built, model training is carried out to obtain the optimal diagnostic model, specifically: The model adopts the Sequential structure, including 5 convolutional blocks and 3 fully connected layers. Each convolutional block consists of a convolutional layer, ReLU activation, and a max pooling layer. The number of channels gradually increases from 3 to 512. The fully connected layers include an input layer with 25,088 dimensions, two hidden layers with 512 dimensions each. After each hidden layer, ReLU activation and Dropout(0.5) are connected. The final output is 2 classes.

[0063] The training parameters include a learning rate of 0.05, a batch size of 32, and 100 training epochs, and the SGD optimizer is selected.

[0064] In the preferred solution, the Shapley value analysis module is specifically as follows: S401: Use the GradientExplainer method in the Shap library to calculate the Shapley value. The formula is: GradientExplainer adopts the method of expected gradient, and the calculation formula is as follows: ; In the formula, is the SHAP value of feature i; is the input sample to be explained (target sample); is the reference input sample (baseline) sampled from the background dataset; is the probability distribution of the background data; is the gradient (partial derivative) of the model with respect to the input; is the increment of input feature i.

[0065] S402: The visualization process includes using the matplotlib library tool to overlay the Shapley value on the original hand image in the form of a heatmap: First, convert the image to RGB format, then perform color mapping on each pixel of the image according to the size of the Shapley value, set an appropriate color mapping table, and display the key areas of concern. The color mapping formula for the heatmap is: ; In the formula, v is the Shapley value, and c is the color mapping function.

[0066] In the preferred solution, the verification module builds a Web-app application system using Python and the Streamlit framework, including obtaining prediction results, an operation page, an example image display page, data storage, and a SHAP analysis page. The prediction results include "RA" and "NORMAL", that is, the judgment results of being ill and normal, and can also include the type and location identification of the disease.

[0067] Such as Figure 9As shown in the figure, in this embodiment, a Web-app application is built using Python and the Streamlit framework, which is divided into a login page, an example image display page, and a result prediction and SHAP analysis page, realizing a complete process from data processing, model building to actual application.

[0068] The test of the image dataset based on the 100-epoch VGG model in this embodiment is as follows. In Table 2-5, "0" represents the X-ray film of a normal human hand, and "1" represents the X-ray film of a RA patient's hand: As Figure 4 shown in Table 2, based on the cases collected in the early stage, 945 hand joint images were predicted and evaluated through the pre-trained VGG model. The AUC value of the model in the training set was 0.99, and the overall prediction accuracy (accuracy) was calculated to be 0.95.

[0069] As Figure 5 shown in Table 3, in order to verify the performance of the model, 95 images that were not determined in the dataset were selected as the validation set for testing. The AUC value of the model in the validation set was 0.81, and the accuracy was 0.78, indicating that the model can also maintain relatively stable prediction ability on unknown data.

[0070] Table 2 Classification report of the 100-epoch VGG model on 945 training images

[0071] Table 3 Classification report on the dataset of 95 training images

[0072] As Figure 6 shown in Table 4, when deeply testing 480 images from another source, the previous validation dataset (39 images) was incorporated into the training dataset. Using the previous pre-trained weights, after dividing the images into normal and RA, a training dataset and a test dataset were divided. Finally, the model was trained on 519 images, and its accuracy was 0.97.

[0073] As Figure 7 shown in Table 5, a 100% accurate recognition rate was achieved on 41 test images, verifying the effectiveness of the strategy in this embodiment.

[0074] Table 4 Classification report on the 519 training datasets

[0075] Table 5 Classification report on the 41 test datasets

[0076] AsFigures 8-9 As shown, it is the analysis of model interpretability: perform SHAP visualization analysis on RA patients. Overall, in the SHAP analysis, the red parts in the image highlight the possible lesion areas of the hand joints, which have an important impact on the model's recognition of "RA". The blue parts emphasize the areas of normal bone structure, which have a strong supporting effect on the "NORMAL" classification. Through the visualization of SHAP values, we can more intuitively understand how the model makes "RA" and "NORMAL" judgments based on different image regions. According to observations, the key areas of concern for the model are highly correlated with clinical concerns.

[0077] Building a Web app based on the Streamlit framework: This study carried out an X-ray classification task based on the VGG model and used Python and the Streamlit framework to build a Web app. This app is divided into a login page, a sample image display page, and a result prediction and SHAP analysis page. As Figure 9 shown, it realizes a complete process from data processing, model building to practical application. At the same time, interpretability analysis was carried out to deeply explore the model decision-making mechanism, and a comprehensive framework covering data processing, model construction, application deployment, and analysis and evaluation was built, providing a reference example for subsequent related research and applications.

[0078] The above embodiments are only the preferred technical solutions of the present invention and should not be regarded as limitations on the present invention. The protection scope of the present invention should be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A disease image recognition system based on Shapley value guidance, characterized in that: Includes the following modules: An image acquisition module, used for acquiring a first image of a diseased part; An image preprocessing module, used for preprocessing the first image acquired by the image acquisition module to obtain a second image; The diagnostic model building module is based on the VGG16 network structure and uses the VGG feature extraction and RA classification modules to build a diagnostic model for inputting the second image, extracting the pathological features of the second image through deep learning, and performing RA classification; The Shapley value analysis module obtains the RA classification results of the diagnostic model building module and performs Shapley value analysis and visualization processing; The verification module obtains the image recognition result based on the results of the Shapley value analysis module and the current clinical data.

2. The disease image recognition system based on Shapley value guidance according to claim 1, characterized in that: The first image of the diseased part is acquired, based on the pathological characteristics of rheumatoid arthritis, and the X-ray films of the corresponding joints on both sides are obtained according to the principle of bilateral symmetry, specifically: For the proximal interphalangeal joints, radiographs were taken of the corresponding proximal interphalangeal joints of the left and right hands; For the metacarpophalangeal joints, X-rays were collected for each metacarpophalangeal joint symmetrically on both sides; For the wrist and knee, take anteroposterior and lateral x-rays of the right and left joints.

3. The disease image recognition system based on Shapley value guidance according to claim 2, characterized in that: The image acquisition module acquires images of the diseased part by X-ray scanning, with the voltage set to 60 kV, the exposure dose to 3 ms, the film focal length to 100 cm, and the images are taken at two angles: frontal and lateral. The pathological features include: joint structure-related features, soft tissue-related features and symmetry features, among which: Characteristics related to joint structure include changes in joint space, erosion of articular surfaces, and bone hyperplasia; Soft tissue-related features include synovial thickening and soft tissue swelling. Symmetry features include the symmetry of joint involvement, which is identified by comparing the image features of corresponding joints on the left and right sides.

4. The disease image recognition system based on Shapley value guidance according to claim 1, characterized in that: The image preprocessing module preprocesses the first image, including: First, the image is scaled, center-cropped, vectorized, symn8 wavelet transformed, and globally normalized; Then, the shape features related to the pathological features are extracted to obtain valid data: a similarity calculation based on deep learning is performed to compare the shape similarity of the two extracted left-right symmetrical images. When the waveform similarity of the two images is within the preset shape similarity threshold range, it is valid data; Finally, the valid data is passed through the VGG module for end-to-end second image feature representation and classified using a fully connected layer.

5. The disease image recognition system based on Shapley value guidance according to claim 4, characterized in that: Before the image preprocessing module preprocesses the first image, the method further includes: according to the data type of the first image, cropping and brightening the first image for excessive unfilled parts, abnormal angles, and darkened colors, specifically: S201: all images are resized to (256, 256) pixels, and a central area of ​​(224, 224) pixels is cropped out of them; S202: The image is then converted into a PyTorch Tensor and the pixel values ​​are normalized to [0, 1]; S203: Standardize each channel using mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]; S204: Convert the image data into a unified preset format.

6. The disease image recognition system based on Shapley value guidance according to claim 5, characterized in that: Also includes: A1: Use data augmentation to randomly change the brightness, contrast, and rotation angle of the image. Specifically: The brightness adjustment formula is: ； In the formula, α is the contrast adjustment coefficient, and β is the brightness adjustment value; The rotation angle adjustment formula in data enhancement is: ； Where (x, y) is the pixel coordinate of the original image, (x′, y′) is the pixel coordinate after rotation, (xc, yc) is the rotation center point (usually the center of the image), and θ is the rotation angle; A2: Use Gaussian filtering to remove Gaussian noise in the image, and use histogram equalization to increase the contrast of the image, where: The two-dimensional Gaussian function formula is: ； In the formula, G(x,y) is the Gaussian function value at the point (x,y), σ is the standard deviation, is the normalization coefficient; The Gaussian filter formula is: ； Where I(x,y) is the pixel value of the original image at point (x,y)(x,y), I′(x,y) is the pixel value of the convolved image at point (x,y), G(i,j) is the value of the Gaussian kernel at position (i,j), and k is the radius of the Gaussian kernel; The histogram equalization formula is: ； Where rk is the grayscale of the original image (k = 0, 1, 2, …, L−1); sk is the grayscale after equalization; L is the total number of grayscales; nj is the number of pixels of grayscale rj; N is the total number of pixels in the image; T(rk) is the transformation function that maps the original grayscale rk to the new grayscale sk.

7. The disease image recognition system based on Shapley value guidance according to claim 1, characterized in that: The diagnostic model building module is specifically: Based on the VGG16 network structure, 5 convolution blocks are used in the convolution layer, including 8 convolution layers. The size of each convolution kernel is mainly 3×3, the step size is 1, the padding is 1, the activation function is ReLU, and the number of stacking times of convolution blocks is adjusted; The pooling layer uses maximum pooling, the pooling kernel size is 2×2, and the stride is 2; The number of output nodes of the last fully connected layer is set to 2; The Softmax function is used as the activation function for classification.

8. The disease image recognition system based on Shapley value guidance according to claim 7, characterized in that: The diagnostic model building module performs model training after building to obtain the optimal diagnostic model, specifically: The model adopts a Sequential structure, including 5 convolutional blocks and 3 fully connected layers. Each convolutional block consists of a convolutional layer, a ReLU activation and a maximum pooling layer. The number of channels gradually increases from 3 to 512. The fully connected layer includes a 25088-dimensional input layer and two 512-dimensional hidden layers. Each hidden layer is followed by ReLU activation and Dropout (0.5). The final output is 2 categories. The training parameters include learning rate 0.05, batch size 32, and number of training rounds 100, and the SGD optimizer is selected.

9. The disease image recognition system based on Shapley value guidance according to claim 1, characterized in that: The Shapley value analysis module is specifically: S401: Use the Kernel SHAP method in the Shap library to calculate the Shapley value. The formula is: GradientExplainer uses the expected gradient method, and the calculation formula is as follows: ； In the formula, is the SHAP value of feature i, is the input sample to be explained, is the reference input sample sampled from the background dataset; is the probability distribution of background data; is the gradient of the model f(x) with respect to the input x; is the increment of input feature i; S402: The visualization process includes using the matplotlib library tool to overlay the Shapley value on the original hand image in the form of a heat map: first, the image is converted into RGB format, and then each pixel of the image is color mapped according to the size of the Shapley value, and a suitable color mapping table is set to display the focus area. The heat map color mapping formula is: ； Where v is the Shapley value and c is the color mapping function.

10. The disease image recognition system based on Shapley value guidance according to claim 1, characterized in that: The verification module uses Python and Streamlit framework to build a Web-app application system, including obtaining prediction results, operation pages, sample image display pages, data storage and SHAP analysis pages.

Images