Computer data mining system based on big data

Abstract

The invention discloses a computer data mining system based on big data. The system comprises a data preprocessing layer which is responsible for collecting, cleaning, integrating and standardizing medical health data; the model training layer is used for training an X-CNN model by using the preprocessed data, and verifying and testing the X-CNN model; the data mining layer is used for mining and analyzing medical health data by applying an X-CNN model, introducing a feature selection method based on a graph theory, learning and extracting most representative features, and generating a feature visualization graph and an importance scoring graph by utilizing a visualization technology; the result explanation layer is used for explaining and explaining the decision of the model, dividing a decision tree into a plurality of layers, representing different stages of the decision of the model by each layer, and explaining the significance and importance of each decision node by gradually displaying the decision process and result of each layer and adding annotations and labels on each layer; when different hierarchies of the decision tree are displayed, local interpretations generated by the LIME are provided at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining, and in particular, to a computer data mining system based on big data. Background Art

[0002] In the field of medical health, with the rapid development of big data technology and the continuous improvement of medical informatization level, medical health data has shown explosive growth. The data contains rich information on disease diagnosis, treatment, and prognosis, which is of great significance for improving the quality and efficiency of medical services.

[0003] In the prior art, doctors and other medical professionals need to be able to understand the logic and basis behind the decisions made by the model. Lack of transparency may lead to trust issues and limit the application of the model in actual clinical decision-making. Therefore, a computer data mining system based on big data is proposed. Summary of the Invention

[0004] The purpose of the present invention is to solve the disadvantages existing in the prior art, where doctors and other medical professionals need to be able to understand the logic and basis behind the decisions made by the model. Lack of transparency may lead to trust issues and limit the application of the model in actual clinical decision-making, and to propose a computer data mining system based on big data.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A computer data mining system based on big data, comprising:

[0007] Data preprocessing layer: responsible for collecting, cleaning, integrating, and standardizing medical health data;

[0008] Model training layer: using the preprocessed data to train the X-CNN model to ensure the accuracy and generalization ability of the model, verifying and testing the X-CNN model to ensure its stability and reliability in actual applications;

[0009] Data mining layer: applying the X-CNN model to mine and analyze medical health data, introducing a feature selection method based on graph theory, learning and extracting the most representative features, improving the robustness and accuracy of the model, and using visualization technology to generate feature visualization graphs and importance scoring graphs to provide data insights for doctors;

[0010] Result Interpretation Layer: Provide an interactive interface. Based on the visualization graph and decision tree graph generated by the Data Mining Layer, interpret and explain the decisions of the model, and allow doctors to adjust the visualization parameters and the depth of interpretation as needed. Divide the decision tree into multiple levels, with each level representing a different stage of the model's decision-making. By gradually showing the decision-making process and results of each level, add annotations and labels at each level to explain the meaning and importance of each decision node. When showing different levels of the decision tree, provide local explanations generated by LIME at the same time.

[0011] The above technical solution further includes:

[0012] Furthermore, the Data Preprocessing Layer uses web crawler technology to automatically capture image data from the medical imaging system, uses an image processing library (such as OpenCV) for image denoising and repair, uses a database management system (such as MySQL, MongoDB) to store and manage image data, and uses image normalization and standardization algorithms for data standardization processing.

[0013] Furthermore, the specific steps for the Model Training Layer to train the X-CNN model using the preprocessed data are as follows:

[0014] Data Preparation: Perform missing value filling, outlier detection and handling, and data standardization;

[0015] Model Initialization: Initialize the parameters of the X-CNN model, including the size, number, and stride of the convolutional kernels, the number of neurons in the fully connected layer, and the activation function;

[0016] Define the Loss Function and Optimizer: The loss function is used to measure the gap between the model's prediction results and the true labels, and the optimizer is responsible for adjusting the model parameters to minimize the loss function;

[0017] Model Training: Input the preprocessed data into the X-CNN model, perform forward propagation to calculate the prediction results, then calculate the gap between the prediction results and the true labels through the loss function. Next, use the optimizer for backpropagation to update the model parameters, and perform multiple iterations until the model converges or reaches the preset number of iterations.

[0018] Furthermore, the Data Mining Layer introduces a graph theory-based feature selection method to learn and extract the most representative features, including the following steps;

[0019] Construct a Feature Graph: Extract a series of features from the medical and health data, and represent the extracted features as a feature graph G=(V, E), where V represents the set of feature nodes, and each node represents a feature; E represents the set of feature edges, and each edge represents the association relationship between features;

[0020] Calculating Feature Importance: Use centrality measures in graph theory to evaluate the importance of features. The centrality measure adopts degree centrality, and the degree centrality of a feature node is defined as the number of neighbor nodes of this node, expressed as DC(v) = ∑ u∈N(v) 1, where DC(v) represents the degree centrality of feature node v, and N(v) represents the set of neighbor nodes of feature node v;

[0021] Selecting the Most Representative Features: After calculating the importance of features, select the top k most representative features according to the importance ranking;

[0022] Optimizing the Feature Subset: Use particle swarm optimization to optimize the initially selected feature subset.

[0023] Furthermore, the specific steps of using particle swarm optimization to optimize the initially selected feature subset are as follows:

[0024] Initializing the Particle Swarm: Initialize a particle swarm, where each particle represents a possible feature subset. The position vector of the particle is expressed as a feature selection vector, that is, a binary vector, where 1 represents selecting this feature and 0 represents not selecting. For example, for a dataset containing 10 features, the position vector of a particle may be [1, 0, 1, 1, 0, 0, 1, 0, 0, 1], indicating that the 1st, 3rd, 4th, 7th, and 10th features are selected. At the same time, each particle also has a velocity vector, which represents the moving direction and speed of the particle in the feature space, and the initial value of the velocity vector is randomly generated;

[0025] Defining the Fitness Function: The fitness function is used to evaluate the performance of each particle (i.e., each feature subset), and the fitness function is the accuracy of the model on the validation set;

[0026] Updating the Particle Position and Velocity: The particle searches for the optimal solution by iteratively updating its position and velocity. The update formulas include the velocity update formula and the position update formula. The velocity update formula is expressed as

[0027]

[0028] where, represents the value of the velocity of particle i in the d-th dimension at the (t + 1)-th iteration; w is the inertia weight, which is used to control the tendency of the particle to maintain its original velocity; c1 and c2 are learning factors, which respectively represent the weights of the particle learning from its own historical optimal position and the global optimal position; r1 and r2 are random numbers, which are used to increase the randomness of the algorithm; represents the historical optimal position of particle i in the d-th dimension; represents the global optimal position; represents the current position of particle i in the d-th dimension;

[0029] The position update formula is expressed as where represents the value of the position of particle i in the d-th dimension at the (t + 1)-th iteration;

[0030] Evaluate the fitness and update the optimal position: In each iteration, evaluate the fitness of each particle, and update the historical optimal position and the global optimal position of the particle. If the fitness of the current particle is better than the fitness of its historical optimal position, then update its historical optimal position; if the fitness of the current global optimal position is surpassed by the fitness of a certain particle, then update the global optimal position;

[0031] Iteratively search until the stopping condition is met: Set a stopping condition, such as reaching the maximum number of iterations or the fitness converging to a certain range. When the stopping condition is met, the algorithm stops iterating and outputs the global optimal position as the optimal feature subset.

[0032] Furthermore, the specific steps for the result interpretation layer to interpret and explain the decisions of the model are as follows:

[0033] Hierarchical division of the decision tree: Abstract the decision logic in the trained X-CNN model into a decision tree, divide the decision tree into multiple levels, each level representing a stage of the model's decision. Starting from the root node, gradually delve into the leaf nodes;

[0034] Show the decision process and results: For each level, show the decision process and results at that level. The doctor clicks or selects a certain level through the interactive interface, and then shows the input features, output categories, and decision paths of all decision nodes at that level;

[0035] Add annotations and labels: At each level, add annotations and labels to explain the meaning and importance of each decision node, including: Explanation of input features: Explain the role and importance of each input feature in the model's decision; Explanation of output categories: Explain the meaning and possible clinical significance represented by each output category; Explanation of decision paths: Explain the decision path from the root node to the current node, and the judgment basis for each decision node;

[0036] Provide LIME local explanations: When showing different levels of the decision tree, provide local explanations generated by LIME (Local Interpretable Model-agnostic Explanations).

[0037] Furthermore, the provision of LIME local explanations includes the following steps:

[0038] Select specific cases: Select one or more specific cases from the medical and health dataset as the objects to be explained;

[0039] Generate a local linear model: Use the LIME tool to generate a local linear model for the selected cases. The local linear model will attempt to approximate the decision boundary of the X-CNN model near the selected cases;

[0040] Extract model coefficients and intercepts: Extract the coefficients (weights) and intercepts from the local linear model. The coefficients represent the degree of influence of each input feature on the model's decision;

[0041] Display local explanations: Present the coefficients and intercepts of the local linear model to the doctor in a visual form. At the same time, display the comparison between the prediction results of the local linear model and the actual prediction results of the X-CNN model to verify the accuracy of the local explanations;

[0042] Explain decision nodes: When presenting different levels of the decision tree, associate the local explanations generated by LIME with each decision node, explain the meaning and importance of each decision node, and how it makes decisions based on the input features;

[0043] Adjust visualization parameters and explanation depth: Allow the doctor to adjust the visualization parameters (such as the width, height, color, etc. of the bar chart) and the explanation depth (such as how many of the most important features to display, whether to display the intercept, etc.) as needed.

[0044] Furthermore, the specific steps for generating the local linear model are as follows:

[0045] Initialize the LIME interpreter: Create an interpreter object using the LIME toolkit. This object will be used to generate the local linear model, specifying the model to be explained (the X-CNN model) and the number of features to be explained;

[0046] Generate perturbed samples: The LIME interpreter makes slight perturbations to the input features of the selected cases to generate a series of perturbed samples, which will be used to observe the decision boundary of the X-CNN model near the selected cases;

[0047] Predict the perturbed samples: Use the X-CNN model to predict the generated perturbed samples to obtain the prediction results for each perturbed sample;

[0048] Construct the local linear model: The LIME interpreter uses the perturbed samples and their prediction results to construct a local linear model. The local linear model will attempt to approximate the decision boundary of the X-CNN model near the selected cases. The construction of the local linear model is based on the following formula:

[0049]

[0050] where: g(z') is a local linear model that takes the perturbed sample z' as input, f is the model we want to explain (the X-CNN model), G is the hypothesis space of the local linear model (a linear regression model in this example), L(f, g, π2) is the local loss function that measures the prediction difference between the model f and the local linear model g on the perturbed sample, Ω(g) is the regularization term used to control the complexity of the local linear model, and π2 is the distribution of the perturbed samples, which determines how to generate perturbed samples near the selected cases.

[0051] The present invention has the following beneficial effects:

[0052] In the present invention, through the hierarchical interpretation method and LIME-assisted interpretation, the complex model decision-making process is transformed into an easy-to-understand hierarchical structure and local interpretation. This not only helps doctors deeply understand the diagnostic logic of the model but also enhances the communication between doctors and patients and improves the patients' trust and acceptance of the diagnostic results. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a system block diagram of a computer data mining system based on big data proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0055] Please refer to Figure 1 As shown, the present invention is a computer data mining system based on big data, including:

[0056] Data preprocessing layer: responsible for collecting, cleaning, integrating, and standardizing medical and health data;

[0057] Model training layer: using the preprocessed data to train the X-CNN model to ensure the accuracy and generalization ability of the model, validating and testing the X-CNN model to ensure its stability and reliability in practical applications;

[0058] Data mining layer: applying the X-CNN model to mine and analyze medical and health data, introducing a graph theory-based feature selection method to learn and extract the most representative features, improving the robustness and accuracy of the model, and using visualization technology to generate feature visualization graphs and importance scoring graphs to provide data insights for doctors;

[0059] Result Interpretation Layer: Provide an interactive interface. Based on the visualization graphs and decision tree graphs generated by the Data Mining Layer, interpret and explain the decisions of the model, and allow doctors to adjust visualization parameters and the depth of interpretation as needed. Divide the decision tree into multiple levels, with each level representing a different stage of the model's decision-making. By gradually showing the decision-making process and results at each level, add annotations and labels at each level to explain the meaning and importance of each decision node. When showing different levels of the decision tree, simultaneously provide local interpretations generated by LIME.

[0060] In one embodiment, the Data Preprocessing Layer uses web crawler technology to automatically scrape image data from a medical imaging system, uses an image processing library (such as OpenCV) for image denoising and restoration, uses a database management system (such as MySQL, MongoDB) to store and manage image data, and uses image normalization and standardization algorithms for data standardization processing.

[0061] Write a web crawler program and set the target URL (i.e., the access address of the radiology imaging system). Configure the request header information of the web crawler program to simulate the access behavior of a normal user. Parse the response data in HTML or JSON format and extract the URLs of lung CT scan images. Download the image data and save it locally or in cloud storage.

[0062] Read the lung CT scan images. Apply denoising algorithms such as Gaussian filtering or median filtering to remove Gaussian noise or salt-and-pepper noise in the images. Use morphological operations (such as dilation, erosion, opening, closing, etc.) to repair problems such as small holes, breaks, or uneven edges in the images. Save the cleaned image data.

[0063] Design the database table structure, including fields such as image ID, patient ID, diagnosis results, etc. Convert the cleaned image data into a suitable storage format (such as JPEG, PNG), and calculate the hash value of the image to ensure uniqueness. Insert the image data and related information into the database table. Establish indexes and query optimizations to improve data retrieval efficiency. Read the image data and obtain the pixel value range of the image.

[0064] According to the pixel value range, apply linear transformation or non-linear transformation to standardize the pixel values to the range of 0 - 1. Save the standardized image data.

[0065] In one embodiment, the specific steps for the Model Training Layer to train the X-CNN model using the preprocessed data are as follows:

[0066] Data Preparation: Perform missing value filling, outlier detection and handling, and data standardization;

[0067] Model Initialization: Initialize the parameters of the X-CNN model, including the size, number, and stride of the convolutional kernels, the number of neurons in the fully connected layer, and the activation function;

[0068] Define the loss function and optimizer: The loss function is used to measure the gap between the model's prediction results and the true labels, and the optimizer is responsible for adjusting the model parameters to minimize the loss function;

[0069] Model Training: Input the preprocessed data into the X-CNN model, perform forward propagation to calculate the prediction results, then calculate the gap between the prediction results and the true labels through the loss function. Next, use the optimizer for backpropagation to update the model parameters, and iterate multiple times until the model converges or reaches the preset number of iterations.

[0070] Data Preprocessing: Perform operations such as denoising and normalization on the lung CT scan images to improve the model's ability to extract image features.

[0071] Model Initialization: Set the convolutional kernel size of the X-CNN model to 3x3, the number to 32, and the stride to 1; the number of neurons in the fully connected layer to 128; and the activation function to ReLU.

[0072] Define the loss function and optimizer: Select the cross-entropy loss function and the Adam optimizer.

[0073] Model Training: Input the preprocessed image data into the X-CNN model, perform forward propagation to calculate the prediction results. Then, calculate the gap between the prediction results and the true labels through the cross-entropy loss function. Next, use the Adam optimizer for backpropagation to update the model parameters. This process was iterated 50 times, and the entire dataset was used for training in each iteration (i.e., batch gradient descent).

[0074] In one embodiment, the data mining layer introduces a feature selection method based on graph theory to learn and extract the most representative features, including the following steps;

[0075] Construct a feature graph: Extract a series of features from the medical and health data, and represent the extracted features as a feature graph G=(V, E), where V represents the set of feature nodes, and each node represents a feature; E represents the set of feature edges, and each edge represents the association relationship between features;

[0076] Calculate feature importance: Use centrality measures in graph theory to evaluate the importance of features. The centrality measure uses degree centrality, and the degree centrality of a feature node is defined as the number of its neighbor nodes, expressed as DC(v)=∑ u∈N(v) 1, where DC(v) represents the degree centrality of feature node v, and N(v) represents the set of neighbor nodes of feature node v;

[0077] Select the most representative features: After calculating the importance of the features, select the top k most representative features according to the importance ranking.

[0078] Optimize the feature subset: Use particle swarm optimization to optimize the initially selected feature subset.

[0079] Suppose there is a medical and health dataset containing 10 features: age, gender, blood pressure, heart rate, blood sugar, cholesterol, BMI, smoking history, family history, and imaging features. Use the feature selection method based on graph theory to select the most representative features from these features.

[0080] Construct a feature graph: I represent these features as nodes in the graph and construct edges according to their association relationships (such as correlation, causality, etc.). For example, age may be related to physiological indicators such as blood pressure and heart rate, so edges can be constructed between these features.

[0081] Calculate the feature importance: Use degree centrality to evaluate the importance of the features. By calculating the degree centrality of each feature, their ranking is obtained. Suppose the ranking result is: imaging features > blood pressure > cholesterol > age >... (other features).

[0082] Select the most representative features: According to the importance ranking, select the top 3 most representative features: imaging features, blood pressure, and cholesterol. These features will be used as the input for the subsequent X-CNN model training.

[0083] Optimize the feature subset: To further improve the quality of the feature subset, use particle swarm optimization to optimize the initially selected feature subset. By iteratively searching and selecting the optimal feature combination, a better feature subset is obtained.

[0084] In one embodiment, the specific steps of using particle swarm optimization to optimize the initially selected feature subset are as follows:

[0085] Initialize the particle swarm: Initialize a particle swarm, where each particle represents a possible feature subset. The position vector of the particle is represented as a feature selection vector, that is, a binary vector, where 1 indicates that the feature is selected and 0 indicates that the feature is not selected. For example, for a dataset containing 10 features, the position vector of a particle may be [1, 0, 1, 1, 0, 0, 1, 0, 0, 1], indicating that the 1st, 3rd, 4th, 7th, and 10th features are selected. At the same time, each particle also has a velocity vector, which represents the moving direction and speed of the particle in the feature space. The initial value of the velocity vector is randomly generated.

[0086] Define the fitness function: The fitness function is used to evaluate the performance of each particle (i.e., each feature subset), and the fitness function is the accuracy of the model on the validation set.

[0087] Update the particle position and velocity: The particle searches for the optimal solution by iteratively updating its position and velocity. The update formulas include the velocity update formula and the position update formula. The velocity update formula is expressed as

[0088]

[0089] where represents the value of the velocity of particle i in the d-th dimension at the (t + 1)-th iteration; w is the inertia weight, which is used to control the tendency of the particle to maintain its original velocity; c1 and c2 are learning factors, which respectively represent the weights of the particle learning from its own historical optimal position and the global optimal position; r1 and r2 are random numbers, which are used to increase the randomness of the algorithm. represents the historical optimal position of particle i in the d-th dimension; represents the global optimal position; represents the current position of particle i in the d-th dimension;

[0090] The position update formula is expressed as where represents the value of the position of particle i in the d-th dimension at the (t + 1)-th iteration;

[0091] Evaluate the fitness and update the optimal position: In each iteration, evaluate the fitness of each particle, and update the historical optimal position and the global optimal position of the particle. If the fitness of the current particle is better than the fitness of its historical optimal position, then update its historical optimal position; if the fitness of the current global optimal position is surpassed by the fitness of a certain particle, then update the global optimal position.

[0092] Iteratively search until the stop condition is met: Set a stop condition, such as reaching the maximum number of iterations or the fitness converging to a certain range. When the stop condition is met, the algorithm stops iterating and outputs the global optimal position as the optimal feature subset.

[0093] In one embodiment, the specific steps for the result interpretation layer to interpret and explain the decisions of the model are as follows:

[0094] Decision tree hierarchical division: Abstract the decision logic in the trained X-CNN model into a decision tree, divide the decision tree into multiple levels, and each level represents a stage of the model decision. Starting from the root node, gradually go deep into the leaf nodes.

[0095] Display the decision-making process and results: For each level, display the decision-making process and results at that level. The doctor clicks or selects a certain level through the interactive interface, and then displays the input features, output categories, and decision-making paths of all decision nodes at that level.

[0096] Add annotations and labels: At each level, add annotations and labels to explain the meaning and importance of each decision node, including: Explanation of input features: Explain the role and importance of each input feature in the model's decision-making; Explanation of output categories: Explain the meaning and possible clinical significance of each output category; Explanation of decision-making paths: Explain the decision-making path from the root node to the current node, and the judgment basis for each decision node.

[0097] Provide LIME local explanations: When displaying different levels of the decision tree, provide local explanations generated by LIME (Local Interpretable Model-agnostic Explanations).

[0098] Suppose there is a medical and health dataset that contains features such as the age, gender, blood pressure, and cholesterol level of patients, as well as a binary classification output (such as whether a certain disease exists). We trained an X-CNN model to predict this output.

[0099] In the result interpretation layer, divide the decision tree into the following levels:

[0100] The first level: Conduct a preliminary screening based on age and gender.

[0101] The second level: On the basis of the first level, further consider blood pressure and cholesterol level.

[0102] The third level: Make a final prediction by integrating all features.

[0103] Now, suppose there is a patient whose age is 60 years old, gender is male, blood pressure is high, and cholesterol level is normal. In the interactive interface, the doctor selects to view the decision-making process of this patient.

[0104] At the first level, the system will display the results of the screening based on age and gender, and explain why this patient enters the next level.

[0105] At the second level, the system will display the impact of blood pressure and cholesterol level on the decision-making, and explain why this patient is classified into a certain specific subcategory.

[0106] At the third level, the system will display the final prediction result after integrating all features, and provide the local explanation generated by LIME to illustrate which features have the most important impact on the prediction result.

[0107] For example, LIME may generate the following local explanations:

[0108] Age: has a positive impact on the prediction result (i.e., the older the age, the higher the risk of disease).

[0109] Gender: has a negative impact on the prediction result (i.e., the risk of disease is lower for males compared to females).

[0110] Blood pressure: has a positive impact on the prediction result (i.e., the higher the blood pressure, the higher the risk of disease).

[0111] Cholesterol level: has a relatively small impact on the prediction result (i.e., in this specific case, the cholesterol level is not the main decision-making factor).

[0112] In one embodiment, providing the LIME local explanation includes the following steps:

[0113] Select a specific case: Select one or more specific cases from the medical and health dataset as the object of explanation;

[0114] Generate a local linear model: Use the LIME tool to generate a local linear model for the selected case, and the local linear model will attempt to approximate the decision boundary of the X-CNN model near the selected case;

[0115] Extract the model coefficients and intercept: Extract the coefficients (weights) and intercept (intercept) from the local linear model, and the coefficients represent the degree of influence of each input feature on the model decision;

[0116] Display the local explanation: Display the coefficients and intercept of the local linear model in a visual form to the doctor. At the same time, display the comparison between the prediction result of the local linear model and the prediction result of the actual X-CNN model to verify the accuracy of the local explanation;

[0117] Explain the decision nodes: When displaying different levels of the decision tree, associate the local explanations generated by LIME with each decision node, explain the meaning and importance of each decision node, and how it makes decisions based on the input features;

[0118] Adjust the visualization parameters and the depth of explanation: Allow the doctor to adjust the visualization parameters (such as the width, height, color, etc. of the bar chart) and the depth of explanation (such as how many of the most important features to display, whether to display the intercept, etc.) as needed.

[0119] In one embodiment, the specific steps of generating the local linear model:

[0120] Initialize the LIME interpreter: Create an interpreter object using the LIME toolkit. This object will be used to generate a local linear model. Specify the model to be explained (the X-CNN model) and the number of features to be explained.

[0121] Generate perturbed samples: The LIME interpreter makes small perturbations to the input features of the selected case to generate a series of perturbed samples. These perturbed samples will be used to observe the decision boundary of the X-CNN model near the selected case.

[0122] Predict the perturbed samples: Use the X-CNN model to predict the generated perturbed samples and obtain the prediction results for each perturbed sample.

[0123] Construct a local linear model: The LIME interpreter uses the perturbed samples and their prediction results to construct a local linear model. This local linear model will attempt to approximate the decision boundary of the X-CNN model near the selected case. The construction of the local linear model is based on the following formula:

[0124]

[0125] where: g(z') is the local linear model, which takes the perturbed sample z' as input, f is the model we want to explain (the X-CNN model), G is the hypothesis space of the local linear model (in this case, a linear regression model), L(f,g,π2) is the local loss function, which measures the prediction difference between the model f and the local linear model g on the perturbed samples, Ω(g) is the regularization term, used to control the complexity of the local linear model, and π2 is the distribution of the perturbed samples, which determines how to generate perturbed samples near the selected case.

[0126] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A computer data mining system based on big data, characterized in that, Including: Data preprocessing layer: responsible for collecting, cleaning, integrating, and standardizing medical and health data; Model training layer: using the preprocessed data to train the X-CNN model, validating and testing the X-CNN model; Data mining layer: applying the X-CNN model to mine and analyze medical and health data, introducing a feature selection method based on graph theory, learning and extracting the most representative features, and using visualization techniques to generate feature visualization graphs and importance scoring graphs; Result interpretation layer: providing an interactive interface, based on the visualization graphs and decision tree graphs generated by the data mining layer, interpreting and explaining the decisions of the model, and allowing doctors to adjust visualization parameters and interpretation depth as needed. The decision tree is divided into multiple levels, each level representing a different stage of the model's decision. By gradually showing the decision-making process and results of each level, adding annotations and labels at each level to explain the meaning and importance of each decision node. When showing different levels of the decision tree, local explanations generated by LIME are provided simultaneously.

2. The computer data mining system based on big data according to claim 1, characterized in that, The data preprocessing layer uses web crawler technology to automatically capture image data from the medical imaging system, uses an image processing library for image denoising and restoration, uses a database management system to store and manage image data, and uses image normalization and standardization algorithms for data standardization processing.

3. The computer data mining system based on big data according to claim 1, wherein, The specific steps for the model training layer to use the preprocessed data to train the X-CNN model are as follows: Data preparation: performing missing value filling, outlier detection and handling, and data standardization; Model initialization: initializing the parameters of the X-CNN model, including the size, number, and stride of the convolutional kernels, the number of neurons in the fully connected layer, and the activation function; Defining the loss function and optimizer: The loss function is used to measure the gap between the model's prediction results and the true labels, and the optimizer is responsible for adjusting the model parameters to minimize the loss function; Model training: inputting the preprocessed data into the X-CNN model, performing forward propagation to calculate the prediction results, then calculating the gap between the prediction results and the true labels through the loss function. Next, using the optimizer for backpropagation to update the model parameters, and performing multiple iterations until the model converges or reaches the preset number of iterations.

4. A computer data mining system based on big data according to claim 1, characterized in that The data mining layer introduces a feature selection method based on graph theory, learning and extracting the most representative features, including the following steps; Constructing a feature graph: extracting a series of features from the medical and health data, representing the extracted features as a feature graph G=(V, E), where V represents the set of feature nodes, and each node represents a feature; E represents the set of feature edges, and each edge represents the association relationship between features; Calculating Feature Importance: Using centrality measures in graph theory to evaluate the importance of features. The centrality measure adopts degree centrality. The degree centrality of a feature node is defined as the number of neighbor nodes of this node, expressed as DC(v) = ∑ u∈N(v) 1, where DC(v) represents the degree centrality of feature node v, and N(v) represents the set of neighbor nodes of feature node v; Selecting the most representative features: after calculating the importance of the features, selecting the top k most representative features according to the importance ranking; Optimizing the feature subset: using particle swarm optimization to optimize the initially selected feature subset.

5. A computer data mining system based on big data according to claim 4, characterized in that, The specific steps for using particle swarm optimization to optimize the initially selected feature subset are as follows: Initialize the particle swarm: Initialize a particle swarm, where each particle represents a possible subset of features. The position vector of the particle is represented as a feature selection vector, i.e., a binary vector, where 1 indicates the selection of the feature and 0 indicates non-selection. At the same time, each particle also has a velocity vector, which represents the moving direction and speed of the particle in the feature space, and the initial value of the velocity vector is randomly generated; Define the fitness function: The fitness function is used to evaluate the performance of each particle (i.e., each subset of features). The fitness function is the accuracy of the model on the validation set; Update the particle position and velocity: The particle searches for the optimal solution by iteratively updating its position and velocity. The update formulas include the velocity update formula and the position update formula. The velocity update formula is expressed as Among them, represents the value of the velocity of particle i in the d-th dimension at the (t + 1)-th iteration; w is the inertia weight, which is used to control the tendency of the particle to maintain its original velocity; c1 and c2 are learning factors, which respectively represent the weights of the particle learning from its own historical best position and the global best position; r1 and r2 are random numbers, which are used to increase the randomness of the algorithm; represents the historical best position of particle i in the d-th dimension; represents the global best position; represents the current position of particle i in the d-th dimension; The position update formula is expressed as where represents the value of the position of particle i in the d-th dimension at the (t + 1)-th iteration; Evaluate the fitness and update the optimal position: In each iteration, evaluate the fitness of each particle, and update the historical optimal position and the global optimal position of the particle. If the fitness of the current particle is better than the fitness of its historical optimal position, then update its historical optimal position; if the fitness of the current global optimal position is surpassed by the fitness of a certain particle, then update the global optimal position; Iteratively search until the stopping condition is met: Set a stopping condition. When the stopping condition is met, the algorithm stops iterating and outputs the global optimal position as the optimal subset of features.

6. A computer data mining system based on big data according to claim 1, characterized in that The specific steps for the result interpretation layer to interpret and explain the decisions of the model: Hierarchical division of the decision tree: Abstract the decision logic in the trained X-CNN model into a decision tree, and divide the decision tree into multiple levels. Each level represents a stage of the model's decision. Starting from the root node, gradually go deep into the leaf node; Show the decision process and results: For each level, show the decision process and results at that level. The doctor clicks or selects a certain level through the interactive interface, and then shows the input features, output categories, and decision paths of all decision nodes at that level; Add annotations and labels: At each level, add annotations and labels to explain the meaning and importance of each decision node, including: Explanation of input features: Explain the role and importance of each input feature in the model's decision; Explanation of output categories: Explain the meaning and possible clinical significance represented by each output category; Explanation of decision paths: Explain the decision path from the root node to the current node, and the judgment basis for each decision node; Provide LIME local explanations: When showing different levels of the decision tree, provide the local explanations generated by LIME.

7. A computer data mining system based on big data according to claim 6, characterized in that, The providing of LIME local explanations includes the following steps: Select specific cases: Select one or more specific cases from the medical and health dataset as the objects to be explained; Generate a local linear model: Use the LIME tool to generate a local linear model for the selected cases. The local linear model will try to approximate the decision boundary of the X-CNN model near the selected cases; Extract the model coefficients and intercepts: Extract the coefficients (weights) and intercepts (intercept) from the local linear model. The coefficients represent the degree of influence of each input feature on the model's decision; Local Explanation Display: Visualize the coefficients and intercepts of the local linear model for doctors, and at the same time, display the comparison between the prediction results of the local linear model and the actual prediction results of the X-CNN model; Explanation of Decision Nodes: When showing different levels of the decision tree, associate the local explanations generated by LIME with each decision node, explain the meaning and importance of each decision node, and how it makes decisions based on the input features; Adjustment of Visualization Parameters and Explanation Depth: Allow doctors to adjust visualization parameters and explanation depth as needed.

8. A computer data mining system based on big data according to claim 7, characterized in that, Specific Steps for Generating the Local Linear Model: Initialization of the LIME Interpreter: Use the LIME toolkit to create an interpreter object, which will be used to generate the local linear model, specifying the model to be explained (X-CNN model) and the number of features to be explained; Generation of Perturbed Samples: The LIME interpreter makes small perturbations to the input features of the selected case to generate a series of perturbed samples, which will be used to observe the decision boundary of the X-CNN model near the selected case; Prediction of Perturbed Samples: Use the X-CNN model to predict the generated perturbed samples to obtain the prediction results of each perturbed sample; Construction of the Local Linear Model: The LIME interpreter uses the perturbed samples and their prediction results to construct a local linear model, which will attempt to approximate the decision boundary of the X-CNN model near the selected case. The construction of the local linear model is based on the following formula: Where: g(z') is the local linear model, which takes the perturbed sample z' as input, f is the model we want to explain (X-CNN model), G is the hypothesis space of the local linear model, L(f,g,π2) is the local loss function, which measures the prediction difference between the model f and the local linear model g on the perturbed samples, Ω(g) is the regularization term, used to control the complexity of the local linear model, and π2 is the distribution of the perturbed samples, which determines how to generate perturbed samples near the selected case.

Images