Method, system, medium, program product for predicting psychological index using machine learning method

Abstract

The invention relates to a method, a system, a medium and a program product for predicting psychological indexes by using a machine learning method, and the method comprises the steps: reading data related to psychological happiness of a teacher from a data set, removing abnormal data by using an IQR method, and replacing an abnormal value with an NAN value; replacing the NAN value with a median, carrying out clustering analysis to identify a psychological state mode of the teacher, carrying out standardization processing, and processing a class imbalance problem by using an SMOTE technology; training a plurality of machine learning models, and selecting the machine learning model with the highest test precision; and outputting the SHAP value of each characteristic variable, and visualizing the relationship between the independent variable and the dependent variable to display the influence factor of the teacher happiness index. Compared with the prior art, the method has the advantages that the prediction process of the visual model can be realized, and the influence of each characteristic variable on the prediction result is explained, so that researchers and decision makers can better understand and trust the prediction of the model.

Description

Technical Field

[0001] The present invention relates to a data processing method, and in particular to a method, system, medium and program product for predicting a psychology index using a machine learning method. Background Art

[0002] Traditional research methods often rely on questionnaires and subjective reports, which are biased and difficult to fully capture the complex factors that affect happiness.

[0003] With the development of machine learning technology, we can now understand happiness more objectively and deeply by analyzing multi-dimensional data. Machine learning algorithms can process and analyze large amounts of data and identify patterns and associations in the data, which provides new perspectives and tools for studying happiness indexes. However, existing machine learning software usually only provides analysis of a single model and lacks consideration of data diversity and complexity.

[0004] In order to improve the accuracy and reliability of predictions, a method is needed that can integrate multiple machine learning models and automatically select the best model. In addition, data preprocessing is a key step in the machine learning workflow, which directly affects the training effect of the model and the final prediction performance. Therefore, it is necessary to develop a system that can automatically perform data cleaning, processing, and clustering analysis to ensure the quality of the data.

[0005] In terms of model training and evaluation, existing technologies often lack a mechanism to compare and select the performance of different models. An excellent system should be able to automatically train multiple models, evaluate their performance, and select the best model for prediction. Such a system can greatly improve the accuracy of predictions and provide researchers with more reliable results.

[0006] Finally, the interpretability of the model is crucial for the understanding and application of research results. However, many machine learning models are considered "black boxes" and their internal working mechanisms are difficult to explain. Therefore, a method is needed to visualize the prediction process of the model and explain the impact of each feature variable on the prediction results, so that researchers and decision makers can better understand and trust the model's predictions. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method, system, medium, and program product for predicting psychological indexes using machine learning methods, which can realize the prediction process of the visualization model and explain the influence of each characteristic variable on the prediction result, so that researchers and decision makers can better understand and trust the prediction of the model.

[0008] The purpose of the present invention can be achieved by the following technical solutions: The first aspect of the present invention provides a method for predicting a psychology index using a machine learning method, comprising the following steps: S1: Data reading and cleaning: Read data related to teachers’ psychological well-being from the dataset, and use the IQR method to remove abnormal data and replace abnormal values ​​with NAN values ​​(Not a Number). S2: Data preprocessing: Use the median to replace NAN values, perform cluster analysis to identify the teacher's psychological state pattern, perform standardization, and use SMOTE technology (Synthetic Minority Oversampling Technique is a technology used to deal with sample imbalance in classification problems. It solves the problem of class imbalance in the data set by synthesizing new minority class samples, thereby improving the generalization ability and accuracy of the model) to deal with the class imbalance problem; S3: Model training and evaluation: Train multiple machine learning models, calculate the training and test accuracy of each model, and then select the machine learning model with the highest test accuracy; S4: Result output and visualization: Based on the machine learning model selected in S3, the SHAP value of each feature variable is output, and the relationship between the independent variable and the dependent variable is visualized to show the influencing factors of the teacher happiness index.

[0009] Furthermore, S1 specifically includes the following steps: Data reading: reading data related to teachers’ psychological well-being from pre-prepared datasets, which includes extracting datasets from database files and loading them into the analysis environment; Data cleaning: The IQR (interquartile range) method is used to detect outliers on the data, identify outliers that do not conform to the normal data distribution, and replace the detected outliers with NAN values, that is, mark them as missing values ​​in the data set, so that they can be properly processed in subsequent steps.

[0010] Furthermore, S1 specifically includes the following steps: Missing value processing: Use the median to replace the NAN value in the data set; the purpose of this step is to handle missing data to avoid bias in subsequent analysis while maintaining the integrity and distribution characteristics of the data; Data conversion and balancing: Conduct cluster analysis to identify patterns in the teachers’ psychological states; cluster analysis is an unsupervised learning method that helps us discover natural groupings or patterns in data, thereby better understanding the structure of the data Standardization is performed to make different features have the same dimension, thus improving the stability and efficiency of model training. Use SMOTE to handle class imbalance problems.

[0011] Furthermore, S3 specifically includes the following steps: Model training: Multiple machine learning models are trained on the preprocessed data. This step involves fitting the model using the training dataset and adjusting the model parameters to learn patterns in the data.

[0012] Model evaluation: Calculate and store the training and test accuracy (accuracy) for each trained model. This usually involves splitting the dataset into a training set and a test set, using the training set to train the model, and then using the test set to evaluate the model's performance. The evaluation metric is usually accuracy, which is the ratio of correctly predicted samples to the total number of samples. Finally, the model with the highest test accuracy is selected from all models as the optimal model for further analysis and application.

[0013] Furthermore, in S3, the models used include one of logistic regression, naive Bayes, decision tree, random forest, K nearest neighbor, and LightGBM.

[0014] Furthermore, S4 specifically includes the following steps: Calculation of SHAP values ​​of feature variables: Based on the machine learning model selected in S3, calculate the SHAP value of each feature variable; SHAP value is a method to explain model predictions, which quantifies the contribution of each feature to the model prediction results. The purpose of this step is to identify and quantify the impact of each feature on the prediction results of the teacher happiness index.

[0015] Visualization of the relationship between independent variables and dependent variables: The SHAP values ​​obtained in S1 are used to visualize the relationship between independent variables and dependent variables. The purpose of this step is to intuitively display the influencing factors of the teacher happiness index and help researchers better understand the relationship behind the model's prediction results and data.

[0016] Furthermore, a visualization of the teacher happiness index is achieved by generating a chart to show the degree and pattern of the influence of different characteristic variables on the teacher happiness index; The graph includes one of a feature importance graph, a dependency graph, and a box plot.

[0017] A second aspect of the present invention provides a computer system comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-mentioned method of predicting a psychological index using a machine learning method.

[0018] A third aspect of the present invention is a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for predicting a psychological index using a machine learning method.

[0019] A fourth aspect of the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned method of predicting a psychological index using a machine learning method.

[0020] Compared with the prior art, the present invention has the following technical advantages: 1) The present invention is not only applicable to the prediction of teachers' happiness index, but can also be extended to the study of the psychological state of other groups, and has broad application prospects.

[0021] 2) Through automated data cleaning, model training and evaluation processes, the present invention significantly improves the efficiency of psychology research, and researchers can quickly analyze large amounts of data, thereby saving valuable time and resources.

[0022] 3) The visualization tools and model interpretation capabilities provided by this invention enable decision makers with non-technical backgrounds to understand and use the model prediction results. This ease of use helps to transform research results into practical applications, thus having a positive impact in the fields of education management, policy making, and human resource planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The figure is a schematic diagram of the process of predicting the psychological index using the machine learning method in the present invention. DETAILED DESCRIPTION

[0026] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Component models, material names, connection structures, control methods, algorithms and other features not clearly described in this technical solution are all considered to be common technical features disclosed in the prior art.

[0027] Example 1

[0028] This embodiment provides a method for predicting a psychology index using a machine learning method, comprising the following steps: S1: Data reading and cleaning: Read data related to teachers’ psychological well-being from the dataset, and use the IQR method to remove abnormal data and replace abnormal values ​​with NAN values ​​(Not a Number). Furthermore, S1 specifically includes the following steps: Data reading: reading data related to teachers’ psychological well-being from pre-prepared datasets, which includes extracting datasets from database files and loading them into the analysis environment; Data cleaning: The IQR (interquartile range) method is used to detect outliers on the data, identify outliers that do not conform to the normal data distribution, and replace the detected outliers with NAN values, that is, mark them as missing values ​​in the data set, so that they can be properly processed in subsequent steps.

[0029] In the specific implementation, S1 specifically includes the following steps: Missing value processing: Use the median to replace the NAN value in the data set; the purpose of this step is to handle missing data to avoid bias in subsequent analysis while maintaining the integrity and distribution characteristics of the data; Data conversion and balancing: Conduct cluster analysis to identify patterns in the teachers’ psychological states; cluster analysis is an unsupervised learning method that helps us discover natural groupings or patterns in data, thereby better understanding the structure of the data Standardization is performed to make different features have the same dimension, thus improving the stability and efficiency of model training. Use SMOTE to handle class imbalance problems.

[0030] S2: Data preprocessing: Use the median to replace NAN values, perform cluster analysis to identify the teacher's psychological state pattern, perform standardization, and use SMOTE technology (Synthetic Minority Oversampling Technique is a technology used to deal with sample imbalance in classification problems. It solves the problem of class imbalance in the data set by synthesizing new minority class samples, thereby improving the generalization ability and accuracy of the model) to deal with the class imbalance problem; In the data reading and cleaning stage, this solution first extracts the data set related to teachers' psychological well-being from the database file and loads it into the analysis environment to ensure the source and initial availability of the data. Then, the IQR method is used to remove abnormal data and replace abnormal values ​​with NAN values. This step helps to identify and process outliers that affect the accuracy of the analysis results, thereby improving the reliability of the data.

[0031] In the missing value processing step, the software uses the median to replace the NAN value, which is an effective missing data processing method that can reduce the bias caused by missing data while maintaining the integrity and distribution characteristics of the data set. The data transformation and balancing processing steps further enhance the quality of the data and the predictive performance of the model. Through cluster analysis, the software is able to identify patterns in the psychological state of teachers, which helps to discover the inherent structure and association in the data. Standardization ensures that different features are compared under the same dimension, improving the stability and efficiency of model training. Finally, the SMOTE technology is used to deal with the problem of class imbalance, ensuring that the model will not be biased due to too many or too few samples in certain categories in the data set, thereby improving the generalization ability and prediction accuracy of the model.

[0032] S3: Model training and evaluation: Train multiple machine learning models, calculate the training and test accuracy of each model, and then select the machine learning model with the highest test accuracy; In the specific implementation, S3 specifically includes the following steps: Model training: Multiple machine learning models are trained on the preprocessed data. This step involves fitting the model using the training dataset and adjusting the model parameters to learn patterns in the data.

[0033] Model evaluation: Calculate and store the training and test accuracy (accuracy) for each trained model. This usually involves splitting the dataset into a training set and a test set, using the training set to train the model, and then using the test set to evaluate the model's performance. The evaluation metric is usually accuracy, which is the ratio of correctly predicted samples to the total number of samples. Finally, the model with the highest test accuracy is selected from all models as the optimal model for further analysis and application.

[0034] In the specific implementation, in S3, the models used include logistic regression, naive Bayes, decision tree, random forest, K nearest neighbor, and LightGBM.

[0035] S4: Result output and visualization: Based on the machine learning model selected in S3, the SHAP value of each feature variable is output, and the relationship between the independent variable and the dependent variable is visualized to show the influencing factors of the teacher happiness index.

[0036] In terms of conceptual logic, the logic of the present invention is to ensure the accuracy of model predictions and the generalization ability of the model. In the model training and evaluation step (S3), the software first trains multiple machine learning models, fits the models with preprocessed data, and adjusts the model parameters to learn patterns in the data. This process involves using a training data set to train the model and minimizing the prediction error by adjusting the parameters. Subsequently, the training and test accuracy of each model is calculated, which is usually done by dividing the data set into a training set and a test set, where the training set is used for model training and the test set is used to evaluate the performance of the model. The evaluation indicator is usually the accuracy, that is, the proportion of correctly predicted samples to the total number of samples. Finally, the software selects the model with the highest test accuracy from all trained models as the optimal model for further analysis and application.

[0037] The design logic of the present invention is also reflected in its meticulous treatment of model selection. In specific implementation, the models used in S3 include logistic regression, naive Bayes, decision tree, random forest, K nearest neighbor and LightGBM. These models cover a variety of algorithms from simple to complex, each with its own characteristics and applicable scenarios. For example, logistic regression is suitable for binary classification problems, decision trees are intuitive and easy to understand and easy to explain, and random forests improve prediction performance and control overfitting by constructing multiple decision trees. In this way, it can be ensured that the selected model not only performs well on the training data, but also generalizes well to new and unseen data. This strategy of comprehensively considering multiple models and selecting the best model improves the accuracy of the prediction results and the practicality of the model.

[0038] S4 specifically includes the following steps: Calculation of SHAP values ​​of feature variables: Based on the machine learning model selected in S3, calculate the SHAP value of each feature variable; SHAP value is a method to explain model predictions, which quantifies the contribution of each feature to the model prediction results. The purpose of this step is to identify and quantify the impact of each feature on the prediction results of the teacher happiness index.

[0039] Visualization of the relationship between independent variables and dependent variables: The SHAP values ​​obtained in S1 are used to visualize the relationship between independent variables and dependent variables. The purpose of this step is to intuitively display the influencing factors of the teacher happiness index and help researchers better understand the relationship behind the model's prediction results and data.

[0040] In specific implementation, a visualization of the teacher happiness index is achieved by generating a chart to show the degree and pattern of the influence of different characteristic variables on the teacher happiness index; In a specific implementation, the icon includes one of a feature importance graph, a dependency graph, and a box-and-whisker graph.

[0041] In terms of conceptual logic, the design logic of the present invention is to use advanced machine learning technology to improve the accuracy and transparency of the prediction of the teacher happiness index. In step S4, the contribution of each feature to the model prediction results is first quantified by calculating the SHAP value of each feature variable. The key role of this step is to provide a transparent method to identify and measure which factors have a significant impact on teacher happiness. Then, these complex data relationships are converted into intuitive charts, such as feature importance graphs, dependency graphs, and box plots, through visualization technology, so that non-professional users can also understand and analyze the prediction results of the model, thereby deepening their understanding of the dynamics of the teacher happiness index.

[0042] In the SHAP value calculation step of the feature variable, the SHAP value is used to quantify the contribution of each feature to the model prediction results. This method is based on the Shapley value in game theory and can fairly distribute the marginal contribution of each feature to the prediction results. Then, in the visualization step of the relationship between the independent variable and the dependent variable, by generating visual charts such as feature importance graphs, dependency graphs, and box plots, the degree and pattern of the influence of different feature variables on the teacher happiness index are intuitively displayed. These charts not only show the influence of the features, but also reveal the potential interactions between the features, thereby helping researchers to deeply understand the complex relationships behind the data.

[0043] Example 2

[0044] This embodiment provides a computer system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-mentioned method for predicting a psychology index using a machine learning method.

[0045] The memory is the data storage center of the system, which is responsible for saving computer programs, raw data sets, preprocessed data, model parameters, and final prediction results. The memory also contains a library of multiple machine learning models that can be used for training and prediction processes, such as logistic regression, naive Bayes, decision tree, random forest, K nearest neighbor, and LightGBM.

[0046] The processor is the computing engine of the system, which executes computer programs in the memory and manages and coordinates the work of various components. The processor first reads the data set related to teachers' psychological well-being from the memory. These data include information on teachers' personal background, working conditions, social environment, career satisfaction and other dimensions. Then, the processor performs a data cleaning step, uses the IQR method to remove abnormal data, marks the abnormal values ​​as NAN values, and then replaces these missing values ​​with the median to ensure the integrity and accuracy of the data.

[0047] In the data preprocessing stage, the processor not only performs data cleaning, but also performs data transformation and balancing processing, such as identifying patterns of teachers' mental states through cluster analysis, unifying the dimensions of different features through standardization, and using SMOTE technology to deal with category imbalance problems, thereby preparing high-quality data for model training.

[0048] The processor is also responsible for executing the model training and evaluation steps. It calls the machine learning model library in the memory, trains multiple models using preprocessed data, and calculates the training and test accuracy of each model. The processor will compare the performance of these models and select the model with the highest test accuracy as the optimal model for the final prediction.

[0049] In addition, the processor also performs the result output and visualization step, which calculates the SHAP value of each feature variable and uses these values ​​to visualize the relationship between the independent variable and the dependent variable. The visualization charts generated by the processor, such as feature importance graphs, dependency graphs, and box plots, can help users intuitively understand the model's prediction results and the relationship behind the data.

[0050] Finally, the processor saves the prediction results and visualization charts back to the memory for further analysis and reporting by the user. The design of this computer system not only improves the accuracy of predicting the teacher happiness index, but also enhances the interpretability of the model, providing strong data support for educational decision-making.

[0051] Example 3

[0052] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for predicting a psychological index using a machine learning method are implemented.

[0053] The computer program stored on the medium includes the relevant algorithms and instruction sets in the present invention, which, when executed by the processor, can implement steps such as data reading, cleaning, preprocessing, model training, evaluation, and result visualization. The program first reads a data set related to teachers' psychological well-being from a database or file, then removes abnormal data through the IQR method, and fills missing values ​​with the median to ensure the quality and integrity of the data.

[0054] Next, the program performs data preprocessing steps, including data standardization and SMOTE technology processing, as well as cluster analysis to identify potential patterns in the data. Subsequently, the program trains multiple machine learning models and calculates their training and test accuracy to select the best model for prediction.

[0055] In addition, the program also calculates the SHAP value of each feature variable and generates various visualization charts, such as feature importance charts, dependency charts, and box plots, which intuitively show the impact of different features on the teacher happiness index. These visualization results not only help to understand the prediction results of the model.

[0056] Example 4

[0057] This embodiment provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for predicting a psychology index using a machine learning method.

[0058] The computer program product includes a data preprocessing module, a machine learning model training module, a prediction result evaluation module and a visualization output module. The data preprocessing module is responsible for reading the data set, cleaning the data, processing missing values ​​and outliers, and performing necessary data conversion and standardization. The machine learning model training module contains a variety of algorithms that can train the model and determine the best prediction model. The prediction result evaluation module is used to evaluate the performance of the model, select the most accurate model, and calculate the accuracy of the model.

[0059] The program product also includes a visualization output module that generates intuitive charts such as feature importance graphs, dependency graphs, and box plots, which help explain the model's prediction results and reveal the key factors that affect the teacher happiness index. These visualization tools not only make complex data relationships easy to understand, but also enhance the transparency and credibility of the model.

[0060] In addition, the computer program product includes a user interface that allows the user to easily input data, select model parameters, start the forecasting process, and view the results. The user interface is designed with ease of use for non-technical users in mind.

[0061] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for predicting a psychological index using a machine learning method, characterized in that: The following steps are involved: S1: Data reading and cleaning: Read data related to teachers’ psychological well-being from the dataset, and use the IQR method to remove abnormal data and replace abnormal values ​​with NAN values; S2: Data preprocessing: Use the median to replace NAN values, perform cluster analysis to identify the teacher's psychological state patterns, perform standardization, and use SMOTE technology to deal with class imbalance problems; S3: Model training and evaluation: Train multiple machine learning models, calculate the training and test accuracy of each model, and then select the machine learning model with the highest test accuracy; S4: Result output and visualization: Based on the machine learning model selected in S3, the SHAP value of each feature variable is output, and the relationship between the independent variable and the dependent variable is visualized to show the influencing factors of the teacher happiness index.

2. A method for predicting a psychological index using a machine learning method according to claim 1, characterized in that: S1 specifically includes the following steps: Data reading: reading data related to teachers’ psychological well-being from pre-prepared datasets, which includes extracting datasets from database files and loading them into the analysis environment; Data cleaning: The IQR method is used to detect outliers on the data, identify outliers that do not conform to the normal data distribution, and replace the detected outliers with NAN values, that is, mark them as missing values ​​in the data set so that they can be properly processed in subsequent steps.

3. The method for predicting a psychology index using a machine learning method according to claim 1, characterized in that: In S1, specific The following steps are involved: Missing value processing: Use the median to replace the NAN values ​​in the data set; Data conversion and balancing: Cluster analysis was conducted to identify patterns in teachers’ psychological states; Standardize the features so that they have the same dimension, thus improving the stability and efficiency of model training. Use SMOTE to handle class imbalance problems.

4. The method for predicting a psychology index using a machine learning method according to claim 1, characterized in that: S3 specifically includes the following steps: Model training: training multiple machine learning models on preprocessed data; Model evaluation: Calculate and store the training and test accuracy for each trained model, where the dataset is divided into a training set and a test set, use the training set to train the model, and then use the test set to evaluate the performance of the model.

5. The method for predicting a psychology index using a machine learning method according to claim 1, characterized in that: In S3, the models used include logistic regression, naive Bayes, decision tree, random forest, K nearest neighbor, and LightGBM.

6. The method for predicting a psychology index using a machine learning method according to claim 1, characterized in that: S4 specifically includes the following steps: SHAP value calculation of feature variables: Based on the machine learning model selected in S3, the SHAP value of each feature variable is calculated; Visualization of the relationship between independent variables and dependent variables: The SHAP values ​​obtained in S1 are used to visualize the relationship between independent variables and dependent variables.

7. A method for predicting a psychology index using a machine learning method according to claim 6, characterized in that: By generating charts, a visual display of the teacher happiness index is achieved to show the degree and pattern of the influence of different characteristic variables on the teacher happiness index; The graph includes one of a feature importance graph, a dependency graph, and a box plot.

8. A computer system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for predicting a psychological index using a machine learning method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for predicting a psychological index using a machine learning method as described in any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for predicting a psychological index using a machine learning method as described in any one of claims 1 to 7 are implemented.

Images