Feature detection method, device, equipment and medium
The feature data of the regulated objects are preprocessed, clustered and verified through machine learning methods, which solves the problem of relying on human subjective experience in existing technologies and realizes objective, scientific and explainable feature identification and classification of regulated objects.
Patent Information
- Application Number
- CN202510776197.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-11
- Publication Date
- 2025-09-19
AI Technical Summary
Existing regulatory object classification and feature detection methods mainly rely on human subjective experience and lack objectivity and personalized intervention measures, which makes it difficult to share information and quickly extract key feature factors.
A machine learning method is used to collect the initial feature data of the regulatory objects for preprocessing, clustering is performed using the KMeans model, and verification is combined with the XGBoost model. Finally, the SHAP value theory is used to explain and demonstrate feature differences.
It achieves objective, scientific and explainable feature identification and classification of regulatory objects, provides a scientific basis for supervision, improves processing capabilities and avoids the limitations of subjective classification.
Smart Images
Figure CN120671004A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine learning technology, and in particular to a feature detection method, device, equipment and medium. Background Art
[0002] According to reported research, the classification and feature identification of supervision objects in most parts of the country are basically based on manual subjective experience, mainly relying on reading the files of supervision objects, the experience of police officers with rich years of experience, the results of relevant psychological scales, and the transformation performance of supervision objects.
[0003] Despite the efforts made by relevant management agencies in transforming regulatory objects as described above, the existing classification and feature detection methods for regulatory objects are still subjective, which is not conducive to the sharing of regulatory object information and the rapid extraction of key feature factors. In addition, due to objective factors such as manpower, the conditions for formulating personalized intervention measures are not available. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and to provide a feature detection method, device, equipment and medium.
[0005] The present invention provides the following technical solutions: In a first aspect, the present invention provides a feature detection method, comprising: Collecting initial feature data of the supervised subject, preprocessing the initial feature data to obtain preprocessed optimized feature data, and dividing the optimized feature data into a training set and a test set according to a preset ratio, wherein the initial feature data includes supervisory feature data, demographic feature data, psychological feature data, original family feature data, and educational feature data; Performing principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, using the dimensionality reduction result to train a KMeans model, using the trained KMeans model to cluster the test set to obtain a clustering result, and displaying the characteristic differences of different clusters in the clustering result through a radar chart; Using the training set to train the XGBoost model, and using the trained XGBoost model to verify the clustering results, to obtain the optimal XGBoost model; The optimal XGBoost model is interpreted using SHAP value theory to obtain SHAP feature interpretation results to verify the feature differences of the radar chart.
[0006] In an optional embodiment, preprocessing the initial feature data to obtain preprocessed optimized feature data includes: Determining unstructured feature data in the initial feature data, and converting the unstructured feature data using a binarization method or a dummy variable method to obtain structured feature data; Determining categorical feature data and numerical feature data in the structured feature data, interpolating missing values in the categorical feature data using a Random Forest method, and interpolating missing values in the numerical feature data using a mode, median, or mean, to obtain interpolated structured feature data; The interpolated structured feature data is pre-classified using the K-Means algorithm to obtain a pre-classification result, and a chi-square test is performed on the pre-classification result to obtain a test result. Data with a test result less than a preset test threshold is used as the pre-processed optimized feature data.
[0007] In an optional embodiment, performing principal component dimensionality reduction on the training set to obtain a dimensionality reduction result includes: Determining the sample size and feature quantity of the training set, forming an initial matrix according to the sample size and the feature quantity, and performing standardization processing on the initial matrix to obtain a standardized matrix; Calculating the covariance matrix of the standardized matrix, and calculating the eigenvalues and corresponding eigenvectors of the covariance matrix, arranging the eigenvectors in descending order according to the eigenvalues, and constructing an orthogonal matrix; Determine the projection matrices corresponding to the first preset number of principal components in the orthogonal matrix, construct a dimensionality reduction matrix according to the normalization matrix and the projection matrix, and use the dimensionality reduction matrix as the dimensionality reduction result.
[0008] In an optional embodiment, the using the dimensionality reduction result to train a KMeans model, and using the trained KMeans model to cluster the test set to obtain a clustering result, includes: randomly selecting a second preset number of samples from the sample volume as a plurality of initial centroids using an elbow method, calculating a distance from each of the samples in the sample volume to each of the initial centroids, and allocating each of the samples according to the distance to obtain a plurality of clusters; The centroid of each cluster is calculated, and each initial centroid is replaced by the centroid of each cluster, and the calculation is repeated until a preset maximum number of iterations is met to obtain the clustering result, which includes multiple sample categories and their corresponding features.
[0009] In an optional embodiment, displaying the characteristic differences of different clusters in the clustering results through a radar chart includes: Assigning a corresponding score value to each feature in the clustering result according to a preset rule, and determining the number of samples for each score value; Calculating the feature weighted scores of different clusters in the clustering results according to the score values of the features and the number of samples of the score values; The radar chart is drawn according to the feature weighted scores of different clusters in the clustering result.
[0010] In an optional embodiment, the training of the XGBoost model using the training set, and verifying the clustering results using the trained XGBoost model to obtain the optimal XGBoost model, includes: Constructing the XGBoost model, setting initial parameters of the XGBoost model, optimizing the initial parameters using a Bayesian algorithm to obtain multiple optimal hyperparameter combinations, and reconstructing multiple optimized XGBoost models using each of the optimal hyperparameter combinations; Training each of the optimized XGBoost models on the training set to obtain a plurality of trained optimized XGBoost models; Each of the trained optimized XGBoost models is evaluated on the test set to obtain an evaluation index of each of the trained optimized XGBoost models, and the optimal XGBoost model is determined by comparing the evaluation index of each of the trained optimized XGBoost models.
[0011] In an optional embodiment, the optimal XGBoost model is interpreted using SHAP value theory to obtain a SHAP feature interpretation result, including: Based on the optimal XGBoost model, the SHAP value of each feature is calculated using the SHAP value theory, where the SHAP value represents the degree of influence of each feature on each type of sample; According to the positive or negative SHAP value, each feature is divided into a positive feature and a negative feature, and a visualization chart is generated using the SHAP value of each positive feature and the negative feature, and the SHAP feature interpretation result is obtained through the visualization chart.
[0012] In a second aspect, the present invention provides a feature detection device, comprising: A preprocessing module is used to collect initial feature data of the supervised object, preprocess the initial feature data to obtain optimized feature data after preprocessing, and divide the optimized feature data into a training set and a test set according to a preset ratio; A clustering module is used to perform principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, train a KMeans model using the dimensionality reduction result, cluster the test set using the trained KMeans model to obtain a clustering result, and display the characteristic differences of different clusters in the clustering result through a radar chart; A verification module is used to train the XGBoost model using the training set, and verify the clustering results using the trained XGBoost model to obtain the optimal XGBoost model; The interpretation module is used to interpret the optimal XGBoost model using SHAP value theory to obtain SHAP feature interpretation results to verify the feature differences of the radar chart.
[0013] In a third aspect, an embodiment of the present disclosure provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the feature detection method described in the first aspect are implemented.
[0014] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the feature detection method described in the first aspect are implemented.
[0015] Beneficial effects of this application: The feature detection method provided in the embodiment of the present application collects initial feature data of the supervised object and preprocesses the initial feature data to obtain preprocessed optimized feature data, and divides the optimized feature data into a training set and a test set according to a preset ratio, wherein the initial feature data includes supervisory feature data, demographic feature data, psychological feature data, native family feature data and educational feature data; performs principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, uses the dimensionality reduction result to train a KMeans model, uses the trained KMeans model to cluster the test set to obtain a clustering result, and displays the feature differences of different clusters in the clustering result through a radar chart; uses the training set to train an XGBoost model, and uses the trained XGBoost model to verify the clustering result to obtain an optimal XGBoost model; uses SHAP value theory to interpret the optimal XGBoost model to obtain a SHAP feature interpretation result to verify the feature differences of the radar chart. This application can use machine learning models to analyze past data and then classify and identify the characteristics of regulatory objects, providing relevant personnel and institutions with a basis for the classification and supervision of regulatory objects, and demonstrating the characteristics of different categories of regulatory objects. It also provides a scientific basis for the formulation of intervention measures, avoids the limitations of subjective classification, improves processing capabilities for problems with many features and large amounts of data, and has the advantages of objectivity, scientificity, and strong interpretability.
[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be considered as limiting the scope. A person of ordinary skill in the art can also derive other relevant drawings based on these drawings without inventive effort. Similar components are numbered similarly in the various drawings.
[0018] Figure 1 A flow chart of a feature detection method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of a broken line of the elbow method provided in an embodiment of the present application is shown; Figure 3 A scatter plot of KMeans classification results provided in an embodiment of the present application is shown; Figure 4 A visual radar chart of different classification features provided by an embodiment of the present application is shown; Figure 5 A schematic structural diagram of a feature detection device provided in an embodiment of the present application is shown; Figure 6 A structural diagram of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0019] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.
[0020] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.
[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the template description herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0022] Example 1 like Figure 1 FIG. 1 is a flow chart of a feature detection method according to an embodiment of the present application. The feature detection method according to an embodiment of the present application includes the following steps: Step S110: Collect initial feature data of the supervised object, preprocess the initial feature data to obtain optimized feature data after preprocessing, and divide the optimized feature data into a training set and a test set according to a preset ratio, wherein the initial feature data includes supervisory feature data, demographic feature data, psychological feature data, native family feature data, and educational feature data.
[0023] Understandably, supervised subjects will regularly collect information through questionnaires or professional diagnostic tests, and the collected data will be stored in a supervision database. In this embodiment, initial feature data from multiple aspects of supervised subjects is collected from the supervision database, including but not limited to supervisory feature data, demographic feature data, psychological feature data, family of origin feature data, and educational feature data. This information reflects various aspects of the supervised subject, such as whether the individual has a family (yes or no), marital status (single, married, or divorced), the integrity of the family of origin (intact, single-parent, reconstituted, empty nest, etc.), adverse childhood experiences (yes or no), educational level (no education, elementary school, junior high school, high school, college, etc.), awards received during school (yes or no), personality disorders (yes or no), and mental illness (yes or no). After data collection, the uniqueness and validity of the data are confirmed based on the supervised subject's ID (or other unique identifier), and all eligible data are included in the initial feature data. This embodiment considers comprehensive features, which facilitates K-Means identification of potential clusters with similar features, making the divided clusters more similar and facilitating the exploration of relationships between features.
[0024] Determine the unstructured feature data in the initial feature data, which generally refers to information that has no predefined data model or is not organized in a fixed format.
[0025] When unstructured data contain categorical information but have too many sub-values or are sensitive (e.g., adverse childhood experiences), simplification is necessary to improve interpretability. Multi-categorical variables can be merged into binary variables ("yes" or "no") using manual rules or simple algorithms (e.g., keyword matching).
[0026] When categorical variables in unstructured data need to be independently analyzed for their impact, they must be converted into structured features. For example, family upbringing styles are categorized into "authoritarian," "permissive," and "democratic." Using dummy variables, these variables can be converted into three new features: "Authoritarian Family Upbringing," "Permissive Family Upbringing," and "Democratic Family Upbringing." For an authoritarian supervisor, the corresponding features are "Authoritarian Family Upbringing" (yes), "Permissive Family Upbringing" (no), and "Democratic Family Upbringing" (no). This avoids the implicit order interference between multiple categorical variables in the model.
[0027] The above two conversion methods map unstructured feature data into structured feature data, which is convenient for subsequent machine learning model processing.
[0028] Furthermore, since some features may have missing values, features with a missing value ratio exceeding a preset ratio (e.g., 25%) are directly deleted to avoid excessive missing values from significantly impacting data quality and analysis results. For categorical feature data with missing values below a preset ratio (e.g., 25%), the random forest method is used for interpolation. Random forest is an ensemble learning algorithm that can predict the value of missing values based on other features in the data. After interpolation is completed, a chi-square test is performed on the samples before and after interpolation to test whether the interpolation effect is reasonable. If the P value of the test result is less than the first preset threshold (e.g., 0.05), it indicates that there is a significant difference in the distribution of the feature before and after interpolation, and the corresponding feature will be deleted. For numerical feature data with missing values below a preset ratio (e.g., 25%), common statistics such as mode, median, and mean can be used for interpolation. These statistics can reasonably fill in the missing values based on the central tendency or intermediate level of the data.
[0029] The above method for handling missing values avoids the significant impact of too many missing values on data quality and analysis results, and ensures the integrity and validity of the data.
[0030] In order to improve model performance, feature screening is required before data training the model to screen out features that can improve model performance, remove irrelevant or redundant features, reduce data dimensions, and improve model training efficiency and accuracy. Feature screening should be achieved through pre-classification and chi-square test. Specifically, the K-Means algorithm is first used to pre-classify the samples and divide the samples into different categories to obtain pre-classification results. Then, based on the pre-classification results, the chi-square test is used to test the differences between each feature in different categories. If the test P value of the inter-class feature difference is less than the second preset threshold (for example, 0.001), it is considered that the feature has a significant impact on the sample classification and is retained for model training. Otherwise, it is eliminated to obtain the pre-processed optimized feature data.
[0031] Finally, the preprocessed, optimized feature data is randomly divided into a training set and a test set according to a preset ratio, for example, 70% for training and 30% for testing. This way, during the subsequent feature detection process, the training set has sufficient data for model training and parameter adjustment, and the test set has sufficient data to verify the model's stability on unseen data.
[0032] The above method can remove irrelevant or redundant features through feature screening, reduce data dimensions, improve model training efficiency and accuracy, and help avoid overfitting problems.
[0033] In step S120, principal component dimensionality reduction is performed on the training set to obtain a dimensionality reduction result, a KMeans model is trained using the dimensionality reduction result, the test set is clustered using the trained KMeans model to obtain a clustering result, and the characteristic differences of different clusters in the clustering result are displayed through a radar chart.
[0034] Understandably, since the dimensions and value ranges of the features in the training set are likely to be different, standardization is required to eliminate the impact of these differences on subsequent analysis. Specifically, the sample size and feature count of the training set are determined, and the initial matrix is formed based on the sample size and feature count. For example, if the sample size M is 889 and the feature count N is 61, the initial matrix for: Then, each column of the initial matrix is normalized to obtain the normalized matrix ,in, is the mean vector of each column of X, is the standard deviation vector of each column of X, calculated as formula (1): (1) Where, represents the mean of the jth feature, M represents the sample size, represents the jth eigenvalue of the i-th sample, represents the standard deviation of the jth feature, and N represents the feature quantity.
[0035] Calculate the covariance matrix C of the standardized matrix Z according to formula (2): (2) Next, calculate the eigenvalues of the covariance matrix C and the corresponding eigenvectors ,satisfy: , arrange the eigenvectors in descending order of eigenvalues and construct an orthogonal matrix The size of the eigenvalue represents the amount of information contained in the corresponding principal component. The larger the eigenvalue, the more important the information contained in the corresponding principal component.
[0036] Further select the first preset number k principal components in the orthogonal matrix, and the corresponding projection matrix for: , construct the dimension reduction matrix based on the normalization matrix and projection matrix , as shown in formula (3): (3) And the reduced dimension matrix As a result of dimensionality reduction, the data is reduced in dimension, the main information components in the data are retained, the data dimension is reduced, the amount of calculation and storage space are reduced, and at the same time, it helps to improve the training speed and performance of the model.
[0037] Furthermore, using the results of principal component dimensionality reduction Train the KMeans model. First, by calculating the variance and SSE (Sum of Squared Errors) under different cluster numbers K, we determine the second preset number K, which is the optimal number of clusters. Initially, the K value range is set between 1-g (g ≥ 1). Generally, multiple K values are tried, such as starting from K = 1 and gradually increasing. The KMeans model is used to cluster different K values, and then the corresponding SSE is calculated according to formula (4): (4) Where, represents the jth sample, represents the centroid of the i-th cluster.
[0038] The method for determining the optimal K value is: when the optimal number of clusters appears, SSE will drop significantly, and a clear inflection point will appear in the visual line graph with SSE as the vertical axis and the number of clusters K as the horizontal axis. In this embodiment, the optimal number of clusters is determined to be 3 by the elbow method. Figure 2 As shown in the figure, an inflection point appears when K=3, indicating that the clustering effect is better at this time. It can not only reasonably divide the data into different clusters, but also ensure that the samples within the cluster have high similarity and the samples between clusters have large differences.
[0039] After the K value is determined, K samples are randomly selected from the sample size as multiple initial centroids and the iteration begins. The iterative process is: ①For each sample in the sample size The distance to each initial centroid is compared by formula (5) and the sample Assigned to the nearest centroid , forming clusters In this way, all samples will be divided into the cluster that is most similar to them (closest distance). Formula (5) is as follows: (5) Where, represents the lth centroid.
[0040] ② Recalculate the new centroids of each cluster , the calculation method is to take the mean of all samples in the cluster, as shown in formula (6): (6) Where, Show cluster The sample size of .
[0041] The initial centroids Replaced with the new centroids of each cluster , repeat the above steps ①-② until the preset maximum number of iterations f is met. The value of f can be set according to actual conditions and experience, and this embodiment does not limit this.
[0042] Preferably, in this embodiment, under the condition of determining the optimal number of clusters, the silhouette coefficient can be further used to verify and compare the trained KMeans model, and it is found that the silhouette coefficient of the model performs well, which further determines that the model is the optimal model. The silhouette coefficient is an indicator that combines the cohesion within the cluster and the separation between clusters, and is used to evaluate the clustering effect. Its core idea is: the closer the sample is to its own cluster, the better, and the farther it is separated from other clusters, the better. Silhouette coefficient The calculation method is as follows: (7) Among them, N is the sample size, , belongs to cluster C, and its silhouette coefficient The calculation method is: , It is a sample The average distance to other samples in the same cluster (intra-cluster cohesion), ; It is a sample The average distance to all samples in the nearest other cluster (inter-cluster separation), The value range of the silhouette coefficient is between [-1, 1]. The larger the value, the better the clustering effect, that is, the closer the sample is to its own cluster and the farther it is separated from other clusters.
[0043] In a preferred embodiment, KMeans classification is used to achieve Figure 3 By observing the effect Figure 3In the KMeans classification scatter plot, the samples are divided into three clusters (Cluster 1, Cluster 2, and Cluster 3). The classification profiles for the training set (left) and the test set (right) are similar, indicating that the model performs consistently and stably across different datasets, indicating good generalization and stability. Based on the clustering results of the optimal KMeans model, the corresponding categories for different samples are recorded.
[0044] The above clustering results can divide samples into different cluster categories and discover the potential structure and patterns in the data, which will be used for subsequent analysis, such as further exploring the differences in various characteristics of samples of different categories.
[0045] Preferably, in this embodiment, after the clustering results are obtained, they can be displayed using a radar chart. The radar chart visualization can intuitively show the differences in various features of different cluster categories, making it easier to compare and analyze the features of samples of different categories. Specifically, first, a corresponding score value is assigned to each feature in the clustering result according to a preset rule. For example, the feature "yes" is regarded as 1 and "no" is regarded as 0. Then, the feature weighted scores of different clusters in the clustering result are calculated according to formula (8): (8) in, represents the weighted score of the kth class on feature X, Indicates the total number of samples of the Kth class. In feature X, there are different score values (In order from smallest to largest). The number of samples for each score value is , and satisfy . The higher the score, the more samples with feature X are in cluster K.
[0046] According to the weighted scores of the features of the above different clusters, a radar chart is drawn, such as Figure 4 As shown in , each dimension represents a feature, and the weighted scores of different clustering categories on each feature dimension are presented in the form of different lines or areas. Figure 4 It can be clearly observed that the representative features of different categories are different. For example, a category may have a higher weighted score on some features and a lower score on other features, thus intuitively presenting the characteristics and differences of samples in each category.
[0047] The above method intuitively displays the differences in various features of different clustering categories through radar charts, which facilitates the comparison and analysis of the features of samples of different categories and helps to discover patterns and characteristics in the data.
[0048] Step S130: Using the training set to train the XGBoost model, and using the trained XGBoost model to verify the clustering result, to obtain the optimal XGBoost model.
[0049] Understandably, we first build the XGBoost model and set its initial parameters, including but not limited to the maximum depth, learning rate, number of trees, etc. Then we define the hyperparameter search space, including: ① Maximum depth max_depth, the range is set to [3, 10]. This parameter controls the maximum depth of the decision tree. A larger max_depth will make the model more complex and prone to overfitting, while a smaller one may underfit. It is necessary to find the optimal value within a reasonable range.
[0050] ② The learning rate learning_rate is set in the range of [0.01, 0.3]. It determines the magnitude of the weight update at each iteration of the model. If the learning rate is too small, the training speed will be very slow, and if it is too large, it may not converge.
[0051] ③ The number of trees n_estimators takes a value of [50, 100, 200], which indicates the total number of decision trees constructed. More trees may improve model performance, but also increase computational cost and overfitting risk.
[0052] ④ The subsample ratio is set in the range of [0.5, 1.0], which is the ratio of data samples sampled each time a decision tree is built, to prevent overfitting and speed up training.
[0053] ⑤ Feature sampling ratio colsample_bytree, set in the range of [0.5, 1.0], refers to the ratio of feature sampling when building each decision tree. Reasonable setting helps reduce overfitting and improve training efficiency.
[0054] ⑥ Pruning parameter gamma, set in the range of [0, 5], is used to control whether pruning is performed. The larger the gamma value, the stricter the pruning is to prevent overfitting.
[0055] Bayesian optimization is a hyperparameter optimization method based on Bayes' theorem and probabilistic models. It constructs a surrogate model of the objective function (such as Gaussian process regression) and uses historical evaluation results to predict the hyperparameter combination most likely to achieve better results for the objective function (in this case, model performance metrics such as accuracy and F1 score). This allows for a more efficient search for optimal hyperparameters in the hyperparameter search space. Based on the multiple optimal hyperparameter combinations obtained by Bayesian optimization, multiple optimized XGBoost models are rebuilt using these optimal hyperparameter combinations.
[0056] The optimized XGBoost model, after re-parameterizing, is trained on the training set to learn the patterns and regularities in the data. After training, the trained optimized XGBoost model is evaluated on the test set, calculating metrics such as precision, recall, and F1 score. These metrics measure model performance from different perspectives. Accuracy reflects the proportion of correct predictions made by the model; recall focuses on the model's ability to identify positive examples; and the F1 score is the harmonic mean of precision and recall, taking both into account.
[0057] By comprehensively comparing the results of various evaluation indicators such as accuracy, recall rate, F1 score, etc., the optimal XGBoost model is determined and output as the final model. This final model can be used for subsequent actual prediction and analysis work. At the same time, its performance can also reflect the rationality of KMeans classification to a certain extent. If the XGBoost model can achieve good results on the test set, it means that the category labels obtained by KMeans clustering have certain reliability and rationality.
[0058] By building an XGBoost model and optimizing its hyperparameters, this method improves the model's predictive accuracy and generalization capabilities, enabling it to better fit the data and predict the categories of new samples. By validating the clustering results using the trained XGBoost model, we can assess their rationality and reliability, thereby verifying the correctness of the KMeans clustering.
[0059] Step S140 , using SHAP value theory to interpret the optimal XGBoost model, and obtaining a SHAP feature interpretation result to verify the feature difference of the radar chart.
[0060] Based on the optimal XGBoost model, the SHAP (SHapley Additive exPlanations) value theory is used to visualize the optimal XGBoost model to reflect the impact of features on different categories, thereby improving the interpretability of the model and enabling people to understand how the model makes classification decisions based on various features.
[0061] Specifically, features are categorized as positive or negative based on their impact on the model's predictions, i.e., the sign of the SHAP value. A positive feature is one in which the larger the value of a feature is for a particular category, the more likely the model is to classify the sample as belonging to that category. For example, the larger the value of feature 1, the more likely the model is to classify the sample as belonging to category 1. Therefore, for category 1, feature 1 is a positive feature. The opposite is true for a negative feature. For example, the smaller the value of feature 2, the more likely the model is to classify the sample as belonging to category 2. Therefore, for category 2, feature 2 is a negative feature.
[0062] The SHAP value assigns a value to each feature to measure the contribution of the feature to the model prediction result. samples , the first The feature is , the model samples The predicted value is , the baseline prediction of the model is , then the calculation of SHAP value obeys formula (9): (9) Where, The SHAP value represents the contribution of the j-th feature to the prediction result. The size of the SHAP value reflects the role played by the feature in the model prediction, and the positive and negative signs indicate the direction of the feature's influence on the prediction result.
[0063] Using the visualization tools provided by SHAP, we can plot a visual graph of the features for each category. In this graph, we can see the influence of each feature's value on the tendency of the sample to be predicted as the corresponding category, thus helping people to gain a deeper understanding of the model's decision logic and the importance of each feature.
[0064] The above method can intuitively reflect the impact of features in the radar chart on different categories through SHAP interpretation, improve the interpretability of the model, enable people to understand how the model makes classification decisions based on various features, help discover important features and key factors in the model, and provide a basis for further analysis and decision-making.
[0065] The feature detection method provided in the embodiment of the present application collects initial feature data of the supervised object, preprocesses the initial feature data, obtains preprocessed optimized feature data, and divides the optimized feature data into a training set and a test set according to a preset ratio, wherein the initial feature data includes supervisory feature data, demographic feature data, psychological feature data, native family feature data and educational feature data; performs principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, uses the dimensionality reduction result to train a KMeans model, uses the trained KMeans model to cluster the test set to obtain a clustering result, and displays the clustering result through a radar chart; uses the training set to train an XGBoost model, and uses the trained XGBoost model to verify the clustering result to obtain an optimal XGBoost model; uses the SHAP value theory to interpret the optimal XGBoost model to obtain a feature impact result. This application can use machine learning models to analyze past data and then classify and identify the characteristics of regulatory objects, providing relevant personnel and institutions with a basis for the classification and supervision of regulatory objects, and demonstrating the characteristics of different categories of regulatory objects. It also provides a scientific basis for the formulation of intervention measures, avoids the limitations of subjective classification, improves processing capabilities for problems with many features and large amounts of data, and has the advantages of objectivity, scientificity, and strong interpretability.
[0066] Example 2 like Figure 5 FIG. 5 is a schematic diagram of a feature detection device 500 according to an embodiment of the present application, wherein the device includes: A preprocessing module 510 is configured to collect initial feature data of the supervised subject, preprocess the initial feature data to obtain optimized feature data after preprocessing, and divide the optimized feature data into a training set and a test set according to a preset ratio, wherein the initial feature data includes supervisory feature data, demographic feature data, psychological feature data, original family feature data, and educational feature data; A clustering module 520 is configured to perform principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, train a KMeans model using the dimensionality reduction result, cluster the test set using the trained KMeans model to obtain a clustering result, and display the characteristic differences of different clusters in the clustering result using a radar chart; A verification module 530 is configured to train an XGBoost model using the training set, and verify the clustering results using the trained XGBoost model to obtain an optimal XGBoost model; The interpretation module 540 is used to interpret the optimal XGBoost model using SHAP value theory to obtain a SHAP feature interpretation result to verify the feature difference of the radar chart.
[0067] The feature detection device provided in the embodiment of the present application can implement each process of the feature detection method corresponding to Example 1 and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0068] The feature detection device provided in the embodiment of the present application can use a machine learning model to analyze past data and then classify and identify the features of the regulatory objects, provide relevant personnel and institutions with a basis for the classification and supervision of the regulatory objects, and display the characteristics of different categories of regulatory objects. It also provides a scientific basis for the formulation of intervention measures, avoids the limitations of subjective classification, improves the processing capability for problems with many features and large amounts of data, and has the advantages of objectivity, scientificity, and strong interpretability.
[0069] Example 3 The present application also provides a computer device. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.
[0070] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.
[0071] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.
[0072] The memory 61 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or D slot compatibility test memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as the hard disk or internal memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. Of course, the memory 61 may also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions for the slot compatibility test method. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.
[0073] In some embodiments, the processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other feature detection chip. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to execute computer-readable instructions or process data stored in the memory 61, such as computer-readable instructions for executing the slot compatibility testing method.
[0074] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.
[0075] The computer device provided in this embodiment can execute the above-mentioned feature detection method, which can be the feature detection method of each of the above-mentioned embodiments.
[0076] Example 4 This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the feature detection method in the embodiment are implemented.
[0077] In this embodiment, the computer-readable storage medium includes flash memory, hard disks, multimedia cards, card-type memories (e.g., SD or DX memories), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk equipped with the computer device, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Of course, the computer-readable storage medium may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are about to be output.
[0078] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.
[0079] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
[0080] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium can be a non-volatile storage medium or a volatile storage medium. For example, the storage medium can be: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.
[0081] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.
Claims
1. A feature detection method, characterized in that: The method comprises: Collecting initial feature data of the supervised subject, preprocessing the initial feature data to obtain preprocessed optimized feature data, and dividing the optimized feature data into a training set and a test set according to a preset ratio, wherein the initial feature data includes supervisory feature data, demographic feature data, psychological feature data, original family feature data, and educational feature data; Performing principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, using the dimensionality reduction result to train a KMeans model, using the trained KMeans model to cluster the test set to obtain a clustering result, and displaying the characteristic differences of different clusters in the clustering result through a radar chart; Using the training set to train the XGBoost model, and using the trained XGBoost model to verify the clustering results, to obtain the optimal XGBoost model; The optimal XGBoost model is interpreted using SHAP value theory to obtain SHAP feature interpretation results to verify the feature differences of the radar chart.
2. The feature detection method according to claim 1, wherein: The preprocessing of the initial feature data to obtain preprocessed optimized feature data includes: Determining unstructured feature data in the initial feature data, and converting the unstructured feature data using a binarization method or a dummy variable method to obtain structured feature data; Determining categorical feature data and numerical feature data in the structured feature data, interpolating missing values in the categorical feature data using a Random Forest method, and interpolating missing values in the numerical feature data using a mode, median, or mean, to obtain interpolated structured feature data; The interpolated structured feature data is pre-classified using the K-Means algorithm to obtain a pre-classification result, and a chi-square test is performed on the pre-classification result to obtain a test result. Data with a test result less than a preset test threshold is used as the pre-processed optimized feature data.
3. The feature detection method according to claim 1, wherein: The performing principal component dimensionality reduction on the training set to obtain a dimensionality reduction result includes: Determining the sample size and feature quantity of the training set, forming an initial matrix according to the sample size and the feature quantity, and performing standardization processing on the initial matrix to obtain a standardized matrix; Calculating the covariance matrix of the standardized matrix, and calculating the eigenvalues and corresponding eigenvectors of the covariance matrix, arranging the eigenvectors in descending order according to the eigenvalues, and constructing an orthogonal matrix; Determine the projection matrices corresponding to the first preset number of principal components in the orthogonal matrix, construct a dimensionality reduction matrix according to the normalization matrix and the projection matrix, and use the dimensionality reduction matrix as the dimensionality reduction result.
4. The feature detection method according to claim 3, wherein: The method of training a KMeans model using the dimensionality reduction result and clustering the test set using the trained KMeans model to obtain a clustering result includes: randomly selecting a second preset number of samples from the sample volume as a plurality of initial centroids using an elbow method, calculating a distance from each of the samples in the sample volume to each of the initial centroids, and allocating each of the samples according to the distance to obtain a plurality of clusters; The centroid of each cluster is calculated, and each initial centroid is replaced by the centroid of each cluster, and the calculation is repeated until a preset maximum number of iterations is met to obtain the clustering result, which includes multiple sample categories and their corresponding features.
5. The feature detection method according to claim 4, characterized in that: The radar chart displays the characteristic differences of different clusters in the clustering results, including: Assigning a corresponding score value to each feature in the clustering result according to a preset rule, and determining the number of samples for each score value; Calculating the feature weighted scores of different clusters in the clustering results according to the score values of the features and the number of samples of the score values; The radar chart is drawn according to the feature weighted scores of different clusters in the clustering result.
6. The feature detection method according to claim 1, wherein: The method of training the XGBoost model using the training set and verifying the clustering results using the trained XGBoost model to obtain the optimal XGBoost model includes: Constructing the XGBoost model, setting initial parameters of the XGBoost model, optimizing the initial parameters using a Bayesian algorithm to obtain multiple optimal hyperparameter combinations, and reconstructing multiple optimized XGBoost models using each of the optimal hyperparameter combinations; Training each of the optimized XGBoost models on the training set to obtain a plurality of trained optimized XGBoost models; Each of the trained optimized XGBoost models is evaluated on the test set to obtain an evaluation index of each of the trained optimized XGBoost models, and the optimal XGBoost model is determined by comparing the evaluation index of each of the trained optimized XGBoost models.
7. The feature detection method according to claim 5, wherein: The optimal XGBoost model is interpreted using the SHAP value theory to obtain a SHAP feature interpretation result, including: Based on the optimal XGBoost model, the SHAP value of each feature is calculated using the SHAP value theory, where the SHAP value represents the degree of influence of each feature on each type of sample; According to the positive or negative SHAP value, each feature is divided into a positive feature and a negative feature, and a visualization chart is generated using the SHAP value of each positive feature and the negative feature, and the SHAP feature interpretation result is obtained through the visualization chart.
8. A feature detection device, characterized in that: For executing the feature detection method according to any one of claims 1 to 7, the device comprises: A preprocessing module is used to collect initial feature data of the supervised object, preprocess the initial feature data to obtain optimized feature data after preprocessing, and divide the optimized feature data into a training set and a test set according to a preset ratio; A clustering module is used to perform principal component dimensionality reduction on the training set to obtain a dimensionality reduction result, train a KMeans model using the dimensionality reduction result, cluster the test set using the trained KMeans model to obtain a clustering result, and display the characteristic differences of different clusters in the clustering result through a radar chart; A verification module is used to train the XGBoost model using the training set, and verify the clustering results using the trained XGBoost model to obtain the optimal XGBoost model; The interpretation module is used to interpret the optimal XGBoost model using SHAP value theory to obtain SHAP feature interpretation results to verify the feature differences of the radar chart.
9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the feature detection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the feature detection method according to any one of claims 1 to 7 are implemented.