Cluster prediction model training method and cluster prediction method

Abstract

The application relates to the technical field of big data processing, and relates to a cluster prediction model training method, a cluster prediction method, a cluster prediction device, computer equipment, a storage medium and a computer program product. The cluster prediction model training method comprises the following steps: acquiring multiple groups of reference data; encoding the multiple groups of reference data according to cluster labels to obtain multiple reference variables; establishing a correlation coefficient of the reference variables and all cluster labels; screening reference data that meets a preset condition as a training sample of a cluster prediction model according to the correlation coefficient; and training the cluster prediction model according to the training sample to obtain a trained cluster prediction model. The method can parameterize the reference data, and screen the data according to the correlation coefficient, so that the cluster prediction classification result of the finally trained cluster prediction model is more accurate and has higher robustness.

Description

Technical Field

[0001] This application relates to the field of big data processing technology, and in particular to a cluster prediction model training method, cluster prediction method, apparatus, computer equipment, storage medium and computer program product. Background Technology

[0002] Clustering, or information labeling, involves grouping reference data with similar attributes into a single group for subsequent analysis. The principle behind clustering is to divide different reference data based on historical data, categorize reference data exhibiting certain patterns, and then observe the specific characteristics of each cluster. Clustering the reference data makes the differences in data characteristics between different clusters more apparent, while ensuring that the data characteristics of reference data within the same group are similar.

[0003] Typically, the classification of large amounts of reference data is mostly done by business personnel based on experience, which incorporates many subjective judgments, resulting in significant deviations in the final classification results. If traditional models such as RFM are used to simply divide the reference data, there will also be situations where there are few evaluation conditions for the reference data and the accuracy of the results is not high. Existing reference data classification methods cannot uncover deeper information within the data, and therefore the accuracy of cluster division in existing technologies is also low. Summary of the Invention

[0004] Therefore, it is necessary to provide a cluster prediction model training method, cluster prediction method, device, computer equipment, storage medium, and computer program product that can improve the accuracy of cluster partitioning, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for training a cluster prediction model, including:

[0006] Multiple sets of reference data are obtained; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label, which is used to identify the cluster type.

[0007] Based on the cluster label, the multiple sets of reference data are encoded to obtain multiple reference variables;

[0008] Establish the correlation coefficients between the reference variable and all cluster labels;

[0009] Based on the correlation coefficient, reference data that meets the preset conditions are selected as training samples for the cluster prediction model.

[0010] Based on the training samples, the cluster prediction model is trained to obtain the trained cluster prediction model.

[0011] In one embodiment, the step of encoding the multiple sets of reference data according to the cluster label to obtain multiple reference variables includes:

[0012] The number of parameter bits is determined based on the type of cluster label, and the number of parameter bits matches the number of types.

[0013] For each set of reference data, according to the preset cluster label encoding rules, the encoding position corresponding to the cluster label carried by the reference data is encoded as a first preset value to obtain the reference variable corresponding to the reference data. The cluster label encoding rules include the association between the cluster label and the encoding position.

[0014] In one embodiment, establishing the correlation coefficient between the reference variable and all cluster labels includes:

[0015] According to the preset partitioning rules, all the reference variables are divided into multiple sets;

[0016] Calculate the relationship between the reference variable in each set and multiple preset values, and use the result as the correlation coefficient between the reference variable in the current set and all cluster labels.

[0017] In one embodiment, the step of selecting reference data that meets preset conditions based on the correlation coefficient as training samples for the cluster prediction model includes:

[0018] Determine the correlation coefficient threshold based on the correlation coefficient;

[0019] The reference data whose corresponding correlation coefficient reaches the correlation coefficient threshold are used as the training samples.

[0020] In one embodiment, training the cluster prediction model based on the training samples to obtain the trained fault diagnosis model includes:

[0021] The training samples are input into the cluster prediction model to obtain the predicted cluster classification results;

[0022] Based on the predicted cluster classification results, determine the model loss of the cluster prediction model;

[0023] The model parameters of the cluster prediction model are adjusted based on the model loss, and training continues until the cluster prediction model converges, resulting in a trained cluster prediction model.

[0024] In one embodiment, adjusting the model parameters of the cluster prediction model based on the model loss and continuing training until the cluster prediction model converges to obtain a trained cluster prediction model includes:

[0025] Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding training samples, the model parameters of the cluster prediction model are adjusted until training is completed and a trained cluster prediction model is obtained.

[0026] Secondly, this application also provides a cluster prediction method, including:

[0027] Collect reference data and input the reference data into the cluster prediction model trained using the cluster prediction model training method described in any of the above embodiments to obtain the cluster type corresponding to the reference data.

[0028] Thirdly, this application also provides a cluster prediction model training device, comprising:

[0029] The acquisition module is used to acquire multiple sets of reference data; wherein each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label, the cluster label being used to identify the cluster type;

[0030] The first processing module is used to encode the multiple sets of reference data according to the cluster label to obtain multiple reference variables;

[0031] The second processing module is used to establish the correlation coefficients between the reference variable and all cluster labels;

[0032] The filtering module is used to filter out reference data that meets preset conditions based on the correlation coefficient, and use them as training samples for the cluster prediction model.

[0033] The model generation module is used to train the cluster prediction model based on the training samples to obtain the trained cluster prediction model.

[0034] In one embodiment, the first processing module is further configured to:

[0035] The number of parameter bits is determined based on the type of cluster label, and the number of parameter bits matches the number of types.

[0036] For each set of reference data, according to the preset cluster label encoding rules, the encoding position corresponding to the cluster label carried by the reference data is encoded as a first preset value to obtain the reference variable corresponding to the reference data. The cluster label encoding rules include the association between the cluster label and the encoding position.

[0037] In one embodiment, the second processing module is further configured to:

[0038] According to the preset partitioning rules, all the reference variables are divided into multiple sets;

[0039] Calculate the relationship between the reference variable in each set and multiple preset values, and use the result as the correlation coefficient between the reference variable in the current set and all cluster labels.

[0040] In one embodiment, the filtering module is further configured to:

[0041] Determine the correlation coefficient threshold based on the correlation coefficient;

[0042] The reference data whose corresponding correlation coefficient reaches the correlation coefficient threshold are used as the training samples.

[0043] In one embodiment, the model generation module is further configured to:

[0044] The training samples are input into the cluster prediction model to obtain the predicted cluster classification results;

[0045] Based on the predicted cluster classification results, determine the model loss of the cluster prediction model;

[0046] The model parameters of the cluster prediction model are adjusted based on the model loss, and training continues until the cluster prediction model converges, resulting in a trained cluster prediction model.

[0047] In one embodiment, the model generation module is further configured to:

[0048] Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding training samples, the model parameters of the cluster prediction model are adjusted until training is completed and a trained cluster prediction model is obtained.

[0049] Fourthly, this application also provides a cluster prediction device, comprising:

[0050] The data acquisition module is used to collect reference data.

[0051] The prediction module is used to input the reference data into the cluster prediction model trained by the cluster prediction model training method described in any of the above embodiments, and to obtain the cluster type corresponding to the reference data.

[0052] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the cluster prediction model training method or cluster prediction method described in any of the above embodiments.

[0053] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the cluster prediction model training method or cluster prediction method described in any of the above embodiments.

[0054] Seventhly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the cluster prediction model training method or cluster prediction method described in any of the above embodiments.

[0055] The aforementioned cluster prediction model training method, cluster prediction method, device, computer equipment, storage medium, and computer program product parameterize each set of reference data into reference variables, enabling reference data with multiple differences to be converted into numbers according to unified processing rules. This makes it easier to filter and eliminate reference data. Furthermore, the correlation coefficient algorithm is used to calculate the correlation coefficient between the reference variables in each set and multiple pre-set values, and reference data with low correlation coefficients and high correlation are eliminated, while reference data with high correlation coefficients are retained as training samples. This completes the cleaning and filtering of model training sample data. Furthermore, the filtered reference data is used to train the cluster prediction model, making the cluster prediction classification results of the finally trained cluster prediction model more accurate and robust. Attached Figure Description

[0056] Figure 1 This is a diagram illustrating the application environment of a cluster prediction model training method in one embodiment.

[0057] Figure 2 This is a flowchart illustrating the cluster prediction model training method in the first embodiment;

[0058] Figure 3 This is a flowchart illustrating the cluster prediction model training method in the second embodiment;

[0059] Figure 4 This is a flowchart illustrating the cluster prediction model training method in the third embodiment;

[0060] Figure 5 This is a flowchart illustrating the cluster prediction model training method in the fourth embodiment;

[0061] Figure 6 This is a flowchart illustrating the cluster prediction model training method in the fifth embodiment;

[0062] Figure 7 This is a flowchart illustrating the cluster prediction model training method in the sixth embodiment;

[0063] Figure 8 This is a flowchart illustrating a cluster prediction method in one embodiment;

[0064] Figure 9 This is a structural block diagram of a cluster prediction model training device in one embodiment;

[0065] Figure 10 This is a structural block diagram of a cluster prediction device in one embodiment;

[0066] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] The cluster prediction model training method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown.

[0069] For example, the cluster prediction model training method is applied to terminal 102. Terminal 102 acquires multiple sets of reference data; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label, which is used to identify the cluster type. Terminal 102 encodes the multiple sets of reference data according to the cluster labels to obtain multiple reference variables; then, for each of the multiple reference variables, it establishes the correlation coefficient between the current reference variable and all cluster labels; and based on the correlation coefficient, it selects reference data that meets preset conditions as training samples for the cluster prediction model; finally, terminal 102 trains the cluster prediction model based on the training samples to obtain the trained cluster prediction model. The trained cluster prediction model is then stored in memory. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers. Terminal 102 and server 104 can be connected directly or indirectly via wired or wireless communication, such as through a network connection.

[0070] For example, the cluster prediction model training method is applied to server 104. Server 104 acquires multiple sets of reference data; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label, which is used to identify the cluster type. Server 104 encodes the multiple sets of reference data according to the cluster labels to obtain multiple reference variables; then, for each of the multiple reference variables, it establishes the correlation coefficient between the current reference variable and all cluster labels; and based on the correlation coefficient, it selects reference data that meets preset conditions as training samples for the cluster prediction model; finally, server 104 trains the cluster prediction model based on the training samples to obtain the trained cluster prediction model. The trained cluster prediction model is then stored in memory. It can be understood that the memory can be an independent storage device, or the memory can be located on the server, or the memory can be located on another terminal.

[0071] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. In one embodiment, a cluster prediction model training method is provided. This embodiment uses the application of this cluster prediction model training method to a processor as an example for illustration. It can be understood that the processor can be located on a terminal or a server. Figure 2 As shown, the training method for this cluster prediction model includes:

[0072] Step 202: Obtain multiple sets of reference data; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label, which is used to identify the cluster type.

[0073] Reference data can be the operation data of the target object. For example, reference data can be login data, resource interaction data, resource size data, protocol data, etc.

[0074] Cluster labels can consist of at least one of letters, characters, or numbers. A cluster label is used to uniquely identify a cluster type. Cluster labels can be automatically labeled by the processor according to pre-set key information judgment rules, or they can be labeled manually. Key information judgment rules can be key fields corresponding to different cluster types that are manually entered. When a set of reference data contains a key field corresponding to a certain cluster type, the processor marks the corresponding cluster label on the reference data.

[0075] The types of cluster labels depend on the types of clusters. Cluster types can be freely defined. For example, when all target objects are divided into four cluster types—Type 1, Type 2, Type 3, and Type 4—there are four types of cluster labels on all the corresponding reference data to identify these four cluster types. The importance of each cluster type differs, for example, decreasing sequentially: data in Type 1 clusters is the most important, and data in Type 4 clusters is the least important.

[0076] In this embodiment, the processor first collects multiple sets of reference data and extracts the cluster label carried on each set of reference data, thereby obtaining the cluster type corresponding to each set of reference data.

[0077] Step 204: Encode multiple sets of reference data according to the cluster labels to obtain multiple reference variables.

[0078] As an example, the processor divides all reference data into multiple sets based on the cluster label carried by each set of reference data. Each set corresponds to a cluster label. Then, the processor encodes the reference data in each set in turn to obtain the reference variable corresponding to each set of reference data.

[0079] Reference variables can consist of at least one of letter strings, number strings, and code strings. Reference variables are used to parameterize reference data so that reference data containing different content can be transformed into a format with the same pattern.

[0080] In this embodiment, the processor encodes the reference data according to the cluster label carried by each set of reference data to obtain the reference variable corresponding to each set of reference data. It can also be understood that each reference variable has a corresponding cluster label, which means that each reference variable has a corresponding cluster type.

[0081] Step 206: For each of the multiple reference variables, establish the correlation coefficient between the current reference variable and all cluster labels.

[0082] As an example, when the reference variable consists of numbers, the processor in this embodiment can pre-set different random numbers for different cluster types. The random numbers corresponding to all cluster types can be continuous variables, that is, different cluster labels correspond to different random numbers, and all random numbers are continuous. The processor can use one of the Pearson correlation coefficient algorithm, Spearman correlation coefficient algorithm, or Kendall correlation coefficient algorithm to establish the correlation coefficient between each reference variable and all random numbers.

[0083] In this embodiment, the processor processes each reference variable to obtain its correlation coefficient with all cluster labels. This correlation coefficient is used to identify the correlation between different reference variables and all cluster types.

[0084] Step 208: Based on the correlation coefficient, select reference data that meet the preset conditions as training samples for the cluster prediction model.

[0085] The preset condition can be a numerical threshold for the correlation coefficient. If a correlation coefficient exceeds the corresponding numerical threshold, it is considered to meet the preset condition. All reference data corresponding to the correlation coefficients that exceed the numerical threshold are selected to form training samples.

[0086] In this embodiment, the processor sets a preset condition in advance and matches each correlation coefficient obtained in step 206 with the preset condition to determine whether each correlation coefficient meets the preset condition. If it does, the reference data corresponding to the correlation coefficients that meet the preset condition are selected to form a set of training samples. If not, the reference data corresponding to the correlation coefficients that do not meet the preset condition are removed to improve the accuracy of the training samples of the cluster prediction model and thus improve the training accuracy of the cluster prediction model.

[0087] Step 210: Train the cluster prediction model based on the training samples to obtain the trained cluster prediction model.

[0088] The cluster prediction model can be trained by calling at least one of the following machine learning algorithms: decision tree, neural network, XGboost, and LightGBM. The cluster prediction model can be one of the following: deep neural network model, convolutional neural network model, and long short-term memory neural network model. This embodiment does not limit the specific type of model used in the cluster prediction model.

[0089] As an example, the processor randomly divides all training samples in a 7:3 ratio, using 70% of the training samples as the training set and 30% as the test set. The processor inputs the training set into the cluster prediction model to obtain a trained cluster prediction model. Then, it inputs the test set into the trained cluster prediction model to obtain the output predicted cluster partitioning results. The processor compares the predicted cluster partitioning result corresponding to each training sample with the cluster label corresponding to that training sample, calculates the prediction accuracy corresponding to the test set, and determines whether the prediction accuracy reaches the processor's preset accuracy threshold. If it does, the final trained cluster prediction model is obtained; if not, the cluster prediction model is retrained using the training set until the prediction accuracy of the cluster prediction model reaches the accuracy threshold.

[0090] In this embodiment, the processor randomly divides the training samples into a training set and a test set. First, the cluster prediction model is trained using the training set. Then, the test set is input into the trained cluster prediction model to obtain the predicted cluster classification result corresponding to each training sample in the test set. The prediction accuracy is calculated based on the predicted cluster classification result. When the prediction accuracy reaches the accuracy threshold preset by the processor, the cluster prediction model at this time is taken as the cluster prediction model that has been trained.

[0091] In the above-described cluster prediction model training method, the processor first retrieves operation data corresponding to multiple target objects from memory, generating multiple sets of reference data carrying cluster labels. These cluster labels identify the cluster type corresponding to the current set of reference data. Then, all reference data are encoded into multiple corresponding reference variables, and the correlation coefficient between each reference variable and all cluster labels is calculated. Reference data with high correlation coefficients are selected to train the cluster prediction model, resulting in the trained cluster prediction model. This method parameterizes different reference data, calculates correlation coefficients, and only uses reference data with high correlation coefficients to train the cluster prediction model. This makes the final trained cluster prediction model more accurate and robust in its cluster prediction classification results.

[0092] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. For example... Figure 3 As shown, in some possible embodiments, step 204 includes:

[0093] Step 2042: Determine the number of parameter bits based on the type of cluster label, ensuring that the number of parameter bits matches the number of cluster label types;

[0094] Step 2044: For each set of reference data, according to the preset cluster label encoding rules, the encoding position corresponding to the cluster label carried by the reference data is encoded as the first preset value to obtain the reference variable corresponding to the reference data. The cluster label encoding rules include the association between the cluster label and the encoding position.

[0095] Specifically, in this embodiment, the processor encodes the reference data into reference variables composed of numbers, where the parameter bit length refers to the total number of bits in the reference variable.

[0096] As an example, when all target objects are divided into four cluster types—first type, second type, third type, and fourth type—there are four cluster labels on all the corresponding reference data. In step 2042, the parameter bit length is matched with the type of cluster label to obtain a parameter bit length of 4.

[0097] Furthermore, the processor iterates through all the reference data and, according to the cluster label encoding rules, encodes the parameter positions corresponding to the cluster labels carried by each set of reference data into a first preset value, and encodes the parameter positions corresponding to the cluster labels not carried into a second preset value, thereby obtaining the reference variables corresponding to each set of reference data.

[0098] As an example, the cluster label encoding rule pre-associates cluster labels for four cluster types—Type 1, Type 2, Type 3, and Type 4—to the first, second, third, and fourth bits of the parameter bit depth. When the processor encounters reference data carrying a cluster label for Type 1 clusters, it encodes the first parameter associated with the cluster label for Type 1 clusters as 1, and encodes the second, third, and fourth parameters as 0, thus obtaining the reference variable 1000 corresponding to the current reference data.

[0099] In this embodiment, the processor parameterizes each group of reference data into reference data according to the cluster label encoding rules, so that reference data with various differences can be converted into numbers according to a unified processing rule. This makes it easier to filter and remove reference data.

[0100] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. For example... Figure 4 As shown, in some possible embodiments, step 206 includes:

[0101] Step 2062: Divide all reference variables into multiple sets according to the preset partitioning rules.

[0102] Step 2064: Calculate the relationship between the reference variable in each set and multiple preset values, and use it as the correlation coefficient between the reference variable in the current set and all cluster labels.

[0103] Specifically, the correlation coefficient can be one of Spearman's rank correlation coefficient, Pearson's correlation coefficient, or Kendall's correlation coefficient.

[0104] As an example, the processor first divides all reference variables into four sets according to the criteria of first-class clusters, second-class clusters, third-class clusters and fourth-class clusters, and then sorts the reference variables in each set.

[0105] Furthermore, for each set, multiple pre-set values ​​are subtracted from the sorted reference variables to obtain multiple differences that form the difference set corresponding to the current set. The pre-set values ​​are, for example, multiple natural numbers that increase or decrease.

[0106] Furthermore, based on the difference sets, the Spearman rank correlation coefficient between the reference variable and the setpoint in each set is calculated using the following formula, and is used as the correlation coefficient:

[0107]

[0108] Where, ρ i d represents the Spearman rank correlation coefficient for the reference variable in the i-th set, where i ∈ N, N represents the number of sets, and d j Let j represent the j-th difference, and n represent the total number of differences in the difference set corresponding to the current set.

[0109] In this embodiment, the processor divides the reference variables corresponding to all reference data into multiple sets according to the cluster type corresponding to each set of reference data, and uses the Spearman rank correlation coefficient algorithm to calculate the correlation coefficient between the reference variables in each set and multiple preset values.

[0110] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. For example... Figure 5 As shown, in some possible embodiments, step 208 includes:

[0111] Step 2082: Determine the correlation coefficient threshold based on the correlation coefficient;

[0112] Step 2084: Use the reference data whose corresponding correlation coefficient reaches the correlation coefficient threshold as training samples.

[0113] As an example, the processor calculates the average of the correlation coefficients for multiple sets and uses this average as the correlation coefficient threshold.

[0114] Furthermore, the processor filters out the set of reference variables whose correlation coefficients do not reach the correlation coefficient threshold, and removes the reference data corresponding to the reference variables in the filtered set. Then, it performs collinearity analysis on the remaining multiple sets of reference data, deletes the reference data corresponding to highly collinear correlation coefficients, and uses the final remaining reference data as training samples.

[0115] In this embodiment, the processor takes the average of the multiple correlation coefficients obtained from actual calculation as the correlation coefficient threshold, removes reference data with low correlation coefficients and high correlation, and retains reference data with high correlation coefficients as training samples.

[0116] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. For example... Figure 6 As shown, in some possible embodiments, step 210 includes:

[0117] Step 2102: Input the training samples into the cluster prediction model to obtain the predicted cluster classification results;

[0118] Step 2104: Determine the model loss of the cluster prediction model based on the predicted cluster classification results;

[0119] Step 2106: Adjust the model parameters of the cluster prediction model based on the model loss, and continue training until the cluster prediction model converges to obtain the trained cluster prediction model.

[0120] As an example, the processor uses the stochastic gradient descent algorithm to update the model parameters of the cluster prediction model according to the preset step size. When the model loss of the updated cluster prediction model is less than the preset loss threshold, the updated cluster prediction model is considered to have converged. At this time, the updated cluster prediction model is taken as the cluster prediction model that has been trained and the model training ends.

[0121] In this embodiment, the model loss of the current cluster prediction model can be determined based on the predicted cluster classification results, and the model parameters of the cluster prediction model can be adjusted according to the model loss until the model loss of the cluster prediction model after parameter adjustment is small. At this point, the training ends, and the cluster prediction model after parameter adjustment is taken as the trained cluster prediction model.

[0122] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. In some possible embodiments, step 2086 includes:

[0123] Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding training samples, the model parameters of the cluster prediction model are adjusted until training is completed and a well-trained cluster prediction model is obtained.

[0124] In this embodiment, the processor can adjust the model loss of the cluster prediction model based on the deviation between the predicted cluster classification results and the cluster labels carried by the training samples.

[0125] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. For example... Figure 7 As shown, in this embodiment, the cluster prediction model training method includes:

[0126] Step 702: Obtain multiple sets of reference data;

[0127] Step 704: Determine the number of bits in the parameters based on the type of cluster label carried in the reference data;

[0128] Step 706: For each set of reference data, based on the cluster label carried by the current reference data, encode the corresponding encoding position as the first preset value to obtain the reference variables corresponding to all reference data;

[0129] Step 708: Divide all reference variables into multiple sets according to the cluster label carried by the reference data corresponding to each reference variable;

[0130] Step 710: Calculate the relationship between the reference variable in each set and multiple preset values, and use them as the correlation coefficient between the reference variable in the current set and all cluster labels;

[0131] Step 712: Calculate the average correlation coefficient of each set as the correlation coefficient threshold;

[0132] Step 714: For each set, determine whether the correlation coefficient has reached the correlation coefficient threshold;

[0133] Step 716: If the condition is not met, remove the reference data corresponding to the reference variable in the current set.

[0134] Step 718: If the threshold is reached, the reference data corresponding to the reference variables in the set that have reached the correlation coefficient threshold will be input into the cluster prediction model to obtain the predicted cluster classification result.

[0135] Step 720: Determine the model loss of the cluster prediction model based on the predicted cluster classification results;

[0136] Step 722: Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding reference data, adjust the model parameters of the cluster prediction model using the gradient descent method until training is completed and a trained cluster prediction model is obtained.

[0137] The above-mentioned cluster prediction model training method first uses a preset cluster label encoding rule to encode each group of reference data, so as to parameterize different reference data to obtain reference variables, making the selection of reference data not only more intuitive, but also faster and more convenient. Then, the correlation coefficient of different reference variables is calculated, and the reference data corresponding to the reference variables with lower correlation coefficients are proposed. The remaining reference data is used to train the cluster prediction model, so that the final cluster prediction model has stronger robustness and higher prediction accuracy.

[0138] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0139] Based on the same inventive concept, such as Figure 8 As shown in the embodiments of this application, a cluster prediction method is also provided, including:

[0140] Step 802: Collect reference data;

[0141] Step 804: Input the reference data into the cluster prediction model trained by the cluster prediction model training method in any of the above embodiments to obtain the predicted cluster classification result corresponding to the reference data.

[0142] The above-described cluster prediction method uses the cluster prediction model training method in any of the above embodiments to train the cluster prediction model. This cluster prediction model selects training samples through correlation coefficients, resulting in higher prediction accuracy and better robustness.

[0143] Based on the same inventive concept, this application also provides a cluster prediction model training apparatus for implementing the aforementioned cluster prediction model training method. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more embodiments of the cluster prediction model training apparatus provided below can be found in the limitations of the cluster prediction model training method described above, and will not be repeated here.

[0144] In one embodiment, such as Figure 9 As shown, a cluster prediction model training device 900 is provided, including: an acquisition module 902, a first processing module 904, a second processing module 906, a filtering module 908, and a model generation module 910, wherein:

[0145] The acquisition module 902 is used to acquire multiple sets of reference data; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label, which is used to identify the cluster type;

[0146] The first processing module 904 is used to encode multiple sets of reference data according to the cluster label to obtain multiple reference variables;

[0147] The second processing module 906 is used to establish the correlation coefficients between the reference variable and all cluster labels;

[0148] The filtering module 908 is used to filter out reference data that meets preset conditions based on the correlation coefficient, and use it as training samples for the cluster prediction model.

[0149] The model generation module 910 is used to train the cluster prediction model based on the training samples to obtain the trained cluster prediction model.

[0150] In some possible embodiments, the first processing module 904 is further configured to:

[0151] The number of parameter bits is determined based on the type of cluster label, and the number of parameter bits matches the number of types.

[0152] For each set of reference data, according to the preset cluster label encoding rules, the encoding position corresponding to the cluster label carried by the reference data is encoded as the first preset value to obtain the reference variable corresponding to the reference data. The cluster label encoding rules include the association between the cluster label and the encoding position.

[0153] In some possible embodiments, the second processing module 906 is further configured to:

[0154] According to the preset partitioning rules, all reference variables are divided into multiple sets;

[0155] Calculate the relationship between the reference variable in each set and multiple preset values, and use this relationship as the correlation coefficient between the reference variable in the current set and all cluster labels.

[0156] In some possible embodiments, the filtering module 908 is further configured to:

[0157] Determine the correlation coefficient threshold based on the correlation coefficient;

[0158] Reference data whose correlation coefficients reach the correlation coefficient threshold are used as training samples.

[0159] In some possible embodiments, the model generation module 910 is further configured to:

[0160] Input the training samples into the cluster prediction model to obtain the predicted cluster classification results;

[0161] Based on the predicted cluster classification results, determine the model loss of the cluster prediction model;

[0162] The model parameters of the cluster prediction model are adjusted based on the model loss, and training continues until the cluster prediction model converges, resulting in a fully trained cluster prediction model.

[0163] In some possible embodiments, the model generation module 910 is further configured to:

[0164] Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding training samples, the model parameters of the cluster prediction model are adjusted until training is completed and a well-trained cluster prediction model is obtained.

[0165] Based on the same inventive concept, this application also provides a cluster prediction apparatus for implementing the cluster prediction method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more cluster prediction apparatus embodiments provided below can be found in the limitations of the cluster prediction method described above, and will not be repeated here.

[0166] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. In one embodiment, such as... Figure 6 As shown, a cluster prediction device 1000 is provided, including: a data acquisition module 1002 and a prediction module 1004, wherein:

[0167] The acquisition module 1002 is used to acquire reference data;

[0168] The prediction module 1004 is used to input the reference data into the cluster prediction model trained by the cluster prediction model training method in any of the above embodiments, and obtain the cluster type corresponding to the reference data.

[0169] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a cluster prediction model training method or a cluster prediction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0170] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0171] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the various steps of the above-described data writing method.

[0172] In this embodiment, the acquisition, storage, use, and processing of data all comply with relevant national laws and regulations. In one embodiment, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the various steps of the aforementioned cluster prediction model training method or cluster prediction method.

[0173] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described cluster prediction model training method or cluster prediction method embodiments. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a cluster prediction model, characterized in that, include: Obtain multiple sets of reference data; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label. The cluster label is used to identify the cluster type. The reference data is the operation data of the target object, and the operation data includes login data, resource interaction data, resource size data, and protocol data. Based on the cluster label, the multiple sets of reference data are encoded to obtain multiple reference variables; Establish the correlation coefficient between the reference variable and all cluster labels, wherein different random numbers are pre-set for different cluster types, different cluster labels correspond to different random numbers, and all random numbers are continuous, and establish the correlation coefficient between each reference variable and all random numbers; Based on the correlation coefficient, reference data that meets the preset conditions are selected as training samples for the cluster prediction model, including: if each obtained correlation coefficient exceeds the corresponding numerical threshold, the reference data is determined to meet the preset conditions. Based on the training samples, the cluster prediction model is trained to obtain the trained cluster prediction model.

2. The method according to claim 1, characterized in that, The process involves encoding the multiple sets of reference data based on the cluster label to obtain multiple reference variables, including: The number of parameter bits is determined based on the type of cluster label, and the number of parameter bits matches the number of types. For each set of reference data, according to the preset cluster label encoding rules, the encoding position corresponding to the cluster label carried by the reference data is encoded as a first preset value to obtain the reference variable corresponding to the reference data. The cluster label encoding rules include the association between the cluster label and the encoding position.

3. The method according to claim 1, characterized in that, The process of establishing the correlation coefficient between the reference variable and all cluster labels includes: According to the preset partitioning rules, all the reference variables are divided into multiple sets; Calculate the relationship between the reference variable in each set and multiple preset values, and use the result as the correlation coefficient between the reference variable in the current set and all cluster labels.

4. The method according to claim 1, characterized in that, The step of selecting reference data that meets preset conditions based on the correlation coefficient as training samples for the cluster prediction model includes: Determine the correlation coefficient threshold based on the correlation coefficient; The reference data whose corresponding correlation coefficient reaches the correlation coefficient threshold are used as the training samples.

5. The method according to claim 1, characterized in that, The step of training the cluster prediction model based on the training samples to obtain the trained fault diagnosis model includes: The training samples are input into the cluster prediction model to obtain the predicted cluster classification results; Based on the predicted cluster classification results, determine the model loss of the cluster prediction model; The model parameters of the cluster prediction model are adjusted based on the model loss, and training continues until the cluster prediction model converges, resulting in a trained cluster prediction model.

6. The method according to claim 5, characterized in that, The process of adjusting the model parameters of the cluster prediction model based on the model loss and continuing training until the cluster prediction model converges to obtain a trained cluster prediction model includes: Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding training samples, the model parameters of the cluster prediction model are adjusted until training is completed and a trained cluster prediction model is obtained.

7. A cluster prediction method, characterized in that, include: Collect reference data and input the reference data into the cluster prediction model trained using the method described in any one of claims 1 to 6 to obtain the cluster type corresponding to the reference data.

8. A cluster prediction model training device, characterized in that, include: The acquisition module is used to acquire multiple sets of reference data; each set of reference data carries a unique cluster label, and all reference data carry at least one cluster label. The cluster label is used to identify the cluster type. The reference data is the operation data of the target object, and the operation data includes login data, resource interaction data, resource size data, and protocol number. The first processing module is used to encode the multiple sets of reference data according to the cluster label to obtain multiple reference variables; The second processing module is used to establish the correlation coefficient between the reference variable and all cluster labels. Different random numbers are preset for different cluster types, different cluster labels correspond to different random numbers, and all random numbers are continuous. The correlation coefficient between each reference variable and all random numbers is established. The filtering module is used to filter out reference data that meet preset conditions based on the correlation coefficients, and use them as training samples for the cluster prediction model. The filtering module includes: if each obtained correlation coefficient exceeds the corresponding numerical threshold, the reference data is determined to meet the preset conditions. The model generation module is used to train the cluster prediction model based on the training samples to obtain the trained cluster prediction model.

9. The apparatus according to claim 8, characterized in that, The first processing module is also configured to: The number of parameter bits is determined based on the type of cluster label, and the number of parameter bits matches the number of types. For each set of reference data, according to the preset cluster label encoding rules, the encoding position corresponding to the cluster label carried by the reference data is encoded as a first preset value to obtain the reference variable corresponding to the reference data. The cluster label encoding rules include the association between the cluster label and the encoding position.

10. The apparatus according to claim 8, characterized in that, The second processing module is also configured to: According to the preset partitioning rules, all the reference variables are divided into multiple sets; Calculate the relationship between the reference variable in each set and multiple preset values, and use the result as the correlation coefficient between the reference variable in the current set and all cluster labels.

11. The apparatus according to claim 8, characterized in that, The filtering module is also configured to: Determine the correlation coefficient threshold based on the correlation coefficient; The reference data whose corresponding correlation coefficient reaches the correlation coefficient threshold are used as the training samples.

12. The apparatus according to claim 8, characterized in that, The model generation module is also configured to: The training samples are input into the cluster prediction model to obtain the predicted cluster classification results; Based on the predicted cluster classification results, determine the model loss of the cluster prediction model; The model parameters of the cluster prediction model are adjusted based on the model loss, and training continues until the cluster prediction model converges, resulting in a trained cluster prediction model.

13. The apparatus according to claim 12, characterized in that, The model generation module is also configured to: Based on the deviation between the predicted cluster classification results and the cluster labels carried by the corresponding training samples, the model parameters of the cluster prediction model are adjusted until training is completed and a trained cluster prediction model is obtained.

14. A cluster prediction device, characterized in that, include: The data acquisition module is used to collect reference data. The prediction module is used to input the reference data into a cluster prediction model trained using the method described in any one of claims 1 to 6, and obtain the cluster type corresponding to the reference data.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6 or the method according to claim 7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6 or the steps of the method of claim 7.

17. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method of any one of claims 1 to 6 or the steps of the method of claim 7.

Images