Classification model training method and personnel classification method

Abstract

The embodiment of the invention provides a classification model training method and a personnel classification method, and relates to the technical field of computers. The training method comprises the following steps: acquiring training sample data and truth value data; inputting the training sample data into the classification model, and obtaining a network layer activation result of the classification model; calculating a loss value of the classification model by using a preset loss function; if the loss value does not meet the preset condition, for each neural network layer, calculating the gradient of a preset loss function at the weight matrix of the neural network layer; calculating the current momentum and the current variance of the neural network layer based on the gradient; updating a weight matrix of the neural network layer based on the gradient, the current momentum, the current variance, a preset learning rate and a preset weight attenuation coefficient; returning to the step of obtaining the training sample data and the truth value data; and if the loss value meets a preset condition, ending the training. On the basis, by applying the scheme provided by the embodiment of the invention, the accuracy of a personnel classification result can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a classification model training method and a personnel classification method. Background Art

[0002] Generally, a person may have multiple characteristics such as gender, age, social activities, frequency of appearing in a specific place, and frequency of contact with specific people. These characteristics contain rich information, so that the person can be classified by using the characteristics of the person to divide the person into different groups. For example, based on the person's travel time, it can be determined whether the person is a night worker. For another example, based on the person's social activities, activity venues, and frequency of contact with ACG enthusiasts, it can be determined whether the person is also an ACG enthusiast.

[0003] In the related art, the personnel characteristics and personnel categories of sample personnel are usually used to train a classification model, and then the category of the person to be classified is predicted by learning the personnel characteristics of the person to be classified through the trained classification model.

[0004] However, in the above-mentioned related technologies, since different personnel characteristics of the sample personnel have different importance for analyzing the characteristics and behaviors of the personnel, different personnel characteristics play different roles in the classification model's category prediction process, and the accuracy of determining the personnel category often depends more on the personnel characteristics with higher importance. In this way, in the training process of the above-mentioned classification model, since the importance of different personnel characteristics is not distinguished, some personnel characteristics with lower importance often interfere with the training effect of the classification model, affecting the prediction accuracy of the classification model, thereby resulting in a lower accuracy of the personnel classification results obtained by the trained classification model. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a classification model training method and a personnel classification method, so as to reduce the interference of less important personnel features on personnel classification and improve the accuracy of the obtained personnel classification results. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for training a classification model, the method comprising:

[0007] Acquire training sample data and true value data; wherein the training sample data includes personnel characteristics of sample personnel, and the true value data includes personnel categories of the sample personnel;

[0008] Input the training sample data into the classification model to be trained, and obtain the network layer activation result of the classification model; wherein the classification model includes a hidden layer, and the hidden layer includes multiple neural network layers, each neural network layer is used to use its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain the prediction result of the neural network layer, and the network layer activation result includes the prediction results of the multiple neural network layers;

[0009] Using a preset loss function, based on the gap between the prediction result of each neural network layer and the true value data, the loss value of the classification model is calculated;

[0010] If the loss value does not meet the preset conditions, then for each neural network layer, calculate the gradient of the preset loss function at the weight matrix of the neural network layer; based on the gradient, calculate the current momentum and current variance of the neural network layer; based on the gradient, the current momentum, the current variance, the preset learning rate and the preset weight decay coefficient, update the weight matrix of the neural network layer; after the weight matrix of each neural network layer is updated, return to the step of obtaining training sample data and true value data; wherein, the current momentum is used to characterize the cumulative result of the gradient of the preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the volatility of the preset result of the neural network layer during the training process of the classification model;

[0011] If the loss value meets the preset condition, the training is terminated to obtain the trained classification model.

[0012] Optionally, in a specific implementation, updating the weight matrix of the neural network layer based on the gradient, the current momentum, the current variance and a preset weight decay coefficient includes:

[0013] Performing deviation correction on the current momentum to obtain a momentum correction result, and performing deviation correction on the current variance to obtain a variance correction result;

[0014] Based on the gradient, the momentum correction result, the variance correction result and the preset weight attenuation coefficient, the weight matrix of the neural network layer is updated.

[0015] Optionally, in a specific implementation, updating the weight matrix of the neural network layer based on the gradient, the momentum correction result, the variance correction result and a preset weight attenuation coefficient includes:

[0016] The weight matrix of the neural network layer is updated using the first formula; wherein the first formula is:

[0017] ;

[0018] in, is the first The updated weight matrix of the neural network layer is For the said The weight matrix of the neural network layer before updating, For the said The momentum correction result of the neural network layer is For the said The variance correction result of the neural network layer is is the preset learning rate, is a preset constant. is the preset weight attenuation coefficient.

[0019] Optionally, in a specific implementation manner, performing deviation correction on the current momentum to obtain a momentum correction result, and performing deviation correction on the current variance to obtain a variance correction result, includes:

[0020] Using the second formula, the current momentum is subjected to deviation correction to obtain a momentum correction result, and using the third formula, the current variance is subjected to deviation correction to obtain a variance correction result;

[0021] Wherein, the second formula is:

[0022] ;

[0023] in, For the said The momentum correction result of the neural network layer is For the said The current momentum of the neural network layer, for of Second power, is the preset momentum coefficient;

[0024] The third formula is:

[0025] ;

[0026] in, For the said The variance correction result of the neural network layer is For the said The current variance of the neural network layer, for of Second power, is the preset variance coefficient.

[0027] Optionally, in a specific implementation, the calculating the current momentum and the current variance of the neural network layer based on the gradient includes:

[0028] Using the fourth formula, the current momentum of the neural network layer is calculated, and using the fifth formula, the current variance of the neural network layer is calculated;

[0029] Wherein, the fourth formula is:

[0030] ;

[0031] in, For the said The current momentum of the neural network layer, For the said The previous current momentum of the neural network layer, For the said The gradient of the weight matrix of a neural network layer; is the preset initial momentum, Indicates the current iteration round of the classification model;

[0032] The fifth formula is:

[0033] ;

[0034] in, For the said The current variance of the neural network layer, For the said The previous current variance of the neural network layer, is the preset initial variance.

[0035] Optionally, in a specific implementation, each neural network layer uses its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain a prediction result of the neural network layer, including:

[0036] Each neural network layer uses the sixth formula to calculate the prediction result of the neural network layer; wherein the sixth formula is:

[0037] ;

[0038] in, is the first The prediction results of the neural network layer are For the said The weight matrix of the neural network layer, For the said The bias matrix of the neural network layer, is the activation function, For the said The input data of the neural network layer is hour, is the training sample data, when hour, is the first The prediction results of the neural network layer.

[0039] Optionally, in a specific implementation, the preset loss function is:

[0040] ;

[0041] in, is the loss value, is the total number of neural network layers included in the hidden layer, is the true value data.

[0042] Optionally, in a specific implementation, the method for generating the training sample data includes:

[0043] Obtaining identity information and / or behavior information of sample personnel to obtain personnel information of sample personnel;

[0044] Performing data preprocessing on the personnel information to obtain target information of the sample personnel; wherein the data preprocessing includes at least one or more of: data cleaning processing, data missing value filling processing, and data deduplication processing;

[0045] According to the content of the target information, the preset feature fields are assigned values ​​to obtain the personnel features of the sample personnel as training sample data; wherein the feature fields are used to characterize the feature categories of the personnel features.

[0046] In a second aspect, an embodiment of the present application provides a method for classifying personnel, the method comprising:

[0047] Obtain target personnel characteristics of personnel to be classified;

[0048] The target person characteristics are input into a trained classification model, and the target classification result of the person to be classified output by the classification model is obtained; wherein the classification model is a model trained using the training method of any classification model provided in the first aspect above.

[0049] In a third aspect, an embodiment of the present application provides a training device for a classification model, the device comprising:

[0050] A data acquisition module, used to acquire training sample data and true value data; wherein the training sample data includes personnel characteristics of sample personnel, and the true value data includes personnel categories of the sample personnel;

[0051] A result acquisition module, used for inputting the training sample data into the classification model to be trained, and obtaining the network layer activation result of the classification model; wherein the classification model includes a hidden layer, and the hidden layer includes a plurality of neural network layers, each neural network layer is used for using its own weight matrix to perform category prediction on the sample personnel based on the training sample data, and obtain the prediction result of the neural network layer, and the network layer activation result includes the prediction results of the plurality of neural network layers;

[0052] A loss value calculation module, used to calculate the loss value of the classification model based on the gap between the prediction result of each neural network layer and the true value data using a preset loss function;

[0053] A parameter updating module, for calculating the gradient of the preset loss function at the weight matrix of each neural network layer if the loss value does not meet the preset conditions; calculating the current momentum and current variance of the neural network layer based on the gradient; updating the weight matrix of the neural network layer based on the gradient, the current momentum, the current variance, the preset learning rate and the preset weight decay coefficient; triggering the data acquisition module after the weight matrix of each neural network layer is updated; wherein the current momentum is used to characterize the cumulative result of the gradient of the preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the volatility of the preset result of the neural network layer during the training process of the classification model;

[0054] The model determination module is used to terminate the training if the loss value meets the preset condition, thereby obtaining the trained classification model.

[0055] Optionally, in a specific implementation, the parameter updating module includes:

[0056] A data correction submodule, used to perform deviation correction on the current momentum to obtain a momentum correction result, and to perform deviation correction on the current variance to obtain a variance correction result;

[0057] A parameter updating submodule is used to update the weight matrix of the neural network layer based on the gradient, the momentum correction result, the variance correction result and a preset weight attenuation coefficient.

[0058] Optionally, in a specific implementation, the parameter updating submodule is specifically used to:

[0059] The weight matrix of the neural network layer is updated using the first formula; wherein the first formula is:

[0060] ;

[0061] in, is the first The updated weight matrix of the neural network layer is For the said The weight matrix of the neural network layer before updating, For the said The momentum correction result of the neural network layer is For the said The variance correction result of the neural network layer is is the preset learning rate, is a preset constant. is the preset weight attenuation coefficient.

[0062] Optionally, in a specific implementation, the data correction submodule is specifically used to:

[0063] Using the second formula, the current momentum is subjected to deviation correction to obtain a momentum correction result, and using the third formula, the current variance is subjected to deviation correction to obtain a variance correction result;

[0064] Wherein, the second formula is:

[0065] ;

[0066] in, For the said The momentum correction result of the neural network layer is For the said The current momentum of the neural network layer, for of Second power, is the preset momentum coefficient;

[0067] The third formula is:

[0068] ;

[0069] in, For the said The variance correction result of the neural network layer is For the said The current variance of the neural network layer, for of Second power, is the preset variance coefficient.

[0070] Optionally, in a specific implementation, the parameter updating module is specifically used to:

[0071] Using the fourth formula, the current momentum of the neural network layer is calculated, and using the fifth formula, the current variance of the neural network layer is calculated;

[0072] Wherein, the fourth formula is:

[0073] ;

[0074] in, For the said The current momentum of the neural network layer, For the said The previous current momentum of the neural network layer, For the said The gradient of the weight matrix of a neural network layer; is the preset initial momentum, Indicates the current iteration round of the classification model;

[0075] The fifth formula is:

[0076] ;

[0077] in, For the said The current variance of the neural network layer, For the said The previous current variance of the neural network layer, is the preset initial variance.

[0078] Optionally, in a specific implementation, each neural network layer uses its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain a prediction result of the neural network layer, including:

[0079] Each neural network layer uses the sixth formula to calculate the prediction result of the neural network layer; wherein the sixth formula is:

[0080] ;

[0081] in, is the first The prediction results of the neural network layer are For the said The weight matrix of the neural network layer, For the said The bias matrix of the neural network layer, is the activation function, For the said The input data of the neural network layer is hour, is the training sample data, when hour, is the first The prediction results of the neural network layer.

[0082] Optionally, in a specific implementation, the preset loss function is:

[0083] ;

[0084] in, is the loss value, is the total number of neural network layers of the classification model, is the true value data.

[0085] Optionally, in a specific implementation, the device further includes: a sample generation module; the sample generation module is used to:

[0086] Obtaining identity information and / or behavior information of sample personnel to obtain personnel information of sample personnel;

[0087] Performing data preprocessing on the personnel information to obtain target information; wherein the data preprocessing at least includes: one or more of data cleaning processing, data missing value filling processing and data deduplication processing;

[0088] According to the content of the target information, the preset feature fields are assigned values ​​to obtain the personnel features of the sample personnel as training sample data; wherein the feature fields are used to characterize the feature category to which the target information belongs.

[0089] In a fourth aspect, an embodiment of the present application provides a personnel classification device, the device comprising:

[0090] A feature acquisition module, used to acquire target personnel features of personnel to be classified;

[0091] A target result acquisition module is used to input the target person characteristics into a trained classification model and obtain the target classification result of the person to be classified output by the classification model; wherein the classification model is a model trained using the training method of any classification model provided in the first aspect above.

[0092] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0093] Memory, used to store computer programs;

[0094] The processor is used to implement the steps of the training method of any classification model provided in the first aspect and / or the steps of a personnel classification method provided in the second aspect when executing the program stored in the memory.

[0095] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of any classification model provided in the first aspect and / or the steps of a personnel classification method provided in the second aspect are implemented.

[0096] In the seventh aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps of the training method of any classification model provided in the first aspect above, and / or the steps of a personnel classification method provided in the second aspect above.

[0097] Beneficial effects of the embodiments of the present application:

[0098] As can be seen from the above, by applying the solution provided by the embodiment of the present application, the method automatically extracts features useful for personnel classification by acquiring personnel features of various types of sample personnel, combining the multi-layer architecture in deep learning, and constructing a personnel classification model including a multi-layer neural network. This architecture combines the AdamW optimization algorithm, and directly applies weight decay in the weight matrix update of the neural network layer, thereby avoiding the inconsistency problem between weight decay and gradient update in the traditional Adam optimizer, thereby reducing the interference of multi-dimensional features, minimizing the loss function, and obtaining more accurate personnel classification results. Therefore, the solution provided by the embodiment of the present application can significantly reduce the impact of less important interference features in personnel features on personnel classification.

[0099] Based on this, the classification model trained by the AdamW optimization algorithm can automatically extract effective personnel features from the input data. The extracted personnel feature vector not only contains key information, but also effectively reduces the interference between features and the interference of unimportant features through the weight decay mechanism of the AdamW optimization algorithm. Therefore, the prediction results of the personnel classification model can achieve the best.

[0100] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0102] Figure 1 A flow chart of a classification model training method provided in an embodiment of the present application;

[0103] Figure 2 A flowchart of a method for generating training sample data provided in an embodiment of the present application;

[0104] Figure 3 A flowchart of a technical concept of a classification model training method provided in an embodiment of the present application;

[0105] Figure 4 A flowchart of a personnel classification method provided in an embodiment of the present application;

[0106] Figure 5 A schematic diagram of the structure of a classification model training device provided in an embodiment of the present application;

[0107] Figure 6 A schematic diagram of the structure of a personnel classification device provided in an embodiment of the present application;

[0108] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0109] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0110] In the related technology, during the training process of the classification model, since the importance of different personnel characteristics is not distinguished, some personnel characteristics with lower importance often interfere with the training effect of the classification model and affect the prediction accuracy of the classification model, thereby resulting in lower accuracy of the personnel classification results obtained by the trained classification model.

[0111] In order to solve the above technical problems, an embodiment of the present application provides a training method for a classification model.

[0112] Among them, the training method can be applied to various application scenarios that require the classification of personnel, for example, in a product recommendation scenario, by determining whether a person is a potential consumer of a product, it is decided whether to recommend the product to the person; in a video push scenario, by determining whether a person is a follower of a certain topic, it is decided whether to push the video of the topic to the person; in a group division scenario, by determining the group to which the person belongs through the characteristics of the person, and people belonging to the same group have characteristics with high similarity, etc. In addition, the execution subject of the training method can be various electronic devices, which are specifically used for training models. In specific applications, the electronic device can be a server, a desktop computer, and the electronic device can be a single electronic device or a cluster of electronic devices consisting of multiple electronic devices. Based on this, the embodiment of the present application does not specifically limit the application scenario and execution subject of the training method of the image generation model.

[0113] A classification model training method provided in an embodiment of the present application may include the following steps:

[0114] Acquire training sample data and true value data; wherein the training sample data includes personnel characteristics of sample personnel, and the true value data includes personnel categories of the sample personnel;

[0115] Input the training sample data into the classification model to be trained, and obtain the network layer activation result of the classification model; wherein the classification model includes a hidden layer, and the hidden layer includes multiple neural network layers, each neural network layer is used to use its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain the prediction result of the neural network layer, and the network layer activation result includes the prediction results of the multiple neural network layers;

[0116] Using a preset loss function, based on the gap between the prediction result of each neural network layer and the true value data, the loss value of the classification model is calculated;

[0117] If the loss value does not meet the preset conditions, then for each neural network layer, calculate the gradient of the preset loss function at the weight matrix of the neural network layer; based on the gradient, calculate the current momentum and current variance of the neural network layer; based on the gradient, the current momentum, the current variance, the preset learning rate and the preset weight decay coefficient, update the weight matrix of the neural network layer; after the weight matrix of each neural network layer is updated, return to the step of obtaining training sample data and true value data; wherein, the current momentum is used to characterize the cumulative result of the gradient of the preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the volatility of the preset result of the neural network layer during the training process of the classification model;

[0118] If the loss value meets the preset condition, the training is terminated to obtain the trained classification model.

[0119] It should be noted that in the technical solutions provided in the embodiments of the present application, the operations of obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0120] As can be seen from the above, by applying the solution provided by the embodiment of the present application, the method automatically extracts features useful for personnel classification by acquiring personnel features of various types of sample personnel, combining the multi-layer architecture in deep learning, and constructing a personnel classification model including a multi-layer neural network. This architecture combines the AdamW optimization algorithm, and directly applies weight decay in the weight matrix update of the neural network layer, thereby avoiding the inconsistency problem between weight decay and gradient update in the traditional Adam optimizer, thereby reducing the interference of multi-dimensional features, minimizing the loss function, and obtaining more accurate personnel classification results. Therefore, the solution provided by the embodiment of the present application can significantly reduce the impact of less important interference features in personnel features on personnel classification.

[0121] Based on this, the classification model trained by the AdamW optimization algorithm can automatically extract effective personnel features from the input data. The extracted personnel feature vector not only contains key information, but also effectively reduces the interference between features and the interference of unimportant features through the weight decay mechanism of the AdamW optimization algorithm. Therefore, the prediction results of the personnel classification model can achieve the best.

[0122] Below, in conjunction with the accompanying drawings, a training method for a classification model provided in an embodiment of the present application is specifically described.

[0123] Figure 1 A flow chart of a training method for a classification model provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the training method may include the following steps S101-S105.

[0124] S101: Obtain training sample data and true value data.

[0125] The training sample data includes the personnel characteristics of the sample personnel, and the true value data includes the personnel categories of the sample personnel.

[0126] In an embodiment of the present application, before training the classification model, first determine the feature categories of the personnel characteristics involved in the application scenario according to the application scenario of the trained classification model. For example, the feature categories of the personnel characteristics involved in the commodity recommendation scenario may include gender, age, purchase record, purchase time, etc., and the feature categories of the personnel characteristics involved in the video push scenario may include gender, age, education, daily browsing time, browsing record, comment content, like video category, etc. Then, the sample personnel can be determined, and further according to the above feature categories, the personnel characteristics of the sample personnel and the personnel category to which the sample personnel belong are determined. The personnel category to which the above sample personnel belongs is the real personnel category of the sample personnel determined according to the classification rules in the application scenario and the personnel characteristics of the sample personnel. Therefore, when the personnel characteristics of the sample personnel are used as training sample data, the personnel category of the sample personnel can be used as the true value of the training sample data in the classification model training process.

[0127] Based on this, when training a classification model, training sample data and true value data may be first obtained, and the training sample data include personnel characteristics of sample personnel, and correspondingly, the true value data include personnel categories of sample personnel.

[0128] In order to make the text clear, examples will be given later on to illustrate the specific method of obtaining the above training sample data.

[0129] S102: Input the training sample data into the classification model to be trained, and obtain the network layer activation result of the classification model.

[0130] Among them, the classification model includes a hidden layer, the hidden layer includes multiple neural network layers, each neural network layer is used to use its own weight matrix to predict the category of sample personnel based on training sample data, and obtain the prediction result of the neural network layer. The network layer activation result includes the prediction results of multiple neural network layers.

[0131] After obtaining the above training sample data and true value data, the above training sample data can be input into the classification model to be trained, and the network layer activation result of the classification model to be trained can be obtained.

[0132] Among them, in the embodiments of the present application, the above-mentioned classification model can be various neural network models that can achieve target classification, for example, MLP (Multi-Layer Perceptron) neural network model, CNN (Convolutional Neural Network) model, RNN (Recurrent Neural Network) model, etc. In this regard, the embodiments of the present application do not limit the specific types of the above-mentioned classification models.

[0133] In the case of MLP model classification, usually, the main structure of the above classification model includes an input layer, a hidden layer and an output layer. Among them, the input layer is used to obtain the input data of the model. The hidden layer includes multiple neural network layers, each of which is used to process the input data based on its own weight matrix, bias matrix and other parameters to obtain the prediction results for the input data, and the process of model training is the process of adjusting the parameters of each neural network layer in the above hidden layer. The output layer is used to convert the final output of the multiple neural network layers included in the hidden layer after processing the input data into a probability distribution through an activation function, as the output result of the classification model, and the output result is the classification result obtained by the classification model for classification prediction of the input data, and the above probability distribution is used to characterize the probability that the categories of the input data are each set category.

[0134] Among them, when the above-mentioned training sample data is input into the classification model to be trained, the above-mentioned training sample data can be used as the input data of the above-mentioned classification model. Usually, the input data of the model is data that has been processed by feature engineering such as normalization and standardization. In an embodiment of the present application, for example, there are 20 training sample data, and each training sample data includes 100 personnel features. When the above-mentioned training sample data is input into the above-mentioned model to be classified, the data form of the input data is a matrix or vector with a dimension of 20×100.

[0135] In the above classification model to be trained, the weight matrix can represent the importance of each personnel feature included in each training sample data to the output result of the classification model. In addition, the weight matrix will vary with the number of layers of the neural network layer, and each neural network layer has its own weight matrix. The size of the weight matrix of each neural network layer depends on the input dimension and output dimension of the neural network layer. For example, in the case where each training sample data includes 100 personnel features, if a neural network layer can receive the 100 personnel features included in each training sample data as input and output 20 activation values ​​(i.e., output values), then the weight matrix of the neural network layer can be a matrix with a dimension of 100×20. During the training process of the above classification model, the weight matrix of each neural network layer can be continuously updated through the back propagation algorithm, so that the above classification model can better fit the training data.

[0136] The output result of the above classification model is the classification result obtained by classifying and predicting the input data, which can be called the classification prediction result. Generally, the above classification prediction result is the probability distribution obtained by converting the final output after the input data is processed by multiple neural network layers through the output layer of the above classification model through the activation function. The probability distribution includes multiple set categories and the probability values ​​corresponding to each category. The probability distribution is used to characterize the probability that the category of the input data is each set category, and the probability corresponding to each set category in the probability is the probability that the category of the input data is the set category.

[0137] For example, in a binary classification problem, the output layer of the above classification model may output a probability distribution such as [0.3, 0.7], indicating that the probability that the input data category is the first category is 0.3, and the probability that the input data category is the second category is 0.7, which means that the input data category is the second category is the most likely classification result.

[0138] The above is classified by MLP model. Optionally, the overall calculation process of the above classification model during model training is as follows:

[0139] 1. The input layer obtains the training sample data as the input data of the model.

[0140] 2. The input layer inputs the input data to the first neural network layer included in the hidden layer. The input data is first multiplied by the weight matrix of the first neural network layer to generate the activation value (i.e., output value) of the first neural network layer. The activation value of the first neural network layer is used as the input of the second neural network layer, and the activation value is multiplied by the weight matrix of the second neural network layer to generate the activation value of the second neural network layer. The activation value of the second neural network layer is used as the input of the third neural network layer, and the activation value is multiplied by the weight matrix of the third neural network layer to generate the activation value of the third neural network layer. This cycle is repeated until the activation value of the last neural network layer is obtained.

[0141] 3. The output of the last neural network layer is used as the input of the input layer. The input layer converts the activation value of the last neural network layer into a probability distribution through the activation function, giving the classification prediction result of the above classification model.

[0142] As mentioned above, for the hidden layer of the above classification model, the first neural network layer calculates the activation value based on the input data (i.e., training sample data) and its own weight matrix, and each neural network layer except the first neural network layer calculates the activation value based on the activation value of the previous neural network layer and its own weight matrix. Obviously, the activation value obtained by each neural network layer will be affected by the input data under the premise of being affected by its own weight matrix. Even for each neural network layer except the first neural network layer, the activation value obtained by the neural network layer will be affected by the weight matrix of the previous neural network layers. Therefore, for each neural network layer, it can be considered that the neural network layer uses its own weight matrix to obtain the activation value of the neural network layer based on the input data. Since multiple neural network layers jointly participate in the category prediction of the input data, for each neural network layer, it can be considered that the process of calculating the activation value of the neural network layer is the process of class prediction of the input data, so the activation value obtained by the neural network layer can be called the prediction result of the neural network layer. Furthermore, the activation values ​​obtained by multiple neural network layers can be called the network layer activation results of the classification model. Obviously, the network layer activation results of the above classification model include the activation values ​​obtained by multiple neural network layers, that is, the network layer activation results of the above classification model include the prediction results of multiple neural network layers.

[0143] Based on this, in the embodiment of the present application, the above classification model includes a hidden layer, and the hidden layer includes multiple neural network layers, so that when the training sample data is input into the classification model to be trained, each neural network layer included in the hidden layer can use its own weight matrix to predict the category of the sample personnel based on the training sample data to obtain the prediction result of the neural network layer. Therefore, the network layer activation result of the classification model obtained above includes the prediction result obtained by each neural network included in the above hidden layer.

[0144] Optionally, in a specific implementation, each neural network layer uses its own weight matrix to predict the category of the sample personnel based on the training sample data to obtain the prediction result of the neural network layer, including:

[0145] Each neural network layer uses the sixth formula to calculate the prediction result of the neural network layer; wherein the sixth formula is:

[0146] ;

[0147] in, The above classification model The prediction results of the neural network layer are For the above The weight matrix of the neural network layer, For the above The bias matrix of the neural network layer, is the activation function, The above classification model The input data of the neural network layer is hour, is the above training sample data, when hour, The above classification model The prediction results of the neural network layer.

[0148] S103: Using the preset loss function, based on the difference between the prediction result of each neural network layer and the true value data, the loss value of the classification model is calculated. If the loss value does not meet the preset conditions, step S104 is executed; if the loss value meets the preset conditions, step S105 is executed.

[0149] In the embodiment of the present application, it is hoped that the classification model can accurately classify personnel based on their personal characteristics, and in an ideal state, for the personnel to be classified, the classification result output by the classification model is the true category of the personnel to be classified. Therefore, in the embodiment of the present application, it is often hoped that the output result of the output layer of the classification model has as high a probability as possible that the true category of the personnel to be classified is output, and since the output result of the output layer of the classification model is determined based on the prediction result of each neural network included in the hidden layer of the classification model, it is hoped that the difference between the prediction result of each neural network and the true value data is as small as possible.

[0150] Based on this, in the embodiment of the present application, the core goal of model training for the above-mentioned classification model is to minimize the loss value, so whether the above-mentioned classification model has converged can be judged by monitoring the loss value. In order to make the above-mentioned classification model gradually converge to complete the training of the above-mentioned classification model, in the training process of the above-mentioned classification model, after completing an iterative training of each pair of the above-mentioned training sample data, the preset loss function can be used to calculate the loss value of the classification model based on the gap between the prediction result of each neural network layer and the true value data, and adjust the parameters of each neural network layer according to the relationship between the loss value and the preset conditions.

[0151] That is, in the training process of the above classification model, after each pair of the above training sample data completes an iterative training, the parameters of each neural network layer are adjusted once. In the embodiment of the present application, since the prediction result of each neural network layer is obtained by using the weight matrix of each neural network layer, in the embodiment of the present application, the essence of adjusting the parameters of each neural network layer is to adjust the weight matrix of each neural network layer.

[0152] Optionally, in a specific implementation, the preset loss function is:

[0153] ;

[0154] in, is the loss value, is the total number of neural network layers included in the above hidden layer, is the above true value data.

[0155] S104: For each neural network layer, calculate the gradient of the preset loss function at the weight matrix of the neural network layer; based on the gradient, calculate the current momentum and current variance of the neural network layer; based on the gradient, current momentum, current variance, preset learning rate and preset weight decay coefficient, update the weight matrix of the neural network layer; after the weight matrix of each neural network layer is updated, return to the acquisition step S101.

[0156] Among them, the current momentum is used to characterize the decay rate of the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the decay degree of the weight matrix of the neural network layer during the training process of the classification model.

[0157] In an embodiment of the present application, in order to make the output result of the above-mentioned classification model more accurate, that is, in order for the above-mentioned classification model to make the classification prediction of the person to be classified closer to the actual situation of the person to be classified, it is necessary to minimize the above-mentioned loss value, that is, to optimize the model parameters of each of the above-mentioned neural network layers.

[0158] Among them, in the embodiment of the present application, since the prediction result of each of the above-mentioned neural network layers is obtained by using the weight matrix of each neural network layer, therefore, in the embodiment of the present application, the essence of optimizing the model parameters of each of the above-mentioned neural network layers is to optimize the weight matrix of each of the above-mentioned neural network layers.

[0159] Based on this, when the above-mentioned loss value does not meet the preset conditions, in order to make the weight matrix of each of the above-mentioned neural network layers reach the optimal value, the gradient of the above-mentioned preset loss function to the weight matrix of each neural network layer can be first calculated through the back propagation algorithm, that is, for each neural network layer, the gradient of the preset loss function at the weight matrix of the neural network layer is calculated.

[0160] Among them, the physical meaning of the gradient is as follows: the gradient is a vector that points to the direction in which the function value grows fastest, and its modulus represents the rate of change in that direction. In the embodiment of the present application, for each neural network layer, the gradient of the preset loss function at the weight matrix of the neural network layer refers to the partial derivative of the loss function with respect to the weight matrix of the neural network layer. In the model training process, the gradient is the core output of the back propagation algorithm, which is used to guide the update direction of the weight matrix, which can directly affect the adjustment of the weight matrix, thereby affecting the performance of the model.

[0161] For the convenience of description, in the subsequent description, the gradient of the preset loss function calculated for each neural network layer at the weight matrix of the neural network layer is referred to as the gradient of each neural network layer.

[0162] In the process of model training, the interference of unimportant features is reduced mainly through weight attenuation, which is achieved by penalizing the weight matrix of each neural network layer to suppress the weight values ​​corresponding to the features that have little impact on the classification prediction results of the classification model (that is, the weight values ​​corresponding to unimportant features). In this way, when the classification model makes classification predictions on the input data, the contribution of unimportant features to the classification prediction results will be weakened or even ignored, thereby improving the accuracy and robustness of the classification prediction results of the classification model.

[0163] Among them, weight decay is a regularization technique used to prevent model overfitting. It is achieved by adding a penalty term proportional to the weight size to the loss function. This penalty term encourages the model to choose a smaller weight value, thereby reducing the complexity of the model.

[0164] In the AdamW optimization algorithm, weight decay is not applied as part of the gradient, but is directly performed when the weight is updated, that is, each time the weight is updated, the weight value is reduced according to a fixed ratio (i.e., the weight decay coefficient). In this way, the model can converge to a smaller weight value more stably during the training process, thereby improving the generalization ability of the model.

[0165] Based on this, since the application of the AdamW optimization algorithm can not only accelerate the convergence speed of the model, but also effectively reduce the interference between features and the interference of unimportant features through its unique weight decay mechanism, the use of the AdamW optimization algorithm can decouple weight decay (L2 regularization) from gradient updates.

[0166] As mentioned above, in the embodiment of the present application, when optimizing the weight matrix of each neural network layer, the AdamW optimization algorithm can be used. Among them, for each neural network layer, the weight matrix of the neural network layer is optimized using the AdamW optimization algorithm, which can include the following two steps.

[0167] Step 1: For each neural network layer, after obtaining the above gradient, the current momentum and current variance of the neural network layer can be calculated based on the above gradient.

[0168] Among them, the above-mentioned current momentum is used to characterize the cumulative result of the gradient of the above-mentioned preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the above-mentioned current variance is used to characterize the volatility of the preset results of the neural network layer during the training process of the classification model.

[0169] Optionally, in a specific implementation, for each neural network layer, calculating the current momentum and current variance of the neural network layer based on the gradient may include the following step 1:

[0170] Step 1: For each neural network layer, use the fourth formula to calculate the current momentum of the neural network layer, and use the fifth formula to calculate the current variance of the neural network layer;

[0171] Among them, the fourth formula is:

[0172] ;

[0173] in, For the The current momentum of the neural network layer, For the The previous current momentum of the neural network layer, For the The gradient of the weight matrix of a neural network layer; is the preset initial momentum, Indicates the current iteration of the classification model;

[0174] The fifth formula is:

[0175] ;

[0176] in, For the The current variance of the neural network layer, For the The previous current variance of the neural network layer, is the preset initial variance.

[0177] In the above step 1, since in the training process of the above classification model, after each pair of the above training sample data completes an iterative training, each neural network layer is adjusted once, therefore, the first The current momentum of the neural network layer , which can be understood as When The current momentum of the neural network layer, that is, when calculating the above When the above classification model completes the first Iterations. Correspondingly, the above The previous current momentum of the neural network layer , which can be understood as the above classification model completing the first At the iteration, the calculated The current momentum of the neural network layer, and considering , the classification model completes the first iteration of the training sample data, and the calculation of the The current vector of the neural network layer needs to be used , you can pre-set the The initial value of the current vector of the neural network layer, that is, setting is the preset initial momentum, that is, A preset fixed value.

[0178] Similarly, the above calculated The current variance of the neural network layer , which can be understood as When The current variance of the neural network layer, and the above The previous current variance of the neural network layer , it can be understood that the above classification model completes the first At the iteration, the calculated The current variance of the neural network layer, and pre-set the The initial value of the current variance of the neural network layer, that is, setting is the preset initial variance, that is, A preset fixed value.

[0179] Step 2: For each neural network layer, after obtaining the above-mentioned gradient, current momentum and current variance, the weight matrix of the neural network layer can be updated based on the gradient, current momentum, current variance, preset learning rate and preset weight decay coefficient.

[0180] For each neural network layer, the weight matrix of the neural network model can be updated according to the current gradient information, momentum information, variance information, preset learning rate and weight decay term of each neural network layer. This process not only accelerates the convergence of the model, but also effectively suppresses the interference of unimportant features through the weight decay mechanism.

[0181] Among them, the momentum simulates the concept of inertia in physics. For each neural network layer, the momentum of the neural network layer accumulates the gradient of the neural network layer calculated in multiple iterations including the current iteration, which is used to accelerate the training of the above classification model and reduce model turbulence. Among them, momentum can help the above classification model jump out of the local minimum and converge to the global optimal solution faster. When the weight matrix of each neural network layer is updated by momentum, not only the gradient of the neural network layer calculated in the current iteration is considered, but also the gradient direction and size of the neural network layer in each iteration before the current iteration are considered, so that the update of the weight matrix of each neural network layer is more reasonable. In the AdamW optimization algorithm, momentum is usually implemented by first-order moment estimation (i.e., mean estimation), that is, momentum calculates the exponential moving average of the gradient, and this average is the embodiment of momentum.

[0182] In addition, in the embodiments of the present application, the above-mentioned variance generally refers to the volatility of the model prediction results. In the AdamW optimization algorithm, the variance generally refers to the second-order moment estimate (i.e., the uncentered variance estimate), which is used to measure the fluctuation of the gradient of the neural network layer to adjust the size of the preset learning rate during the update process of the weight matrix of the neural network layer.

[0183] The preset learning rate is used to control the step size of the weight matrix of each neural network layer in the gradient direction, thereby directly affecting the speed and stability of the weight matrix update of each neural network layer. The preset learning rate can determine the amplitude of each update of the weight matrix of each neural network layer together with the above-mentioned gradient. A too large preset learning rate may cause the model to oscillate or diverge, while a too small preset learning rate may make the model training process very slow.

[0184] For each neural network layer, when its gradient fluctuates greatly, the variance will increase accordingly. At this time, the above preset learning rate will decrease to avoid instability caused by excessive update step size. Conversely, when its gradient fluctuates less, the variance will decrease accordingly. At this time, the above preset learning rate will increase to accelerate convergence.

[0185] Through the above-mentioned equivalence and variance, the AdamW optimization algorithm can adaptively adjust the learning rate, making the optimization process of the weight matrix of each neural network layer of the above-mentioned classification model more robust and efficient during model training.

[0186] The preset weight decay coefficient is used to control the degree of decay of the weight matrix of each neural network layer. A larger preset weight decay coefficient means a larger penalty granularity, causing the weight matrix of each neural network layer to tend to a smaller absolute value. Conversely, a smaller preset weight decay coefficient means a smaller penalty intensity, thereby allowing the weight matrix of each neural network layer to take a larger value.

[0187] That is, in the embodiment of the present application, when adjusting the weight matrix of each neural network layer included in the hidden layer of the above classification model, the AdamW optimization algorithm is used. Compared with the prior art, the core improvement of using the AdamW optimization algorithm for classification model training is that the AdamW optimization algorithm decouples weight decay from gradient update and applies it independently in each iteration. This change enables the classification model to more effectively reduce the weight of unimportant features during the optimization process, thereby reducing the interference of unimportant features on the final classification results.

[0188] At the same time, because the AdamW optimization algorithm applies weight decay independently in each iteration, the classification model can more flexibly adjust the weights of different features during the training process, thereby more effectively reducing the interference between features. That is, when the weight of a feature is identified as unimportant (that is, when its gradient is small or there is a strong correlation with other features), the AdamW optimization algorithm will gradually reduce its weight through the weight decay term, thereby reducing its influence in subsequent classification model predictions. This process helps the model focus more on those features that are closely related to the final task, thereby improving the accuracy and robustness of feature extraction.

[0189] Optionally, in a specific implementation, the above-mentioned updating of the weight matrix of each neural network layer based on the gradient, current momentum, current variance and preset weight attenuation coefficient for each neural network layer may include the following steps 2-3.

[0190] Step 2: For each neural network layer, perform bias correction on the current momentum to obtain a momentum correction result, and perform bias correction on the current variance to obtain a variance correction result.

[0191] In a specific implementation, for each neural network layer, the obtained current momentum can first be bias-corrected to obtain a momentum correction result, and the obtained current variance can be bias-corrected to obtain a variance correction result.

[0192] Optionally, in a specific implementation, the above step 2 may include the following step 21.

[0193] Step 21: using the second formula to perform deviation correction on the current momentum to obtain a momentum correction result, and using the third formula to perform deviation correction on the current variance to obtain a variance correction result;

[0194] Among them, the second formula is:

[0195] ;

[0196] in, For the The momentum correction result of the neural network layer is For the The current momentum of the neural network layer, for of Second power, is the preset momentum coefficient;

[0197] The third formula is:

[0198] ;

[0199] in, For the The variance correction result of the neural network layer is For the The current variance of the neural network layer, for of Second power, is the preset variance coefficient.

[0200] Similar to step 1 above, the calculated The momentum correction result of the neural network layer can be understood as When The momentum correction result of the neural network layer is the above calculated Variance correction results for neural network layers , which can be understood as When Variance correction results for neural network layers.

[0201] Step 3: For each neural network layer, update the weight matrix of the neural network layer based on the gradient, momentum correction result, variance correction result and preset weight attenuation coefficient.

[0202] For each neural network layer, after obtaining the above momentum correction result and variance correction result, the weight matrix of the neural network layer can be updated based on the above gradient, the above momentum correction result, the above variance correction result and the preset weight attenuation coefficient.

[0203] Optionally, in a specific implementation, the above step 3 may include the following step 31.

[0204] Step 31: Use the first formula to update the weight matrix of the neural network layer; wherein the first formula is:

[0205] ;

[0206] in, For the classification model The updated weight matrix of the neural network layer is For the The weight matrix of the neural network layer before updating, For the The momentum correction result of the neural network layer is For the The variance correction result of the neural network layer is is the preset learning rate, is a preset constant. is the preset weight decay coefficient.

[0207] Among them, the preset constant ϵ is a small offset, the main purpose of which is to ensure that the denominator will not be zero to avoid numerical instability.

[0208] Typically, for the above three parameters, you can usually start with some common default values, such as 0.01, 0.001, 0.0001, etc., and then adjust them according to the performance of the trained classification model on the validation set or test set.

[0209] In this specific implementation, in the prior art, the weight matrix of each neural network layer is updated as follows: Obviously, in the prior art, when updating the weights of each neural network layer, the learning rate is fixed. , which leads to slow convergence of the classification model, easy to fall into local minimum, sensitive to hyperparameters, lack of momentum mechanism and regularization. In contrast, in this specific implementation, the weight decay term Each time the weight is updated, the size of the current weight matrix will be reduced, and for those features that are not very relevant to the final task (i.e., unimportant features), their weights will gradually decrease, and their role in subsequent classification model predictions will gradually weaken. Therefore, in this specific implementation, the AdamW optimization algorithm optimizes the deficiencies in the above-mentioned existing technologies through adaptive learning rate, momentum, and weight decay, providing a more stable and efficient optimization process.

[0210] It can be seen from the above that after each pair of the above training sample data completes an iterative training, if the loss value of the above classification model does not meet the preset conditions, the above step 104 can be executed to complete the adjustment of the weight matrix of each neural network layer included in the hidden layer of the above classification model. Furthermore, after the weight matrix of each neural network layer included in the hidden layer of the above classification model is updated, it is possible to return to the above step S101 again, and use the updated weight matrix of each neural network layer included in the hidden layer of the above classification model to perform the next iterative training on the above training sample data. The cycle is repeated in sequence to realize continuous training of the above classification model until the loss value of the above classification model meets the preset conditions.

[0211] Optionally, during the model training process, verification sample data and verification true value data may be added to determine the model fitting status of the classification model during the training process. At the end of each iteration round of the model training, the verification sample data may be input into the classification model, and the classification prediction result of the classification model may be obtained. Based on the gap between the classification prediction result and the verification true value data, the verification loss value of the classification model may be calculated. Thus, the fitting status of the classification model may be determined based on the change in the loss value of the classification model and the change in the verification loss value of the classification model.

[0212] For example, the loss value of the above classification model continues to decrease, while the verification accuracy and verification loss value continue to increase in the opposite direction. This means that the above classification model performs well on the training set (i.e., training sample data), but performs poorly on the verification set (i.e., verification sample data), and it may be overfitting.

[0213] S105: End the training and obtain a trained classification model.

[0214] During the training process of the above-mentioned classification model, when one iterative training of the above-mentioned training sample data is completed, the loss value of the obtained classification model meets the preset conditions, and the above-mentioned classification model converges at this time. Therefore, it can be considered that the classification effect of the above-mentioned classification model on personnel meets the requirements, and the training of the above-mentioned classification model can be terminated at this time to obtain the trained classification model.

[0215] Optionally, after obtaining the trained classification model, test sample data and test true value data can be obtained, the test sample data includes the personnel characteristics of the test personnel, and the test true value data includes the personnel category of the test personnel. Then, the test sample data is input into the trained classification model, and the classification prediction result of the classification model is obtained. Based on the gap between the classification prediction result and the test true value data, the prediction accuracy of the trained classification model is calculated. In this way, the effect of the trained classification model can be judged by the above test sample data and test true value data.

[0216] As can be seen from the above, by applying the solution provided by the embodiment of the present application, the method automatically extracts features useful for personnel classification by acquiring personnel features of various types of sample personnel, combining the multi-layer architecture in deep learning, and constructing a personnel classification model including a multi-layer neural network. This architecture combines the AdamW optimization algorithm, and directly applies weight decay in the weight matrix update of the neural network layer, thereby avoiding the inconsistency problem between weight decay and gradient update in the traditional Adam optimizer, thereby reducing the interference of multi-dimensional features, minimizing the loss function, and obtaining more accurate personnel classification results. Therefore, the solution provided by the embodiment of the present application can significantly reduce the impact of less important interference features in personnel features on personnel classification.

[0217] Based on this, the classification model trained by the AdamW optimization algorithm can automatically extract effective personnel features from the input data. The extracted personnel feature vector not only contains key information, but also effectively reduces the interference between features and the interference of unimportant features through the weight decay mechanism of the AdamW optimization algorithm. Therefore, the prediction results of the personnel classification model can achieve the best.

[0218] The following is a detailed description of the method for generating the above training sample data.

[0219] Optionally, a specific implementation method, such as Figure 2 As shown, the method for generating the above training sample data may include the following steps S201-S203.

[0220] S201: Acquire the identity information and / or behavior information of the sample personnel to obtain the personnel information of the sample personnel.

[0221] S202: Perform data preprocessing on the personnel information to obtain target information of the sample personnel.

[0222] Among them, data preprocessing includes at least one or more of: data cleaning processing, data missing value filling processing and data deduplication processing.

[0223] S203: Assign values ​​to preset feature fields according to the target information of the sample personnel to obtain personnel features of the sample personnel as training sample data.

[0224] Among them, the feature field is used to characterize the feature category of personnel characteristics.

[0225] In this specific implementation, before the above-mentioned classification model is trained, the feature category of the personnel characteristics involved in the application scenario can be determined according to the application scenario of the trained classification model. Thus, the feature field can be set according to the feature category of the personnel characteristics involved in the application scenario, and the feature field is used to characterize the feature category of the personnel characteristics.

[0226] For example, the feature categories of personnel features involved in the video push scenario may include gender, age, education level, daily browsing time, browsing history, comment content, and liked video categories, and the names of the above feature categories may be set as feature fields.

[0227] Then, the identity information and / or behavior information of the sample personnel is obtained to obtain the personnel information of the sample personnel. The personnel information of the sample personnel can be obtained from a variety of data sources, and the personnel information of the sample personnel is obtained with the authorization of the sample personnel. Considering that the personnel information of the sample personnel obtained is often raw data, it may have problems such as invalid data, data duplication, and data missing. Therefore, the personnel information of each sample person can be preprocessed to obtain the target information of each sample person. Then, according to the content of the target information of each sample person, the preset feature fields can be assigned to obtain the personnel characteristics of each sample person as training sample data.

[0228] Among them, for each feature field, a numerical value can be set for each situation according to the possible situations of the personnel characteristics corresponding to the feature field. Then, after obtaining the target data of each sample person, for each feature field, the situation to which the data corresponding to the feature field belongs in the above target data can be first determined, and then the numerical value of the situation to which it belongs can be used to assign a value to the feature field, and finally the personnel characteristics of each sample person are obtained, and then the training sample data is obtained.

[0229] For example, the characteristic fields include: age, gender and education level. For the characteristic field age, it can be divided into two cases: under 18 years old and over 18 years old, and the value 0 is set for under 18 years old, and the value 1 is set for over 18 years old; for the characteristic field gender, the value 0 is set for female, and the value 1 is set for male; for the characteristic field education level, it can be divided into five cases: illiterate, primary school, junior high school, senior high school and undergraduate and above, and the value 0 is set for illiterate, the value 1 is set for primary school, the value 2 is set for junior high school, the value 3 is set for senior high school, and the value 4 is set for undergraduate and above.

[0230] The target information of sample person 1 is: 20 years old, male, high school graduate. After assigning values ​​to the feature fields based on the target information of sample person 1, the personnel characteristics of sample person 1 are: [1 1 3].

[0231] The target information of sample person 2 is: 30 years old, female, undergraduate. Then, according to the target information of sample person 2, after assigning values ​​to the feature fields, the personnel characteristics of sample person 2 are obtained as: [0 0 4].

[0232] Optionally, in some cases, after obtaining the personnel characteristics of the above-mentioned sample personnel, only the personnel characteristics of some of the sample personnel can be selected from the obtained personnel characteristics of the sample personnel as training sample data; and the personnel characteristics of the remaining unselected sample personnel can be used as test sample data. At this time, the unselected sample personnel can be called test personnel, and the accuracy of the classification model obtained by the above-mentioned training can be tested using the above-mentioned test sample data to determine whether the accuracy of the classification model obtained by the above-mentioned training meets the requirements.

[0233] For example, the personnel characteristics of 80% of all sample personnel can be used as training sample data, and the personnel characteristics of the remaining 20% ​​of the sample personnel can be used as test sample data.

[0234] In summary, taking the MLP neural network model as an example, it can be seen that the technical concept of a classification model training method provided in the embodiment of the present application is as follows: Figure 3 As shown, the technical concept may include the following steps S301-S304.

[0235] S301: Obtain personnel information of sample personnel, perform data preprocessing, and generate a group label data set.

[0236] That is, obtaining the personnel information of the sample personnel; performing data preprocessing on the above personnel information to obtain the target information of the sample personnel; assigning values ​​to the preset feature fields according to the content of the target information of the sample personnel to obtain the personnel features of the sample personnel as the training sample data. Among them, the above group label data set is the above training sample data.

[0237] S302: Build a multi-layer perceptron neural network model as a classification model.

[0238] S303: The above classification model is combined with an optimization algorithm to accelerate the convergence speed of the classification model and reduce the interference between the characteristics of each person and the interference of unimportant characteristics in the above training sample data.

[0239] Among them, the optimization algorithm in the above step S303 is the AdamW optimization algorithm, that is, the essence of the above step S303 is: the above classification model is combined with the AdamW optimization algorithm to accelerate the convergence speed of the classification model and reduce the interference between the characteristics of each person and the interference of unimportant characteristics in the above training sample data.

[0240] S304: Obtain the optimal parameters of the classification model and the prediction results of the classification model for the test person.

[0241] That is, the optimal weight matrix of each neural network layer included in the hidden layer of the classification model is obtained, and the trained classification model is used to perform classification prediction on the test personnel, and the output result of the output layer of the classification model is obtained as the personnel classification result of the personnel to be classified.

[0242] In summary, by constructing a multi-layer neural network, the representation ability of the classification model is increased, and the complex patterns and nonlinear relationships in the personnel characteristics included in the training sample data can be better captured. The AdamW optimization algorithm accurately decays the weights during the classification model training process, which can better control the complexity of the model and reduce overfitting, making the output results of the classification model more consistent in the training data and the verification data, and reducing the interference between group characteristics; the multi-layer perceptron combined with the AdamW optimization can more accurately evaluate the importance of each personnel feature during the training process, and make the model parameters more sparse, thereby reducing the negative interference of unimportant features on the group feature extraction model.

[0243] Corresponding to the training method of a classification model provided in the above-mentioned embodiment of the present application, the embodiment of the present application also provides a personnel classification method.

[0244] Figure 4 A flow chart of a personnel classification method provided in an embodiment of the present application is as follows: Figure 4 As shown, the personnel classification method may include the following steps S401-S402.

[0245] S401: Obtain target personnel characteristics of personnel to be classified.

[0246] Among them, the method for generating the above-mentioned target personnel characteristics is the same as the method for generating the personnel characteristics of the sample personnel included in the training sample data in the training method of the classification model provided in the above-mentioned embodiment of the present application.

[0247] S402: Input the target person's characteristics into the trained classification model, and obtain the target classification result of the person to be classified output by the classification model.

[0248] Among them, the above-mentioned classification model is a model trained using the training method of any classification model provided in the above-mentioned embodiments of the present application.

[0249] As can be seen from the above, by applying the solution provided by the embodiment of the present application, the classification model trained by the AdamW optimization algorithm can automatically extract effective personnel features from the input data, and the extracted personnel feature vector not only contains key information, but also effectively reduces the interference between features and the interference of unimportant features through the weight decay mechanism of the AdamW optimization algorithm. Therefore, the prediction result of the personnel classification model can achieve the best. Thus, when the above classification model is used to perform classification prediction on the personnel to be classified, the interference of personnel features with lower importance on personnel classification can be reduced, and the accuracy of the personnel classification results of the personnel to be classified can be improved.

[0250] Corresponding to the training method of a classification model provided in the above-mentioned embodiment of the present application, the embodiment of the present application also provides a training device for a classification model.

[0251] Figure 5 A structural diagram of a training device for a classification model provided in an embodiment of the present application, such as Figure 5 As shown, the training device may include the following modules:

[0252] The data acquisition module 510 is used to acquire training sample data and true value data; wherein the training sample data includes personnel characteristics of sample personnel, and the true value data includes personnel categories of the sample personnel;

[0253] The result acquisition module 520 is used to input the training sample data into the classification model to be trained, and obtain the network layer activation result of the classification model; wherein the classification model includes a hidden layer, and the hidden layer includes multiple neural network layers, each neural network layer is used to use its own weight matrix to predict the category of the sample personnel based on the training sample data, and obtain the prediction result of the neural network layer, and the network layer activation result includes the prediction results of the multiple neural network layers;

[0254] A loss value calculation module 530, used to calculate the loss value of the classification model based on the gap between the prediction result of each neural network layer and the true value data using a preset loss function;

[0255] The parameter updating module 540 is used for calculating the gradient of the preset loss function at the weight matrix of each neural network layer if the loss value does not meet the preset conditions; calculating the current momentum and current variance of the neural network layer based on the gradient; updating the weight matrix of the neural network layer based on the gradient, the current momentum, the current variance, the preset learning rate and the preset weight decay coefficient; after the weight matrix of each neural network layer is updated, triggering the data acquisition module; wherein the current momentum is used to characterize the cumulative result of the gradient of the preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the volatility of the preset result of the neural network layer during the training process of the classification model;

[0256] The model determination module 550 is used to terminate the training if the loss value meets the preset condition, thereby obtaining the trained classification model.

[0257] As can be seen from the above, by applying the solution provided by the embodiment of the present application, the method automatically extracts features useful for personnel classification by acquiring personnel features of various types of sample personnel, combining the multi-layer architecture in deep learning, and constructing a personnel classification model including a multi-layer neural network. This architecture combines the AdamW optimization algorithm, and directly applies weight decay in the weight matrix update of the neural network layer, thereby avoiding the inconsistency problem between weight decay and gradient update in the traditional Adam optimizer, thereby reducing the interference of multi-dimensional features, minimizing the loss function, and obtaining more accurate personnel classification results. Therefore, the solution provided by the embodiment of the present application can significantly reduce the impact of less important interference features in personnel features on personnel classification.

[0258] Based on this, the classification model trained by the AdamW optimization algorithm can automatically extract effective personnel features from the input data. The extracted personnel feature vector not only contains key information, but also effectively reduces the interference between features and the interference of unimportant features through the weight decay mechanism of the AdamW optimization algorithm. Therefore, the prediction results of the personnel classification model can achieve the best.

[0259] Optionally, in a specific implementation, the parameter updating module 540 includes:

[0260] A data correction submodule, used to perform deviation correction on the current momentum to obtain a momentum correction result, and to perform deviation correction on the current variance to obtain a variance correction result;

[0261] A parameter updating submodule is used to update the weight matrix of the neural network layer based on the gradient, the momentum correction result, the variance correction result and a preset weight attenuation coefficient.

[0262] Optionally, in a specific implementation, the parameter updating submodule is specifically used to:

[0263] The weight matrix of the neural network layer is updated using the first formula; wherein the first formula is:

[0264] ;

[0265] in, is the first The updated weight matrix of the neural network layer is For the said The weight matrix of the neural network layer before updating, For the said The momentum correction result of the neural network layer is For the said The variance correction result of the neural network layer is is the preset learning rate, is a preset constant. is the preset weight attenuation coefficient.

[0266] Optionally, in a specific implementation, the data correction submodule is specifically used to:

[0267] Using the second formula, the current momentum is subjected to deviation correction to obtain a momentum correction result, and using the third formula, the current variance is subjected to deviation correction to obtain a variance correction result;

[0268] Wherein, the second formula is:

[0269] ;

[0270] in, For the said The momentum correction result of the neural network layer is For the said The current momentum of the neural network layer, for of Second power, is the preset momentum coefficient;

[0271] The third formula is:

[0272] ;

[0273] in, For the said The variance correction result of the neural network layer is For the said The current variance of the neural network layer, for of Second power, is the preset variance coefficient.

[0274] Optionally, in a specific implementation, the parameter updating module 540 is specifically used to:

[0275] Using the fourth formula, the current momentum of the neural network layer is calculated, and using the fifth formula, the current variance of the neural network layer is calculated;

[0276] Wherein, the fourth formula is:

[0277] ;

[0278] in, For the said The current momentum of the neural network layer, For the said The previous current momentum of the neural network layer, For the said The gradient of the weight matrix of a neural network layer; is the preset initial momentum, Indicates the current iteration round of the classification model;

[0279] The fifth formula is:

[0280] ;

[0281] in, For the said The current variance of the neural network layer, For the said The previous current variance of the neural network layer, is the preset initial variance.

[0282] Optionally, in a specific implementation, each neural network layer uses its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain a prediction result of the neural network layer, including:

[0283] Each neural network layer uses the sixth formula to calculate the prediction result of the neural network layer; wherein the sixth formula is:

[0284] ;

[0285] in, is the first The prediction results of the neural network layer are For the said The weight matrix of the neural network layer, For the said The bias matrix of the neural network layer, is the activation function, For the said The input data of the neural network layer is hour, is the training sample data, when hour, is the first The prediction results of the neural network layer.

[0286] Optionally, in a specific implementation, the preset loss function is:

[0287] ;

[0288] in, is the loss value, is the total number of neural network layers of the classification model, is the true value data.

[0289] Optionally, in a specific implementation, the device further includes: a sample generation module; the sample generation module is used to:

[0290] Obtaining identity information and / or behavior information of sample personnel to obtain personnel information of sample personnel;

[0291] Performing data preprocessing on the personnel information to obtain target information; wherein the data preprocessing at least includes: one or more of data cleaning processing, data missing value filling processing and data deduplication processing;

[0292] According to the content of the target information, the preset feature fields are assigned values ​​to obtain the personnel features of the sample personnel as training sample data; wherein the feature fields are used to characterize the feature category to which the target information belongs.

[0293] Corresponding to the personnel classification method provided in the above-mentioned embodiment of the present application, the embodiment of the present application also provides a personnel classification device.

[0294] Figure 6 A schematic diagram of a personnel classification device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the personnel classification device may include the following modules:

[0295] The feature acquisition module 610 is used to acquire the target person features of the person to be classified;

[0296] The target result acquisition module 620 is used to input the target person characteristics into the trained classification model and obtain the target classification result of the person to be classified output by the classification model; wherein the above-mentioned classification model is a model trained using the training method of any classification model provided in the above-mentioned embodiments of the present application.

[0297] As can be seen from the above, by applying the solution provided by the embodiment of the present application, the classification model trained by the AdamW optimization algorithm can automatically extract effective personnel features from the input data, and the extracted personnel feature vector not only contains key information, but also effectively reduces the interference between features and the interference of unimportant features through the weight decay mechanism of the AdamW optimization algorithm. Therefore, the prediction result of the personnel classification model can achieve the best. Thus, when the above classification model is used to perform classification prediction on the personnel to be classified, the interference of personnel features with lower importance on personnel classification can be reduced, and the accuracy of the personnel classification results of the personnel to be classified can be improved.

[0298] Corresponding to a classification model training method and a personnel classification method provided in an embodiment of the present application, an electronic device is also provided in an embodiment of the present application, such as Figure 7 As shown, including:

[0299] Memory 701, used for storing computer programs;

[0300] The processor 702 is used to implement the steps of the training method of any classification model provided in the above-mentioned embodiments of the present application, and / or the steps of a personnel classification method provided in the above-mentioned embodiments of the present application when executing the program stored in the memory 701.

[0301] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 702, the communication interface, and the memory 701 communicate with each other via the communication bus.

[0302] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0303] The communication interface is used for communication between the above electronic device and other devices.

[0304] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0305] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0306] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, the steps of the training method of any classification model provided in the above-mentioned embodiments of the present application and / or the steps of a personnel classification method provided in the above-mentioned embodiments of the present application are implemented.

[0307] In another embodiment provided in the present application, a computer program product comprising instructions is also provided. When the computer program product is run on a computer, the computer executes the steps of the training method of any classification model provided in the above-mentioned embodiments of the present application, and / or the steps of a personnel classification method provided in the above-mentioned embodiments of the present application.

[0308] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0309] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0310] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, electronic device embodiment, computer-readable storage medium embodiment, and computer storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0311] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A classification model training method, characterized in that: The method comprises: Acquire training sample data and true value data; wherein the training sample data includes personnel characteristics of sample personnel, and the true value data includes personnel categories of the sample personnel; Input the training sample data into the classification model to be trained, and obtain the network layer activation result of the classification model; wherein the classification model includes a hidden layer, and the hidden layer includes multiple neural network layers, each neural network layer is used to use its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain the prediction result of the neural network layer, and the network layer activation result includes the prediction results of the multiple neural network layers; Using a preset loss function, based on the gap between the prediction result of each neural network layer and the true value data, the loss value of the classification model is calculated; If the loss value does not meet the preset conditions, then for each neural network layer, calculate the gradient of the preset loss function at the weight matrix of the neural network layer; based on the gradient, calculate the current momentum and current variance of the neural network layer; based on the gradient, the current momentum, the current variance, the preset learning rate and the preset weight decay coefficient, update the weight matrix of the neural network layer; after the weight matrix of each neural network layer is updated, return to the step of obtaining training sample data and true value data; wherein, the current momentum is used to characterize the cumulative result of the gradient of the preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the volatility of the preset result of the neural network layer during the training process of the classification model; If the loss value meets the preset condition, the training is terminated to obtain the trained classification model.

2. The method according to claim 1, characterized in that: The updating of the weight matrix of the neural network layer based on the gradient, the current momentum, the current variance and a preset weight decay coefficient includes: Performing deviation correction on the current momentum to obtain a momentum correction result, and performing deviation correction on the current variance to obtain a variance correction result; Based on the gradient, the momentum correction result, the variance correction result and the preset weight attenuation coefficient, the weight matrix of the neural network layer is updated.

3. The method according to claim 2, characterized in that The updating of the weight matrix of the neural network layer based on the gradient, the momentum correction result, the variance correction result and the preset weight attenuation coefficient includes: The weight matrix of the neural network layer is updated using the first formula; wherein the first formula is: ； in, is the first The updated weight matrix of the neural network layer is For the The weight matrix of the neural network layer before updating, For the The momentum correction result of the neural network layer is For the The variance correction result of the neural network layer is is the preset learning rate, is a preset constant. is the preset weight attenuation coefficient.

4. The method according to claim 3, characterized in that: The performing deviation correction on the current momentum to obtain a momentum correction result, and performing deviation correction on the current variance to obtain a variance correction result, includes: Using the second formula, the current momentum is subjected to deviation correction to obtain a momentum correction result, and using the third formula, the current variance is subjected to deviation correction to obtain a variance correction result; Wherein, the second formula is: ； in, For the The momentum correction result of the neural network layer is For the The current momentum of the neural network layer, for of Second power, is the preset momentum coefficient; The third formula is: ； in, For the The variance correction result of the neural network layer is For the The current variance of the neural network layer, for of Second power, is the preset variance coefficient.

5. The method according to claim 4, characterized in that The calculating the current momentum and the current variance of the neural network layer based on the gradient includes: Using the fourth formula, the current momentum of the neural network layer is calculated, and using the fifth formula, the current variance of the neural network layer is calculated; Wherein, the fourth formula is: ； in, For the The current momentum of the neural network layer, For the The previous current momentum of the neural network layer, For the The gradient of the weight matrix of a neural network layer; is the preset initial momentum, Indicates the current iteration round of the classification model; The fifth formula is: ； in, For the The current variance of the neural network layer, For the The previous current variance of the neural network layer, is the preset initial variance.

6. The method according to any one of claims 1 to 5, characterized in that: Each neural network layer uses its own weight matrix to perform category prediction on the sample personnel based on the training sample data to obtain the prediction result of the neural network layer, including: Each neural network layer uses the sixth formula to calculate the prediction result of the neural network layer; wherein the sixth formula is: ； in, is the first The prediction results of the neural network layer are For the said The weight matrix of the neural network layer, For the said The bias matrix of the neural network layer, is the activation function, For the said The input data of the neural network layer is hour, is the training sample data, when hour, is the first The prediction results of the neural network layer.

7. The method according to claim 6, characterized in that The preset loss function is: ； in, is the loss value, is the total number of neural network layers included in the hidden layer, is the true value data.

8. The method according to claim 1, characterized in that: The method for generating the training sample data includes: Obtaining identity information and / or behavior information of sample personnel to obtain personnel information of sample personnel; Performing data preprocessing on the personnel information to obtain target information of the sample personnel; wherein the data preprocessing includes at least one or more of: data cleaning processing, data missing value filling processing, and data deduplication processing; According to the content of the target information, the preset feature fields are assigned values ​​to obtain the personnel features of the sample personnel as training sample data; wherein the feature fields are used to characterize the feature categories of the personnel features.

9. A personnel classification method, characterized in that: The method comprises: Obtain target personnel characteristics of personnel to be classified; The target person characteristics are input into a trained classification model, and the target classification result of the person to be classified output by the classification model is obtained; wherein the classification model is a model trained using the method described in any one of claims 1-8.

10. A training device for a classification model, characterized in that: The device comprises: A data acquisition module, used to acquire training sample data and true value data; wherein the training sample data includes personnel characteristics of sample personnel, and the true value data includes personnel categories of the sample personnel; A result acquisition module, used for inputting the training sample data into the classification model to be trained, and obtaining the network layer activation result of the classification model; wherein the classification model includes a hidden layer, and the hidden layer includes a plurality of neural network layers, each neural network layer is used for using its own weight matrix to perform category prediction on the sample personnel based on the training sample data, and obtain the prediction result of the neural network layer, and the network layer activation result includes the prediction results of the plurality of neural network layers; A loss value calculation module, used to calculate the loss value of the classification model based on the gap between the prediction result of each neural network layer and the true value data using a preset loss function; A parameter updating module, for calculating the gradient of the preset loss function at the weight matrix of each neural network layer for each neural network layer if the loss value does not meet the preset conditions; calculating the current momentum and current variance of the neural network layer based on the gradient; updating the weight matrix of the neural network layer based on the gradient, the current momentum, the current variance, the preset learning rate and the preset weight decay coefficient; triggering the data acquisition module after the weight matrix of each neural network layer is updated; wherein the current momentum is used to characterize the cumulative result of the gradient of the preset loss function at the weight matrix of the neural network layer during the training process of the classification model, and the current variance is used to characterize the volatility of the preset result of the neural network layer during the training process of the classification model; The model determination module is used to terminate the training if the loss value meets the preset condition, thereby obtaining the trained classification model.

11. A personnel classification device, characterized in that: The device comprises: A feature acquisition module, used to acquire target personnel features of personnel to be classified; A target result acquisition module is used to input the target person characteristics into a trained classification model and obtain the target classification result of the person to be classified output by the classification model; wherein the classification model is a model trained by the method described in any one of claims 1-8.

12. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images