User behavior prediction method and system based on BiLSTM network

Abstract

The invention discloses a BiLSTM network-based user behavior prediction method and system, and relates to the technical field of deep learning. According to the method, an improved BiLSTM network is utilized to predict user behaviors, before the BiLSTM network is trained, an improved starvation game search algorithm is utilized to optimize network hyper-parameters, an optimal hyper-parameter combination is determined, and then the BiLSTM with the optimal hyper-parameter combination is trained; in the training process, an improved focus loss function is adopted to carry out error loss calculation on an output predicted value and a real label; wherein the starvation game search algorithm is optimized and improved based on the problems of insufficient diversity of an initial population and premature convergence, so that the BiLSTM with the optimal hyper-parameter combination is superior to other existing models when predicting user behaviors; the improved focus loss function has obvious advantages in stability and convergence. The method has better application significance in predicting the re-purchase behavior of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular, to a user behavior prediction method and system based on a BiLSTM network. Background Art

[0002] Behavior prediction is the prediction and inference of the future action state or uncertain situation of an individual or a group. Specifically at the operational level, it relies on historical information, subjective experiences, and combines the analysis of theoretical and computational models to form a quantitative or qualitative judgment on future unknown events. This process reveals the inherent regularity of behavior occurrence and provides a scientific basis for future decision-making and strategy planning. Currently, the commonly used methods mainly analyze log data through technologies such as machine learning and data mining to construct a high-precision prediction model.

[0003] Compared with classical machine learning methods, deep learning learns the high-level feature representations of data through non-linear activation functions and complex network structures. Deep learning usually involves convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their derivative network types - bidirectional long short-term memory networks (BiLSTMs). Each layer of the neural network experiences two steps: forward propagation and backward propagation.

[0004] In summary, how to construct a prediction model based on deep learning to achieve high-precision user behavior prediction is worthy of research. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a user behavior prediction method and system based on a BiLSTM network.

[0006] According to one aspect of the present invention, a user behavior prediction method based on a BiLSTM network is proposed. The method includes:

[0007] Obtain a user behavior data set;

[0008] Preprocess the user behavior data;

[0009] Train a user behavior prediction model based on an improved BiLSTM network based on the preprocessed user behavior data set;

[0010] Use the trained user behavior prediction model to predict the behavior data to be predicted.

[0011] Furthermore, the user behavior dataset includes user basic information, user behavior logs, and user repurchase labels; among them, user basic information includes user ID, gender, and age; user behavior logs contain user operation behavior information and corresponding commodity information; user repurchase labels contain user repurchase behavior information; the preprocessing includes the processing of missing data, abnormal data, and duplicate data.

[0012] Furthermore, training the user behavior prediction model based on the improved BiLSTM network using the preprocessed user behavior dataset includes:

[0013] Optimizing the learning rate, number of neurons, and dropout rate of the BiLSTM network based on the improved Hunger Games Search Algorithm to determine the optimal hyperparameter combination;

[0014] Inputting the preprocessed user behavior dataset into the BiLSTM network with the optimal model hyperparameter combination for training. During the training process, the improved focal loss function is used to calculate the error loss between the predicted values output by the network and the true labels in the user behavior dataset to optimize the model training; obtaining the trained user behavior prediction model.

[0015] Furthermore, the improvements of the Hunger Games Search Algorithm include: generating reverse individuals using reverse learning and selecting the better ones to retain to enhance the diversity of the initial population; using a parameter adjustment strategy based on the normal distribution to adaptively adjust the control strategy selection parameter l in the iterative update formula for individual foraging.

[0016] Furthermore, the improvements of the Hunger Games Search Algorithm also include: introducing a randomness factor and logarithmic spiral in reverse learning to construct a new search mechanism. In the new search mechanism, first, a solution in the opposite direction is generated through the reverse learning strategy, then the randomness factor is introduced for fine-tuning, and then, using the characteristics of the logarithmic spiral, starting from the current solution, jumps are made along the path of the logarithmic spiral.

[0017] Furthermore, optimizing the learning rate, number of neurons, and dropout rate of the BiLSTM network based on the improved Hunger Games Search Algorithm to determine the optimal hyperparameter combination includes:

[0018] Generating an initial population within the search area; generating reverse individuals X using reverse learning based on the initial population op , retaining the better ones according to the fitness function; sorting according to the fitness size and updating the global optimal position Optimal fitness BF and worst fitness WF; update the control strategy selection parameter l according to the current iteration number t using a parameter adjustment strategy based on the normal distribution; each individual in the population selects an iterative update strategy according to the iterative update formula; calculate the control parameter E selected by the strategy and generate a random parameter R; calculate the sum of the starvation weights; calculate the starvation degree of each individual and sum them up to obtain the total starvation degree; update the population position; if the optimal fitness BF has not been updated after more than n iterative updates, perturb the population using the new search mechanism and select the best ones to retain; if the current iteration number t is greater than the set maximum iteration number T, return the optimal fitness BF and the optimal solution Otherwise, continue with iterative updates.

[0019] Further, the formula for generating the reverse individual X using reverse learning based on the initial population is as follows op :

[0020] X op = UB + LB - X;

[0021] In the formula, UB and LB respectively represent the upper and lower limits of the search area, and X represents the current individual;

[0022] The formula for selecting and retaining the better one according to the fitness function is as follows

[0023]

[0024] In the formula, f(.) represents the fitness function.

[0025] Further, the formula for updating the control strategy selection parameter l according to the current iteration number t using a parameter adjustment strategy based on the normal distribution is as follows

[0026]

[0027] In the formula, μ represents the mean of the normal distribution; σ 2 represents the variance of the normal distribution;

[0028] The formula for perturbing the population using the new search mechanism is as follows

[0029] X new = |X - X op '| × d k × cos(2πk);

[0030] In the formula, X new represents the perturbed solution; k = 2 × r3 - 1; X op ' = r4 × UB + r5 × LB - X; r3, r4, and r5 are all random numbers between 0 and 1;

[0031] And the formula for selecting the best ones to retain is as follows

[0032]

[0033] Further, the formula of the improved focal loss function is as follows:

[0034]

[0035] In the formula, C represents the total number of sample categories; α k represents the weight coefficient; p k represents the probability of correctly predicting the k-th class sample.

[0036] According to another aspect of the present invention, a user behavior prediction system based on a BiLSTM network is proposed. The system includes:

[0037] A data acquisition module configured to acquire a user behavior data set;

[0038] A preprocessing module configured to preprocess the user behavior data;

[0039] A model training module configured to train a user behavior prediction model based on an improved BiLSTM network based on the preprocessed user behavior data set;

[0040] A behavior prediction module configured to predict the behavior data to be predicted by using the trained user behavior prediction model.

[0041] The present invention has the following technical effects:

[0042] The present invention proposes a user behavior prediction method and system based on a BiLSTM network, which uses an improved bidirectional long short-term memory network to predict user behavior; before training the bidirectional long short-term memory network, an improved Hunger Games Search Algorithm is used to optimize the network hyperparameters, including the learning rate, the number of neurons, and the dropout rate, to determine the optimal hyperparameter combination; then, the bidirectional long short-term memory network with the optimal hyperparameter combination is trained, and during the training process, an improved focal loss function is used to calculate the error loss between the predicted value and the true label of the output, optimizing the model training. Among them, the Hunger Games Search Algorithm is optimized and improved based on the problems of insufficient diversity of the initial population and premature convergence, so that the bidirectional long short-term memory network with the optimal hyperparameter combination is superior to other existing models in predicting user behavior; the improved focal loss function has obvious advantages in stability and convergence. The present invention has better application significance in predicting user repurchase behavior. Description of the Drawings

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of a user behavior prediction method based on a BiLSTM network according to an embodiment of the present invention.

[0045] Figure 2 It is a schematic structural diagram of a BiLSTM network in an embodiment of the present invention.

[0046] Figure 3 It is a schematic diagram of the dropout mechanism of a BiLSTM network in an embodiment of the present invention.

[0047] Figure 4 It is a flowchart of an improved Hunger Games Search algorithm in an embodiment of the present invention.

[0048] [[ID=!19]] Figure 5 It is a schematic structural diagram of a user behavior prediction system based on a BiLSTM network according to an embodiment of the present invention. Specific Embodiments

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope protected by the present invention.

[0050] An embodiment of the present invention proposes a user behavior prediction method based on a BiLSTM network. As Figure 1 shown, the method includes:

[0051] S1. Obtain a user behavior data set;

[0052] S2. Preprocess the user behavior data;

[0053] S3. Train a user behavior prediction model based on an improved BiLSTM network based on the preprocessed user behavior data set;

[0054] S4. Use the trained user behavior prediction model to predict the behavior data to be predicted.

[0055] The method starts with S1. In S1, a user behavior data set is obtained.

[0056] According to an embodiment of the present invention, the data set is derived from the real purchase behavior data of users on a certain online platform. The data set contains user information and behavior data of millions of anonymous merchants and multiple anonymous buyers corresponding to these merchants. Through these data, the repurchase behavior of users is studied to explore the laws and potential values therein, and further deeply excavate the consumption behavior habits of users. The data set mainly consists of three parts, namely user basic information, user behavior logs, and user repurchase labels. Among them, the user basic information includes user ID, gender, and age; the user behavior logs contain the operation behavior information and corresponding commodity information that occur when the user uses the online platform for shopping, such as click, favorite, add to cart, and purchase; the user repurchase labels contain the repurchase behavior information that occurs when the user uses the online platform for shopping, and the user repurchase labels include user ID, commodity ID, and whether it is a repurchase.

[0057] Then, S2 is executed. In S2, the user behavior data is preprocessed.

[0058] According to an embodiment of the present invention, the preprocessing mainly includes the processing of data missing, data anomalies, and data duplicates, and operations such as deletion, filling, replacement, and deduplication are performed on the identified missing values, outliers, and duplicate values to ensure the quality of the data in the subsequent modeling process.

[0059] 1) Missing value processing: The common processing methods mainly include two methods: filling missing values and deleting missing values. If the amount of missing data is large and the data structure has a certain regularity, data filling is considered. Data filling usually uses statistical quantities such as mean, median, and mode that reflect the central tendency for filling; if the amount of default data is small and deleting it will not affect the data structure, direct deletion is considered, that is, a certain row or column where the identified missing data is located is deleted, so that there are no missing values in the finally retained data.

[0060] 2) Outlier processing: The user behavior data is checked for duplicates based on extreme situations or impulse consumption situations to screen and identify whether a user repeatedly purchases the same commodity in large quantities at the same time. The upper limit of the number of repetitions is set to 10 times or more, and when the repeated purchase behavior exceeds 10 times, it is excluded; screen and identify whether a user repeatedly clicks on the same commodity in large quantities at the same time. The upper limit of the number of repetitions is set to 50 times, and when the repeated click behavior exceeds 50 times, it is excluded.

[0061] 3) Duplicate value handling: Use Python to check for duplicates in the user's personal information and user behavior information. There are no duplicate data in the user's personal information, that is, the unique user ID and corresponding basic information; there are more than ten thousand duplicate data in the user behavior information, accounting for a relatively large proportion of the total data. Considering that the user's repeated behaviors mainly focus on repeated clicks, collections, and purchases of goods, which conform to the actual shopping scenario, the duplicate data existing in the user behavior log is retained.

[0062] Then execute S3. In S3, based on the preprocessed user behavior data set, a user behavior prediction model based on the improved BiLSTM network is trained.

[0063] According to the embodiments of the present invention, a bidirectional long short-term memory network (BiLSTM) is used as the basic model for user behavior prediction. BiLSTM belongs to an extension of the recurrent neural network. It enhances the ability to understand sequence data by combining forward and backward information flows. Different from the unidirectional information processing of the traditional long short-term memory network (LSTM), the bidirectional long short-term memory network can comprehensively integrate sequential and reverse sequence information through bidirectional parsing. This method significantly enhances the model's understanding of context information. BiLSTM realizes the reverse propagation of information by introducing an additional layer of LSTM, thus realizing an information flow from the future to the past. This bidirectional architecture enables the network to have a comprehensive perspective of the entire input sequence and can reasonably integrate the information of the forward and backward LSTM layers. Figure 2 The network structure of BiLSTM is shown.

[0064] However, the original BiLSTM network is difficult to solve the problems of difficult-to-distinguish samples and misclassification costs during the classification prediction process. Therefore, the embodiments of the present invention use the improved Hunger Games Search Algorithm and the focal loss function to optimize and improve the original BiLSTM network.

[0065] 1) Improved Hunger Games Search Algorithm

[0066] The model hyperparameters in the neural network are divided into structural hyperparameters and training hyperparameters. The embodiments of the present invention use the Hunger Games Search Algorithm to optimize the number of neurons and dropout rate in the structural hyperparameters of the BiLSTM network and optimize the learning rate in the training hyperparameters. Among them, neurons are the computing units of the LSTM neural network, receiving inputs and generating outputs. The hidden layer composed of neurons is the basis for the neural network to extract features. Too few neurons will cause the network to be unable to reasonably express the general laws of sample data, which will seriously affect the generalization ability of the model; too many neurons will lead to a high degree of data fitting during the training process, which will cause the network to overly focus on the unique special laws of the training samples and thus be unable to apply to new data samples, that is, the overfitting phenomenon. Therefore, a reasonable number of neurons needs to be determined. AsFigure 3 As shown, the dropout mechanism is a common neural network regularization method, that is, in each iteration process, the training results of a part of neurons are randomly discarded. The discarding and retaining of neurons are randomly determined by probability. Therefore, the composition of neurons that the neural network can utilize in each iteration during the training process is different. The dropout mechanism can prevent the model from having too strong mutual dependence on the relationships between certain nodes during training, improve the robustness and generalization ability of the model, and the dropout mechanism is of great significance for improving the performance of the neural network. The training process of the neural network is a process in which network parameters continuously move towards the optimal solution according to the difference between the predicted value and the true value. The step size of the movement is the learning rate used to train the network. Therefore, the learning rate controls the convergence progress of the network. Using a larger learning rate can accelerate the convergence speed of the network, but too large a learning rate may cause the parameters to oscillate near the optimal solution or even miss the optimal solution; on the contrary, too small a learning rate will cause the network to converge too slowly or fall into a local optimum. Determining a reasonable learning rate can train a deep learning model with the best effect.

[0067] The Hunger Games Search algorithm (HGS) is a new intelligent optimization algorithm proposed in 2021. This algorithm innovatively simulates the foraging behavior of animals driven by hunger, not only integrating the activity characteristics of animals, but also considering the influence of animal psychological factors on foraging behavior. This design makes the HGS algorithm closer to the real animal foraging logic, so that it can more accurately simulate and solve complex optimization problems. The execution process of the Hunger Games Search algorithm is as follows:

[0068] Step 1: Set the basic parameters of the algorithm;

[0069] Step 2: Randomly generate an initial population within the search area;

[0070] Step 3: Calculate the fitness of all individuals in the population;

[0071] Step 4: Sort according to the fitness size and update the global optimal position The best fitness BF and the worst fitness WF;

[0072] Step 5: Each individual in the population selects an iterative update strategy according to the iterative update formula; The iterative update formula is as follows:

[0073]

[0074] In the formula, Game1 simulates the behavior of animals foraging alone, while Game2 and Game3 simulate the behavior of animals foraging in groups; represents the current individual position; is a random number between [-a, a], which is used to simulate the balance between exploration and exploitation that may exist during the foraging process of animals. a is the convergence factor, which linearly decreases from 2 to 0 as the number of iterations increases, simulating the behavioral change of animals gradually shifting from random exploration to focusing more on existing food sources during the foraging process; both r1 and r2 are random numbers between [0, 1]; randn(1) is a random number that follows the standard normal distribution, which is used to introduce randomness to simulate the uncertainty in the real world; and represents the hunger weight; represents the global optimal position, that is, the best food position found so far; the parameter l is a constant used to control the strategy selection, and its value is set to 0.08; E represents the control parameter for strategy selection;

[0075] Step 6: Calculate the control parameter E for strategy selection according to the existing formula, and generate a random parameter R:

[0076] Step 7: Calculate the hunger weight according to the existing formula and

[0077] Step 8: Calculate the hunger degree of each individual according to the existing formula, and sum them up to get the total hunger degree;

[0078] Step 9: Update the population position through Steps 3 - 8;

[0079] Step 10: If the current iteration number t is greater than the set maximum iteration number T, then return the optimal fitness BF and the optimal solution Otherwise, return to Step 3.

[0080] However, the above existing Hunger Games Search Algorithm has the following deficiencies: First, the lack of diversity in the initial population leads to limitations in the search process of the algorithm, affecting the exertion of its global search ability; Second, premature convergence makes it difficult for the algorithm to jump out after finding the local optimal solution, easily falling into the local optimum, resulting in search stagnation, thus restricting the performance improvement of the algorithm.

[0081] In view of the above deficiencies of the Hunger Games Search Algorithm, the embodiments of the present invention propose the following improvement methods:

[0082] 11) Introduce reverse learning to enhance the diversity of the initial population

[0083] The core of the reverse learning strategy lies in that when seeking solutions, not only the current candidate solutions are concerned, but also their reverse solutions, that is, the solutions that are opposite to the current solutions within the feasible region, are considered simultaneously. By comparing the performance of the two, individuals that are more likely to approach the optimal solution can be selected, thereby exploring the search space more effectively, reducing the possibility of falling into local optima, and enhancing the global search ability. The implementation process of reverse learning is as follows: map the current solution to the opposite position in the feasible region, that is, calculate the reverse solution X according to the current solution X and the upper and lower bounds UB and LB of the feasible region (search area). op :

[0084] X op = UB + LB - X (2)

[0085] However, this reverse mapping does not always lead to better solutions. Therefore, the reverse learning adopts a greedy strategy and only retains those solutions with improved fitness. That is:

[0086]

[0087] In the formula, f(.) represents the fitness function; X represents the current individual.

[0088] The algorithm will compare the fitness of the current solution X with the reverse solution X op and select the better one as the candidate solution for the next generation population. This strategy ensures that the population can continuously develop in a better direction during the evolution process. The reverse learning strategy changes the traditional random initial population strategy, increases the diversity of the population, and enables the algorithm to have stronger competitiveness from the very beginning.

[0089] Although the reverse learning strategy improves the performance of the algorithm to a certain extent, there are still certain limitations. It may cause the algorithm to fall into a new local optimum and cannot fully explore the entire solution space. To overcome this limitation, the embodiments of the present invention introduce a randomness factor and a logarithmic spiral. The logarithmic spiral is a unique shape that starts from the center and gradually expands outwards, covering a wide range of areas. Further combine the logarithmic spiral with reverse learning to construct a more efficient and flexible search mechanism. In the new search mechanism, first generate a solution in the opposite direction through the reverse learning strategy, then introduce the randomness factor for fine-tuning, and then utilize the characteristics of the logarithmic spiral to jump along the path of the logarithmic spiral starting from the current solution. This jumping method not only maintains the directionality of the jump but also can cover a wider solution space, enabling the local optimum jumping mechanism to more comprehensively explore possible solutions. The formula is expressed as follows:

[0090] X new = |X - X op '| × e k × cos(2πk) (4)

[0091] Wherein, X new represents the perturbed solution; k = 2×r3 - 1; X op ' = r4×UB + r5×LB - X; r3, r4, and r5 are all random numbers between 0 and 1.

[0092] Adopt a greedy strategy to only retain those solutions whose fitness has improved after the position change, expressed as:

[0093]

[0094] 12) A parameter adjustment strategy based on the normal distribution is proposed to solve the problem that the algorithm falls into a local optimum

[0095] The parameter adjustment strategy proposed in the embodiments of the present invention is to upgrade the originally fixed parameters to parameters that can change dynamically with the algorithm iteration process by adjusting the control parameters inside the algorithm. This dynamic adjustment strategy can make the algorithm more flexible to select the best convergence method according to the actual situation or increase the probability of the algorithm developing towards the optimal solution. In the parameter adjustment strategy based on the normal distribution, the parameter l is dynamically adjusted with the change of the iteration number t, expressed as follows:

[0096]

[0097] Wherein, l represents the control strategy selection parameter for individual foraging in the iterative update strategy; T represents the maximum number of iterations; μ represents the mean of the normal distribution; σ 2 represents the variance of the normal distribution.

[0098] After testing, when μ = 0.5 and σ 2 = 0.45, the algorithm has the best optimization effect. This choice not only improves the optimization efficiency but also ensures the stability and accuracy of the algorithm. The comprehensive experimental results also show that the improved Hunger Games Search algorithm has surpassed the original Hunger Games Search algorithm and other optimization algorithms (such as Particle Swarm Optimization algorithm, Genetic Optimization algorithm, Sparrow Search algorithm) in performance, and it demonstrates excellent stability and effectiveness.

[0099] As Figure 4 shown, the execution process of the improved Hunger Games Search algorithm after the above improvement is as follows:

[0100] Step 1': Generate an initial population within the search area;

[0101] Step 2': Generate the reverse individual X according to the initial population according to Equation (2) op ;

[0102] Step 3': Select and retain the better ones according to the fitness function according to Equation (3);

[0103] Step 4': Sort according to the fitness value and update the global optimal position The optimal fitness BF and the worst fitness WF;

[0104] Step 5': Update the control strategy selection parameter l according to the current iteration number t according to Equation (6);

[0105] Step 6': Each individual in the population selects an iterative update strategy according to the iterative update formula (1);

[0106] Step 7': Calculate the control parameter E selected by the strategy according to the existing formula and generate a random parameter R;

[0107] Step 8': Calculate the sum of the starvation weights according to the existing formula;

[0108] Step 9': Calculate the starvation degree of each individual according to the existing formula and sum them up to obtain the total starvation degree;

[0109] Step 10': Calculate and update the population position through Steps 3' to 8';

[0110] Step 11': If the optimal fitness BF has not been updated after more than n times (for example, n = 8) of iterative updates, perturb the population using Equation (4) and select the best according to Equation (5);

[0111] Step 12': If the current iteration number t is greater than the set maximum iteration number T, then return the optimal fitness BF and the optimal solution Otherwise, return to Step 3'.

[0112] Use the improved Hunger Games Search algorithm above to optimize the learning rate, the number of neurons, and the dropout rate of the BiLSTM network to determine the optimal hyperparameter combination. Each individual in the population has different BiLSTM learning rates, the number of neurons, and dropout rates, and the root mean square error between the model prediction value and the true value is used as the fitness function.

[0113] 2) Improved focal loss function

[0114] The role of the loss function is the error between the predicted value of the training sample model and the true value. However, the calculation of the traditional loss function (root mean square error) is easily affected by unstable samples, resulting in difficult sample discrimination, thus affecting the classification accuracy of real samples and spoof samples. To solve this problem, the focal loss function is used to replace the traditional loss function. The focal loss function is an improvement of the cross-entropy loss function. By introducing adjustment parameters and weight coefficients, it has the advantages of solving sample imbalance, increasing the weight of small-number target categories, and strengthening the weight of misclassified samples. The formula of the focal loss function is as follows:[[]]

[0115] FL = -α k(1 - p k ) γ log(p k ) (7)

[0116] In the formula, FL represents the focal loss function, γ represents the adjustment parameter; α k represents the weight coefficient; p k represents the probability of correct prediction of the k-th class sample. It can be seen from the above formula that the closer the value of p k is to 1, the smaller the value of the focal loss function FL, and vice versa, the larger the FL value. When the value of the adjustment parameter γ is in the range of 0 to 5, it has the characteristics of dynamically adjusting the sample loss value and balancing the weights of samples in the region. When a sample is misclassified, its prediction probability p k is small. At this time, the adjustment parameter γ is close to 1, so that the loss value of this sample is hardly affected. As the model iterates, the model will effectively learn this sample by adjusting the parameters, making its p k value gradually increase. At the same time, the adjustment parameter γ will gradually decrease, resulting in a decrease in the loss value of this sample. Through this dynamic change, the adjustment parameter will adaptively adjust the gradient optimization direction, promoting the model to more reasonably allocate attention during the training process. The adjustment parameter γ enables the classifier to pay more attention to difficult-to-classify samples, thereby improving its classification accuracy; the weight coefficient α k helps to adjust the balance between samples, thus avoiding affecting the loss calculation of easy-to-classify and difficult-to-classify samples.

[0117] However, the increase of α k and the decrease of γ will both lead to an increase in the function value, which further increases the complexity of the parameter search process. Therefore, unifying or simplifying hyperparameters is one of the effective methods to reduce the difficulty of parameter search. Thus, the embodiment of the present invention introduces a power function term and a correction term to improve the focal loss. The improved focal loss function is expressed as follows:

[0118]

[0119] In the formula, C represents the total number of sample categories. Compared with the focal loss function before improvement, when the sample has a serious classification error, the loss value is large. As p k increases, the loss decays at a relatively fast rate, which ensures that the loss value can converge quickly in the initial stage of training; and when p k continues to increase, the function curve becomes smooth, ensuring the stability in the middle and late stages of model training. In addition, the improved focal loss function needs to be calculated once for each sample, and there is no nested loop or complex mathematical operation in the calculation process. The algorithm complexity is the same as that of the focal loss function. Therefore, the improved focal loss has more advantages in stability and convergence without increasing the algorithm complexity.

[0120] Input the preprocessed user behavior dataset into the above - constructed BiLSTM network with optimal hyperparameters for training. During the training process, use the above - improved focal loss function to calculate the error loss between the predicted values output by the model and the true labels in the user behavior dataset to optimize the model training.

[0121] Then execute S4. In S4, use the trained user behavior prediction model to predict the behavior data to be predicted.

[0122] According to the embodiments of the present invention, the behavior data to be predicted includes user basic information and user behavior logs. Input the behavior data to be predicted into the trained user behavior prediction model, and the output result is the probability that the user will repurchase a certain product.

[0123] Further verify the technical effects of the present invention through experiments.

[0124] In the experiment, use the above - constructed user behavior dataset as input data, and use the BiLSTM model, LightGBM model, and the BiLSTM and CNN fusion model as comparison algorithms. As can be seen from Table 1, the prediction effect of the improved BiLSTM network proposed by the present invention has been greatly improved compared with other models, and it is significantly better than other models in terms of recall rate. This indicates that the improved model proposed by the present invention has a lower false negative rate and can better capture positive examples. In contrast, other models are prone to performance degradation when facing high - dimensional features and are difficult to accurately predict user repurchase behavior. In addition, the relatively high F1 value also reflects to a certain extent that the improved model proposed by the present invention has relatively high comprehensive performance, while other models may cause overfitting problems due to over - interpreting the user purchase behavior feature data, such as the BiLSTM and CNN fusion model.

[0125] Table 1

[0126] BiLSTM LightGBM BiLSTM+CNN The present invention Accuracy rate 0.990 0.992 0.993 0.995 Precision 0.920 0.942 0.945 0.945 Recall rate 0.978 0.988 0.984 0.994 F1 value 0.948 0.967 0.965 0.969

[0127] Another embodiment of the present invention proposes a user behavior prediction system based on a BiLSTM network, as Figure 5 shown. The system includes:

[0128] A data acquisition module 510 configured to acquire a user behavior dataset;

[0129] A pre - processing module 520 configured to pre - process the user behavior data;

[0130] A model training module 530 configured to train a user behavior prediction model based on an improved BiLSTM network based on the pre - processed user behavior dataset;

[0131] The behavior prediction module 540 is configured to predict the behavior data to be predicted by using the trained user behavior prediction model.

[0132] For the unspecified parts of the user behavior prediction system based on the BiLSTM network according to the embodiments of the present invention, please also refer to the specific descriptions of the above method embodiments.

[0133] It should be noted that the terms used in the present invention are only for describing specific embodiments and do not limit the scope of the present application. As shown in the specification of the present invention, unless the context clearly indicates an exception, words such as "a", "an", "one" and / or "the" do not specifically refer to the singular and may also include the plural. The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method or device including the element.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A user behavior prediction method based on the BiLSTM network, characterized in that, Including: Obtain a user behavior data set; Preprocess the user behavior data; Train a user behavior prediction model based on an improved BiLSTM network based on the preprocessed user behavior data set; Use the trained user behavior prediction model to predict the behavior data to be predicted.

2. The user behavior prediction method based on the BiLSTM network according to claim 1, characterized in that, The user behavior data set includes user basic information, user behavior logs, and user repurchase labels; among them, user basic information includes user ID, gender, and age; user behavior logs contain user operation behavior information and corresponding commodity information; The user repurchase label contains user repurchase behavior information; The preprocessing includes the processing of missing data, abnormal data, and duplicate data.

3. A user behavior prediction method based on the BiLSTM network according to claim 1, characterized in that, The training of the user behavior prediction model based on the improved BiLSTM network based on the preprocessed user behavior data set includes: Optimize the learning rate, number of neurons, and dropout rate of the BiLSTM network based on the improved Hunger Games Search Algorithm to determine the optimal combination of hyperparameters; Input the preprocessed user behavior data set into the BiLSTM network with the optimal model hyperparameter combination for training. During the training process, use the improved focal loss function to calculate the error loss between the predicted value output by the network and the true label in the user behavior data set to optimize the model training; obtain the trained user behavior prediction model.

4. The user behavior prediction method based on the BiLSTM network according to claim 3, wherein The improvements of the Hunger Games Search Algorithm include: generating reverse individuals using reverse learning and selecting the better ones to retain to enhance the diversity of the initial population; using a parameter adjustment strategy based on the normal distribution to adaptively adjust the control strategy selection parameter l in the iterative update formula for individual foraging.

5. A user behavior prediction method based on the BiLSTM network according to claim 4, characterized in that The improvements of the Hunger Games Search Algorithm also include: introducing a randomness factor and logarithmic spiral in reverse learning to construct a new search mechanism. In the new search mechanism, first generate a solution in the opposite direction through the reverse learning strategy, then introduce the randomness factor for fine-tuning, and then use the characteristics of the logarithmic spiral to jump along the path of the logarithmic spiral starting from the current solution.

6. The user behavior prediction method based on the BiLSTM network according to claim 5, characterized in that, The optimization of the learning rate, number of neurons, and dropout rate of the BiLSTM network based on the improved Hunger Games Search Algorithm to determine the optimal combination of hyperparameters includes: Generate an initial population within the search area; generate the reverse individual X using reverse learning based on the initial population op , select and retain the better ones according to the fitness function; sort according to the fitness size and update the global optimal position The optimal fitness BF and the worst fitness WF; update the control strategy selection parameter l using the parameter adjustment strategy based on the normal distribution according to the current iteration number t; each individual in the population selects the iterative update strategy according to the iterative update formula; calculate the control parameter E selected by the strategy and generate the random parameter R; calculate the sum of the starvation weights; calculate the starvation degree of each individual and sum them up to obtain the total starvation degree; update the population position; if the optimal fitness BF has not been updated after more than n iterative updates, perturb the population using the new search mechanism and retain the better ones; if the current iteration number t is greater than the set maximum iteration number T, return the optimal fitness BF and the optimal solution Otherwise, continue the iterative update.

7. A user behavior prediction method based on a BiLSTM network according to claim 6, characterized in that, The reverse individual X is generated by using reverse learning based on the initial population op The formula is as follows: X op = UB + LB - X; In the formula, UB and LB respectively represent the upper and lower limits of the search area, and X represents the current individual; The formula for selecting the better ones to retain according to the fitness function is: In the formula, f(.) represents the fitness function.

8. A user behavior prediction method based on a BiLSTM network according to claim 7, characterized in that The formula for updating the control strategy selection parameter l according to the current iteration number t using the parameter adjustment strategy based on the normal distribution is: where μ represents the mean of the normal distribution; σ 2 represents the variance of the normal distribution; The formula for perturbing the population using the new search mechanism is: X new = |X - X op '| × e k × cos(2πk); where X new represents the perturbed solution; k = 2×r3 - 1; X op ' = r4×UB + r5×LB - X; r3, r4, and r5 are all random numbers between 0 and 1; And the formula for selecting the best ones to retain is:

9. A user behavior prediction method based on the BiLSTM network according to claim 3, characterized in that The formula for the improved focal loss function is: where C represents the total number of sample categories; α k represents the weight coefficient; p k represents the probability of correct prediction of the k-th category of samples.

10. A user behavior prediction system based on a BiLSTM network, characterized in that, Including: A data acquisition module configured to obtain a user behavior data set; A preprocessing module configured to preprocess the user behavior data; A model training module configured to train a user behavior prediction model based on an improved BiLSTM network based on the preprocessed user behavior data set; A behavior prediction module configured to use the trained user behavior prediction model to predict the behavior data to be predicted.

Images