User behavior prediction method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a user behavior prediction method, electronic equipment and a computer readable storage medium, belongs to the field of computers, and solves the problem that prediction is inaccurate due to the fact that user behavior prediction depends on artificial experience in related technologies. The method comprises the steps of obtaining target behavior data of a target user, the target behavior data being obtained based on multi-dimensional data, the multi-dimensional data comprises at least two of user basic information of the target user, position information of the target user, user ordering behavior information of the target user, user use behavior information of the target user and user consumption behavior information of the target user; based on the target behavior data and a pre-trained target deep learning model, obtaining a behavior prediction result of the target user; wherein the target deep learning model is a time sequence prediction model adopting an encoder-decoder structure. The method is used for behavior prediction.

Description

Technical Field

[0001] This application belongs to the field of computer science, and specifically relates to a user behavior prediction method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In recent years, with the rapid development of Internet technology, the number of Internet users has increased exponentially compared with the past. At the same time, with the emergence and increasing maturity of the concept and technology of big data, more and more companies are building big data application platforms based on their own business scope to collect user-related data and predict user behavior based on the collected user-related data.

[0003] Related technologies for predicting user behavior often rely on business analysts' understanding and experience of the business. For example, when predicting user ordering behavior, they might choose common metrics such as the number of historical orders or the total order amount. Due to limitations in understanding, some potentially important factors may be overlooked, thus failing to comprehensively consider all factors influencing user behavior.

[0004] It is evident that the relevant technologies, which rely on human experience to predict user behavior, suffer from inaccurate predictions. Summary of the Invention

[0005] This application provides a user behavior prediction method, an electronic device, and a computer-readable storage medium, which can solve the problem of inaccurate prediction in related technologies that rely on human experience for user behavior prediction.

[0006] In a first aspect, embodiments of this application provide a user behavior prediction method, including: Obtain target user behavior data, which is obtained based on multi-dimensional data, including at least two of the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information. Based on the target behavior data and the pre-trained target deep learning model, the behavior prediction result of the target user is obtained; The target deep learning model is a time series prediction model with an encoder-decoder structure.

[0007] In a second aspect, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores a program or instructions that run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0008] Thirdly, embodiments of this application provide a computer-readable storage medium storing a program or instructions that, when executed, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a computer program product, the computer program product including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] In this embodiment, target user behavior data is obtained. This target behavior data is based on multi-dimensional data, including at least two of the following: the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information. Based on the target behavior data and a pre-trained target deep learning model, a behavior prediction result for the target user is obtained. The target deep learning model is a time series prediction model employing an encoder-decoder structure. Thus, by using target behavior data obtained from multi-dimensional data and a pre-trained target deep learning model for user behavior prediction, the accuracy of the predicted behavior results can be guaranteed, solving the problem of inaccurate predictions in related technologies that rely on human experience for user behavior prediction. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a user behavior prediction method provided in an embodiment of this application; Figure 2 This is a flowchart of another user behavior prediction method provided in the embodiments of this application; Figure 3 This is an overall architecture diagram of the user behavior analysis and prediction model based on digital twins in the embodiments of this application; Figure 4 This is a schematic diagram of the data storage and processing flow in the embodiments of this application; Figure 5 This is a schematic diagram of the internal structure of the user behavior analysis and prediction model based on digital twins, as described in an embodiment of this application. Figure 6 This is a structural block diagram of a user behavior prediction device provided in an embodiment of this application.

[0013] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0016] Figure 1 This is a flowchart illustrating a user behavior prediction method provided in an embodiment of this application. The user behavior prediction method provided in this embodiment can be executed by an electronic device. (Refer to...) Figure 1 The user behavior prediction method provided in this application includes: Step 110: Obtain target behavior data of the target user. The target behavior data is obtained based on multi-dimensional data, which includes at least two of the following: the target user's basic information, the target user's location information, the target user's ordering behavior information, the target user's usage behavior information, and the target user's consumption behavior information. Among them, user consumption behavior information can be information about a user's formal purchase behavior. Specifically, user consumption behavior information can include the purchaser, purchase time, and details of the purchased service.

[0017] Step 120: Based on the target behavior data and the pre-trained target deep learning model, obtain the behavior prediction result of the target user; The target deep learning model is a time series prediction model employing an encoder-decoder structure. This deep learning model can be a user behavior analysis and prediction model based on digital twins.

[0018] In step 120, the target behavior data can be input into a pre-trained target deep learning model. The pre-trained target deep learning model can then be used to predict and analyze the target behavior data to obtain the behavior prediction results of the target user.

[0019] In this embodiment, target user behavior data is obtained. This target behavior data is based on multi-dimensional data, including at least two of the following: the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information. Based on the target behavior data and a pre-trained target deep learning model, a behavior prediction result for the target user is obtained. The target deep learning model is a time series prediction model employing an encoder-decoder structure. Thus, by using target behavior data obtained from multi-dimensional data and a pre-trained target deep learning model for user behavior prediction, the accuracy of the predicted behavior results can be guaranteed, solving the problem of inaccurate predictions in related technologies that rely on human experience for user behavior prediction.

[0020] In some embodiments of this application, step 110, obtaining target user behavior data, includes: obtaining multi-dimensional data of the target user, wherein the multi-dimensional data includes the target user's basic user information, the target user's location information, the target user's ordering behavior information, the target user's usage behavior information, and the target user's consumption behavior information; each type of data in the multi-dimensional data contains a unique user identifier; based on the unique user identifier contained in each type of data in the multi-dimensional data, the multi-dimensional data is fused to obtain target behavior data. In this way, multi-dimensional data can be conveniently fused to obtain target behavior data for behavior prediction.

[0021] User behavior data is typically long-sequence, multivariate time-series data. This type of data often faces the following challenges: long-term dependency, meaning a user's current behavior may be influenced by an event from days or even weeks ago; sequence length, meaning capturing long-term patterns often requires very long historical sequences, leading to enormous computational costs; and multivariate correlation, meaning different behaviors have complex mutual influences. Therefore, in some embodiments of this application, the time-series prediction model includes an Informer model. That is, an Informer model is used for user behavior prediction. Compared to traditional models, the Informer model can efficiently handle long sequences, solving the computational bottleneck of the standard Transformer model. Furthermore, the Informer model in this application embodiment can employ a self-attention distillation mechanism to further compress the sequence, highlighting important information and reducing memory usage and computational cost. In addition, the Informer model in this application embodiment can employ a generative decoder, capable of predicting the entire output sequence at once, rather than predicting step by step, which greatly accelerates inference speed and avoids accumulated errors.

[0022] In some embodiments of this application, the Informer model employs a probability-based sparse self-attention mechanism. This sparse self-attention mechanism selects the top k query vectors sorted by importance from highest to lowest, where k is a positive integer. This significantly reduces computational complexity and improves computational efficiency. Standard self-attention mechanisms compute for all position pairs in the input sequence. For a sequence of length L, its computational complexity is O(L^2 / 2). 2 This is because an L×L attention matrix needs to be computed (i.e., each query interacts with all keys). This becomes a huge computational and memory burden when processing long sequences (such as documents, images, or audio). For most query vectors, their attention distribution is highly sparse, meaning it is only significantly correlated with a few keys, while its correlation with other keys is extremely low, approaching a uniform distribution. Therefore, in the embodiments of this application, by quickly identifying those important queries whose attention distribution is "far from a uniform distribution" and only computing the full attention of these queries, the computational load can be significantly reduced.

[0023] Since the magnitude of KL divergence directly reflects the "specificity" of a query, the larger the KL divergence, the more "specific" the query. In this embodiment, important queries can be determined using KL divergence. Specifically, for each query in the sequence, the KL divergence between its attention distribution and the uniform distribution can be calculated as its "importance score." All queries are then sorted based on this "importance score." Only the top k queries with the highest scores are selected, for example, k = 5 * log(L). The set of these k queries constitutes the "active queue."

[0024] In some embodiments of this application, the time series prediction model includes an Informer model that employs a self-attention distillation mechanism. This self-attention distillation mechanism is used to progressively and selectively compress the sequence length within the encoder. This allows for information condensation, significantly reducing computational load and improving computational efficiency. In traditional Transformer encoders, each layer outputs a sequence identifier with the same length as the input sequence. When processing long sequences, this leads to enormous computational and memory overhead. The self-attention distillation mechanism employed in this application can progressively and selectively compress the sequence length within the encoder while retaining the most significant information. By merging features along the time dimension, the attention mechanisms of subsequent layers can focus on more refined and global contextual information.

[0025] In some embodiments of this application, the time series prediction model includes an Informer model, which employs a generative decoder to generate the entire output sequence at once. In these embodiments, the input to the Informer model's decoder may include two parts: a start token for the target sequence and a placeholder for the sequence to be predicted. The start token is a zero-based vector serving as a guiding signal for decoding; the placeholder for the sequence to be predicted is a sequence in which the target sequence appears, but its content is unknown during inference. By constructing the Informer model's decoder input as a start token and a partially known sequence, masked attention can be used to ensure temporal causality. The model can generate the entire output sequence in parallel and at once without autoregression. This completely eliminates the error accumulation problem during the inference stage, significantly improving the accuracy and efficiency of long sequence prediction.

[0026] In some embodiments of this application, after obtaining the behavior prediction results of the target users, the method further includes: generating an indicator system based on the behavior prediction results of the target users; and visually presenting the indicators. In this step, the output of the prediction model (raw data) is transformed into measurable business indicators (business language), and then presented intuitively to business stakeholders or decision-makers through visualization (visual language), thereby supporting strategy formulation and evaluation. The role of the indicator system is to quantify and evaluate the value and impact of the prediction results; it should be closely aligned with business objectives. For example, if user behavior prediction is "predicting the probability of user purchases in the next 30 days," then the following dimensions of indicators can be constructed: A. User value and segmentation indicators, specifically, high-value user scale: the total number of users with a predicted purchase probability greater than 0.8; user structure distribution: the number and percentage of users divided according to predicted probability intervals (e.g., high, medium, low); number of users at potential churn risk: the number of users predicted to exhibit "churn" behavior. B. Business revenue prediction indicators, specifically, predicted total revenue; expected order growth; high-value user contribution ratio: predicted total revenue of the high-value user group / overall predicted total revenue. C. Operational efficiency metrics, specifically: Marketing target user pool concentration: the proportion of high-value users in the target user pool; Marketing reach efficiency: the minimum number of users needed to reach to achieve business goals. D. Risk and stability metrics, specifically: Model prediction confidence: the average confidence level of the model for these predictions; Prediction uncertainty: the proportion of users whose prediction probability is near a critical value (e.g., around 0.5). The goal of visualization is to clearly, intuitively, and easily understandably convey these metrics to the audience.

[0027] In some embodiments of this application, before acquiring the target user's target behavior data, the user behavior prediction method provided in this application further includes: acquiring training data, wherein the training data includes real-time data and historical data; and iteratively training a deep learning model for user behavior prediction based on the training data to obtain a target deep learning model; wherein both real-time data and historical data are data related to user behavior. The deep learning model is a virtual digital twin model.

[0028] A digital twin is a high-fidelity digital model created in virtual space using data from a physical entity. This "twin" is not static, but rather dynamically simulates, predicts, monitors, and controls the entire lifecycle of the physical entity through real-time data interaction and feedback.

[0029] A complete digital twin system typically includes three core elements: (1) Entities in physical space: These can be any tangible object, such as an engine, an airplane, a building, or a city; or intangible processes, such as a production line or a logistics system. (2) Digital models in virtual space: This is a digital mapping of the physical entity, typically integrating geometric models (3D shape), physical models (following physical laws, such as stress and thermodynamics), behavioral models (operational logic), and rule models (business rules). (3) Two-way data connection and integration: This is the "lifeline" of the digital twin. Real-time data of the physical entity (such as temperature, vibration, location, and operating status) is continuously transmitted to the digital model through sensors, IoT devices, business systems, etc. At the same time, after analyzing, simulating, and optimizing the model in virtual space, instructions or feedback (such as optimization parameters and early warning information) can be sent back to the physical entity to guide its operation or maintenance.

[0030] Figure 2 This is a flowchart of another user behavior prediction method provided in an embodiment of this application. (Refer to...) Figure 2 The user behavior prediction method provided in this application includes: Step 210: Obtain multi-dimensional data of the target user, including the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information; each type of data in the multi-dimensional data contains a unique user identifier. Step 220: Based on the unique user identifier contained in each type of data in the multi-dimensional data, the multi-dimensional data is fused to obtain target behavior data; In this embodiment, before step 210 or step 220, training data, including real-time data and historical data, can be acquired first. Based on the training data, a deep learning model for user behavior prediction is iteratively trained to obtain a target deep learning model. The deep learning model is a virtual digital twin model. This allows for a more accurate target deep learning model, facilitating subsequent use of the trained model to obtain behavior prediction results.

[0031] Step 230: Based on the target behavior data and the pre-trained target deep learning model, obtain the behavior prediction result of the target user; wherein, the target deep learning model is a time series prediction model with an encoder-decoder structure. Step 240: Generate an indicator system based on the behavior prediction results of the target users; Step 250: Visualize the indicators.

[0032] The relevant content of steps 210-250 can be referred to in the preceding and following text. Moreover, steps 240 and 250 are optional steps in some embodiments.

[0033] In this embodiment, target user behavior data is obtained. This target behavior data is based on multi-dimensional data, including at least two of the following: the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information. Based on the target behavior data and a pre-trained target deep learning model, a behavior prediction result for the target user is obtained. The target deep learning model is a time series prediction model employing an encoder-decoder structure. Thus, by using target behavior data obtained from multi-dimensional data and a pre-trained target deep learning model for user behavior prediction, the accuracy of the predicted behavior results can be guaranteed, solving the problem of inaccurate predictions in related technologies that rely on human experience for user behavior prediction.

[0034] To better understand the user behavior prediction method provided in the embodiments of this application, further explanation is provided below.

[0035] Related technologies employ user behavior prediction based on specific metrics. This method primarily involves collecting, organizing, and analyzing metrics such as basic information, ordering behavior, location behavior, and usage behavior. It then manually selects one or more specific user metrics to assess future user behavior. However, this approach has problems in real-world production applications, mainly due to its somewhat one-sided data application. While this method offers advantages such as flexibility, simplicity, and clear logic, it suffers from several issues in practice. First, the selection of specific metrics often relies on the business analyst's understanding and experience. For example, when predicting mobile user ordering behavior, common metrics like historical order frequency and order amount might be chosen. Due to cognitive limitations, some potentially important factors may be overlooked, failing to comprehensively consider all factors influencing user behavior. Second, it lacks dynamic adaptability. User behavior is typically dynamic, with new behaviors and influencing factors constantly emerging, but metric-based prediction methods cannot capture these changes in a timely manner. For example, with the popularization of mobile payment, mobile users usually use online payment methods to order products, and payment habits have changed significantly. If the predictive model is based on traditional credit card payment-related indicators, it will be unable to adapt to this new payment behavior pattern, resulting in inaccurate predictions.

[0036] With the establishment of big data platforms, the user behavior data currently collected contains more and more dimensions and multi-domain data. At the same time, in order to better understand user behavior, enterprises also collect other peripheral data besides user behavior. Therefore, dozens or hundreds of dimensions of data are formed around users. However, current prediction models, whether based on indicators or machine learning, can only learn for one or a few dimensions and cannot integrate and learn for large-scale dimensional data. If learning is required, feature engineering of the data is needed, but this will lead to a certain degree of lack of data features, thus affecting the final effect of the model.

[0037] The overall process in this application embodiment can be as follows: Through detailed analysis and explanation of the current user behavior analysis and prediction methods, it can be seen that the relevant methods mainly have common problems such as one-sided data application dimensions and inability to integrate multi-domain data. In response to these two problems, this application embodiment proposes a user behavior prediction method. Specifically, it can be a user behavior analysis and prediction method based on digital twins. This method mainly includes four parts: data collection, data storage and preprocessing, virtual entity construction, and indicator system. The data acquisition section primarily relies on the enterprise's big data platform to collect multi-domain, multi-dimensional data in real time through data pipelines, and performs simple preprocessing on the data during acquisition. Next is the data storage and processing section, which mainly stores and processes the collected data. The processing flow is divided into real-time processing, batch processing, and predictive data processing. The real-time processing mainly performs filtering, mapping, and aggregation operations on newly collected data from the data pipeline, relying on the enterprise's big data platform's streaming processing architecture. The batch processing mainly processes historical data, including data storage and multi-table concatenation. Finally, the predictive data processing flow integrates the data to form data tags. This process mainly uses database languages ​​to develop tags based on business needs.

[0038] The third part is the virtual entity component, which simulates the physical entity. This part mainly includes various data models, each representing a physical entity. These models can be static or dynamic behavioral models, and they can be matched and aligned using unique user codes and time to form a complete virtual digital twin model. Finally, there is the indicator system. Its main function is to develop relevant indicators based on the data output by the virtual entity according to the needs of business personnel. These indicators are then categorized and correlated according to their dimensions and granularity to form an indicator system. Finally, this system is displayed through a large-screen visualization, allowing operations personnel to intuitively view the model's status and aid in decision-making. The overall architecture of the user behavior analysis and prediction model based on digital twins is as follows: Figure 3 As shown.

[0039] The overall process may involve the construction of a digital twin data management system. The construction process of a digital twin data management system may include: data storage and processing, and data indicator visualization.

[0040] The data storage and processing can be summarized as follows: Since digital twin models require a large amount of data for modeling, how to store and effectively process this collected data is a crucial aspect of establishing a digital twin model. To efficiently store and process data, the first step is to classify the data collected by the enterprise. Based on model requirements, this can be divided into offline data and real-time data. Offline data is primarily used for offline analysis tasks and can be collected using mature data collection tools like Sqoop. Then, Flume receives and caches the data at different intervals in Kafka, distributes it to a Hive database, and stores it in HDFS. Subsequently, HQL can be used to call the Hadoop distributed computing framework to perform preprocessing operations such as data cleaning, data fusion, topic classification, and data standardization. Offline data has lower real-time requirements, so the processing flow is relatively simple. However, processing real-time data is more complex.

[0041] Real-time data acquisition requires a real-time API. Data is periodically cached in Kafka by Flume, but Kafka distributes the data to a Spark in-memory database, requiring preprocessing by a specialized computing framework. The data is ultimately queried via MongoDB and sent to the front-end for use and display. During data processing, components such as Zookeeper, Azkaban, and Atlas are used for load balancing, task scheduling, and data governance, ultimately forming a complete data storage and processing workflow. This workflow can be described as follows: Figure 4 As shown.

[0042] The content of data indicator visualization can be as follows: After storing and processing the collected data, it is also necessary to visualize the current data. How to visualize large-scale data is an important issue. With the development of front-end technology, various data visualization tools have emerged, which can comprehensively display data and transform data into charts, graphs, maps, or other visual elements to more intuitively understand and convey the relationships, patterns, and trends of the data. To visualize digital twin models, this application proposes a visualization method for an indicator system based on user behavior. This method first organizes and classifies the indicators related to user behavior to form a complete indicator system. Then, it uses large-screen visualization technology to display the indicators, allowing operators to view them intuitively and assist in decision-making.

[0043] The overall process may involve the construction of a digital twin virtual space model. The construction of the digital twin virtual space model may involve physical space components, virtual space components, and user behavior prediction algorithms based on Informer.

[0044] The physical space component is as follows: Since twin digital technology simulates the entire process of real objects in physical space, but the number and complexity of entities in real physical space are large, it is necessary to filter and delete unnecessary entities according to requirements to form a relative physical space. Then, a hierarchical modeling design is performed on the current physical space. Therefore, in this embodiment of the invention, the physical space simulation can separately establish user basic information, user order information, user behavior information, and user consumption information to create a user model. Then, it is also necessary to simulate product information, product sales, etc., to form a product model. Simultaneously, data generated throughout the entire process of user browsing, purchasing, and consuming through the application can be collected. All collected data can be processed in real time through the Kafka distributed publish-subscribe messaging system.

[0045] The virtual space section comprises the following: In relevant time series forecasting, recurrent neural networks are primarily used to extract patterns from time series data. By describing user browsing, purchasing, and consumption behaviors within the application, a high-fidelity digital virtual model corresponding to real user behavior is established, comprehensively simulating the behavior of multiple real users in the digital world. In the virtual space for goods, corresponding geometric models are established for different goods, and corresponding physical and behavioral rules are added to accurately reflect the real situation of user purchasing behavior. Based on the user behavior model, future behavior is predicted to approximate reality as closely as possible. In the multi-user virtual space, real-time collection and prediction of user behavior data are mainly carried out. On the one hand, data on future user behavior is collected; on the other hand, the collected data is integrated and analyzed, and mapped into the virtual space. Simultaneously, the predicted data is input into the Informer behavior prediction model for online calculation, ultimately predicting future user behavior.

[0046] The content of the user behavior prediction algorithm based on Informer is as follows: Since the user behavior model generated in the virtual space of a digital twin model can be composed of multi-dimensional data output from multiple models such as the user's basic information model, user ordering behavior model, and user usage behavior model, predicting user behavior can be achieved by fusing multiple dimensional models and inputting them into a time series algorithm for learning. Based on these characteristics, this application uses an improved Informer user behavior prediction algorithm based on the Transformer architecture. This algorithm improves upon the problems of the Transformer model, such as its inability to be used for long-sequence time series prediction, its time complexity being proportional to the sequence length, and its high memory usage during prediction. It forms a long-sequence time series prediction model that surpasses the Transformer, and its model architecture is as follows: Figure 5 As shown. Among them, Figure 5 In this context, MLP (Multilayer Perceptron) refers to a multilayer perceptron; Add and Normalize refer to two consecutive operations: residual connections and layer normalization. Residual connections, also called "skip connections," directly add the input and output of a submodule. Residual connections alleviate the vanishing gradient problem: in very deep networks, gradients can become very small during backpropagation, causing parameters near the input layer to receive almost no updates. Residual connections provide a "highway" that allows gradients to propagate directly and losslessly to earlier layers, greatly improving the model's training stability. Residual connections also preserve original information: each layer in the network may not need to drastically change the input. Residual connections allow the model to easily learn an "identity mapping," meaning that if the current layer is not used, its output can learn to approach zero, thus preserving information for the next layer. This makes the network easier to train and optimize. Layer normalization standardizes a sample along one feature dimension, making its mean 0 and variance 1. Then, learnable scaling and offset parameters are introduced to increase the model's expressive power. Layer normalization can stabilize the training process: the magnitude and distribution of the output values ​​of intermediate layers in a deep learning model can change drastically during training (called "internal covariate shift"). Normalization can stabilize the distribution of these outputs within a relatively fixed range, allowing for a larger learning rate and accelerating model convergence. Layer normalization can also reduce training time: by stabilizing the distribution of activation values, layer normalization makes the training process smoother and typically significantly reduces the required training time.

[0047] The model structure is the same as the transformer model, both using an encoder-decoder structure, but the following three improvements have been made to address the problems in the original transformer model structure: The first improvement addresses the high computational time complexity of the multiple self-attention mechanism in the original transformer model by proposing a probability-based sparse self-attention mechanism. The key detail of this mechanism is the use of KL divergence to measure the distribution within the attention mechanism. The distance between the uniform distribution and the active queue is used to identify the active queue, and the formula is as follows:

[0048] Where q is the query vector. j The attention probability distribution, where p is a uniform distribution; i and j are indices; k i q is the i-th key vector, and n is the total number of keys; j It is the j-th query vector; This is the dot product value.

[0049] It is a scaling factor. It is the dimension of the key vector.

[0050] The second improvement addresses the issue of excessive memory consumption in the original transformer model by introducing a self-attention distillation mechanism to reduce dimensionality and the number of network parameters. The formula is shown below:

[0051] in This is the output of the previous layer, i.e., the j-th layer encoder. This is the input to the next layer, the (j+1)th layer encoder, where j is a positive integer. ELU stands for Exponential Linear Unit, a commonly used activation function in deep learning, and MaxPool is a one-dimensional max-pooling operation. It is a one-dimensional convolution operation.

[0052] The third improvement is a decoder for long sequence prediction that predicts multiple labels at once. Traditional decoders predict labels one by one, then substitute the currently generated label into the sequence before predicting the next label. However, this suffers from the problem of accumulated loss. Each label prediction incurs a certain loss, and the accumulation of loss over long sequences leads to a significant decrease in the model's predictive ability. Therefore, a decoder for long sequence prediction is proposed that predicts multiple labels at once.

[0053] By improving upon these three points, the model maintains its performance while reducing the time complexity of training and prediction, as well as the system memory consumed during runtime, making it easier to deploy and apply in production environments.

[0054] This application proposes a user behavior analysis and prediction method based on digital twins. The method consists of four parts: data collection, data storage and processing, virtual space construction, and indicator system display. First, multi-dimensional user behavior data is collected, stored, and preprocessed. A virtual space is formed by simulating the physical space using a data model. User behavior is predicted using an Informer algorithm. Finally, an indicator system is generated and visualized. This method realizes the function of simulating and analyzing user behavior throughout the entire lifecycle, while providing business operation personnel with auxiliary analysis and decision-making capabilities.

[0055] Figure 6 This is a structural block diagram of a user behavior prediction device provided in an embodiment of this application. (Refer to...) Figure 6 The user behavior prediction device 600 provided in this application embodiment includes: an acquisition module 610 and a processing module 620.

[0056] The acquisition module 610 is used to acquire target behavior data of the target user. The target behavior data is obtained based on multi-dimensional data, which includes at least two of the following: the target user's basic information, the target user's location information, the target user's ordering behavior information, the target user's usage behavior information, and the target user's consumption behavior information. The processing module 620 is used to obtain the behavior prediction result of the target user based on the target behavior data and the pre-trained target deep learning model; wherein the target deep learning model is a time series prediction model with an encoder-decoder structure.

[0057] In this embodiment, target user behavior data is obtained. This target behavior data is based on multi-dimensional data, including at least two of the following: the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information. Based on the target behavior data and a pre-trained target deep learning model, a behavior prediction result for the target user is obtained. The target deep learning model is a time series prediction model employing an encoder-decoder structure. Thus, by using target behavior data obtained from multi-dimensional data and a pre-trained target deep learning model for user behavior prediction, the accuracy of the predicted behavior results can be guaranteed, solving the problem of inaccurate predictions in related technologies that rely on human experience for user behavior prediction.

[0058] In some embodiments of this application, during the process of acquiring target user target behavior data, the acquisition module 610 is specifically used for: Acquire multi-dimensional data of the target user, including the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information; each type of data in the multi-dimensional data contains a unique user identifier. Based on the unique user identifier contained in each of the multi-dimensional data, the multi-dimensional data is fused to obtain target behavior data.

[0059] In some embodiments of this application, the time series prediction model includes an Informer model, which employs a probability-based sparse self-attention mechanism to select the top k query vectors sorted from high to low importance, where k is a positive integer.

[0060] In some embodiments of this application, the time series prediction model includes an informer model that employs a self-attention distillation mechanism for progressively and selectively compressing the sequence length within the encoder.

[0061] In some embodiments of this application, the time series prediction model includes an informer model, which employs a generative decoder to generate the entire output sequence at once.

[0062] In some embodiments of this application, after the processing module 620 obtains the behavior prediction result of the target user, the processing module 620 is further configured to: generate an indicator system based on the behavior prediction result of the target user. The user behavior prediction device provided in this application embodiment may also include a display module for visually presenting the indicators.

[0063] In some embodiments of this application, before the acquisition module 610 acquires the target user's target behavior data, the acquisition module 610 can also be used to acquire training data, which includes real-time data and historical data. Accordingly, the user behavior prediction device provided in the embodiments of this application may further include: a training module, used to iteratively train a deep learning model for user behavior prediction based on the training data to obtain a target deep learning model; wherein, the deep learning model is an established virtual digital twin model.

[0064] It should be understood that the relevant content described in the user behavior prediction device provided in the embodiments of this application can be referred to the content discussed in the preceding method embodiments.

[0065] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of this application. For example... Figure 7 As shown in the illustration, this application also provides an electronic device 700, which can be various types of computers, etc. The electronic device 700 includes a processor 710 and a memory 720. The memory 720 stores programs or instructions, which, when executed by the processor 710, implement the steps of any of the methods described above. For example, when the program is executed by the processor 710, it implements the following process: acquiring target user behavior data, which is obtained based on multi-dimensional data, including at least two of the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information; obtaining a behavior prediction result for the target user based on the target behavior data and a pre-trained target deep learning model; wherein the target deep learning model is a time series prediction model using an encoder-decoder structure. Thus, by using target behavior data obtained from multi-dimensional data and a pre-trained target deep learning model for user behavior prediction, the obtained behavior prediction result can be guaranteed to be relatively accurate, solving the problem of inaccurate predictions in related technologies that rely on human experience for user behavior prediction.

[0066] This application also provides a computer-readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described method embodiments and achieve the same technical effects. To avoid repetition, these will not be described again here.

[0067] The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0068] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0069] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0070] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0072] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A user behavior prediction method, characterized in that, include: Obtain target user behavior data, which is obtained based on multi-dimensional data, including at least two of the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information. Based on the target behavior data and the pre-trained target deep learning model, the behavior prediction result of the target user is obtained; The target deep learning model is a time series prediction model with an encoder-decoder structure.

2. The user behavior prediction method according to claim 1, characterized in that, The acquisition of target user behavior data includes: Acquire multi-dimensional data of the target user, including the target user's basic information, location information, ordering behavior information, usage behavior information, and consumption behavior information; each type of data in the multi-dimensional data contains a unique user identifier. Based on the unique user identifier contained in each of the multi-dimensional data, the multi-dimensional data is fused to obtain target behavior data.

3. The user behavior prediction method according to claim 1, characterized in that, The time series prediction model includes an Informer model, which employs a probability-based sparse self-attention mechanism. This sparse self-attention mechanism is used to select the top k query vectors sorted from highest to lowest importance, where k is a positive integer.

4. The user behavior prediction method according to claim 1, characterized in that, The time series prediction model includes an Informer model, which employs a self-attention distillation mechanism to progressively and selectively compress the sequence length within the encoder.

5. The user behavior prediction method according to claim 1, characterized in that, The time series prediction model includes an informer model, which employs a generative decoder to generate the entire output sequence at once.

6. The user behavior prediction method according to any one of claims 1-5, characterized in that, After obtaining the behavior prediction result of the target user, the method further includes: Based on the predicted behavior of the target users, an indicator system is generated. The aforementioned indicators will be presented visually.

7. The user behavior prediction method according to any one of claims 1-5, characterized in that, Before acquiring the target user's target behavior data, the method further includes: Acquire training data, which includes real-time data and historical data; Based on the training data, the deep learning model used for user behavior prediction is iteratively trained to obtain the target deep learning model. The deep learning model is a virtual digital twin model.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that run on the processor, the program or instructions which, when executed by the processor, implement the steps of the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The medium stores a program or instructions that, when executed, implement the steps of the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images