User behavior prediction method and device, equipment and storage medium

Abstract

The invention provides a user behavior prediction method and device, equipment and a storage medium, and the method comprises the steps: obtaining behavior event data of a user within a preset time, inputting the behavior event data into a pre-training model, obtaining first prediction data through the pre-training model, the pre-training model comprises a plurality of self-attention mechanism layers; collecting user data of the user in a target scene, wherein the user data comprises multi-modal data; according to the target scene and the user data, carrying out transfer learning on the pre-training model to obtain a behavior prediction model; and outputting a behavior prediction result of the user in the target scene through the behavior prediction model based on the user data and the first prediction data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a user behavior prediction method, device, equipment and storage medium. Background Art

[0002] Data analysis and modeling are crucial in many industries, covering data analysis, risk assessment, fraud detection, actuarial science, etc. Through efficient data processing and precise modeling, companies can gain a deep understanding of customer needs, accurately assess risks, and develop effective strategies to provide better products and services.

[0003] In the existing technology, first of all, in terms of data utilization, traditional methods are usually based on single-scenario modeling, resulting in low data utilization. Secondly, traditional methods are based on specific task modeling, and the model is only applicable to specific tasks. It can neither draw on other tasks nor inspire other tasks, and lacks generalization capabilities. Thirdly, each scenario needs to be modeled separately, and data processing, feature extraction and algorithm design must be done from scratch each time. A lot of time and energy are consumed in data processing and feature engineering, which invisibly increases development costs. At the same time, due to the lack of data utilization and generalization capabilities of traditional methods, the model effect is often not ideal. Finally, traditional methods lack the ability to iterate step by step, each modeling is one-time, and the model is difficult to reuse.

[0004] Therefore, in view of the shortcomings of the prior art solutions, the present invention provides a user behavior prediction method. Summary of the invention

[0005] Based on this, it is necessary to provide a user behavior prediction method, device, equipment and storage medium to address the above technical problems.

[0006] On the one hand, a user behavior prediction method is provided, the method comprising: obtaining behavior event data of a user within a preset time, inputting the behavior event data into a pre-trained model, and obtaining first prediction data through the pre-trained model, wherein the behavior event data corresponds to a timestamp, and the pre-trained model comprises a multi-layer self-attention mechanism layer; collecting user data of the user in a target scene, wherein the user data comprises multimodal data; performing transfer learning on the pre-trained model according to the target scene and the user data to obtain a behavior prediction model; and outputting a behavior prediction result of the user in the target scene through the behavior prediction model based on the user data and the first prediction data.

[0007] Optionally, before obtaining the user's behavior event data within the preset time, the method further includes: determining a user, a preset time, and a first range, where the first range includes multiple scenarios; collecting the user's behavior event data and timestamps within the first range during the preset time to obtain the user's behavior event data.

[0008] Optionally, for obtaining the user's behavior event data within the preset time and inputting the behavior event data into a pre-trained model, it includes: sorting the behavior event data according to the timestamps to obtain a behavior sequence; performing vector embedding on the behavior sequence to obtain a behavior vector; performing vector embedding on the timestamps to obtain a time vector; fusing the behavior vector and the time vector to obtain an input vector, and inputting the input vector into the pre-trained model.

[0009] Optionally, after sorting the behavior event data according to the timestamps to obtain a behavior sequence, the method further includes: comparing the length of the behavior sequence according to a preset sequence length; when the length of the behavior sequence is greater than the preset sequence length, obtaining the behavior sequence and the corresponding timestamps of the most recent time to obtain a target behavior sequence and the corresponding timestamps; when the length of the behavior sequence is less than the preset sequence length, determining the positions to be filled; filling the filling data into the positions to be filled to obtain the target behavior sequence and the corresponding timestamps.

[0010] Optionally, for inputting the behavior event data into a pre-trained model and obtaining first prediction data through the pre-trained model, it includes: collecting the static data of the user; processing the static data of the user to obtain a static data vector and inputting it into the pre-trained model; based on the input vector and the static data vector, obtaining the first prediction data through the pre-trained model.

[0011] Optionally, before obtaining the user's behavior event data within the preset time and inputting the behavior event data into a pre-trained model, the method further includes: obtaining and parsing a training behavior sequence to obtain multiple behavior events; covering some of the behavior events in the training behavior sequence according to a replacement rule and filling the preset identifier into the covered behavior events to obtain a first behavior sequence; inputting the first behavior sequence into the pre-trained model to obtain predicted behavior events; calculating the difference between the replaced behavior events and the predicted behavior events through a cross-entropy loss function to obtain the loss function value of the pre-trained model; updating the parameters of the pre-trained model according to the loss function value, and completing the training when the loss function value is the smallest.

[0012] Optionally, based on the user data and the first prediction data, the behavior prediction result of the user in the target scenario is output through the behavior prediction model, including: extracting features from the user data to obtain a feature vector of the user data; normalizing the feature vector to obtain a target feature vector; and based on the target feature vector and the first prediction data, outputting the behavior prediction result of the user in the target scenario through the behavior prediction model.

[0013] On the other hand, a user behavior prediction device is provided. The device includes: a preprocessing module, configured to obtain the behavior event data of the user within a preset time, input the behavior event data into a pre-trained model, and obtain first prediction data through the pre-trained model, where the behavior event data corresponds to a time stamp, and the pre-trained model includes multiple self-attention mechanism layers; a collection module, configured to collect the user data of the user in the target scenario, where the user data includes multi-modal data; a learning module, configured to perform transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; and a prediction module, configured to output the behavior prediction result of the user in the target scenario through the behavior prediction model based on the user data and the first prediction data.

[0014] On yet another aspect, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining the behavior event data of the user within a preset time, inputting the behavior event data into a pre-trained model, and obtaining first prediction data through the pre-trained model, where the behavior event data corresponds to a time stamp, and the pre-trained model includes multiple self-attention mechanism layers; collecting the user data of the user in the target scenario, where the user data includes multi-modal data; performing transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; and outputting the behavior prediction result of the user in the target scenario through the behavior prediction model based on the user data and the first prediction data.

[0015] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining the behavior event data of a user within a preset time, inputting the behavior event data into a pre-trained model, and obtaining first prediction data through the pre-trained model, where the behavior event data corresponds to a time stamp, and the pre-trained model includes multiple self-attention mechanism layers; collecting the user data of the user in a target scenario, where the user data includes multi-modal data; performing transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; and based on the user data and the first prediction data, outputting a behavior prediction result of the user in the target scenario through the behavior prediction model.

[0016] For the above user behavior prediction method, device, equipment and storage medium, the method includes: obtaining the behavior event data of a user within a preset time, inputting the behavior event data into a pre-trained model, and obtaining first prediction data through the pre-trained model, where the behavior event data corresponds to a time stamp, and the pre-trained model includes multiple self-attention mechanism layers; collecting the user data of the user in a target scenario, where the user data includes multi-modal data; performing transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; and based on the user data and the first prediction data, outputting a behavior prediction result of the user in the target scenario through the behavior prediction model. In this way, user behavior can be shared between different scenarios, and user behavior prediction in multiple different scenarios can be realized. It can be applied to scenarios with different data volumes, and has the advantages of high data utilization rate, low development cost, strong generalization ability, and great iteration potential. Description of the Drawings

[0017] Figure 1 It is a schematic flow chart of a user behavior prediction method in an embodiment;

[0018] Figure 2 It is a schematic diagram of a behavior sequence of a user behavior prediction method in an embodiment;

[0019] Figure 3 It is a schematic prediction flow chart of a user behavior prediction method in an embodiment;

[0020] Figure 4 It is a structural block diagram of a user behavior prediction device in an embodiment;

[0021] Figure 5 It is an internal structure diagram of a computer device in an embodiment. Detailed Embodiments

[0022] To make the objectives, technical solutions and advantages of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part rather than all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0023] It should be understood that in the description of this application, unless the context clearly requires otherwise, the words such as "including" and "comprising" throughout the specification should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, it is the meaning of "including but not limited to".

[0024] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0025] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing steps and do not particularly refer to the meaning of order or sequence, nor are they used to limit this application. They are only used to conveniently describe the method of this application and cannot be construed as indicating the sequence of steps. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0026] In one embodiment, as Figure 1 shown, a user behavior prediction method is provided, including the following steps:

[0027] S101: Obtain the behavior event data of the user within a preset time, input the behavior event data into a pre-trained model, and obtain first prediction data through the pre-trained model, where the behavior event data corresponds to a time stamp, and the pre-trained model includes multiple self-attention mechanism layers.

[0028] Here, pre-training is a strategy for training a deep learning model. Its core lies in using a large-scale data set to preliminarily train the model so that the model learns general feature representations. This process is similar to the basic learning stage of humans before learning new knowledge, accumulating experience through extensive reading and observation.

[0029] Here, the behavior event data can be the operation behaviors of users such as clicks, purchases, browsing, consultations, registrations, and exposures.

[0030] Among them, the first prediction data can be a vector.

[0031] Among them, the pre-trained model for user behavior events can be a model based on the Transformer architecture, a graph neural network, a model based on contrastive learning, etc.

[0032] Specifically, the pre-trained model can adopt a model based on the Transformer architecture, which includes multiple layers of self-attention mechanism layers. The self-attention mechanism is one of the core ideas of the Transformer model. It calculates the representation of the same sequence by associating elements at different positions in the sequence. The main purpose of the self-attention mechanism is to capture the dependencies between elements in the sequence, so as to better understand and process data. Through the pre-trained model, the context relationship and interaction of user behavior can be obtained.

[0033] Among them, in the pre-trained model, the position of the sequence can also be encoded by rotating position encoding, which can overcome the limitations that traditional position encoding methods may encounter when processing long sequences.

[0034] S102: Collect the user data of the user in the target scenario, where the user data includes multi-modal data.

[0035] Here, multi-modal data refers to data from multiple different sources or types, which jointly describe a complex object or event.

[0036] Among them, the user data can include one or more modalities such as images, texts, behavior events, audio, etc.

[0037] Furthermore, when the time difference between collecting the user data of the target scenario and collecting the user behavior event data exceeds the preset time difference, it is necessary to re-acquire the user behavior event data and recalculate the first prediction data.

[0038] Exemplarily, the target scenario can be a user purchasing a product, a user consulting a question, etc.

[0039] S103: Perform transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model.

[0040] Here, transfer learning is a machine learning method that improves the performance of the model on a new task by applying the knowledge learned on one task to another related but different task.

[0041] Among them, the methods of transfer learning may include parameter-based methods such as fine-tuning and feature extraction, instance-based methods such as domain adaptation and sample reweighting, relationship-based methods such as meta-learning and multi-task learning, and representation-based methods such as adversarial domain adaptation and joint training.

[0042] S104: Based on the user data and the first prediction data, through the behavior prediction model, output the behavior prediction result of the user in the target scenario.

[0043] Among them, at the same time point, when changing different scenarios, the corresponding first prediction data of the user is the same.

[0044] In one embodiment, when the user data does not exist, according to the scenario and the first prediction data, the behavior prediction result can also be output through the behavior prediction model.

[0045] It should be noted that this application can share user behaviors between different scenarios, realize user behavior prediction in multiple different scenarios, can be applied to scenarios with different data volumes, and has the advantages of high data utilization rate, low development cost, strong generalization ability, and great iteration potential.

[0046] In some specific embodiments, before obtaining the behavior event data of the user within the preset time, the method further includes:

[0047] Determine the user, the preset time, and the first range, where the first range includes multiple scenarios;

[0048] Collect the behavior event data and timestamps of the user in the first range within the preset time to obtain the behavior event data of the user.

[0049] Here, the preset time can be the past 7 days, the past 30 days, or the past 360 days, etc.

[0050] Here, the first range can be comprehensive applications, browsers, web pages, etc. Exemplarily, assuming the first range is insurance-related, it can include multiple scenarios such as claims settlement, purchase, repurchase, renewal, surrender, consultation, etc. Assuming the first range is learning-related, it can include scenarios such as homework, tests, interactions, social interactions, browsing courses, course selection, reading learning texts, etc.

[0051] In some specific embodiments, when obtaining the behavior event data of the user within the preset time and inputting the behavior event data into the pre-trained model, it includes:

[0052] Sort the behavior event data according to the timestamps to obtain a behavior sequence;

[0053] Perform vector embedding on the behavior sequence to obtain a behavior vector;

[0054] Perform vector embedding on the timestamp to obtain a time vector;

[0055] Fuse the behavior vector and the time vector to obtain an input vector, and input the input vector into the pre-trained model.

[0056] Here, embedding is a technique that maps data into a low-dimensional vector space, enabling similar data points to be close to each other in the vector space. It is widely used in fields such as natural language processing (NLP) and computer vision to capture and express the internal features and relationships of data. Through embedding, the model can process high-dimensional data more effectively, improving learning efficiency and performance. Among them, the behavior sequence and the timestamp are discrete sequences.

[0057] Here, fusion in the fields of machine learning and artificial intelligence refers to integrating data, model outputs, or feature representations from multiple different sources or types to form a more comprehensive understanding or make more accurate predictions. Fusion can be performed through methods such as concatenation, weighted summation, and attention mechanisms.

[0058] Exemplarily, Figure 2 is a schematic diagram of the behavior sequence in the embodiments of this application. As Figure 2 shown, the behavior sequence consists of multiple behaviors, and each behavior includes: time and behavior event data. Specifically, the behavior event of the user at time T1 is exposure; the behavior event of the user at time T2 is click; the behavior event of the user at time T3 is registration; the behavior event of the user at time T4 is insurance application... The behavior event of the user at time T N is online customer service, and predict the behavior event of the user at time T N+1 .

[0059] In this way, by adding timestamps, the time intervals between different behavior event data can be obtained, improving the prediction accuracy.

[0060] In some specific embodiments, after sorting the behavior event data according to the timestamp to obtain a behavior sequence, the method further includes:

[0061] Compare the length of the behavior sequence according to a preset sequence length;

[0062] When the length of the behavior sequence is greater than the preset sequence length, obtain the behavior sequence and the corresponding timestamp at the most recent time to obtain the target behavior sequence and the corresponding timestamp;

[0063] When the length of the behavior sequence is less than the preset sequence length, determine the position to be filled;

[0064] Fill the data to be filled into the position to be filled, to obtain the target behavior sequence and the corresponding time stamp.

[0065] Specifically, assuming that the preset sequence length is N, sequences with lengths less than N are filled to make the sequence length up to N, and the filled sequences do not participate in the calculations in the model.

[0066] In this way, even when the collected data is scarce, user behavior prediction can still be achieved.

[0067] In some specific embodiments, the inputting the behavior event data into the pre-trained model to obtain the first prediction data by the pre-trained model includes:

[0068] Collect the static data of the user;

[0069] Process the static data of the user to obtain a static data vector, and input it into the pre-trained model;

[0070] Based on the input vector and the static data vector, obtain the first prediction data through the pre-trained model.

[0071] Here, the static data may include the user's gender, age, address, preferences, etc.

[0072] In this way, by combining the static data and the behavior event data for prediction, the prediction accuracy is improved.

[0073] In some specific embodiments, before the obtaining the behavior event data of the user within a preset time and inputting the behavior event data into the pre-trained model, the method further includes:

[0074] Obtain and parse the training behavior sequence to obtain a plurality of behavior events;

[0075] According to the replacement rule, cover part of the behavior events in the training behavior sequence, and fill the preset identifier into the covered behavior events to obtain the first behavior sequence;

[0076] Input the first behavior sequence into the pre-trained model to obtain the predicted behavior events;

[0077] Calculate the difference between the replaced behavior events and the predicted behavior events through the cross-entropy loss function to obtain the loss function value of the pre-trained model;

[0078] Update the parameters of the pre-trained model according to the loss function value, and complete the training when the loss function value is the smallest.

[0079] Here, the cross-entropy loss function is a loss function used in deep learning for classification problems, which is used to measure the difference between two probability distributions. The smaller the value of the cross-entropy loss function, the closer the predicted result is to the true result, and the better the performance of the model.

[0080] In one embodiment, each training randomly re-masks each behavior sequence through a dynamic mask.

[0081] Specifically, 15% of the behavior events in the behavior sequence are selected for masking to obtain masked behavior events. The masked behavior events are divided into three groups according to a preset ratio to obtain the first group of masked behavior events, the second group of masked behavior events, and the third group of masked behavior events. The first group of masked behavior events are replaced with mask tokens, the second group of masked behavior events are not replaced, and the third group of masked behavior events are replaced with a random behavior event to obtain the replaced behavior sequence, and the replaced behavior sequence is trained. Exemplarily, 15% of the behavior events can be randomly masked for each sequence, 80% of the 15% masked behavior events are replaced with a reserved special event, 10% are randomly replaced with the remaining events, and the remaining 10% remain unchanged.

[0082] Specifically, part of the behavior events in the behavior sequence are covered, the covered behavior sequence is input into the pre-trained model, the covered part is predicted through the pre-trained model, the predicted result is compared with the actual value to obtain the loss function value, the model is updated according to the loss function value, and the covering and prediction are performed again until the loss function value is the smallest, and the training is stopped.

[0083] In this way, the pre-trained model is trained to improve the accuracy of the pre-trained model.

[0084] In some specific embodiments, based on the user data and the first prediction data, through the behavior prediction model, outputting the behavior prediction result of the user in the target scenario includes:

[0085] Performing feature extraction on the user data to obtain the feature vector of the user data;

[0086] Normalizing the feature vector to obtain the target feature vector;

[0087] Based on the target feature vector and the first prediction data, through the behavior prediction model, outputting the behavior prediction result of the user in the target scenario.

[0088] Here, normalization is a data preprocessing method used to eliminate the influence of the dimension of different features to improve the comparability and stability of data analysis. Through normalization, the deviation caused by the too large difference in the numerical range between features can be eliminated.

[0089] In this way, it can be applied to multiple application scenarios through transfer learning and feature extraction methods.

[0090] In one embodiment, Figure 3 is a schematic diagram of the prediction process in the embodiments of the present application. As Figure 3 shown, the prediction process in the present application includes: collecting user behavior event data and corresponding timestamps, embedding and fusing the behavior event data and timestamps, using them as input vectors to input into a model, embedding the location and fusing it with the input vectors to obtain a first vector, inputting the first vector into a Transformer encoder, and then inputting it into a prediction layer to obtain a prediction result.

[0091] It should be understood that although Figures 1-3 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figures 1-3 at least a part of the steps in

[0092] In one embodiment, as Figure 4 shown, a user behavior prediction device is provided. The device includes: a preprocessing module 401, configured to obtain user behavior event data within a preset time, input the behavior event data into a pre-trained model, and obtain first prediction data through the pre-trained model, where the behavior event data corresponds to a timestamp, and the pre-trained model includes multiple self-attention mechanism layers; a collection module 402, configured to collect user data of the user in a target scenario, where the user data includes multimodal data; a learning module 403, configured to perform transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; a prediction module 404, configured to output a behavior prediction result of the user in the target scenario based on the user data and the first prediction data through the behavior prediction model.

[0093] As a preferred implementation manner, in the embodiments of the present application, the device further includes: a first processing module, and the first processing module is specifically configured to: determine a user, a preset time, and a first range, where the first range includes multiple scenarios; collect the behavior event data and timestamps of the user in the first range within the preset time to obtain the behavior event data of the user.

[0094] As a preferred implementation manner, in the embodiment of the present application, the preprocessing module 401 is specifically configured to: sort the behavior event data according to the time stamp to obtain a behavior sequence; perform vector embedding on the behavior sequence to obtain a behavior vector; perform vector embedding on the time stamp to obtain a time vector; fuse the behavior vector and the time vector to obtain an input vector, and input the input vector into the pre-trained model.

[0095] As a preferred implementation manner, in the embodiment of the present application, the preprocessing module 401 is specifically configured to: fuse the input vector and the rotational position encoding to obtain a first vector; based on the first vector, obtain the first prediction data through the pre-trained model.

[0096] As a preferred implementation manner, in the embodiment of the present application, the device further includes: an adjustment module, and the adjustment module is specifically configured to compare the length of the behavior sequence according to a preset sequence length; when the length of the behavior sequence is greater than the preset sequence length, obtain the behavior sequence of the most recent time and the corresponding time stamp to obtain a target behavior sequence and the corresponding time stamp; when the length of the behavior sequence is less than the preset sequence length, determine the position to be filled; fill the fill data into the position to be filled to obtain the target behavior sequence and the corresponding time stamp. Based on the input vector and the static data vector

[0097] As a preferred implementation manner, in the embodiment of the present application, the device further includes: a second processing module, and the second processing module is specifically configured to: obtain and parse a training behavior sequence to obtain a plurality of behavior events; cover some of the behavior events in the training behavior sequence according to a replacement rule, and fill a preset identifier into the covered behavior events to obtain a first behavior sequence; input the first behavior sequence into the pre-trained model to obtain predicted behavior events; calculate the difference between the replaced behavior events and the predicted behavior events through a cross-entropy loss function to obtain a loss function value of the pre-trained model; update the parameters of the pre-trained model according to the loss function value, and complete the training when the loss function value is the smallest.

[0098] As a preferred implementation manner, in the embodiment of the present application, the prediction module 404 is specifically configured to: extract features from the user data to obtain a feature vector of the user data; normalize the feature vector to obtain a target feature vector; based on the target feature vector and the first prediction data, output a behavior prediction result of the user in the target scenario through the behavior prediction model.

[0099] For the specific limitations of the user behavior prediction device, reference may be made to the limitations of the user behavior prediction method in the foregoing text, which will not be elaborated here. Each module in the above-mentioned user behavior prediction device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0100] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a user behavior prediction method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the shell of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0101] Those skilled in the art can understand that Figure 5 the structure shown in

[0102] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: S1: Obtain the behavioral event data of the user within a preset time, input the behavioral event data into a pre-trained model, and obtain first prediction data through the pre-trained model, where the behavioral event data corresponds to a timestamp, and the pre-trained model includes multiple self-attention mechanism layers; S2: Collect the user data of the user in the target scenario, where the user data includes multimodal data; S3: Perform transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; S4: Based on the user data and the first prediction data, output the behavior prediction result of the user in the target scenario through the behavior prediction model.

[0103] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Determine the user, the preset time, and a first range, where the first range includes multiple scenarios; Collect the behavioral event data and timestamps of the user in the first range within the preset time to obtain the behavioral event data of the user.

[0104] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Sort the behavioral event data according to the timestamp to obtain a behavior sequence; Perform vector embedding on the behavior sequence to obtain a behavior vector; Perform vector embedding on the timestamp to obtain a time vector; Fuse the behavior vector and the time vector to obtain an input vector, and input the input vector into the pre-trained model.

[0105] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Compare the length of the behavior sequence according to a preset sequence length; When the length of the behavior sequence is greater than the preset sequence length, obtain the behavior sequence and the corresponding timestamp at the most recent time to obtain the target behavior sequence and the corresponding timestamp; When the length of the behavior sequence is less than the preset sequence length, determine the position to be filled; Fill the position to be filled with padding data to obtain the target behavior sequence and the corresponding timestamp.

[0106] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Collect the static data of the user; Process the static data of the user to obtain a static data vector, and input it into the pre-trained model; Based on the input vector and the static data vector, obtain the first prediction data through the pre-trained model.

[0107] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining and parsing a training behavior sequence to obtain a plurality of behavior events; covering some of the behavior events in the training behavior sequence according to a replacement rule, and filling a preset identifier into the covered behavior events to obtain a first behavior sequence; inputting the first behavior sequence into the pre-trained model to obtain predicted behavior events; calculating the difference between the replaced behavior events and the predicted behavior events through a cross-entropy loss function to obtain the loss function value of the pre-trained model; updating the parameters of the pre-trained model according to the loss function value, and completing the training when the loss function value is minimized.

[0108] In one embodiment, when the processor executes the computer program, the following steps are further implemented: extracting features from the user data to obtain a feature vector of the user data; normalizing the feature vector to obtain a target feature vector; based on the target feature vector and the first prediction data, outputting a behavior prediction result of the user in the target scenario through the behavior prediction model.

[0109] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: S1: obtaining the behavior event data of the user within a preset time, inputting the behavior event data into a pre-trained model, and obtaining first prediction data through the pre-trained model, where the behavior event data corresponds to a timestamp, and the pre-trained model includes multiple self-attention mechanism layers; S2: collecting the user data of the user in the target scenario, where the user data includes multi-modal data; S3: performing transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; S4: based on the user data and the first prediction data, outputting a behavior prediction result of the user in the target scenario through the behavior prediction model.

[0110] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determining a user, a preset time, and a first range, where the first range includes multiple scenarios; collecting the behavior event data and timestamps of the user in the first range within the preset time to obtain the behavior event data of the user.

[0111] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: sorting the behavior event data according to the timestamp to obtain a behavior sequence; performing vector embedding on the behavior sequence to obtain a behavior vector; performing vector embedding on the timestamp to obtain a time vector; fusing the behavior vector and the time vector to obtain an input vector, and inputting the input vector into the pre-trained model.

[0112] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: comparing the length of the behavior sequence according to a preset sequence length; when the length of the behavior sequence is greater than the preset sequence length, obtaining the behavior sequence and the corresponding timestamp at the most recent time to obtain the target behavior sequence and the corresponding timestamp; when the length of the behavior sequence is less than the preset sequence length, determining the position to be filled; filling the filling data into the position to be filled to obtain the target behavior sequence and the corresponding timestamp.

[0113] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: collecting the static data of the user; processing the static data of the user to obtain a static data vector and inputting the static data vector into the pre-trained model; based on the input vector and the static data vector, obtaining the first prediction data through the pre-trained model.

[0114] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining and parsing the training behavior sequence to obtain a plurality of behavior events; covering some of the behavior events in the training behavior sequence according to a replacement rule and filling a preset identifier into the covered behavior events to obtain a first behavior sequence; inputting the first behavior sequence into the pre-trained model to obtain the predicted behavior events; calculating the difference between the replaced behavior events and the predicted behavior events through a cross-entropy loss function to obtain the loss function value of the pre-trained model; updating the parameters of the pre-trained model according to the loss function value, and completing the training when the loss function value is the smallest.

[0115] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: extracting features from the user data to obtain a feature vector of the user data; normalizing the feature vector to obtain a target feature vector; based on the target feature vector and the first prediction data, outputting a behavior prediction result of the user in the target scenario through the behavior prediction model.

[0116] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0117] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0118] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A user behavior prediction method, characterized in that: The method comprises: Obtaining behavioral event data of a user within a preset time, inputting the behavioral event data into a pre-trained model, and obtaining first prediction data through the pre-trained model, wherein the behavioral event data corresponds to a timestamp, and the pre-trained model includes a multi-layer self-attention mechanism layer; Collecting user data of the user in a target scenario, wherein the user data includes multimodal data; According to the target scenario and the user data, transfer learning is performed on the pre-trained model to obtain a behavior prediction model; Based on the user data and the first prediction data, the behavior prediction result of the user in the target scenario is output through the behavior prediction model.

2. The user behavior prediction method according to claim 1, characterized in that: Before obtaining the user's behavior event data within the preset time, the method further includes: Determine a user, a preset time, and a first range, wherein the first range includes a plurality of scenes; The behavior event data and timestamp of the user in the first range within the preset time are collected to obtain the behavior event data of the user.

3. The user behavior prediction method according to claim 1, characterized in that: The step of obtaining the user's behavior event data within a preset time and inputting the behavior event data into a pre-trained model includes: Sorting the behavior event data according to the timestamps to obtain a behavior sequence; Embedding the behavior sequence into a vector to obtain a behavior vector; Embed the timestamp into a vector to obtain a time vector; The behavior vector and the time vector are fused to obtain an input vector, and the input vector is input into the pre-trained model.

4. The user behavior prediction method according to claim 3, characterized in that: After the behavior event data is sorted according to the timestamp to obtain the behavior sequence, the method further includes: According to a preset sequence length, comparing the length of the behavior sequence; When the length of the behavior sequence is greater than the preset sequence length, obtaining the behavior sequence of the most recent time and the corresponding timestamp, and obtaining the target behavior sequence and the corresponding timestamp; When the length of the behavior sequence is less than the preset sequence length, determining a position to be filled; Fill the filling data into the position to be filled, and obtain the target behavior sequence and the corresponding timestamp.

5. The user behavior prediction method according to claim 3, characterized in that: The step of inputting the behavior event data into a pre-training model and obtaining first prediction data through the pre-training model includes: Collecting static data of the user; Processing the user's static data to obtain a static data vector, and inputting the static data vector into the pre-trained model; Based on the input vector and the static data vector, the first prediction data is obtained through the pre-trained model.

6. The user behavior prediction method according to claim 3, characterized in that: Before obtaining the behavior event data of the user within the preset time and inputting the behavior event data into the pre-training model, the method further includes: Acquire and parse the training behavior sequence to obtain multiple behavior events; According to the replacement rule, some behavior events in the training behavior sequence are covered, and preset identifiers are filled into the covered behavior events to obtain a first behavior sequence; Inputting the first behavior sequence into the pre-trained model to obtain predicted behavior events; Calculating the difference between the replaced behavior event and the predicted behavior event through a cross entropy loss function to obtain a loss function value of the pre-trained model; According to the loss function value, the parameters of the pre-trained model are updated, and the training is completed when the loss function value is minimized.

7. The user behavior prediction method according to claim 1, characterized in that: The outputting the user's behavior prediction result in the target scenario based on the user data and the first prediction data through the behavior prediction model includes: Extracting features from the user data to obtain a feature vector of the user data; Normalizing the feature vector to obtain a target feature vector; Based on the target feature vector and the first prediction data, the behavior prediction result of the user in the target scenario is output through the behavior prediction model.

8. A user behavior prediction device, characterized in that: The device comprises: A preprocessing module, used for obtaining behavioral event data of a user within a preset time, inputting the behavioral event data into a pretraining model, and obtaining first prediction data through the pretraining model, wherein the behavioral event data corresponds to a timestamp, and the pretraining model includes a multi-layer self-attention mechanism layer; A collection module, used to collect user data of the user in the target scene, wherein the user data includes multimodal data; A learning module, used to perform transfer learning on the pre-trained model according to the target scenario and the user data to obtain a behavior prediction model; A prediction module is used to output a behavior prediction result of the user in the target scenario based on the user data and the first prediction data through the behavior prediction model.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images