User behavior prediction method and device, electronic equipment, medium and program product

Abstract

Embodiments of the present specification disclose a user behavior prediction method and device, electronic equipment, medium and program product. The method comprises: obtaining a current behavior sequence of a target user; the current behavior sequence is composed of a plurality of current interaction records arranged in chronological order; each current interaction record comprises current behavior type information and current point information; each current interaction record in the current behavior sequence is queried based on a point vector library to obtain a current behavior embedding sequence; the current behavior embedding sequence is composed of current point vectors corresponding to each current interaction record arranged in chronological order; and the current behavior embedding sequence is input into a first target model to output a future behavior prediction result corresponding to the target user.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a user behavior prediction method, device, electronic device, medium, and program product. Background Technology

[0002] Currently, user interactions across various applications are becoming increasingly complex and diverse, generating massive amounts of high-dimensional, time-series behavioral data with strong temporal correlations. Faced with this data, traditional behavior prediction systems typically employ two main approaches: one is based on shallow neural network models with ID-type features, such as DIN and SASRec, which can effectively model the dynamic changes in user interests; the other relies on large-scale pre-trained language models, leveraging their powerful representational capabilities to semantically model and predict users' historical behavioral sequences. Summary of the Invention

[0003] This specification provides a user behavior prediction method, device, electronic device, medium, and program product. By combining the target user's current behavior type information with their current location information to embed and represent the current behavior, it can more accurately capture the spatiotemporal characteristics of user behavior, avoiding the problem of insufficient feature dimensions caused by relying solely on a single behavior type or location information, thus improving the accuracy of behavior prediction. Furthermore, it achieves time-complexity mapping queries through a pre-built location vector library, and uses time-complexity vector mapping instead of descriptive text related to the user's current behavior as input to the first target large model, effectively reducing the computational overhead of the behavior sequence vectorization process. This significantly improves the system processing efficiency in large-scale user behavior prediction scenarios. Moreover, by leveraging the powerful temporal modeling capabilities of the large model, it can effectively uncover hidden logical connections and trend patterns in the current behavior sequence, making future behavior prediction results more closely aligned with users' actual behavioral habits. This provides a reliable decision-making basis for subsequent personalized service recommendations, user demand responses, and other scenarios. The above technical solutions are as follows: Firstly, embodiments of this specification provide a user behavior prediction method, including: Obtain the target user's current behavior sequence; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each current interaction record includes current behavior type information and current location information; based on the location vector library, perform a mapping query on each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence; the current behavior embedding sequence consists of the current location vectors corresponding to each current interaction record arranged in order of occurrence timestamp; input the current behavior embedding sequence into the first target large model, and output the future behavior prediction result corresponding to the target user.

[0004] Secondly, embodiments of this specification provide a user behavior prediction device, the device comprising: The first acquisition module is used to acquire the current behavior sequence of the target user; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each of the current interaction records includes current behavior type information and current location information; The mapping query module is used to perform mapping queries on each current interaction record in the above current behavior sequence based on the point vector library to obtain the current behavior embedding sequence; the above current behavior embedding sequence consists of the current point vectors corresponding to each of the above current interaction records arranged in order of occurrence timestamp. The user behavior prediction module is used to embed the current behavior sequence into the first target large model and output the future behavior prediction result corresponding to the target user.

[0005] Thirdly, embodiments of this specification provide an electronic device, including: a processor and a memory; The processor is connected to the memory. The aforementioned memory is used to store executable program code; The processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method provided in the first aspect of the embodiments of this specification.

[0006] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method provided in the first aspect of embodiments of this specification.

[0007] Fifthly, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute the user behavior prediction method provided in the first aspect of embodiments of this specification.

[0008] In this embodiment, on the one hand, a mapping query is performed on each current interaction record in the aforementioned current behavior sequence based on the point vector library to obtain a current behavior embedding sequence. The current behavior embedding sequence consists of the current point vectors corresponding to each current interaction record arranged in order of occurrence timestamp. Each current interaction record includes current behavior type information and current point information. Thus, by combining the target user's current behavior type information and current point information for current behavior embedding representation, the spatiotemporal features of user behavior can be captured more accurately, avoiding the problem of insufficient feature dimensions caused by relying solely on a single behavior type or point information, thereby improving the accuracy of behavior prediction. On the other hand, by using pre-constructed... The point vector library enables time-complexity mapping queries and replaces the descriptive text related to the user's current behavior with time-complexity vector mapping as the input to the first target large model. This effectively reduces the computational overhead of the behavior sequence vectorization process and can significantly improve the system processing efficiency in large-scale user behavior prediction scenarios. On the other hand, by using the first target large model to process the current behavior embedding sequence, and leveraging the powerful temporal modeling capabilities of the large model, the hidden logical connections and trend patterns in the current behavior sequence can be effectively mined. This makes the future behavior prediction results more consistent with the user's real behavior habits, providing a reliable decision-making basis for subsequent personalized service recommendations, user demand responses, and other scenarios. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of the architecture of a user behavior prediction system provided for an exemplary embodiment of this specification; Figure 2 A flowchart illustrating a user behavior prediction method provided as an exemplary embodiment of this specification; Figure 3 A schematic diagram illustrating the implementation process of a mapping query as provided in an exemplary embodiment of this specification; Figure 4 A schematic diagram illustrating the construction process of a point vector library provided for an exemplary embodiment of this specification; Figure 5 A flowchart illustrating another user behavior prediction method provided as an exemplary embodiment of this specification; Figure 6 A flowchart illustrating a model training method provided as an exemplary embodiment of this specification; Figure 7A schematic diagram of the structure of a user behavior prediction device provided for an exemplary embodiment of this specification; Figure 8 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. Detailed Implementation

[0011] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0012] The terms "first," "second," "third," etc., used in this specification, claims, and the foregoing drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0013] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the current behavior sequence and target interaction behavior involved in this specification were obtained with full authorization.

[0014] With the rapid development of mobile internet and digital finance, user operation paths within applications are becoming increasingly complex, generating massive amounts of high-dimensional, sparse, and long-cycle behavioral sequence data. Accurately predicting users' next actions has become a core technical challenge for improving user experience, optimizing product conversion, and controlling business risks. Related technologies mainly rely on ID-based sequence models (such as DIN and SASRec) or directly applying large language models end-to-end for behavioral prediction. However, the former is severely limited by the cold start problem and model capacity, making it difficult to generalize to new pages or functions; the latter suffers from a huge increase in computational cost due to the dramatic increase in length after behavioral sequence textification, failing to meet the latency and throughput requirements of industrial-grade online services. Especially in process-oriented financial businesses, where user paths have many branches and decision-making links are long, traditional methods generally have low prediction accuracy, making it difficult to support the needs of refined operations.

[0015] Based on this, embodiments of this specification provide a user behavior prediction method. On one hand, based on a point vector library, a mapping query is performed on each current interaction record in the aforementioned current behavior sequence to obtain a current behavior embedding sequence. The current behavior embedding sequence consists of current point vectors corresponding to each current interaction record arranged in chronological order of occurrence timestamps. Each current interaction record includes current behavior type information and current point information. Thus, by combining the target user's current behavior type information and current point information for current behavior embedding representation, the spatiotemporal characteristics of user behavior can be captured more accurately, avoiding the problem of insufficient feature dimensions caused by relying solely on a single behavior type or point information, thereby improving the accuracy of behavior prediction. Furthermore, by using a pre-built point vector library to achieve time-complexity mapping queries, and replacing the descriptive text related to the user's current behavior with time-complexity vector mapping as the input of the first target large model, the computational overhead of the behavior sequence vectorization process is effectively reduced, which can significantly improve the system processing efficiency in large-scale user behavior prediction scenarios. On the other hand, by using the first target large model to process the current behavior embedded sequence, and leveraging the powerful temporal modeling capabilities of the large model, the hidden logical connections and trend patterns in the current behavior sequence can be effectively mined, making the future behavior prediction results more in line with the user's real behavior habits, and providing a reliable decision-making basis for subsequent personalized service recommendations, user demand response, and other scenarios.

[0016] Please refer to the following. Figure 1 , Figure 1 This is a schematic diagram of the architecture of a user behavior prediction system provided as an exemplary embodiment of this specification. Figure 1 As shown, the user behavior prediction system may include: a terminal 110 and a server 120. Wherein: Terminal 110 may include one or more user terminals. One or more user-version applications may be installed on terminal 110. Terminal 110 can establish a data relationship with a network and establish a data connection relationship with server 120 through the network, such as sending the current behavior sequence of the target user and receiving user behavior prediction results. Terminal 110 may be, but is not limited to, a mobile phone, tablet computer, laptop computer, or other device with user-version applications installed.

[0017] Specifically, during the process of a user accessing the application through terminal 110, terminal 110 can, but is not limited to, collect the target user's current behavior sequence in real time. This current behavior sequence consists of multiple current interaction records arranged in chronological order of their occurrence timestamps. Each current interaction record includes current behavior type information and current location information. Then, based on a location vector library, a mapping query is performed on each current interaction record in the current behavior sequence to obtain a current behavior embedding sequence. This current behavior embedding sequence consists of the current location vectors corresponding to each current interaction record arranged in chronological order of their occurrence timestamps. Finally, the current behavior embedding sequence is input into the first target large model, outputting the future behavior prediction result corresponding to the target user.

[0018] Optionally, after the target user's current behavior sequence or the behavior prediction instruction triggered by the target user, the terminal 110 can also send the current behavior sequence to the server 120 via the network to trigger the server 120 to perform user behavior prediction based on the target user's current behavior sequence.

[0019] Server 120 can be a server that can provide various user behavior prediction services. Server 120 can be, but is not limited to, a hardware server, a virtual server, a cloud server, etc.

[0020] Optionally, server 120 may, but is not limited to, receive the current behavior sequence corresponding to the target user sent by terminal 110, and perform corresponding user behavior prediction using the user behavior prediction scheme provided in the embodiments of this specification to obtain the future behavior prediction result corresponding to the target user. Simultaneously, server 120 may also return the aforementioned future behavior prediction result to terminal 110.

[0021] It is understood that the above-mentioned user behavior prediction process is not limited to being executed by the server 120, but can also be executed by the terminal 110. This specification does not specifically limit this, and the following embodiments all use the terminal 110 to perform the above-mentioned user behavior prediction as an example.

[0022] The network can be a medium that provides a communication link between server 120 and terminal 110, or it can be the Internet, which includes network devices and transmission media, and is not limited thereto. The transmission media can be a wired link, such as, but not limited to, coaxial cable, fiber optic cable, and digital subscriber line (DSL), or a wireless link, such as, but not limited to, wireless fidelity (WIFI), Bluetooth, and mobile device networks.

[0023] Understandably, Figure 1The number of terminals 110 and servers 120 in the user behavior prediction system shown is merely an example. In a specific implementation, the user behavior prediction system can contain any number of terminals 110 and servers 120. This specification does not specifically limit this. For example, but not limited to, terminal 110 can be a terminal cluster composed of multiple terminals, and server 120 can be a server cluster composed of multiple servers.

[0024] Next, combine Figure 1 This document describes the user behavior prediction method provided in the embodiments of this specification. Please refer to [link / reference] for details. Figure 2 This is a flowchart illustrating a user behavior prediction method provided in an exemplary embodiment of this specification. Figure 2 As shown, this user behavior prediction method includes the following steps: S202, Obtain the current behavior sequence of the target user; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each current interaction record includes current behavior type information and current location information.

[0025] Specifically, the aforementioned current behavior sequence may be, but is not limited to, a sequence of triplets consisting of current behavior type information, current location information, and timestamps corresponding to the interactive behaviors of the target user within the current time window (e.g., but not limited to the day, week, month, year, etc.). The aforementioned current behavior type information is used to identify the specific category of the target user's current behavior, such as, but not limited to, "exposure," "click," and "access." The aforementioned current location information is used to accurately identify the current interaction position of the target user's behavior, and may include, but is not limited to, descriptive text information corresponding to the current location (current interaction position), such as, but not limited to, SPM location information composed of site / business positions, page positions, block positions, element positions, etc., corresponding to the current interaction position.

[0026] Optionally, the target user's current behavior sequence can be generated based on, but is not limited to, the target user's interaction data with the system or application within the current time window. For example, but not limited to, if the current time window is the most recent 24 hours, the system will collect all the target user's interactive operations during this period, including but not limited to browsing product detail pages, clicking product purchase buttons, etc. (current interaction records), and sort them sequentially from earliest to latest according to the timestamps of the actions (current interaction records), forming an ordered current behavior sequence. By obtaining the target user's current behavior sequence, the system can comprehensively and accurately capture the target user's recent behavioral trajectory, providing basic data support for subsequent behavior prediction.

[0027] It is understood that the above-mentioned acquisition of the target user's current behavior sequence may be triggered after receiving the target user's input target behavior prediction request, or it may be automatically triggered when the target user's current interactive behavior is detected in the system or application, etc. The embodiments in this specification do not limit this.

[0028] S204, based on the point vector library, perform a mapping query on each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence; the current behavior embedding sequence consists of the current point vectors corresponding to each current interaction record arranged in order of occurrence timestamp.

[0029] Specifically, the point vector library is a pre-built structured database used to store the feature representations (point vectors) of all interactive points in a system or application. Each interactive point (such as, but not limited to, the "Add to Cart" button on the product details page, the "Recommended Products" block on the homepage, etc.) corresponds to a dense vector (point vector), which can capture the core features of the point from multiple dimensions such as semantics, function, and user intent.

[0030] Optionally, the point vectors in the aforementioned point vector library can be, but are not limited to, pre-encoded using the second objective large model based on the full point information of the system or application. During mapping queries, the system can first parse the current point information (such as SPM point encoding) in the current interaction record, and then use this current point information as an index to perform precise matching in the point vector library, quickly locating the corresponding current point vector. For example, if the current point information corresponding to a certain current interaction record is "product details page - buy now button," the system will find the vector (current point vector) corresponding to that button in the point vector library. This vector may contain numerical representations of feature dimensions such as "purchase intent," "product conversion stage," and "details page scenario." By sequentially performing the above mapping query operation on each interaction record in the current behavior sequence, the originally discrete interaction behaviors are transformed into continuous vector forms, ultimately forming a current behavior embedding sequence arranged in timestamp order, providing computable input data for the subsequent behavior prediction model (first objective large model).

[0031] In some possible embodiments, such as Figure 3 As shown, the above S204, which maps and queries each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence, can also include, but is not limited to, the following steps: S302, based on the global mapping relationship, perform a mapping query on each current interaction record in the current behavior sequence to obtain the current discretized behavior sequence corresponding to the current behavior sequence; the current discretized behavior sequence consists of the current integer identifiers corresponding to each current interaction record arranged in the order of occurrence timestamp.

[0032] Specifically, the aforementioned global mapping relationship is used to represent the mapping relationship between the combination of each behavior type information and each point information and the unique integer identifier. That is, based on this global mapping relationship, the current interaction record can be mapped to a unique current integer identifier. For example, but not limited to, if in the global mapping relationship, "click - product details page" corresponds to the unique integer identifier 0015 and "access - product list page" corresponds to the unique integer identifier 0123, then when the current behavior type information in the current interaction record is "click" and the current point information is "product details page", the system will directly map it to the current integer identifier 0015; if the current behavior type information is "access" and the current point information is "product list page", the system will directly map it to the current integer identifier 0123.

[0033] For each current interaction record in the current behavior sequence, the system uses a global mapping relationship to find the corresponding current integer identifier. These current integer identifiers are then arranged in chronological order according to the timestamps of the interactions, forming the current discretized behavior sequence. In this way, long sequences of interaction records, originally existing in text or encoded form, are transformed into integer sequences that the model can understand. This preserves the behavior type and location information corresponding to the user's interaction while achieving discretization input compatibility, laying the foundation for subsequent mapping into vector form.

[0034] S304, based on the current integer identifier corresponding to each current interaction record, query the point vector library to obtain the current point vector corresponding to each current interaction record.

[0035] Specifically, the point vector library is pre-built, storing the mapping relationship between integer identifiers and corresponding point vectors for all possible point information corresponding to interaction records (combinations of point information and behavior type information). The system uses each current integer identifier in the current discretized behavior sequence as an index to perform precise matching in the point vector library, thereby quickly obtaining the current point vector corresponding to that current integer identifier.

[0036] In the embodiments of this specification, on the one hand, efficient mapping is achieved through the global mapping relationship between interaction records (a combination of location information and behavior type information) and integer identifiers, transforming unstructured behavior data into low-dimensional dense vectors and avoiding the high computational cost of text sequence processing during behavior prediction; on the other hand, through the combined design of integer identifiers and location vector libraries, new interaction points only need to be assigned a unique integer identifier to quickly access the system, solving the feature loss problem in cold start scenarios; furthermore, through standardized vector mapping query methods, the consistency of feature spaces for different types of interaction records is ensured, providing a unified input format for subsequent large model processing and improving the generalization ability and stability of the prediction model (the first target large model).

[0037] S306, Arrange and combine the current point vectors corresponding to each current interaction record in the current behavior sequence according to the order of occurrence timestamps to obtain the current behavior embedding sequence.

[0038] Specifically, the current behavior sequence consists of multiple interaction records occurring sequentially in time. Each interaction record corresponds to a unique current integer identifier, and each current integer identifier is associated with a corresponding current point vector in the point vector library. The system first extracts the timestamp information of each current interaction record in the current behavior sequence to ensure that these records are arranged strictly in chronological order, avoiding any inaccurate reflection of the user's actual behavior logic due to disordered chronological order. Then, based on the sorted current interaction records, the system retrieves the corresponding current point vector from the point vector library one by one using the current integer identifier, and concatenates these chronologically retrieved current point vectors to form an ordered vector sequence, i.e., the current behavior embedding sequence. This current behavior embedding sequence not only preserves the chronological characteristics of the target user's behavior but also transforms the discrete current interaction records into a continuous feature representation in vector form, providing structured and sequential input data for the subsequent prediction of the target user's behavior by the primary target model.

[0039] In some possible embodiments, after obtaining the target user's current behavior sequence, and before performing a mapping query on each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence, the current behavior sequence may be filtered first (e.g., but not limited to, removing empty sequences or samples with inconsistent lengths, filtering dirty data with empty or erroneous point information, etc.) to obtain the target current behavior sequence. The process of performing a mapping query on each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence may include, but is not limited to, performing a mapping query on each current interaction record in the target current behavior sequence based on the point vector library to obtain the current behavior embedding sequence.

[0040] Please continue to refer to the following. Figure 2 ,like Figure 2 As shown, in step S204 above, after mapping and querying each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence, this user behavior prediction method may also include, but is not limited to: S206: Embed the current behavior sequence into the first target large model and output the prediction result of the future behavior of the target user.

[0041] Specifically, the aforementioned first target large model can be, but is not limited to, a pre-trained large language model. After the current behavior embedding sequence is input into the first target large model, the encoder layer of the model performs context encoding on each current point vector in the current behavior embedding sequence to capture the correlation features between target user interaction behaviors at different time points, such as, but not limited to, the potential intent association of the target user clicking on different functional modules consecutively; the decoder layer then generates future behavior prediction results within the future time window based on the context behavior feature representation output by the encoder and combined with the user behavior patterns learned during the model training.

[0042] In this embodiment, on the one hand, a mapping query is performed on each current interaction record in the aforementioned current behavior sequence based on the point vector library to obtain a current behavior embedding sequence. The current behavior embedding sequence consists of the current point vectors corresponding to each current interaction record arranged in order of occurrence timestamp. Each current interaction record includes current behavior type information and current point information. Thus, by combining the target user's current behavior type information and current point information for current behavior embedding representation, the spatiotemporal features of user behavior can be captured more accurately, avoiding the problem of insufficient feature dimensions caused by relying solely on a single behavior type or point information, thereby improving the accuracy of behavior prediction. On the other hand, by using pre-constructed... The point vector library enables time-complexity mapping queries and replaces the descriptive text related to the user's current behavior with time-complexity vector mapping as the input to the first target large model. This effectively reduces the computational overhead of the behavior sequence vectorization process and can significantly improve the system processing efficiency in large-scale user behavior prediction scenarios. On the other hand, by using the first target large model to process the current behavior embedding sequence, and leveraging the powerful temporal modeling capabilities of the large model, the hidden logical connections and trend patterns in the current behavior sequence can be effectively mined. This makes the future behavior prediction results more consistent with the user's real behavior habits, providing a reliable decision-making basis for subsequent personalized service recommendations, user demand responses, and other scenarios.

[0043] Optionally, the process of S206 above, which involves inputting the current behavior embedding sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user, may include, but is not limited to: inputting the current behavior embedding sequence into the first target large model, performing inference analysis based on the user behavior vector library and the current behavior embedding sequence, and outputting the future behavior prediction result corresponding to the target user. The aforementioned user behavior vector library includes sample behavior vectors corresponding to multiple sample interaction records with known occurrence timestamps for at least one sample user.

[0044] The system maintains an offline user behavior vector library, which stores sample behavior vector sequences corresponding to complete behavior sequences of massive historical sample users (multiple sample interaction records arranged in chronological order of occurrence timestamps). Each sample behavior vector sequence is bound to its occurrence timestamp. During the inference phase, the primary target model can, but is not limited to, use the input current behavior embedding sequence as a query vector to retrieve the K semantically most similar sample user behavior vector sequences from the user behavior vector library using an approximate nearest neighbor algorithm (such as Faiss). The retrieval process strictly follows the timestamp sorting to ensure that only historical trajectories with "similar behavior patterns" and "consistent temporal evolution paths" are matched, avoiding mismatches across temporal patterns. For each retrieved sample user behavior vector sequence, the next one or more behaviors (i.e., "next steps or multiple actions") that occur after the current behavior embedding sequence are extracted, and their distribution probability among all similar samples is calculated. For example, if 87% of similar sample users clicked "coupon" and then visited the "discount zone," the system outputs this behavior (visiting the discount zone) as a high-confidence prediction result. This process does not rely on user identity identification; it is entirely based on the semantic alignment and temporal continuity of behavioral patterns, enabling accurate prediction of cold-start user behavior.

[0045] In the embodiments of this specification, on the one hand, without relying on the historical behavior accumulation of target users, it can match groups with similar behavioral patterns based on only a small amount of current behavior, effectively solving the prediction dilemma of "no historical data to rely on" for new users or low-activity users, and significantly improving the usability of the system in new scenarios, that is, breaking through the dependence on individual data and achieving efficient prediction for cold starts; on the other hand, by strictly binding timestamps for sequence sorting and comparison, it ensures that the prediction is based on the behavioral evolution path of "actual occurrence order" rather than static feature combination, avoiding misjudgments caused by disordered behavior order, making the prediction results closer to the real user intent, and enhancing the temporal sensitivity and prediction accuracy of behavior sequences; furthermore, the inference logic is replaced by lightweight vector retrieval instead of complex online model recomputation. The first target large model can rely entirely on the pre-built user behavior vector library for behavior prediction, greatly reducing online service latency (millisecond-level response), while supporting efficient indexing of the sample library, with good engineering scalability, reducing model training costs, and improving online inference efficiency.

[0046] Optionally, the aforementioned future behavior prediction results may include, but are not limited to, at least one of the following: future behavior generative prediction results and future behavior discriminative prediction results. The aforementioned future behavior generative prediction results refer to complete behavior path predictions output through sequence generation, which may include, but are not limited to, future behavior prediction sequences corresponding to the target user. These future behavior prediction sequences consist of at least one combination of future predicted interactive behaviors arranged in chronological order of future prediction timestamps. The aforementioned combination of future predicted interactive behaviors includes future behavior prediction type information (e.g., but not limited to, click, input, exposure, etc.) and future behavior prediction point information (e.g., but not limited to, descriptions of points the target user may interact with in the future). The aforementioned future behavior discriminative prediction results refer to multi-option predictions output through probability distribution, which may include, but are not limited to, multiple candidate interactive behavior combinations that the target user may next, and the probability of future interaction occurrence corresponding to each candidate interactive behavior combination. The aforementioned candidate interactive behavior combinations include candidate interactive behavior type information and candidate interactive behavior point information.

[0047] In the embodiments of this specification, on the one hand, the first target large model performs generative prediction to support long-sequence behavioral path planning in the user interaction process (such as, but not limited to, account opening process guidance in financial business), and on the other hand, it performs discriminative prediction to meet real-time decision support in the user interaction process (such as prediction of clicks for wealth management product recommendations), thereby achieving full-scenario coverage from macro-path to micro-decision. The above dual-mode prediction mechanism enables user behavior prediction to adapt to the needs of multiple scenarios. On the other hand, by making the future behavior prediction results include a precise combination of behavior type and location information, it can directly connect to the front-end interaction logic, avoid the semantic parsing cost of traditional text prediction, and improve the response speed of the recommendation system, that is, improve the implementation efficiency through structured output. Furthermore, it can choose to output the probability distribution provided by the discriminative prediction results of future behavior to provide quantitative decision-making basis for risk control scenarios (such as financial transaction security verification), reduce the misjudgment rate, and enhance the reliability of decision-making. Alternatively, it can choose to output the generative prediction results of future behavior to realize multi-step prediction of user future behavior, and strictly follow the timestamp order to ensure the predictive coherence of complex business processes (such as credit application steps), and improve the path completion rate of users in specific interaction processes.

[0048] Please refer to the following. Figure 4 ,like Figure 4 As shown, the construction process of the above-mentioned point vector library may include, but is not limited to, the following steps: S402, acquire multiple preset point information and preset behavior type configuration information corresponding to each preset point information; the preset behavior type configuration information includes at least one preset behavior type information configured for the corresponding preset point.

[0049] Specifically, the aforementioned preset location information may include, but is not limited to, information about user-interactive locations pre-defined in systems or applications that can interact with users. The aforementioned preset behavior type configuration information may include, but is not limited to, pre-defined behavior types (e.g., but not limited to, clicking, accessing) that users can perform at each preset location. The acquisition of multiple preset location information and corresponding preset behavior type configuration information can be achieved, but is not limited to, through a front-end configuration database connected to the interactive system or application. This database stores the attribute information of all locations (such as location name, page path, and interface coordinates) and corresponding behavior type rules, supporting dynamic updates based on business needs—for example, when the platform adds a "Smart Investment Consulting" function location, the identifier of that location can be added to the database simultaneously, and the corresponding user-executable behavior type can be configured, providing basic data support for the subsequent construction of the location vector library.

[0050] S404, based on the global mapping relationship, combine and map the information of each preset point and the corresponding preset behavior type information to obtain the corresponding preset integer identifier.

[0051] Specifically, the aforementioned global mapping relationship is a pre-constructed rule system used to uniquely map the combination of location information and behavior type information to integer identifiers. This rule system can employ multi-level encoding logic, for example, but not limited to, first assigning basic integer segments to all preset location information (e.g., location A corresponds to 1-100, location B corresponds to 101-200), and then assigning sub-identifiers within the corresponding basic integer segments for different preset behavior types under each preset location (e.g., the "click" behavior for location A corresponds to 101, and the "access" behavior corresponds to 102); it can also employ a hash mapping algorithm, generating fixed-length integer identifiers by hashing the combined string of preset location information and preset behavior type information, and ensuring the uniqueness of the mapping through a conflict detection mechanism. For example, when the preset location is the "Smart Investment Consultation" button (identified as P001) and the preset behavior type is "click" (identified as B001), the global mapping relationship can map this combination to the integer 1001.

[0052] S406, input each preset integer identifier into the second target large model, and generate the corresponding preset point vector based on each preset integer identifier.

[0053] Specifically, the aforementioned second target model can be, but is not limited to, a pre-trained vector generation-type large language model. Its input layer receives a preset integer identifier, which is then converted into a low-dimensional dense initial vector through an embedding layer. This initial vector is then processed by a multi-layer Transformer encoder for feature extraction and semantic fusion, ultimately outputting a preset point vector with uniform dimensions. During the generation of this preset point vector, the second target model can, but is not limited to, associative encoding with the corresponding point scene and behavioral attributes behind the preset integer identifier. For example, when the input integer 1001 (corresponding to the "click" action of the "intelligent investment advisory consultation" button) is given, the model automatically captures potential semantic features such as "financial services" and "user-initiated interaction," ensuring that the generated vector not only contains basic information about the point and behavior but also incorporates contextualized associative attributes, providing richer feature support for subsequent behavior prediction.

[0054] S408, associate each preset point vector with its corresponding preset integer identifier and store it in the point vector library.

[0055] Specifically, the aforementioned point vector library adopts a key-value storage structure, where the key is a preset integer identifier and the value is the corresponding preset point vector.

[0056] In the embodiments of this specification, on the one hand, by converting complex interaction combinations (preset location information and preset behavior type information) into corresponding single preset integer identifiers, the vector query complexity is reduced from O(n) to O(1). Combined with the key-value pair storage structure in the location vector library, the call latency of location vectors is compressed to the microsecond level, improving the call efficiency of location vectors in the behavior prediction process. On the other hand, by inputting each preset integer identifier into the second target large model for encoding, the corresponding preset location vector is generated, realizing the conversion from text description to dense vector embedding, and solving the adaptation problem between large language models and tracking data.

[0057] In some possible embodiments, the system may also acquire (e.g., but not limited to, detecting or receiving information sent by the development end) new location information and corresponding new behavior type configuration information at preset time intervals (e.g., but not limited to weekly, daily, monthly, etc.). The new behavior type configuration information includes at least one new behavior type information configured for the new location. Then, based on the global mapping relationship, the new location information and its corresponding combinations of various new behavior type information are combined and mapped to obtain the corresponding new integer identifier. Finally, each new integer identifier is input into the second target large model, and a corresponding new location vector is generated based on each new integer identifier. The new integer identifier and the new location vector are then associated and stored in the location vector library.

[0058] In this embodiment, by periodically acquiring information on newly added points and corresponding configuration information for newly added behavior types, and combining this with global mapping relationships to generate new integer identifiers and encode them into vectors stored in the point vector library, dynamic expansion and real-time updates of the point vector library are achieved. This mechanism ensures that the system can continuously adapt to changes in business scenarios. Whether it is new interactive points added for new functions or new behavior types added to existing points, they can be quickly incorporated into the behavior prediction system, avoiding prediction deviations caused by lagging point information. At the same time, the generation of new point vectors uses the same integer identifier encoding logic as the preset points, ensuring the consistency and compatibility of the vector space, enabling new and old point vectors to work together in the same prediction model, further improving the comprehensiveness and accuracy of behavior prediction. In addition, the reuse of the key-value pair storage structure during dynamic updates continues the O(1) query efficiency advantage. Even as the size of the point vector library continues to increase with business expansion, it can still maintain microsecond-level call latency, ensuring the real-time performance and efficiency of behavior prediction.

[0059] Please refer to the following. Figure 5 This is a flowchart illustrating another user behavior prediction method provided in an exemplary embodiment of this specification. Figure 5 As shown, this user behavior prediction method includes the following steps: S502, Obtain the current behavior sequence of the target user; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each current interaction record includes current behavior type information and current location information.

[0060] Specifically, S502 is the same as S202, and will not be repeated here.

[0061] S504, based on the point vector library, perform a mapping query on each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence; the current behavior embedding sequence consists of the current point vectors corresponding to each current interaction record arranged in order of occurrence timestamp.

[0062] Specifically, S504 is the same as S204, and will not be repeated here.

[0063] S506. Based on the target user identifier corresponding to the target user, query the user behavior vector library to obtain the pre-computed embedding sequence of the target user's historical behavior.

[0064] Specifically, the aforementioned historical behavior pre-computed embedding sequence consists of historical behavior vectors corresponding to multiple historical interaction records of the target user arranged in order of occurrence timestamps; the aforementioned user behavior vector library may include, but is not limited to, multiple historical behavior pre-computed embedding sequences corresponding to each user.

[0065] Optionally, the user behavior vector library is pre-constructed by processing the historical interaction data of each user. Each historical behavior vector is based on the behavior type information and location information in the corresponding historical interaction record. This information is mapped to the location vector library and then fused with the time characteristics and behavior frequency characteristics of the historical interaction record. For example, for a historical interaction record of user A, the corresponding location vector is first obtained from the location vector library based on its location information. Then, features such as the time of day in which the record occurred (e.g., morning peak, off-peak, evening peak) and the frequency of the behavior type in user A's historical behavior are extracted. These features are then concatenated or weighted and summed with the location vector to generate the historical behavior vector corresponding to the historical interaction record. The pre-calculated embedding sequence of historical behavior for each user stored in the user behavior vector library is arranged in order from morning to night according to the timestamp of the historical interaction record and is periodically updated based on newly generated historical interaction data to ensure that the pre-calculated embedding sequence of historical behavior reflects the user's recent behavioral habits.

[0066] Optionally, before querying the user behavior vector library based on the target user identifier corresponding to the target user to obtain the historical behavior pre-computed embedding sequence of the target user, the behavior prediction method may also include, but is not limited to, the following: after detecting the target user's target interaction behavior, performing a mapping query on the target interaction record corresponding to the target interaction behavior based on the global mapping relationship to obtain the target integer identifier corresponding to the target interaction record; then, inputting the target integer identifier into the first target large model to generate the corresponding target behavior vector, and associating and storing the target behavior vector and the target user identifier in the user behavior vector library. During the user's online interaction, the system can monitor the user's interaction behavior in real time and automatically trigger the dynamic update mechanism of the user behavior vector library to ensure that the user's latest behavior representation (target behavior vector) is always synchronously integrated with the historical behavior pre-computed embedding sequence, thereby supporting the subsequent behavior prediction model to accurately capture and respond to the user's short-term intentions.

[0067] The first target model is a pre-trained large language model. Its input is the target integer identifier corresponding to the target interaction record. The output includes a hidden state corresponding to the special [USER] token. This hidden state can be used as the target user's target behavior vector (User Embedding) for subsequent tasks such as target user behavior prediction. When storing the target behavior vector with the target user identifier, the timestamp of the target interaction behavior corresponding to the target behavior vector is also recorded simultaneously. This ensures that when updating the target user's historical behavior pre-computed embedding sequence, the newly generated target behavior vector is inserted into the corresponding position according to the time sequence, ensuring the temporal continuity of the sequence.

[0068] S508 inputs the historical behavior pre-calculated embedding sequence and the current behavior embedding sequence into the first target large model, and outputs the future behavior prediction result corresponding to the target user.

[0069] Specifically, but not limited to, the historical behavior pre-computed embedding sequence can be arranged in timestamp order and concatenated with the current behavior embedding sequence to form a fused sequence containing long-term user behavior trends and real-time behavior characteristics. The first objective big model encodes the fused sequence through a multi-layer attention mechanism to capture the dependencies between behaviors at different time points, such as changes in the frequency of user access to the same type of service at different times and the sequential logic of behavior triggering. During the training phase, the model has learned typical features of user behavior patterns, such as the behavior chain of users adding items to their cart and placing orders after browsing products in e-commerce scenarios, or the interest migration pattern of users from casual reading to in-depth reading in information platforms. During prediction, the first objective big model outputs future behavior prediction results based on the key features extracted from the fused sequence, including information on the type of behavior, location information, behavior probability, and time interval of behavior triggering that the target user may produce within a preset time window.

[0070] In the embodiments of this specification, on the one hand, by pre-computing and storing historical behavior embedding sequences through a user behavior vector library, the high computational cost of real-time processing of massive historical data is avoided. Based on the pre-built point vector library, vector queries with O(1) complexity are realized. With the dual sequence input mechanism of historical behavior pre-computed embedding sequences and current behavior embedding sequences, the overall prediction process response delay can be reduced, the computational bottleneck problem caused by end-to-end processing of traditional large models is solved, and the computational efficiency and system scalability of user behavior prediction are optimized. On the other hand, the joint input of current behavior embedding sequences (real-time interaction) and historical behavior pre-computed embedding sequences (long-term preferences) enables the first target large model to capture both short-term user intentions and long-term behavioral habits at the same time, thereby improving the completion rate of user behavior paths in strong process scenarios such as financial credit applications and wealth management product recommendations.

[0071] Please refer to the following. Figure 6 This is a flowchart illustrating a model training method provided in an exemplary embodiment of this specification. Figure 6 As shown, the model training method includes the following steps: S602, acquire multiple sample behavior sequences; the sample behavior sequence consists of multiple sample interaction records arranged in order of occurrence timestamp; the sample interaction records include sample behavior type information and sample behavior location information.

[0072] Specifically, sample behavior sequences can be obtained through multi-source data collection channels. For example, but not limited to, extracting user operation records within financial applications from application logs of user terminal devices (such as mobile phones and computers), including behavior types such as clicking on "credit application portal" and "wealth management product details page," as well as corresponding behavior points (such as specific page identifiers and functional module codes within the application). Alternatively, but not limited to, synchronizing users' online business transaction records from the financial institution's business system and integrating these records into a continuous sample behavior sequence after sorting by timestamps, can also be used. Each sample interaction record must contain clear sample behavior type information (such as "exposure," "click," "access," etc.) and precise sample behavior point information (such as page URL, function button identifiers, business transaction window numbers, etc.) to ensure that the sample data can fully reflect the behavioral trajectory of sample users in different scenarios, providing a realistic and rich foundation of behavioral features for subsequent model training.

[0073] S604, using the second initial large model to generate sample behavior embedding sequences corresponding to each sample behavior sequence; the sample behavior embedding sequence consists of sample behavior point vectors corresponding to each sample interaction record ordered by the occurrence timestamp.

[0074] Specifically, the second initial large model can, but is not limited to, a pre-trained large language model based on the Transformer architecture. This model, through pre-training on massive amounts of text data from various domains (e.g., but not limited to the financial domain), already possesses the ability to understand the semantics of behavior in different domain scenarios. When generating sample behavior embedding sequences, firstly, each sample interaction record can, but is not limited to, be mapped and queried based on a global mapping relationship to obtain the corresponding sample integer identifier. Subsequently, the sample integer identifier sequence corresponding to each sample behavior sequence is input into the second initial large model. The model uses a multi-layer self-attention mechanism to map and extract the semantic features corresponding to the input sample integer identifier sequence, encoding the specific descriptive information of the behavior points to generate sample behavior point vectors with fixed dimensions. For a sample behavior sequence containing multiple sample interaction records, the sample behavior point vectors corresponding to each record are arranged sequentially according to the timestamp order of each record, ultimately obtaining a sample behavior embedding sequence consistent with the temporal order of the sample behavior sequence. This sample behavior embedding sequence not only preserves the temporal sequence of the behaviors but also transforms the semantic information of the behaviors into a computable vector form, providing a low-dimensional, dense, and semantically rich feature representation for subsequent behavior prediction tasks.

[0075] S606 uses the first initial large model to output the sample future behavior prediction results corresponding to each sample behavior sequence based on the sample behavior embedding sequence.

[0076] Specifically, but not limited to, the sample behavior embedding sequence can be input into the input layer of the first initial large model. The model captures the temporal dependencies and semantic associations of the behavior in the sample behavior embedding sequence through a multi-layer attention mechanism. The hidden layer of the first initial large model performs nonlinear transformation on the captured features through a feedforward neural network to further extract deep patterns of the behavior sequence. Finally, the output layer generates the probability distribution of possible future behavior categories of the sample, or the prediction results of the sample's future behavior, such as behavior type information and location information. That is, for example, but not limited to, generative prediction results of the sample's future behavior and discriminative prediction results of the sample's future behavior.

[0077] S608, based on the predicted future behavior of each sample behavior sequence and the actual future behavior information of the sample, the first initial large model and the second initial large model are jointly trained to obtain the first target large model and the second target large model.

[0078] Specifically, the system acquires the actual future behavior information for each sample, including key data such as the type of behavior, location, and timestamp of the behavior that actually occurs after the sample's behavior sequence. The system compares the predicted future behavior of the sample from the first initial large model with the actual future behavior information, calculating the loss between the two. For example, a cross-entropy loss function is used to measure the difference in probability distribution, enabling the trained first target large model to make discriminative predictions of future behavior. A contrastive learning loss function is used to measure the difference between the generated future behavior and the actual future behavior, enabling the trained first target large model to make generative predictions of future behavior. Simultaneously, a second initial large model is used to supplement the encoding of the features of the sample behavior sequence. Its output is fused with the intermediate features of the first initial large model, and both participate in the loss calculation. During joint training, the parameters of both the first and second initial large models are updated simultaneously using the backpropagation algorithm, allowing the two models to continuously optimize their feature capture ability and prediction accuracy for behavior sequences through collaborative learning.

[0079] Optionally, the joint training process of the first and second initial large models can be optimized end-to-end using only contrastive learning loss, or it can be supervised fine-tuned using only cross-entropy loss. Alternatively, it can combine both for multi-task joint optimization to balance the diversity of generative predictions with the accuracy of discriminative predictions. During training, the loss weights are dynamically adjusted to adaptively balance the contributions of the two types of tasks, ensuring that the model has robust modeling capabilities for both long-term behavioral evolution and sudden behavioral transitions in real-world business scenarios.

[0080] In the embodiments of this specification, the end-to-end optimization of feature generation and behavior prediction is achieved through the above-mentioned collaborative training process. The sample behavior embedding sequence generated by the second initial large model directly serves the prediction task of the first initial large model, making the semantic encoding of the point vector deeply bound to the future behavior prediction target. Compared with the independent training strategy, the overall prediction accuracy of the model is higher. On the other hand, the sample interaction record contains both sample behavior type and sample point information. Through joint training of the two models, heterogeneous features (operation behavior type + spatial location) are fused into a unified vector representation, avoiding feature information loss in traditional single model training, improving feature utilization, and maximizing data utilization efficiency. Furthermore, by pre-generating the sample behavior embedding sequence through the second initial large model, the high computational overhead of the first initial large model directly processing the original text sequence (sample behavior sequence) is avoided, reducing GPU resource consumption during the training phase and supporting efficient training of larger-scale sample data.

[0081] Please refer to the following. Figure 7 , Figure 7 This is a schematic diagram of a user behavior prediction device provided as an exemplary embodiment of this specification. Figure 7 As shown, the user behavior prediction device 700 includes: The first acquisition module 710 is used to acquire the current behavior sequence of the target user; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each of the current interaction records includes current behavior type information and current location information; The mapping query module 720 is used to perform mapping queries on each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence; the current behavior embedding sequence is composed of the current point vectors corresponding to each of the current interaction records arranged in order of occurrence timestamp. The user behavior prediction module 730 is used to embed the current behavior sequence into the first target large model and output the future behavior prediction result corresponding to the target user.

[0082] In one possible implementation, the above mapping query 720 includes: The mapping query unit is used to perform a mapping query on each current interaction record in the current behavior sequence based on the global mapping relationship, so as to obtain the current discretized behavior sequence corresponding to the current behavior sequence; the current discretized behavior sequence is composed of the current integer identifiers corresponding to each of the current interaction records arranged in the order of occurrence timestamps; The query unit is used to query the point vector library based on the current integer identifier corresponding to each of the above-mentioned current interaction records to obtain the current point vector corresponding to each of the above-mentioned current interaction records. The combination unit is used to arrange and combine the current point vectors corresponding to each of the current interaction records in the current behavior sequence according to the occurrence timestamp order to obtain the current behavior embedding sequence.

[0083] In one possible implementation, the user behavior prediction device 700 further includes: The second acquisition module is used to acquire multiple preset point information and preset behavior type configuration information corresponding to each preset point information; the preset behavior type configuration information includes at least one preset behavior type information configured for the corresponding preset point. The combination mapping module is used to perform combination mapping on the combination of each of the above-mentioned preset point information and the corresponding combination of various of the above-mentioned preset behavior type information based on the global mapping relationship, so as to obtain the corresponding preset integer identifier. The point vector encoding module is used to input the aforementioned preset integer identifiers into the second target large model and generate corresponding preset point vectors based on the aforementioned preset integer identifiers. The first associated storage module is used to associate and store each of the above-mentioned preset point vectors with its corresponding preset integer identifier in the above-mentioned point vector library.

[0084] In one possible implementation, the user behavior prediction device 700 further includes: The third acquisition module is used to acquire newly added location information and newly added behavior type configuration information corresponding to the newly added location information at preset time intervals; the newly added behavior type configuration information includes at least one type of newly added behavior information configured for the newly added location. The combination mapping module is used to perform combination mapping on the above-mentioned newly added point information and its corresponding combinations of various newly added behavior type information based on the above-mentioned global mapping relationship, so as to obtain the corresponding newly added integer identifier. The point vector encoding module is used to input the aforementioned newly added integer identifiers into the aforementioned second target large model, and generate corresponding newly added point vectors based on the aforementioned newly added integer identifiers. The first associated storage module is used to associate and store each of the newly added integer identifiers and the newly added point vectors into the point vector library.

[0085] In one possible implementation, the user behavior prediction device 700 further includes: The query module is used to query the user behavior vector library based on the target user identifier corresponding to the target user to obtain the historical behavior pre-computed embedding sequence of the target user; the historical behavior pre-computed embedding sequence is composed of the historical behavior vectors corresponding to multiple historical interaction records of the target user arranged in order of occurrence timestamp; the user behavior vector library includes the historical behavior pre-computed embedding sequences corresponding to multiple users.

[0086] The aforementioned user behavior prediction module 730 is specifically used to: input the aforementioned historical behavior pre-calculated embedding sequence and the aforementioned current behavior embedding sequence into the first target large model, and output the future behavior prediction result corresponding to the aforementioned target user.

[0087] In one possible implementation, the mapping query module 720 is further configured to: after detecting the target user's target interaction behavior, perform a mapping query on the target interaction record corresponding to the target interaction behavior based on the global mapping relationship, and obtain the target integer identifier corresponding to the target interaction record. The aforementioned user behavior prediction device 700 also includes: The behavior vector encoding module is used to input the target integer identifier into the first target large model mentioned above to generate the corresponding target behavior vector; The second associated storage module is used to associate and store the target behavior vector and the target user identifier in the user behavior vector library.

[0088] In one possible implementation, the user behavior prediction module 730 described above is specifically used for: The above-mentioned current behavior embedding sequence is input into the first target large model. Based on the user behavior vector library and the above-mentioned current behavior embedding sequence, inference analysis is performed, and the future behavior prediction result corresponding to the above-mentioned target user is output. The above-mentioned user behavior vector library includes the sample behavior vectors corresponding to multiple sample interaction records with known occurrence timestamps for at least one sample user.

[0089] In one possible implementation, the user behavior prediction device 700 further includes: The fourth acquisition module is used for multiple sample behavior sequences; the above sample behavior sequences consist of multiple sample interaction records arranged in order of occurrence timestamps; the above sample interaction records include sample behavior type information and sample behavior location information; The initial point vector encoding module is used to generate sample behavior embedding sequences corresponding to each of the above sample behavior sequences using the second initial large model; the above sample behavior embedding sequences are composed of sample behavior point vectors corresponding to each of the above sample interaction records ordered by the occurrence timestamp. The initial user behavior prediction module is used to output the future behavior prediction results of each of the above sample behavior sequences based on the above sample behavior embedding sequences using the first initial large model. The joint training module is used to jointly train the first initial large model and the second initial large model based on the sample future behavior prediction results and sample future actual behavior information corresponding to each of the above sample behavior sequences, so as to obtain the first target large model and the second target large model.

[0090] In one possible implementation, the user behavior prediction device 700 further includes: The filtering module is used to filter the above current behavior sequence to obtain the target current behavior sequence.

[0091] The mapping query module 720 is specifically used to: perform mapping queries on each current interaction record in the current behavior sequence of the target based on the point vector library, and obtain the current behavior embedding sequence.

[0092] In one possible implementation, the aforementioned future behavior prediction results include at least one of the following: future behavior generative prediction results and future behavior discriminative prediction results; The aforementioned generative prediction results of future behavior include the predicted sequence of future behavior for the target user; the predicted sequence of future behavior consists of at least one combination of predicted interactive behaviors arranged in chronological order of the predicted timestamps; the combination of predicted interactive behaviors includes information on the type of future behavior prediction and information on the location of future behavior prediction. The aforementioned discriminative prediction results for future behavior include multiple combinations of candidate interactive behaviors that the target user may engage in next, as well as the probability of future interaction occurring for each of these candidate interactive behavior combinations; the aforementioned candidate interactive behavior combinations include candidate interactive behavior type information and candidate interactive behavior location information.

[0093] The division of modules in the above-described user behavior prediction device is for illustrative purposes only. In other embodiments, the user behavior prediction device can be divided into different modules as needed to complete all or part of the functions of the above-described user behavior prediction device. The implementation of each module in the user behavior prediction device provided in the embodiments of this specification can be in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements all or part of the steps of the user behavior prediction method described in the embodiments of this specification.

[0094] Please refer to the following. Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this specification. For example... Figure 8 As shown, the electronic device 800 may include: at least one processor 810, at least one communication bus 820, a user interface 830, at least one network interface 840, and a memory 850. The communication bus 820 can be used to enable communication between the aforementioned components.

[0095] The user interface 830 may include a display screen and a camera. Optionally, the user interface 830 may also include a standard wired interface and a wireless interface.

[0096] The network interface 840 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, etc.

[0097] The processor 810 may include one or more processing cores. The processor 810 connects to various parts within the electronic device 800 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 850, and by calling data stored in the memory 850. Optionally, the processor 810 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 810 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 810 and may be implemented as a separate chip.

[0098] The memory 850 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 850 may include a non-transitory computer-readable medium. The memory 850 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 850 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as fetching functions, user behavior prediction functions, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 850 may also be at least one storage device located remotely from the aforementioned processor 810. Figure 8 As shown, the memory 850, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs.

[0099] Specifically, the processor 810 can be used to call the application stored in the memory 850 and perform the following operations: obtain the current behavior sequence of the target user; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each of the current interaction records includes current behavior type information and current location information; perform a mapping query on each current interaction record in the current behavior sequence based on the location vector library to obtain the current behavior embedding sequence; the current behavior embedding sequence consists of the current location vectors corresponding to each of the current interaction records arranged in order of occurrence timestamp; input the current behavior embedding sequence into the first target large model and output the future behavior prediction result corresponding to the target user.

[0100] In some possible embodiments, when the processor 810 performs the mapping query based on the point vector library for each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence, it specifically performs the following: a mapping query based on the global mapping relationship for each current interaction record in the current behavior sequence to obtain the current discretized behavior sequence corresponding to the current behavior sequence; the current discretized behavior sequence is composed of the current integer identifiers corresponding to each of the current interaction records arranged in the order of their occurrence timestamps; a query is performed in the point vector library based on the current integer identifiers corresponding to each of the current interaction records to obtain the current point vectors corresponding to each of the current interaction records; the current point vectors corresponding to each of the current interaction records in the current behavior sequence are arranged and combined in the order of their occurrence timestamps to obtain the current behavior embedding sequence.

[0101] In some possible embodiments, before the processor 810 performs the mapping query based on the point vector library to each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence, it is further configured to perform: obtaining multiple preset point information and preset behavior type configuration information corresponding to each preset point information; the preset behavior type configuration information includes at least one preset behavior type information configured for the corresponding preset point; performing combination mapping on each preset point information and its corresponding combinations of various preset behavior type information based on a global mapping relationship to obtain a corresponding preset integer identifier; inputting each preset integer identifier into the second target large model, generating a corresponding preset point vector based on each preset integer identifier; and associating each preset point vector with its corresponding preset integer identifier and storing it in the point vector library.

[0102] In some possible embodiments, the processor 810 is further configured to perform: acquiring new location information and corresponding new behavior type configuration information at preset time intervals; the new behavior type configuration information includes at least one new behavior type information configured for the new location; combining and mapping the new location information and its corresponding combinations of various new behavior type information based on the global mapping relationship to obtain corresponding new integer identifiers; inputting each of the new integer identifiers into the second target large model, generating corresponding new location vectors based on each of the new integer identifiers, and storing each of the new integer identifiers and the new location vectors in association in the location vector library.

[0103] In some possible embodiments, after the processor 810 executes the above-mentioned acquisition of the target user's current behavior sequence, and before the above-mentioned embedding the current behavior sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user, it is further configured to execute: Based on the target user identifier corresponding to the target user, a query is performed in the user behavior vector library to obtain the historical behavior pre-computed embedding sequence of the target user; the historical behavior pre-computed embedding sequence consists of the historical behavior vectors corresponding to each of the historical interaction records of the target user arranged in order of occurrence timestamp; the user behavior vector library includes the historical behavior pre-computed embedding sequences corresponding to each of the multiple users. When the processor 810 executes the above-mentioned process of embedding the current behavior sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user, it is specifically used to perform the following: The above-mentioned historical behavior pre-calculated embedding sequence and the above-mentioned current behavior embedding sequence are input into the first target large model, and the future behavior prediction results corresponding to the above-mentioned target users are output.

[0104] In some possible embodiments, before the processor 810 executes the above-mentioned query based on the target user identifier corresponding to the target user in the user behavior vector library to obtain the historical behavior pre-calculated embedding sequence of the target user, it is also used to perform: after detecting the target user's target interaction behavior, performing a mapping query on the target interaction record corresponding to the target interaction behavior based on the global mapping relationship to obtain the target integer identifier corresponding to the target interaction record; inputting the target integer identifier into the first target big model to generate the corresponding target behavior vector, and associating and storing the target behavior vector and the target user identifier in the user behavior vector library.

[0105] In some possible embodiments, when the processor 810 executes the above-mentioned method of inputting the current behavior embedding sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user, it is specifically used to perform: inputting the current behavior embedding sequence into the first target large model, performing inference analysis based on the user behavior vector library and the current behavior embedding sequence, and outputting the future behavior prediction result corresponding to the target user; the user behavior vector library includes sample behavior vectors corresponding to multiple sample interaction records with known occurrence timestamps for at least one sample user.

[0106] In some possible embodiments, before the processor 810 performs the above-mentioned inputting each of the preset integer identifiers into the second target large model and generating the corresponding preset point vectors based on each of the preset integer identifiers, it is also used to perform: Multiple sample behavior sequences are acquired; each sample behavior sequence consists of multiple sample interaction records arranged in chronological order of occurrence timestamps; each sample interaction record includes sample behavior type information and sample behavior location information; a second initial large model is used to generate sample behavior embedding sequences corresponding to each of the above sample behavior sequences; each sample behavior embedding sequence consists of sample behavior location vectors corresponding to each of the above sample interaction records arranged in chronological order of occurrence timestamps; a first initial large model is used to output the sample future behavior prediction results corresponding to each of the above sample behavior sequences based on the above sample behavior embedding sequences; the first initial large model and the second initial large model are jointly trained based on the sample future behavior prediction results corresponding to each of the above sample behavior sequences and the sample future actual behavior information to obtain a first target large model and a second target large model.

[0107] In some possible embodiments, after the processor 810 executes the above-mentioned acquisition of the target user's current behavior sequence, and before the above-mentioned mapping query based on the point vector library to obtain the current behavior embedding sequence, it is also used to execute: The above current behavior sequence is filtered to obtain the target current behavior sequence.

[0108] When the processor 810 executes the mapping query based on the point vector library for each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence, it specifically performs the following: Based on the point vector library, a mapping query is performed on each current interaction record in the current behavior sequence of the above target to obtain the current behavior embedding sequence.

[0109] In some possible embodiments, the above-mentioned future behavior prediction results include at least one of the following: future behavior generative prediction results and future behavior discriminative prediction results; The aforementioned generative prediction results of future behavior include the predicted sequence of future behavior for the target user; the predicted sequence of future behavior consists of at least one combination of predicted interactive behaviors arranged in chronological order of the predicted timestamps; the combination of predicted interactive behaviors includes information on the type of future behavior prediction and information on the location of future behavior prediction. The aforementioned discriminative prediction results for future behavior include multiple combinations of candidate interactive behaviors that the target user may engage in next, as well as the probability of future interaction occurring for each of these candidate interactive behavior combinations; the aforementioned candidate interactive behavior combinations include candidate interactive behavior type information and candidate interactive behavior location information.

[0110] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described user behavior prediction device are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium.

[0111] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means, such as coaxial cable, fiber optic cable, Digital Subscriber Line (DSL), or wireless means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs, DVDs), or semiconductor media (e.g., Solid State Disks, SSDs).

[0112] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0113] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made to the technical solutions of this specification by those skilled in the art without departing from the spirit of this specification should fall within the protection scope defined by the claims.

[0114] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims and specification may be performed in a different order than in the embodiments described in the specification and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A user behavior prediction method, characterized in that, The method includes: Obtain the target user's current behavior sequence; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each current interaction record includes current behavior type information and current location information; Based on the point vector library, a mapping query is performed on each current interaction record in the current behavior sequence to obtain the current behavior embedding sequence; the current behavior embedding sequence consists of the current point vectors corresponding to each current interaction record arranged in order of occurrence timestamp; the current point vector is used to characterize the point feature representation corresponding to the current point information in the current interaction record; The current behavior embedding sequence is input into the first target large model, and the future behavior prediction result corresponding to the target user is output; The step of embedding the current behavior sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user includes: The current behavior embedding sequence is input into the first target large model, and inference analysis is performed based on the user behavior vector library and the current behavior embedding sequence to output the future behavior prediction result corresponding to the target user; the user behavior vector library includes sample behavior vectors corresponding to multiple sample interaction records with known occurrence timestamps for at least one sample user.

2. The method as described in claim 1, characterized in that, The step of mapping and querying each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence includes: Based on the global mapping relationship, a mapping query is performed on each current interaction record in the current behavior sequence to obtain the current discretized behavior sequence corresponding to the current behavior sequence; the current discretized behavior sequence consists of the current integer identifiers corresponding to each current interaction record arranged in order of occurrence timestamp; Based on the current integer identifier corresponding to each current interaction record, a query is performed in the point vector library to obtain the current point vector corresponding to each current interaction record; The current point vectors corresponding to each current interaction record in the current behavior sequence are arranged and combined according to the timestamp order to obtain the current behavior embedding sequence.

3. The method as described in claim 1 or 2, characterized in that, Before performing a mapping query on each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence, the method further includes: Acquire multiple preset location information and preset behavior type configuration information corresponding to each preset location information; the preset behavior type configuration information includes at least one preset behavior type information configured for the corresponding preset location; Based on the global mapping relationship, the combination mapping of each preset point information and its corresponding preset behavior type information is performed to obtain the corresponding preset integer identifier. Input each of the preset integer identifiers into the second target large model, and generate corresponding preset point vectors based on each of the preset integer identifiers; Each preset point vector is associated with its corresponding preset integer identifier and stored in the point vector library.

4. The method as described in claim 3, characterized in that, The method further includes: New location information and corresponding new behavior type configuration information are acquired at preset time intervals; the new behavior type configuration information includes at least one new behavior type configured for the new location. Based on the global mapping relationship, the newly added point information and its corresponding combinations of various newly added behavior type information are combined and mapped to obtain the corresponding newly added integer identifier; Each of the newly added integer identifiers is input into the second target large model, and a corresponding newly added point vector is generated based on each of the newly added integer identifiers. The newly added integer identifiers and the newly added point vectors are then associated and stored in the point vector library.

5. The method as described in claim 1, characterized in that, After obtaining the current behavior sequence of the target user, and before inputting the current behavior embedding sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user, the method further includes: Based on the target user identifier corresponding to the target user, a query is performed in the user behavior vector library to obtain the historical behavior pre-computed embedding sequence of the target user; the historical behavior pre-computed embedding sequence consists of the historical behavior vectors corresponding to each of the historical interaction records of the target user arranged in order of occurrence timestamp; the user behavior vector library includes the historical behavior pre-computed embedding sequences corresponding to each of the multiple users. The step of embedding the current behavior sequence into the first target large model and outputting the future behavior prediction result corresponding to the target user includes: The historical behavior pre-computed embedding sequence and the current behavior embedding sequence are input into the first target large model, and the future behavior prediction result corresponding to the target user is output.

6. The method as described in claim 5, characterized in that, Before querying the user behavior vector library based on the target user identifier corresponding to the target user to obtain the pre-computed embedding sequence of the target user's historical behavior, the method further includes: After detecting the target user's target interaction behavior, a mapping query is performed on the target interaction record corresponding to the target interaction behavior based on the global mapping relationship to obtain the target integer identifier corresponding to the target interaction record. The target integer identifier is input into the first target large model to generate the corresponding target behavior vector, and the target behavior vector and the target user identifier are associated and stored in the user behavior vector library.

7. The method as described in claim 3, characterized in that, Before inputting each of the preset integer identifiers into the second target large model and generating corresponding preset point vectors based on each of the preset integer identifiers, the method further includes: Multiple sample behavior sequences are obtained; each sample behavior sequence consists of multiple sample interaction records arranged in order of occurrence timestamp; each sample interaction record includes sample behavior type information and sample behavior location information. The second initial large model is used to generate sample behavior embedding sequences corresponding to each of the sample behavior sequences; the sample behavior embedding sequences are composed of sample behavior point vectors corresponding to each of the sample interaction records arranged in order of occurrence timestamps; Using the first initial large model, based on the embedded sequence of the sample behavior, output the prediction results of the future behavior of the sample corresponding to each of the sample behavior sequences; Based on the predicted future behavior of each sample behavior sequence and the actual future behavior information of the sample, the first initial large model and the second initial large model are jointly trained to obtain the first target large model and the second target large model.

8. The method as described in claim 1, characterized in that, After obtaining the target user's current behavior sequence, and before performing a mapping query on each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence, the method further includes: The current behavior sequence is filtered to obtain the target current behavior sequence; The step of mapping and querying each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence includes: Based on the point vector library, a mapping query is performed on each current interaction record in the target's current behavior sequence to obtain the current behavior embedding sequence.

9. The method according to any one of claims 1-2 or 5-6, characterized in that, The future behavior prediction results include at least one of the following: future behavior generative prediction results, and future behavior discriminative prediction results; The future behavior generative prediction result includes the future behavior prediction sequence corresponding to the target user; the future behavior prediction sequence consists of at least one combination of future prediction interactive behaviors arranged in order of future prediction timestamps; the combination of future prediction interactive behaviors includes future behavior prediction type information and future behavior prediction location information. The discriminative prediction result of future behavior includes multiple combinations of candidate interaction behaviors that the target user may have in the next step, as well as the probability of future interaction occurrence corresponding to each combination of candidate interaction behaviors. The candidate interaction behavior combination includes candidate interaction behavior type information and candidate interaction behavior location information.

10. A user behavior prediction device, characterized in that, The device includes: The first acquisition module is used to acquire the current behavior sequence of the target user; the current behavior sequence consists of multiple current interaction records arranged in order of occurrence timestamp; each current interaction record includes current behavior type information and current location information; The mapping query module is used to perform mapping queries on each current interaction record in the current behavior sequence based on the point vector library to obtain the current behavior embedding sequence; the current behavior embedding sequence is composed of the current point vectors corresponding to each current interaction record arranged in order of occurrence timestamp; the current point vector is used to characterize the point feature representation corresponding to the current point information in the current interaction record; The user behavior prediction module is used to input the current behavior embedding sequence into the first target large model and output the future behavior prediction result corresponding to the target user. The user behavior prediction module is specifically used to: input the current behavior embedding sequence into the first target large model, perform inference analysis based on the user behavior vector library and the current behavior embedding sequence, and output the future behavior prediction result corresponding to the target user; the user behavior vector library includes sample behavior vectors corresponding to multiple sample interaction records with known occurrence timestamps for at least one sample user.

11. An electronic device, characterized in that, include: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-9.

12. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the steps of the method as described in any one of claims 1-9.

13. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer or processor, it causes the computer or processor to perform the user behavior prediction method as described in any one of claims 1-9.

Images