A customer group evolution prediction method based on large model semantic analysis

Abstract

The application relates to the technical field of semantic analysis, in particular to a customer group evolution prediction method based on large model semantic analysis, which comprises the following steps: obtaining historical unstructured data of users of a streaming media platform, performing semantic analysis on multiple time points by a large model to generate semantic vectors; processing the semantic vectors by a loop module to generate a historical semantic sequence; training a semantic evolution model based on the sequence; in an offline batch processing stage, calculating future semantic vectors of all users in advance by using the evolution model, and constructing a semantic index; when a seed customer group is received online, calculating a customer group centroid vector representing a future trend for the seed customer group; performing approximate neighbor search on the centroid vector in the predictive semantic index, and outputting a predicted evolved customer group. The application improves prediction accuracy through future-to-future matching, and ensures low delay and feasibility of the evolution prediction function through an offline precalculation architecture.

Description

Technical Field

[0001] This invention relates to the field of semantic analysis technology, specifically to a customer group evolution prediction method based on large-scale model semantic analysis. Background Technology

[0002] In internet information service platforms, customer base diffusion technology is widely used for content recommendation and precision marketing. The main goal of this technology is to identify potential users with similar characteristics from a massive pool of candidate users, based on a given seed user set.

[0003] Existing customer base expansion strategies primarily rely on modeling based on users' historical interaction data. In terms of data processing, a common approach is to transform user clicks, browsing, and purchasing behaviors into structured feature vectors. For user-generated text data (such as tags and brief descriptions), current technologies typically employ natural language processing techniques like keyword extraction, topic modeling, or bag-of-words models to map it into discrete labels or low-dimensional features, facilitating input into collaborative filtering or logistic regression models for processing.

[0004] In time series modeling, existing techniques typically employ time window aggregation to process user data. For example, user behavior data is statistically analyzed by day or week, generating a series of state snapshots arranged at fixed time intervals. Based on these snapshots, recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) can be used to capture trends in user interest changes and generate a representation of the user's interest at the current moment.

[0005] In the online application phase, when customer base expansion is required, pre-calculated user current interest representations are typically used to calculate the similarity (such as cosine similarity) between seed users and candidate users in the vector space, thereby selecting the user group with the most similar current features as the expansion result. As the platform's user base grows, how to handle high-dimensional feature matching of massive numbers of users while maintaining response speed is a continuously focusing technical direction in this field.

[0006] To address this, a customer group evolution prediction method based on large-scale model semantic analysis is proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a customer group evolution prediction method based on large-scale model semantic analysis. This method improves prediction accuracy through future-to-future matching and ensures low latency and feasibility of the evolution prediction function through an offline pre-computation architecture. The method includes: acquiring historical unstructured data of users on a streaming media platform; performing semantic analysis at multiple time points using a large-scale model to generate corresponding semantic vectors; processing the semantic vectors through a loop module to generate a historical semantic sequence; training a semantic evolution model based on this sequence; in the offline batch processing stage, using the evolution model, pre-computing the future semantic vectors of all users and constructing a semantic index; when a seed customer group is received online, calculating a centroid vector representing the future trend for the seed customer group; performing an approximate nearest neighbor search on this centroid vector in the predictive semantic index, and outputting the predicted evolved customer group.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A customer group evolution prediction method based on large-scale model semantic analysis includes:

[0010] The system acquires historical data from multiple users on a streaming media platform, including users' historical comments, bullet comments, and search queries; performs semantic analysis on the historical data to generate semantic vectors for each user at multiple time points; processes the semantic vectors in chronological order through a loop module to generate historical semantic sequences for each user, and stores the historical semantic sequences in a time-series database.

[0011] When the model training task is started, the historical semantic sequence is obtained, and the semantic evolution model is trained based on the historical semantic sequence. In the offline batch processing stage, the semantic evolution model is used to calculate the future semantic vector corresponding to each user, and the future semantic vectors of multiple users are stored in the semantic index.

[0012] When a customer base diffusion request is received, a seed customer base is obtained; the corresponding historical semantic sequence of the seed customer base is retrieved from the time series database; the historical semantic sequence of the seed customer base is calculated using a semantic evolution model to generate a customer base centroid vector;

[0013] The centroid vector of the customer group is used as the query vector and input into the semantic index; an approximate nearest neighbor search is performed in the semantic index to identify similar users, which are then output as the predicted evolved customer group.

[0014] Preferably, the process of generating a semantic vector for each user includes: acquiring historical data, including the user's historical comments, bullet comments, and search queries, and extracting the timestamp corresponding to each piece of historical data; for each of the multiple users, constructing the historical data and timestamps into an asynchronous user event stream ordered by timestamp in computer memory; processing individual events in the user event stream one by one according to the chronological order of the user event stream, wherein each individual event corresponds to a comment, a bullet comment, and / or a search query; inputting the text data of the individual event into a pre-trained bidirectional encoder model, wherein the bidirectional encoder model is a specific implementation of a larger model; and performing semantic encoding operations on the text data of the individual event, outputting the final hidden layer output vector of the labeled text data as the semantic vector corresponding to the user at the timestamp.

[0015] Preferably, the process of generating a historical semantic sequence for each user and storing the historical semantic sequence in a time-series database includes: for each of the multiple users, initializing a historical state vector in computer memory, obtaining the semantic vector and timestamp as an asynchronous user event stream, and inputting them one by one into a loop module in chronological order of timestamps; when the loop module receives the current semantic vector and current timestamp, retrieving the previous historical state vector and previous timestamp from the historical semantic sequence and calculating the time interval between the current timestamp and the previous timestamp; the loop module applies a trainable time decay function to decay the influence of the previous historical state vector according to the time interval, generating a decayed historical state; the loop module inputs the semantic vector and the decayed historical state together into a recursive update gate to calculate a new historical state vector; and appends and stores the new historical state vector and timestamp into the time-series database as the latest state of the historical semantic sequence.

[0016] Preferably, the process of the trainable time decay function includes: the trainable time decay function receiving a calculated time interval as input time; the trainable time decay function applying an internal trainable decay parameter vector, the dimension of which matches the dimension of the previous historical state vector; each parameter value in the trainable decay parameter vector being learned and determined during the model training phase through a backpropagation algorithm, each parameter value representing the interest forgetting rate of different dimensions in the historical state vector; the trainable time decay function using the trainable decay parameter vector as the forgetting rate to perform an exponential decay operation on the input time to calculate a decay weight vector; and the trainable time decay function performing element-wise multiplication of the decay weight vector with the previous historical state vector to generate the decayed historical state.

[0017] Preferably, the process of storing the future semantic vectors of multiple users in a semantic index includes: during the model training phase, extracting training sample pairs from a time-series database using a sliding window method, wherein the training sample pairs include historical semantic state sequences as input features and actual future semantic vectors as training labels; inputting the input features and training labels into a semantic evolution model for training, and obtaining the trained semantic evolution model by minimizing the vector distance loss between the future vectors predicted by the model and the training labels; during the offline batch processing phase, starting a periodic background computing task, traversing multiple users, retrieving the latest historical semantic sequence from the time-series database for each user, applying the semantic evolution model to perform an inference calculation, and calculating the future semantic vector corresponding to the user; the background computing task batch importing the calculated future semantic vectors corresponding to all users into a high-dimensional vector index, constructing and outputting the semantic index.

[0018] Preferably, the semantic evolution model includes:

[0019] The time interval embedding sub-network receives the current timestamp and the previous timestamp from the historical semantic sequence; calculates the scalar time interval; inputs the scalar time interval into the time feature extractor, and converts the scalar into a high-dimensional time interval vector; the time feature extractor is a multilayer perceptron.

[0020] The state fusion layer receives a semantic vector, a previous historical state vector, and a high-dimensional time interval vector; it fuses the three vectors into a fusion input vector through vector concatenation.

[0021] A recursive state update layer receives a fused input vector; the recursive state update layer is a gated loop unit; the gated loop unit processes the fused input vector through gate weights to calculate the final historical state vector;

[0022] The prediction output layer, which is a multilayer perceptron, receives the final historical state vector and outputs a predicted future semantic vector.

[0023] Preferably, the process of generating the centroid vector of the customer group includes: in the online query stage, obtaining each seed user included in the seed customer group; for each seed user, retrieving the corresponding historical semantic sequence from the time series database; inputting the historical semantic sequence of each seed user into the semantic evolution model, performing an inference calculation, and generating a corresponding individual future semantic vector; after obtaining the individual future semantic vectors of all seed users in the seed customer group, performing a vector averaging operation on multiple individual future semantic vectors, calculating and outputting the centroid vector of the customer group.

[0024] Preferably, the process of identifying a predetermined number of users with the highest similarity to the centroid vector of the customer group and outputting them as the predicted evolved customer group includes: inputting the centroid vector of the customer group as a query vector into a semantic index; the semantic index is an approximate nearest neighbor index; the approximate nearest neighbor index is configured to perform a candidate set retrieval operation; the candidate set retrieval operation returns a candidate user set with a size larger than the final output size of the predicted evolved customer group and the approximate distances corresponding to the candidate user set;

[0025] For each candidate user in the candidate user set, a future semantic vector is extracted from the semantic index; in the computer processor, the vector distance between the future semantic vector of each candidate user and the centroid vector of the customer group is calculated; based on the calculated vector distance, the candidate user set is reordered, and from the reordered candidate user set, a predetermined number of users with the highest ranking are selected as approximate users for output.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] 1. By using pre-calculated future semantic vectors instead of users' current state vectors when matching customer groups, this design of matching future states allows the evolution of user interests over time (i.e., "interest drift") to be taken into account when expanding customer groups. Compared to methods that only match current states, the expansion results of this invention are more adaptable to the dynamic changes of users, thereby helping to improve the relevance and accuracy of the expanded customer groups at future points in time.

[0028] 2. Through its offline-online separation computing architecture, it provides an efficient and computationally feasible solution for realizing complex "evolutionary prediction" functions. For the computationally expensive task of calculating future semantic vectors for a massive number of users across the entire site, it pre-computes the calculations in the offline batch processing stage. This computational transfer design means that during online queries, only one prediction for the seed customer group and one index search need to be performed. This ensures that the customer group evolution prediction function can be realized while also ensuring low-latency response for online queries, making it suitable for industrial deployment environments with high real-time requirements.

[0029] 3. Through its time-aware loop module design, it improves the modeling accuracy of asynchronous sparse event streams. User historical data (such as comments and bullet screens) are usually irregularly distributed over time. By introducing time interval embedding and a trainable time decay function, the passage of time itself is modeled as a trainable feature. This refined processing of asynchronous sparse data streams enables the construction of historical semantic sequences to more reasonably reflect the forgetting process of user interests as irregular time passes, thus providing higher quality input for semantic evolution models and helping to improve their prediction accuracy. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a customer group evolution prediction method based on large-model semantic analysis according to the present invention.

[0031] Figure 2 This is a schematic diagram illustrating the process of storing historical semantic sequences in a time-series database according to the present invention;

[0032] Figure 3 This is a flowchart illustrating the semantic evolution model of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figures 1 to 3 This invention provides a customer group evolution prediction method based on large-scale model semantic analysis, the technical solution of which is as follows:

[0035] Example 1:

[0036] A customer group evolution prediction method based on large-scale model semantic analysis, the specific process of which is as follows: Figure 1 As shown, it includes.

[0037] The system acquires historical data from multiple users on a streaming media platform, including users' historical comments, bullet comments, and search queries; performs semantic analysis on the historical data to generate semantic vectors for each user at multiple time points; processes the semantic vectors in chronological order through a loop module to generate historical semantic sequences for each user, and stores the historical semantic sequences in a time-series database.

[0038] When the model training task is started, the historical semantic sequence is obtained, and the semantic evolution model is trained based on the historical semantic sequence. In the offline batch processing stage, the semantic evolution model is used to calculate the future semantic vector corresponding to each user, and the future semantic vectors of multiple users are stored in the semantic index.

[0039] When a customer base diffusion request is received, a seed customer base is obtained; the corresponding historical semantic sequence of the seed customer base is retrieved from the time series database; the historical semantic sequence of the seed customer base is calculated using a semantic evolution model to generate a customer base centroid vector;

[0040] The centroid vector of the customer group is used as the query vector and input into the semantic index; an approximate nearest neighbor search is performed in the semantic index to identify similar users, which are then output as the predicted evolved customer group.

[0041] Furthermore, the process of generating semantic vectors for each user includes: acquiring historical data, including the user's historical comments, bullet comments, and search queries, and extracting the timestamp corresponding to each piece of historical data; for each of the multiple users, constructing the historical data and timestamps into an asynchronous user event stream ordered by timestamps in computer memory; processing individual events in the user event stream one by one according to the chronological order of the user event stream, wherein each individual event corresponds to a comment, a bullet comment, and / or a search query; inputting the text data of the individual event into a pre-trained bidirectional encoder model, wherein the bidirectional encoder model is a specific implementation of the larger model; and performing semantic encoding operations on the text data of the individual event, outputting the final hidden layer output vector of the label corresponding to the text data as the semantic vector corresponding to the timestamp of the user.

[0042] Specifically, in the data acquisition and event stream construction stage, this embodiment accesses the backend data stream of the online streaming media platform. The acquired historical unstructured text data specifically includes: (1) the bullet comments sent by the user while watching the video; (2) the historical comments posted by the user in the video comment section or on social media; (3) the search query entered by the user in the platform search box. Not only is the text content acquired, but also the precise timestamp associated with the text event is extracted (for example, accurate to the second). After acquisition, for each user, all their historical text data and their precise timestamps are constructed in the computer memory as an asynchronous user event stream that is strictly ordered by timestamp.

[0043] To ensure the accuracy and consistency of timestamps, the precise timestamps should be obtained in the following ways: Time source: The server's system time should be used as the standard time source. When a user submits a comment, bullet screen, or search request, the server should immediately record the time of receipt of the request, with an accuracy of no less than the second level; Calibration: If the user's client has a local timestamp, it should be calibrated using a time synchronization protocol (such as NTP) after the server receives the request; Furthermore, the system should adopt the following strategies to handle timestamp anomalies: If the time interval between two consecutive events exceeds 365 days, the event should be considered abnormal and excluded; If the timestamps of events are out of order, the system should reorder them according to the physical order of the events in the log.

[0044] In the large model selection stage, in order to achieve semantic analysis, this embodiment adopts a pre-trained bidirectional encoder model (as a specific implementation of the large model). In this embodiment, the pre-trained bidirectional encoder model is specifically a BERT model based on the Transformer architecture. In this embodiment, the bert-base-chinese model is adopted. This model has been pre-trained on a massive Chinese corpus and can deeply understand and capture complex contextual semantic information, making it suitable for processing short texts such as comments, bullet comments, and search terms.

[0045] In the asynchronous event processing phase, this embodiment processes individual events in the user event stream one by one according to the chronological order of the stream. Each individual event corresponds to a comment, a bullet screen comment, or a search query. In the semantic encoding phase, when processing a single event (e.g., the text content is "This show has a great pace"), the following detailed process is executed:

[0046] (Word segmentation): The text data of the single event is input into the word segmenter of the BERT model. The word segmenter converts the text into a sequence of input tokens that the model can recognize and automatically adds a [CLS] (Class) token at the beginning of the sequence.

[0047] (Encoding): The input labeled sequence is fed into the BERT model to perform semantic encoding, i.e., one forward propagation computation.

[0048] (Extraction): In the final hidden layer of the model output, the output vector corresponding to the [CLS] tag is extracted. In this embodiment, the output vector is a 768-dimensional vector. The 768-dimensional vector is output as the semantic vector corresponding to the user at the precise timestamp and waits for processing by the subsequent loop module.

[0049] The semantic vector should have a dimension of no less than 256 and no more than 1024; preferably, the semantic vector dimension is between 512 and 1024. The dimension selection should meet one of the following conditions: a) the output dimension is directly determined based on the pre-trained bidirectional encoder model; b) the original high-dimensional vector is compressed to a specified dimension using a dimension compression algorithm (such as principal component analysis or autoencoder), and the compressed vector should retain at least 85% of the information content of the original vector.

[0050] By constructing user historical data into an asynchronous user event stream and performing semantic encoding on each individual event (such as a bullet comment or search term) within the stream, the method of this invention can capture high-temporal-precision points of interest that are ignored by traditional periodic aggregation methods (such as daily aggregation). This refined processing ensures that even transient or sparse expressions of interest can be effectively captured, providing richer and more accurate instantaneous semantic input for subsequent evolutionary analysis.

[0051] Further, the process of generating a historical semantic sequence for each user and storing the historical semantic sequence in a time-series database includes: for each of the multiple users, initializing a historical state vector in computer memory, obtaining the semantic vector and timestamp as an asynchronous user event stream, and inputting them one by one into a loop module in chronological order of timestamps; when the loop module receives the current semantic vector and current timestamp, retrieving the previous historical state vector and previous timestamp from the historical semantic sequence and calculating the time interval between the current timestamp and the previous timestamp; the loop module applies a trainable time decay function to decay the influence of the previous historical state vector according to the time interval, generating a decayed historical state; the loop module inputs the semantic vector and the decayed historical state together into a recursive update gate to calculate a new historical state vector; and appends and stores the new historical state vector and timestamp into the time-series database as the latest state of the historical semantic sequence. The specific flowchart is as follows: Figure 2 As shown.

[0052] During the initialization phase, this embodiment initializes a historical state vector in the computer memory for each of the multiple users. This vector (e.g., a 768-dimensional zero vector) will serve as the starting point for recursive calculations. During the asynchronous event stream processing phase, the semantic vector and its precise timestamp are obtained and used as an asynchronous user event stream. The data is then input into the loop module one by one in the order of the precise timestamps.

[0053] When the loop module receives the current instantaneous semantic vector and its current timestamp, it executes the following detailed process:

[0054] First, the loop module retrieves the user's previous historical state vector and its previous timestamp from the time-series database. Then, the module calculates the time interval between the current timestamp and the previous timestamp in the processor; this is the scalar time interval. In principle, this scalar time interval represents the silent period or forgetting period between two expressions of interest by the user.

[0055] Next, the loop module applies a trainable time decay function that decays the influence of the previous historical state vector based on the calculated scalar time interval to generate a decayed historical state. This step is used to mathematically simulate the natural forgetting of user interests over time.

[0056] Subsequently, the recurrent module inputs the semantic vector (representing the new interest event) and the decayed historical state (representing the decayed old interest) into a recursive update gate. In this embodiment, the recursive update gate is specifically a gated recurrent unit (GRU). The gating weights of the GRU (learned in the subsequent model training phase) dynamically calculate, through its update gate and reset gate, the extent to which the new interest event should cover the decayed old interest, and calculate and output a new historical state vector.

[0057] Finally, the new historical state vector and its precise timestamp are appended and stored in the time-series database. This new historical state vector serves as the latest state of the historical semantic state sequence and awaits the arrival of the next asynchronous event.

[0058] By introducing time interval calculation and a trainable time decay function into the loop module, the “natural forgetting” process of user interests can be explicitly modeled. This design enables it to reasonably distinguish the different effects of long intervals (leading to interest decay) and short intervals (maintaining interest intensity) when dealing with asynchronous event streams with irregular time intervals, thereby improving the accuracy of constructing historical semantic state sequences and providing high-quality input that is more in line with real time dynamics for subsequent evolutionary prediction models.

[0059] Furthermore, the process of the trainable time decay function includes: the trainable time decay function receives the calculated time interval as the input time; the trainable time decay function applies an internal trainable decay parameter vector, the dimension of which matches the dimension of the previous historical state vector; each parameter value in the trainable decay parameter vector is learned and determined during the model training phase through a backpropagation algorithm, and each parameter value is used to characterize the interest forgetting rate represented by different dimensions in the historical state vector; the trainable time decay function uses the trainable decay parameter vector as the forgetting rate to perform an exponential decay operation on the input time to calculate the decay weight vector; the trainable time decay function performs element-wise multiplication of the decay weight vector with the previous historical state vector to generate the decayed historical state.

[0060] Specifically, in the definition and initialization phase of the model components, this embodiment defines and initializes a trainable decay parameter vector within the semantic evolution model. This vector is a trainable weight matrix whose dimension is strictly set to match the dimension of the previous historical state vector. For example, if the historical state vector is 768-dimensional, then the trainable decay parameter vector is also a vector containing 768 independent floating-point parameters. The specific value of each parameter in the trainable decay parameter vector is not a hyperparameter pre-set by humans, but is learned and determined by the backpropagation algorithm during the model training phase as a trainable part of the semantic evolution model. The technical meaning of this design is that each parameter value is trained to represent the forgetting rate of the specific interest represented by different dimensions (e.g., the i-th dimension) in the historical state vector.

[0061] During the runtime computation phase, when the loop module receives the calculated time interval (a scalar representing the elapsed time between two events) as input time, the trainable time decay function executes the following detailed process: First, the trainable time decay function applies its internal trainable decay parameter vector (as the forgetting rate) to perform an exponential decay operation on the input time, multiplying the input time (scalar) by the trainable decay parameter vector (a 768-dimensional vector) (through a broadcast mechanism, i.e., multiplying the scalar by each element of the vector), and then taking its negative exponent. The output of this operation is a new 768-dimensional vector, which in this embodiment is the decay weight vector. Finally, the trainable time decay function performs element-wise multiplication of the decay weight vector with the previous historical state vector (a 768-dimensional vector). The result of this element-wise multiplication (i.e., multiplying the corresponding dimensions of the two 768-dimensional vectors) is the decayed historical state, which is then fed into the recursive update gate for the next fusion computation.

[0062] The trainable time decay function can be implemented in one of the following ways: exponential decay: suitable for scenarios where user interest forgetting follows an exponential law; piecewise linear decay: the decay function can be defined in segments, for example: rapid decay (slope of 0.1 / day) is used in the first 0-7 days, and slow decay (slope of 0.02 / day) is used in the last 7-30 days, to more finely simulate the forgetting process of interest; higher-order polynomial decay: a second- or third-order polynomial function is used to adapt to more complex interest evolution laws. The selected decay method should be evaluated by validation set performance during the model training phase, and the method that minimizes validation loss should be selected.

[0063] To ensure the stability and effectiveness of model training, the parameter values ​​of the trainable decay parameter vector are set as follows:

[0064] (1) Initialization: Each parameter value is randomly sampled from a uniform distribution within the interval [0.01, 0.1]; (2) Training constraints: During model training, gradient clipping is used to ensure that each parameter value is always within the interval (0, 1) to guarantee the mathematical validity of the exponential decay function; (3) Overflow prevention: Exponential decay operation is used. ,in The time interval is in days. For the attenuation parameter, constraints To prevent numerical underflow.

[0065] In a preferred embodiment, before the step of the recurrent module inputting the semantic vector and the decayed historical state into the recursive update gate, the method further includes: when generating the semantic vector, additionally acquiring and storing the data source type, wherein the data source type represents whether the historical unstructured text data originates from historical comments, bullet comments, or search queries; inputting the semantic vector and the corresponding data source type together into a pre-trained source attention sub-network; the source attention sub-network is configured to calculate a semantic relevance weight between 0 and 1 based on the data source type, wherein the parameters of the source attention sub-network are learned together with the recurrent module during the model training phase; multiplying the semantic relevance weight by the semantic vector to generate a weighted semantic vector; and the step of the recurrent module inputting the semantic vector and the decayed historical state together into the recursive update gate is specifically implemented as: inputting the weighted semantic vector and the decayed historical state together into the recursive update gate to calculate a new historical state vector.

[0066] Specifically, the technical principle of this design lies in the fact that not all text data can equally represent a user's true interests. A search query may only be instantaneous, low-relevance noise, while a long comment represents highly relevant true interests. This embodiment solves this problem by introducing a source attention sub-network. In the data acquisition and event stream construction phase, the method additionally acquires and stores a data source type. When processing a bullet comment, it stores the instantaneous semantic vector, the precise timestamp, and the bullet comment data source type. When processing a search, it stores the instantaneous semantic vector, the precise timestamp, and the search query data source type. In the asynchronous event stream processing phase, when the loop module receives the current semantic vector and its data source type, after executing the decay step and before executing the recursive update gate, it additionally executes the following source attention process.

[0067] The data source types include historical comments, bullet comments, and search query terms; each category is encoded as an independent embedding vector with dimensions ranging from 64 to 256, and 128 dimensions in this embodiment; the source attention subnetwork is a multilayer perceptron, and its structure is as follows: input layer (receiving the concatenation of a 768-dimensional semantic vector and a 128-dimensional source embedding vector, for a total of 896 dimensions) -> hidden layer 1 (256 units, ReLU activation) -> hidden layer 2 (64 units, ReLU activation) -> output layer (1 unit, Sigmoid activation); the semantic relevance weights (scalars) output by the Sigmoid are multiplied element-wise with the semantic vector (768 dimensions) to generate a weighted semantic vector with the dimension maintained at 768 dimensions; in the early stage of model training, the initial weights corresponding to all data source types are set to 0.5 to avoid some data sources being suppressed due to improper initialization.

[0068] The semantic vector and the corresponding data source type are input into a source attention sub-network. In this embodiment, the source attention sub-network is a multilayer perceptron. The input layer of the multilayer perceptron receives the concatenation of the semantic vector and the embedding vector of the data source type. All parameters of the multilayer perceptron, as well as the embedding vectors of data source types such as comments, bullet comments, and searches, are learned and determined during the model training phase through the backpropagation algorithm. During training, the model is forced to learn the contribution of different data sources to the prediction accuracy. The model learns that inputs of search query type should be given lower weights, while inputs of historical comment type should be given higher weights.

[0069] The output layer of the multilayer perceptron uses a sigmoid activation function to ensure that its output is a scalar between 0 and 1. This scalar is the semantic relevance weight. Then, the semantic relevance weight scalar is multiplied with the semantic vector to generate a weighted semantic vector. Finally, the input received by the recursive update gate is no longer the original instantaneous semantic vector, but this weighted instantaneous semantic vector. This vector is fused with the decayed historical state to calculate a new historical state vector.

[0070] By introducing a source attention subnetwork, the system can automatically learn and distinguish the importance of different data sources (such as comments and search terms). This design suppresses the interference of low-relevance noise (such as search terms) by weighting the input, improves the accuracy of constructing historical semantic state sequences, and provides more reliable input for subsequent evolutionary prediction.

[0071] By learning an independent trainable decay parameter for each dimension of the historical state vector, this method can more precisely simulate the differentiated forgetting characteristics of different types of interests. This design enables the model to reasonably distinguish between long-term stable interests (slow forgetting) and short-term hot interests (fast forgetting) when dealing with asynchronous event streams with irregular time intervals, thereby improving the accuracy of constructing historical semantic state sequences and providing higher quality input for subsequent evolutionary prediction models.

[0072] Furthermore, the process of storing the future semantic vectors of multiple users in the semantic index includes: during the model training phase, extracting training sample pairs from the time-series database using the sliding window method, wherein the training sample pairs include historical semantic state sequences as input features and actual future semantic vectors as training labels; inputting the input features and training labels into the semantic evolution model for training, and obtaining the trained semantic evolution model by minimizing the vector distance loss between the future vectors predicted by the model and the training labels; during the offline batch processing phase, starting a periodic background computing task, traversing multiple users, retrieving the latest historical semantic sequence from the time-series database for each user, applying the semantic evolution model to perform an inference calculation, and calculating the future semantic vector corresponding to the user; the background computing task batch imports the calculated future semantic vectors corresponding to all users into a high-dimensional vector index, constructing and outputting the semantic index.

[0073] Specifically, in the model training phase, this embodiment executes the following detailed process: First, in the training sample construction step, training sample pairs are extracted from the time series database using the sliding window method. The parameters of the sliding window method are configured as follows: Historical window length: If a window based on the number of events is used, the length is set to 10 to 20 events; if a window based on the time span is used, the span is set to 14 to 30 days; Future prediction horizon K: K is set to 7 days; When the total number of historical events of a user is less than the set historical window length, a filling strategy is adopted. Specifically, a historical window length (e.g., N=10 events or N=14 days) and a future prediction horizon (e.g., K=7 days) are set in the user's historical semantic sequence. Slide the window upwards to extract the sequence (including semantic vectors and precise timestamps) within the historical window as input features, and extract the actual future semantic vector on the Kth day after the end point of the window as training labels. Then, in the model training step, the input features and the training labels are batch-input into the semantic evolution model. The model is a sequence-to-vector neural network architecture. The model is trained by minimizing the vector distance loss between the future vectors predicted by the model and the training labels (actual future semantic vectors). In this embodiment, the loss function is specifically a cosine similarity loss, which is used to mathematically maximize the directional consistency of two vectors in high-dimensional space. After training, the trained semantic evolution model is obtained.

[0074] In the offline batch processing stage, the following detailed process is executed in this embodiment: First, in the task scheduling step, a periodic background computing task is started. In this embodiment, this task is configured by a task scheduling tool (such as Apache Airflow) to be automatically executed daily (e.g., at 2 AM every day). Next, in the batch inference step, after the background computing task is started, it iterates through all active users on the online streaming media platform. For each user, the task retrieves its latest historical semantic state sequence from the time-series database. Subsequently, the trained general semantic evolution model is loaded, and the latest historical... The semantic state sequence performs an inference computation to pre-calculate the future semantic vector corresponding to the user. Finally, in the index construction step, the background computing task imports the pre-calculated future semantic vectors corresponding to all users into a high-dimensional vector index in batches. In this embodiment, the index is specifically a FAISS index and can adopt the IndexIVFPQ (Inverted File and Product Quantization) structure to balance search speed, accuracy and memory usage. After the FAISS index is constructed, it is loaded into the memory of the online query server as the predictive semantic index to await real-time invocation in the online query stage.

[0075] In a preferred embodiment, the model training step applies a multi-task joint loss function, and the calculation process includes: calculating the cosine similarity loss between the final predicted future semantic vector of the semantic evolution model and the actual future semantic vector of the training label, as a first loss term; calculating the vector mutual information between the high-dimensional time interval vector and the final historical state vector, as a second loss term; and jointly optimizing and outputting the trained semantic evolution model by minimizing the first loss term and simultaneously minimizing the second loss term.

[0076] In this embodiment, when the model training step is performed, the method extracts training sample pairs from the time series database. The training sample pairs are input into the semantic evolution model. In one forward propagation, the model calculates and outputs a high-dimensional time interval vector (output by the time interval embedding sub-network) in parallel. This vector only represents the nonlinear representation of the passage of time. Finally, the historical state vector (output by the recursive state update layer) mainly represents the cumulative memory of semantic content over time.

[0077] At this point, the method applies a multi-task joint loss function, which consists of two loss terms:

[0078] The first loss term is the prediction loss, which is calculated as follows: the final historical state vector is fed into the prediction output layer (MLP) to obtain the final predicted future semantic vector; then, the cosine similarity loss between the predicted vector and the training label (the actual future semantic vector) is calculated. This loss term is the main optimization objective of this model, used to improve prediction accuracy.

[0079] The second loss term is the decoupling loss, which is calculated as follows: obtain the high-dimensional time interval vector and the final historical state vector, and then calculate the vector mutual information between the two vectors (e.g., through a dedicated mutual information estimator network, such as MINE). The goal of this loss term is to minimize the mutual information between the two vectors, i.e., to force them to be orthogonal or statistically independent.

[0080] Finally, in the backpropagation step, all trainable parameters of the semantic evolution model are jointly updated by jointly optimizing the prediction loss and the decoupling loss. The multi-task joint loss function is calculated using a weighted summation method, i.e. ,in Represents the total loss. Represents predicted loss, represents Decoupling loss, This represents the weighted average; to balance prediction accuracy and decoupling effect, the weights are set to... =1.0, =0.5, to eliminate the dimensional difference between the two loss terms. and The two corresponding loss terms are normalized before weighting (e.g., divided by their respective mean in the current batch). This joint optimization, in principle, forces the model to encode temporal and content information into different, unrelated neuronal subspaces while learning how to accurately predict the future.

[0081] By initiating periodic background computation tasks during the offline batch processing phase, the computationally expensive operation of calculating future semantic vectors for a massive number of users is separated from the online query phase. This computational transfer design avoids real-time inference for a massive number of users during online queries, ensuring low-latency response and feasibility of the evolutionary prediction function in industrial-grade deployment. At the same time, by using the sliding window method and cosine similarity loss for model training, optimized embedding representations in the vector direction are provided for subsequent nearest neighbor search tasks, which helps to improve the construction quality of predictive semantic indexes and the final retrieval accuracy.

[0082] Furthermore, the semantic evolution model includes:

[0083] The time interval embedding sub-network receives the current timestamp and the previous timestamp from the historical semantic sequence; calculates the scalar time interval; inputs the scalar time interval into the time feature extractor, and converts the scalar into a high-dimensional time interval vector; the time feature extractor is a multilayer perceptron.

[0084] The state fusion layer receives a semantic vector, a previous historical state vector, and a high-dimensional time interval vector; it fuses the three vectors into a fusion input vector through vector concatenation.

[0085] A recursive state update layer receives a fused input vector; the recursive state update layer is a gated loop unit; the gated loop unit processes the fused input vector through gate weights to calculate the final historical state vector;

[0086] The prediction output layer, which is a multilayer perceptron, receives the final historical state vector and outputs a predicted future semantic vector. The specific process is as follows: Figure 3 As shown.

[0087] Specifically, the internal computational structure of the semantic evolution model is a time-time-aware recursive architecture.

[0088] The first component of this architecture is a time interval embedding subnetwork, which is an input processing module specifically designed to handle non-uniform time intervals. Its internal computation process includes: during runtime, when the current timestamp and the previous timestamp are received from the historical semantic state sequence, the subnetwork performs a subtraction operation in the processor to calculate a scalar time interval. Subsequently, the scalar time interval is input into a temporal feature extractor, which in this embodiment is a multilayer perceptron (MLP). The MLP performs a nonlinear transformation on the scalar, mapping it and outputting it as a high-dimensional time interval vector, which in principle represents the information brought about by the passage of time itself.

[0089] The second component of this architecture is the state fusion layer, which fuses content information (from semantic vectors) with temporal information (from time interval vectors). At runtime, it receives three inputs: a semantic vector (representing the content of the current event), a previous historical state vector (representing past memories), and a high-dimensional time interval vector (representing the passage of time). In this embodiment, the state fusion layer performs a vector concatenation operation, concatenating the three vectors dimensionally into a single, longer, time-aware fused input vector. The dimensionality of this fused input vector consists of three parts: the semantic vector dimension, the previous historical state vector dimension, and the high-dimensional time interval vector. To ensure model scaling consistency, the dimensions of the high-dimensional time interval vector and the previous historical state vector are both set to the same dimension as the semantic vector dimension. The number of hidden layer units in the gated recurrent unit is set to be no less than the same dimension as the semantic vector dimension, and its initial hidden state is initialized to a zero vector.

[0090] The third component of this architecture is a recursive state update layer, which in this embodiment is a gated recurrent unit. The gated recurrent unit only receives the time-aware fused input vector as its input. The gated recurrent unit applies its internal gating weights (i.e., update gate and reset gate) learned during the model training phase to process the time-aware fused input vector to dynamically fuse new information and old memories, and calculates and outputs a new historical state vector.

[0091] The fourth component of this architecture is the prediction output layer, which in this embodiment is a multilayer perceptron. After processing the last event of the input sequence, the final historical state vector output by the recursive state update core is input into the prediction output layer. The multilayer perceptron performs a final nonlinear transformation on it to output the final predicted future semantic vector, which is then used for loss calculation or offline inference.

[0092] By introducing a time interval embedding subnetwork, the irregular time intervals between asynchronous, sparse user events can be explicitly modeled within the model. This design enables the general semantic evolution model to distinguish the different effects of long intervals (which may lead to forgetting of interests) and short intervals (which may indicate enhanced interest), thereby improving the accuracy of constructing historical semantic state sequences and providing high-quality input that better reflects real-time dynamics for subsequent evolutionary predictions.

[0093] Furthermore, the process of generating the centroid vector of the customer group includes: in the online query stage, obtaining each seed user included in the seed customer group; for each seed user, retrieving the corresponding historical semantic sequence from the time series database; inputting the historical semantic sequence of each seed user into the semantic evolution model, performing an inference calculation, and generating a corresponding individual future semantic vector; after obtaining the individual future semantic vectors of all seed users in the seed customer group, performing a vector averaging operation on multiple individual future semantic vectors, calculating and outputting the centroid vector of the customer group.

[0094] Specifically, during the online query request receiving phase, a seed customer group is received from an external application (e.g., an advertising platform or a content operation backend). In this embodiment, the seed customer group is specifically a list of user IDs (e.g., a list containing 1000 seed user IDs). During the historical data retrieval phase, the user ID list is traversed. For each seed user ID, the method initiates a concurrent retrieval to the time-series database. This retrieval operation is used to obtain the latest historical semantic sequence of each seed user (e.g., the event sequence of the user in the last N days). The format of this sequence is consistent with the format of the input features used by the semantic evolution model during training.

[0095] In the batch inference computation stage, all (e.g., 1000) historical semantic sequences retrieved in the previous step are collected into a computation batch. Then, the method inputs the computation batch into the trained general semantic evolution model at once. The semantic evolution model (which has been loaded into the memory or GPU memory of the online inference server) performs inference computation once for each historical semantic state sequence in the batch. The output of this batch inference operation is multiple (e.g., 1000) corresponding individual future semantic vectors.

[0096] In the centroid vector aggregation stage, a vector averaging operation is performed on the multiple individual future semantic vectors generated in the previous step. Specifically, to overcome the potential representativeness issues that may arise from simple averaging operations in high-dimensional space, the calculation of the customer group centroid vector should employ a weighted averaging method, i.e.: ,in The weight corresponding to the i-th seed user. This is its individual future semantic vector. The weights... The strategy should be activity-weighted: The weight of the centroid vector should be proportional to the frequency of the user's historical events, meaning active users have a higher weight. The calculated centroid vector of the customer group should be validated: calculate the average cosine similarity between this vector and the individual vectors of all seed users. This value should not be lower than 0.85. If it is lower than this threshold, it indicates that the differences in interests within the group are too large, and the group should be decomposed into multiple subgroups for separate processing. The specific implementation of the step of decomposing the group into multiple subgroups for separate processing is as follows: Start clustering: when the average cosine similarity is lower than 0.85, the system starts the subgroup decomposition process, processing multiple future semantic vectors (i.e., ...) of the seed customer group. The system performs K-Means clustering; to determine the optimal number of subgroups K, the system uses silhouette coefficients for evaluation. The system iterates within a preset range of K values ​​(e.g., K = 2 to 5) and automatically selects the K value that results in the highest silhouette coefficient as the final number of subgroups; after clustering, the original seed customer group is decomposed into K subgroups. The system should treat each subgroup as an independent new seed customer group and perform the complete steps of calculating centroids and performing search separately for each subgroup, that is: calculate the customer centroid vectors of each of the K subgroups, and perform approximate nearest neighbor search for each of the K centroid vectors, and finally output the K predicted evolutionary customer groups.

[0097] The generation of the centroid vector of the customer group further includes: determining whether the user size of the seed customer group is less than a preset minimum robustness threshold; if it is less, then initiating a vector space enhancement process, inputting multiple future semantic vectors into a pre-trained variational autoencoder; the decoder of the variational autoencoder samples from the latent distribution implied by the future semantic vectors to generate enhanced future semantic vectors; merging the original future semantic vectors with the enhanced future semantic vectors to form an expanded vector set; the vector averaging operation is specifically implemented as: performing a vector averaging operation on the expanded vector set, calculating and outputting the centroid vector of the customer group.

[0098] During the online query request receiving phase, when a seed customer group (a list of user IDs) is obtained, the method of the present invention first determines in the processor whether the user size of the seed customer group is less than a preset minimum robustness threshold. In this embodiment, the minimum robustness threshold is set to 100 users. If the user size is greater than or equal to 100, the enhancement process of this embodiment is skipped, and the standard vector averaging operation is directly executed.

[0099] If the user base is less than 100 (e.g., only 50 seed users are acquired), a vector space augmentation process is initiated to address the bias and instability of the centroid vectors caused by insufficient samples. In this vector space augmentation process, the preceding steps are executed normally: the historical semantic state sequences of the 50 seed users are retrieved from the time-series database, and the trained general semantic evolution model is applied to perform inference calculations to generate 50 corresponding future semantic vectors. Next, these 50 future semantic vectors (as a set) are input into a pre-trained variational autoencoder, which is pre-trained offline using all future semantic vectors stored in the predictive semantic index as unsupervised data.

[0100] The variational autoencoder comprises an encoder and a decoder. The encoder is trained to compress input future semantic vectors (e.g., 768-dimensional) into a low-dimensional latent distribution (e.g., a 32-dimensional mean and variance vector). The decoder is trained to sample from this latent distribution and reconstruct the original 768-dimensional vector. During the execution of the augmentation process, these 50 future semantic vectors are fed into the encoder of the VAE to compute a population latent distribution they collectively represent (e.g., by averaging the mean and variance of the 50 latent distributions).

[0101] The variational autoencoder (VAE) uses future semantic vector samples stored in the predictive semantic index for unsupervised training; the encoder structure of the VAE is as follows: input layer (768 dimensions) -> hidden layer 1 (512 units) -> hidden layer 2 (256 units) -> output layer (64 dimensions in total, i.e., 32-dimensional mean + 32-dimensional variance), and the decoder adopts a reverse-order symmetrical design.

[0102] The training loss function of the VAE is: , This represents the mean squared error (MSE). Represents the KL divergence. Represents the KL divergence weights. The value is set to 0.001 to 0.01, and in this embodiment it is 0.005; the VAE model is retrained periodically (e.g., once a week), and the online query sampling process samples the potential distribution of the population N times, where N is set to 50 to 500 times, and in this embodiment it is 200 times, to generate sufficient augmentation vectors.

[0103] Subsequently, the decoder of the variational autoencoder randomly samples from the latent distribution of the group (e.g., 200 times) to generate a set of enhanced, synthetic future semantic vectors (200 new vectors similar to but not identical to the original vectors). The 50 original individual future semantic vectors are merged with the 200 enhanced, synthetic future semantic vectors to form an expanded vector set (250 vectors). Finally, the vector averaging operation is specifically implemented by performing a vector averaging operation on the expanded vector set (250 vectors) to calculate and output the centroid vector of the customer group. Since the centroid vector of the customer group is calculated based on a larger and more robust vector set, its stability and representativeness are significantly improved.

[0104] By performing independent inference calculations on each user in the seed customer group and then averaging the output individual future vectors, the common predicted evolutionary center point of the seed customer group can be accurately calculated. This "predict first, then average" process avoids information loss and noise interference caused by premature aggregation (averaging) of historical sequences, ensuring that the generated customer group centroid vector can more realistically reflect the common trend of group evolution, thereby providing a more accurate query target for subsequent nearest neighbor search steps.

[0105] Furthermore, the process of identifying the predetermined number of users with the highest similarity to the centroid vector of the customer group and outputting them as the predicted evolved customer group includes: inputting the centroid vector of the customer group as a query vector into a semantic index; the semantic index is an approximate nearest neighbor index; the approximate nearest neighbor index is configured to perform a candidate set retrieval operation; the candidate set retrieval operation returns a candidate user set whose size is larger than the final output size of the predicted evolved customer group and the approximate distances corresponding to the candidate user set;

[0106] For each candidate user in the candidate user set, a future semantic vector is extracted from the semantic index; in the computer processor, the vector distance between the future semantic vector of each candidate user and the centroid vector of the customer group is calculated; based on the calculated vector distance, the candidate user set is reordered, and from the reordered candidate user set, a predetermined number of users with the highest ranking are selected as approximate users for output.

[0107] Specifically, the process is implemented as a two-phase query and reordering architecture:

[0108] In the first stage: candidate set retrieval (coarse recall), the method performs the following process:

[0109] In this embodiment, the predictive semantic index (i.e., semantic index) is a FAISS index, specifically structured as IndexIVFPQ (Inverted File and Product Quantization). This index structure, in principle, uses quantization techniques to compress high-dimensional (e.g., 768-dimensional) future semantic vectors into low-dimensional codes to achieve extremely high retrieval speeds. However, this comes at the cost of sacrificing distance accuracy. The centroid vector of the customer group is used as the query vector and input into the FAISS index. The index is configured to perform a candidate set retrieval operation, which performs an approximate nearest neighbor search in the quantization space and returns a candidate user set larger than the predicted final output size of the evolved customer group (e.g., if 1000 users are ultimately needed, this step is configured to return 10000 candidate users) and their approximate distances.

[0110] The quantization parameters of the approximate nearest neighbor index are configured as follows: the number of inverted files is set to the square root of the total number of users, and constrained to be between 100 and 100,000; for example, for 1 million users, nlist is set to 1,000. Quantization bit depth: the number of bytes for product quantization is set to 4 to 16 bits, and in this embodiment, it is 8 bits; candidate set size: the number of candidate users returned by the candidate set retrieval operation is set to 5 to 20 times the final output number of users, and in this embodiment, it is 10 times; to ensure retrieval quality, after the FAISS index is constructed, a query test is performed on a known seed customer group to verify that the accuracy of Top-K retrieval should not be less than 0.8.

[0111] The configuration of the approximate nearest neighbor index should meet the following accuracy guarantee conditions: After the index is built, it should be verified by comparison test with an exact index (IndexFlat) that the recall rate in the coarse recall stage should not be less than 0.9 (i.e., the top K candidate users should include at least 90% of the truly nearest K users). If the recall rate in the coarse recall stage is less than 0.9, the return size of the candidate set should be increased (e.g., from 10 times to 20 times) or the quantization parameter should be adjusted (e.g., increase the number of n bits) until the recall is not less than 0.9. The system should periodically monitor the accuracy of coarse recall. If a downward trend in the recall rate is observed, the index should be rebuilt.

[0112] In the second phase: precise reordering (fine sorting), the method performs the following process:

[0113] The method initiates a reordering process. The technical principle of this process is that the approximate distance returned in the first stage is lossy. This stage requires precise calculation using complete, uncompressed vectors. For each candidate user in the candidate user set (e.g., 10,000 users), the method extracts its complete, pre-calculated future semantic vector from an independent vector database (e.g., a key-value store or an uncompressed FAISS IndexFlat) that stores complete, precise vectors. Then, in a computer processor, the precise vector distance between each candidate user's complete future semantic vector and the centroid vector of the customer group is precisely calculated. In this embodiment, this distance metric is specifically cosine similarity, which is consistent with the loss function (cosine similarity loss) used by the semantic evolution model during the training phase, ensuring consistency between training and inference. Finally, based on the calculated precise cosine similarity, the candidate user set (10,000 users) is reordered in descending order, and a predetermined number (e.g., 1,000) of users with the highest ranking are selected from the reordered list as the approximate users (i.e., the final predicted evolved customer group) for output.

[0114] By employing a two-stage query architecture of coarse recall using approximate index and precise vector reordering, this method balances the speed of online queries with the accuracy of the final results. First, it uses an approximate nearest neighbor index to quickly filter out a large set of candidate users. Then, it extracts the complete future semantic vectors from only the candidate set and performs precise vector distance calculations. This reordering step corrects the accuracy loss that may be introduced by approximate search, ensuring that the approximate users output in the final output are the group that is truly closest to the centroid vector of the customer group.

[0115] This invention, through an innovative offline pre-computation-online index query approach, shifts the computationally expensive task of calculating future semantic vectors for a massive number of users across the entire site from the online query stage to the offline batch processing stage. This computational shift design is a specific high-performance, low-latency system architecture implementation scheme proposed in the field of streaming media recommendation systems to achieve complex evolutionary prediction functions. Furthermore, this invention does not simply apply FAISS; instead, it explicitly adopts the IndexIVFPQ structure for specific technical scenarios involving high-dimensional vectors and massive user scales, and provides specific configurations for key parameters such as nlist and nbits. This ensures that this invention balances speed, accuracy, and memory usage during industrial-grade deployment.

[0116] Example 2:

[0117] In this embodiment, during the data acquisition and event stream construction phase, the backend data stream of the online streaming media platform is first accessed. The acquired historical unstructured text data specifically includes: (1) long comments published by users; (2) film review articles published by users in the platform's dynamic area or columns; (3) specific analytical or in-depth interpretation related search query terms entered by users in the platform's search box. The method extracts the precise timestamp of each piece of text data and constructs an asynchronous user in-depth comment event stream sorted by timestamp for each user.

[0118] During the generation of instantaneous semantic vectors, individual events (e.g., a 500-word movie review) are processed one by one according to the chronological order of the user's deep comment event stream. The text is input into a pre-trained bidirectional encoder model (a BERT model), which performs semantic encoding operations and extracts the final hidden layer output vector (a 768-dimensional vector) corresponding to its [CLS] tag, as the instantaneous deep comment semantic vector corresponding to the user at that precise timestamp.

[0119] In the recursive generation phase of the historical semantic state sequence, the instantaneous deep comment semantic vector and its precise timestamp are input into a time-interval-aware loop module. This module first retrieves the user's previous historical state vector and previous timestamp, and calculates the scalar time interval between the two events. Then, the module applies a trainable time decay function (which contains a decay parameter vector learned during model training that matches the dimension of the state vector) to perform element-wise multiplication on the previous historical state vector according to the scalar time interval to generate a decayed historical state. Finally, a recursive update gate (a GRU) merges the instantaneous deep comment semantic vector with the decayed historical state, calculates and outputs a new historical state vector, which is appended to the time series database along with its timestamp.

[0120] In the model training and offline index building phases, the method employs a time-interval-aware recursive architecture (such as a structure containing a time-interval embedding sub-network, a state fusion layer, and a recursive state update core) as the semantic evolution model. A sliding window method is used to extract training sample pairs from the time-series database (e.g., using the deep comment sequence of the past 90 days to predict the deep comment interest vector for the next 180 days). Cosine similarity loss is then used to train the model. After training, a periodic background computation task is initiated, which iterates through all active users and applies the trained general semantic evolution model to pre-calculate the future deep comment semantic state vector for each user in the future (e.g., the next 180 days). All these pre-calculated future vectors are then batch-imported into a FAISS (IndexIVFPQ) index to construct a predictive semantic index.

[0121] In the online query and generation of customer centroid vectors stage, when the platform wants to incubate high-value commenters, it obtains a seed customer group (for example, a list of user IDs of the platform's existing Top 500 film review KOLs), iterates through each user in the seed customer group, retrieves their historical in-depth comment semantic sequence, and inputs them into the trained semantic evolution model to perform inference calculations to generate multiple individual future in-depth comment semantic vectors. Subsequently, a vector averaging operation is performed on these multiple individual vectors to calculate and output a customer centroid vector.

[0122] In the online matching and output of the predicted evolving customer group stage, the method uses the customer group centroid vector generated in the previous step as the query vector and inputs it into the predictive semantic index (FAISS index) to perform an approximate nearest neighbor search to quickly return a large-scale candidate user set (e.g., the top 50,000 approximate users). Then, the method enters the precise re-ranking stage: extracting the complete, uncompressed future deep review semantic state vectors of these 50,000 candidate users, and calculating the precise cosine similarity between them and the customer group centroid vector in the processor. The method re-ranks the candidate set according to the precise cosine similarity and selects the users with the highest ranking (e.g., the top 1,000) as the predicted evolving customer group (i.e., potential KOLs) for output.

[0123] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A customer group evolution prediction method based on large-scale model semantic analysis, characterized in that, include: Acquire historical data from multiple users on a streaming platform. The historical data includes users' historical comments, bullet comments, and search queries. Semantic analysis is performed on historical data to generate semantic vectors for each user at multiple time points; the semantic vectors are processed in chronological order through a loop module to generate historical semantic sequences for each user, and the historical semantic sequences are stored in a time-series database. The process of generating a historical semantic sequence for each user and storing the historical semantic sequence in a time-series database includes: for each of the multiple users, initializing a historical state vector in computer memory, obtaining the semantic vector and timestamp as an asynchronous user event stream, and inputting them one by one into a loop module in chronological order of timestamps; when the loop module receives the current semantic vector and current timestamp, retrieving the previous historical state vector and previous timestamp from the historical semantic sequence and calculating the time interval between the current timestamp and the previous timestamp; the loop module applies a trainable time decay function to decay the influence of the previous historical state vector according to the time interval, generating a decayed historical state; the loop module inputs the semantic vector and the decayed historical state together into a recursive update gate to calculate a new historical state vector; and appends and stores the new historical state vector and timestamp into the time-series database as the latest state of the historical semantic sequence. When the model training task is started, the historical semantic sequence is obtained, and the semantic evolution model is trained based on the historical semantic sequence. In the offline batch processing stage, the semantic evolution model is used to calculate the future semantic vector corresponding to each user, and the future semantic vectors of multiple users are stored in the semantic index. When a customer base diffusion request is received, a seed customer base is obtained; the corresponding historical semantic sequence of the seed customer base is retrieved from the time series database; the historical semantic sequence of the seed customer base is calculated using a semantic evolution model to generate a customer base centroid vector; The centroid vector of the customer group is used as the query vector and input into the semantic index. An approximate nearest neighbor search is performed in the semantic index to identify a predetermined number of users with the highest similarity to the centroid vector of the customer group, which are then output as the predicted evolved customer group.

2. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of generating semantic vectors for each user includes: acquiring historical data, including the user's historical comments, bullet comments, and search queries, and extracting the timestamp corresponding to each piece of historical data; for each user among multiple users, constructing the historical data and timestamps into an asynchronous user event stream ordered by timestamps in computer memory; processing individual events in the user event stream one by one according to the chronological order of the user event stream, where each individual event corresponds to a comment, a bullet comment, and / or a search query; inputting the text data of the individual event into a pre-trained bidirectional encoder model, where the bidirectional encoder model is a specific implementation of the larger model; and performing semantic encoding operations on the text data of the individual event, outputting the final hidden layer output vector of the labeled text data as the semantic vector corresponding to the user at the timestamp.

3. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of the trainable time decay function includes: the trainable time decay function receives the calculated time interval as the input time; the trainable time decay function applies an internal trainable decay parameter vector, the dimension of which matches the dimension of the previous historical state vector; each parameter value in the trainable decay parameter vector is learned and determined during the model training phase through backpropagation, and each parameter value is used to characterize the interest forgetting rate represented by different dimensions in the historical state vector; the trainable time decay function uses the trainable decay parameter vector as the forgetting rate to perform an exponential decay operation on the input time to calculate the decay weight vector; the trainable time decay function performs element-wise multiplication of the decay weight vector with the previous historical state vector to generate the decayed historical state.

4. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of storing the future semantic vectors of multiple users in a semantic index includes: during the model training phase, extracting training sample pairs from the time-series database using a sliding window method; the training sample pairs include historical semantic state sequences as input features and actual future semantic vectors as training labels; inputting the input features and training labels into the semantic evolution model for training, and obtaining the trained semantic evolution model by minimizing the vector distance loss between the future vectors predicted by the semantic evolution model and the training labels; during the offline batch processing phase, starting a periodic background computing task, traversing multiple users, retrieving the latest historical semantic sequence from the time-series database for each user, applying the semantic evolution model to perform an inference calculation, and calculating the future semantic vector corresponding to the user; the background computing task batch imports the calculated future semantic vectors corresponding to all users into a high-dimensional vector index, constructing and outputting the semantic index.

5. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 4, characterized in that, The semantic evolution model includes: The time interval embedding sub-network receives the current timestamp and the previous timestamp from the historical semantic sequence; calculates the scalar time interval; inputs the scalar time interval into the time feature extractor, and converts the scalar into a high-dimensional time interval vector; the time feature extractor is a multilayer perceptron. The state fusion layer receives a semantic vector, a previous historical state vector, and a high-dimensional time interval vector; it fuses the three vectors into a fusion input vector through vector concatenation. A recursive state update layer receives a fused input vector; the recursive state update layer is a gated loop unit; the gated loop unit processes the fused input vector through gate weights to calculate the final historical state vector; The prediction output layer, which is a multilayer perceptron, receives the final historical state vector and outputs a predicted future semantic vector.

6. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of generating the centroid vector of the customer group includes: in the online query stage, obtaining each seed user included in the seed customer group; for each seed user, retrieving the corresponding historical semantic sequence from the time series database; inputting the historical semantic sequence of each seed user into the semantic evolution model, performing an inference calculation, and generating a corresponding individual future semantic vector; after obtaining the individual future semantic vectors of all seed users in the seed customer group, performing a vector averaging operation on multiple individual future semantic vectors, calculating and outputting the centroid vector of the customer group.

7. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of identifying a predetermined number of users with the highest similarity to the centroid vector of the customer group and outputting them as the predicted evolved customer group includes: inputting the centroid vector of the customer group as a query vector into a semantic index; the semantic index is an approximate nearest neighbor index; the approximate nearest neighbor index is configured to perform a candidate set retrieval operation; the candidate set retrieval operation returns a candidate user set with a size larger than the final output size of the predicted evolved customer group and the approximate distances corresponding to the candidate user set; For each candidate user in the candidate user set, a future semantic vector is extracted from the semantic index; in the computer processor, the vector distance between the future semantic vector of each candidate user and the centroid vector of the customer group is calculated; based on the calculated vector distance, the candidate user set is reordered, and from the reordered candidate user set, a predetermined number of users with the highest ranking are selected as approximate users for output.

Images