Customer group evolution prediction method based on large model semantic analysis

Abstract

The invention relates to the technical field of semantic analysis, in particular to a customer group evolution prediction method based on large model semantic analysis, which comprises the following steps: acquiring historical unstructured data of streaming media platform users, and executing semantic analysis at multiple time points through a large model to generate semantic vectors; processing the semantic vector through a loop module to generate a historical semantic sequence; training a semantic evolution model based on the sequence; the method comprises the following steps: in an offline batch processing stage, pre-calculating future semantic vectors of all users by utilizing an evolution model, and constructing a semantic index; when a seed customer group is received online, calculating a customer group centroid vector representing a future trend for the seed customer group; and performing approximate neighbor search on the centroid vector in a predictive semantic index, and outputting a predicted evolved customer group. According to the method, the prediction accuracy is improved through future matching, and low delay and feasibility of an evolution prediction function are ensured through an off-line pre-calculation framework.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semantic analysis, in particular to a customer group evolution prediction method based on large model semantic analysis. BACKGROUND

[0002] In the Internet information service platform, customer group diffusion technology is widely used in content recommendation and precision marketing. The main goal of this technology is to find potential users with similar characteristics in a large number of candidate users based on a given set of seed users.

[0003] Existing customer group diffusion schemes are mainly based on user historical interaction data modeling. In terms of data processing, the common practice is to convert user clicks, browsing, purchasing and other behaviors into structured feature vectors. For text data generated by users (such as tags, short descriptions, etc.), existing technologies usually use keyword extraction, topic modeling or bag-of-words model, etc. Natural language processing technology maps it to discrete labels or low-dimensional features to facilitate input into collaborative filtering or logistic regression models for processing.

[0004] In terms of time series modeling, existing technologies usually use time window aggregation to process user data. For example, user behavior data is counted by day or by week to generate a series of state snapshots arranged at fixed time intervals. Based on these snapshots, recurrent neural networks (RNN) or long short-term memory networks (LSTM) can be used to capture user interest trends and generate user current interest representations.

[0005] In the online application stage, when customer group diffusion is needed, the pre-computed user current interest representation is usually used to calculate the similarity (such as cosine similarity) between seed users and candidate users in the vector space, so as to filter out the user group with the closest current features as the diffusion result. With the growth of platform user scale, how to process high-dimensional feature matching of massive users while ensuring response speed is a technical direction that continues to be concerned in this field.

[0006] Therefore, a customer group evolution prediction method based on large model semantic analysis is proposed. SUMMARY

[0007] The application aims to provide a customer group evolution prediction method based on large model semantic analysis, which improves prediction accuracy through future-to-future matching and ensures low latency and feasibility of evolution prediction function through offline pre-computation architecture. The method includes obtaining historical unstructured data of users of a streaming media platform, performing semantic analysis on multiple time points through a large model to generate corresponding semantic vectors; processing the semantic vectors through a loop module to generate a historical semantic sequence; and training a semantic evolution model based on the sequence; in the offline batch processing stage, using the evolution model, pre-computing future semantic vectors of all users, and constructing a semantic index; when receiving a seed customer group online, calculating a customer group centroid vector representing future trends for the seed customer group; performing approximate nearest neighbor search on the centroid vector in the predictive semantic index, and outputting the predicted evolved customer group.

[0008] To achieve the above object, the application provides the following technical scheme. A customer group evolution prediction method based on large model semantic analysis, comprising: Obtaining historical data of multiple users in a streaming media platform, the historical data including historical comments, bullet screens and search query words of the users; performing semantic analysis on the historical data to generate semantic vectors for each user at multiple time points; processing the semantic vectors in time sequence through a loop module to generate a historical semantic sequence for each user, and storing the historical semantic sequence in a time series database; When starting a model training task, obtaining the historical semantic sequence, and training a semantic evolution model based on the historical semantic sequence; in the offline batch processing stage, using the semantic evolution model to calculate the future semantic vector corresponding to each user, and storing the future semantic vectors of the multiple users in a semantic index; When receiving a customer group diffusion request, obtaining a seed customer group; retrieving the corresponding historical semantic sequence of the seed customer group from the time series database; using the semantic evolution model to calculate the historical semantic sequence of the seed customer group to generate a customer group centroid vector; Inputting the customer group centroid vector as a query vector into the semantic index; performing approximate nearest neighbor search in the semantic index to identify approximate users as the predicted evolved customer group for output.

[0009] Preferably, the process of generating semantic vectors for each user comprises: obtaining historical data, the historical data comprising historical comments, bullet screens and search query words of users, and extracting time stamps corresponding to each piece of historical data; for each user in the plurality of users, constructing the historical data and time stamps into a time-stamped asynchronous user event stream in the computer memory; processing single events in the user event stream one by one in chronological order of the user event stream, the single event corresponding to a comment, a bullet screen and / or a search query word; inputting text data of the single event into a pre-trained bidirectional encoder model, the bidirectional encoder model being a specific implementation of a large model; the bidirectional encoder model performing semantic encoding operations on the text data of the single event, and outputting a final hidden layer output vector of the token corresponding to the text data as the semantic vector of the user at the time stamp.

[0010] Preferably, the process of generating historical semantic sequences for each user and storing the historical semantic sequences in the time series database comprises: for each user in the plurality of users, initializing a historical state vector in the computer memory, obtaining the semantic vectors and time stamps as an asynchronous user event stream, and inputting them one by one into a loop module in chronological order of the time stamps, the loop module retrieving a previous historical state vector and a previous time stamp from the historical semantic sequence and calculating a time interval between the current time stamp and the previous time stamp when receiving the current semantic vector and the current time stamp; the loop module applying a trainable time decay function to decay the influence of the previous historical state vector according to the time interval, generating a decayed historical state; the loop module inputting the semantic vector and the decayed historical state into a recursive update gate to calculate a new historical state vector; and appending the new historical state vector and the time stamp to the time series database as the latest state of the historical semantic sequence.

[0011] Preferably, the process of the trainable time decay function comprises: the trainable time decay function receiving the calculated time interval as an input time; the trainable time decay function applying an internal trainable decay parameter vector, the dimension of the trainable decay parameter vector matching the dimension of the previous historical state vector; each parameter value in the trainable decay parameter vector is learned and determined by a backpropagation algorithm during a model training phase, and each parameter value is used to represent an interest forgetting rate of different dimensions in the historical state vector; the trainable time decay function uses the trainable decay parameter vector as a forgetting rate to perform an exponential decay operation on the input time to calculate a decay weight vector; and the trainable time decay function performs element-wise multiplication between the decay weight vector and the previous historical state vector to generate a decayed historical state.

[0012] Preferably, the process of storing the future semantic vectors of multiple users in the semantic index comprises: in a model training phase, extracting training sample pairs from the time series database using a sliding window method, the training sample pairs comprising a historical semantic state sequence as an input feature and an actual future semantic vector as a training label; inputting the input feature and the training label into the semantic evolution model for training, and obtaining the trained semantic evolution model by minimizing the vector distance loss between the future vector predicted by the model and the training label; in an offline batch processing phase, starting a periodic background computing task, traversing multiple users, for each user, retrieving the latest historical semantic sequence from the time series database, and applying the semantic evolution model to perform an inference calculation to calculate the future semantic vector corresponding to the user; the background computing task batch imports all the calculated future semantic vectors corresponding to the users into the high-dimensional vector index, constructs and outputs the semantic index.

[0013] Preferably, the semantic evolution model comprises: a time interval embedding sub-network receiving a current timestamp and a previous timestamp in the historical semantic sequence; calculating a scalar time interval; inputting the scalar time interval into a time feature extractor to convert the scalar into a high-dimensional time interval vector; the time feature extractor is a multilayer perceptron; a state fusion layer receiving the semantic vector, the previous historical state vector and the high-dimensional time interval vector; fusing the three vectors into a fusion input vector through vector splicing operation; a recursive state update layer; receiving the fusion input vector; the recursive state update layer is a gated recurrent unit; the gated recurrent unit processes the fusion input vector through a gating weight to calculate a final historical state vector; a prediction output layer, the prediction output layer is a multilayer perceptron; receiving the final historical state vector and outputting a predicted future semantic vector.

[0014] Preferably, the process of generating the guest group centroid vector comprises: in an online query phase, obtaining each seed user contained in a seed guest group; for each seed user, retrieving the corresponding historical semantic sequence from the time series database; inputting the historical semantic sequence of each seed user into the semantic evolution model respectively to perform an inference calculation to generate an individual future semantic vector corresponding to each seed user; after obtaining the individual future semantic vectors of all seed users in the seed guest group, performing a vector averaging operation on the multiple individual future semantic vectors to calculate and output the guest group centroid vector.

[0015] Preferably, the process of identifying a predetermined number of users with the highest similarity to the centroid vector of the target group as the predicted evolving target group for output comprises: inputting the centroid vector of the target group as a query vector into a semantic index; the semantic index is an approximate neighbor index; the approximate neighbor index is configured to perform a candidate set retrieval operation; the candidate set retrieval operation returns a candidate user set with a size larger than the final output size of the predicted evolving target group and an approximate distance corresponding to the candidate user set; For each candidate user in the candidate user set, a future semantic vector is extracted from the semantic index; in the computer processor, the vector distance between the future semantic vector of each candidate user and the centroid vector of the target group is calculated; the candidate user set is re-ranked according to the calculated vector distance, and a predetermined number of users with the highest ranking in the re-ranked candidate user set are selected as approximate users for output.

[0016] Compared with the prior art, the present application has the following advantages: 1. By using the pre-calculated future semantic vector when matching the target group, instead of the current state vector of the user, this design of matching the future state enables the trend of user interest evolution over time (i.e. "interest drift") to be taken into account when performing target group diffusion. Compared with the method of matching only the current state, the diffusion result of the present application is more adaptable to the dynamic changes of the user, thereby helping to improve the relevance and accuracy of the diffusion target group at future time points.

[0017] 2. By separating the calculation architecture into offline and online, the present application provides an efficient and computationally feasible solution for implementing the complex "evolution prediction" function. By pre-calculating the future semantic vector of the massive users in the entire station in the offline batch processing stage, this design of calculation transfer enables only one prediction for the seed target group and one index search to be performed during online query. This not only ensures that the target group evolution prediction function can be implemented, but also ensures low-latency response of the online query, making it suitable for industrial deployment environments with high real-time requirements.

[0018] 3. By its time-aware recurrent module design, the modeling accuracy of asynchronous sparse event streams is improved. The historical data of users (such as comments and bullet screens) are usually irregularly distributed in time. By introducing time interval embedding and a trainable time decay function, the passage of time itself is modeled as a trainable feature. This fine processing of asynchronous sparse data streams enables the construction of historical semantic sequences to more reasonably reflect the forgetting process of user interest over irregular passage of time, thereby providing a higher quality input for the semantic evolution model and helping to improve the accuracy of its prediction. BRIEF DESCRIPTION OF DRAWINGS

[0019] Fig. 1 A flowchart of a customer group evolution prediction method based on large model semantic analysis of the present application is shown in the figure. Fig. 2 A flowchart of storing historical semantic sequences in a time series database of the present application is shown in the figure. Fig. 3 A flowchart of a semantic evolution model of the present application is shown in the figure. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0021] Please refer to Figs. 1 to 3 The present application provides a customer group evolution prediction method based on large model semantic analysis, and the technical solutions are as follows: Embodiment one: A customer group evolution prediction method based on large model semantic analysis, the specific process is shown in the figure, which includes. Fig. 1

[0022] Obtain historical data of a plurality of users in a streaming media platform, the historical data including historical comments, bullet screens and search query words of the users; perform semantic analysis on the historical data to generate semantic vectors for each user at a plurality of time points; process the semantic vectors in time sequence through a loop module to generate historical semantic sequences for each user, and store the historical semantic sequences in a time series database; When a model training task is started, obtain the historical semantic sequences, and train a semantic evolution model based on the historical semantic sequences; in an offline batch processing stage, calculate the future semantic vectors corresponding to each user by using the semantic evolution model, and store the future semantic vectors of the plurality of users in a semantic index; When a customer group diffusion request is received, obtain a seed customer group; retrieve the corresponding historical semantic sequence of the seed customer group from the time series database; calculate the historical semantic sequence of the seed customer group by using the semantic evolution model to generate a customer group centroid vector; Input the customer group centroid vector as a query vector into the semantic index; perform approximate neighbor search in the semantic index to identify approximate users as the predicted evolved customer group for output.

[0023] ​Further, the process of generating semantic vectors for each user includes: obtaining historical data including historical comments, bullet screens and search query words of the user, and extracting the time stamp corresponding to each piece of historical data; for each user in the plurality of users, constructing the historical data and the time stamp in the computer memory as a timestamp-ordered asynchronous user event stream; processing single events in the user event stream one by one in the chronological order of the user event stream, the single event corresponding to a comment, a bullet screen and / or a search query word; inputting the text data of the single event into a pre-trained bidirectional encoder model, the bidirectional encoder model being a specific implementation of a large model; the bidirectional encoder model performs semantic encoding operation on the text data of the single event, and outputs the final hidden layer output vector of the token corresponding to the text data as the semantic vector of the user at the time stamp.

[0024] Specifically, in the data acquisition and event stream construction stage, the embodiment accesses the backend data stream of the online streaming media platform to obtain historical unstructured text data, which specifically includes: (1) the bullet screen content sent by the user when watching the video; (2) the historical comments published by the user in the video comment area or social media; (3) the search query words input by the user in the platform search box, not only the text content, but also the accurate time stamp (for example, accurate to seconds) associated with the text event. After obtaining, for each user, all the historical text data and the accurate time stamp thereof are constructed in the computer memory as a timestamp-ordered asynchronous user event stream.

[0025] To ensure the accuracy and consistency of the time stamp, the accurate time stamp should be obtained in the following way: time source: use the system time of the server side as the standard time source, when the user submits a comment, a bullet screen or a search request, the server should immediately record the receiving time of the request, and the accuracy should not be less than seconds; calibration: if there is a local time stamp on the user side, it should be calibrated through a time synchronization protocol (such as NTP) after the server receives the request; and the system should use the following strategy to handle time stamp anomalies: if the time interval of two consecutive events exceeds 365 days, the event should be considered as abnormal and excluded; if the time stamp of the event is out of order, the system should reorder it according to the physical order of the event in the log.

[0026] In the large model selection stage, in order to achieve semantic analysis, this embodiment adopts a pre-trained bidirectional encoder model (as a specific implementation of the large model). In this embodiment, the pre-trained bidirectional encoder model is specifically a BERT model based on the Transformer architecture. In this embodiment, the bert-base-chinese model is adopted. This model has been pre-trained on a massive Chinese corpus and can deeply understand and capture complex contextual semantic information, making it suitable for processing short texts such as comments, bullet comments, and search terms.

[0027] In the asynchronous event processing phase, this embodiment processes individual events in the user event stream one by one according to the chronological order of the stream. Each individual event corresponds to a comment, a bullet screen comment, or a search query. In the semantic encoding phase, when processing a single event (e.g., the text content is "This show has a great pace"), the following detailed process is executed: (Word segmentation): The text data of the single event is input into the word segmenter of the BERT model. The word segmenter converts the text into a sequence of input tokens that the model can recognize and automatically adds a [CLS] (Class) token at the beginning of the sequence.

[0028] (Encoding): The input labeled sequence is fed into the BERT model to perform semantic encoding, i.e., one forward propagation computation.

[0029] (Extraction): In the final hidden layer of the model output, the output vector corresponding to the [CLS] tag is extracted. In this embodiment, the output vector is a 768-dimensional vector. The 768-dimensional vector is output as the semantic vector corresponding to the user at the precise timestamp and waits for processing by the subsequent loop module.

[0030] The semantic vector should have a dimension of no less than 256 and no more than 1024; preferably, the semantic vector dimension is between 512 and 1024. The dimension selection should meet one of the following conditions: a) the output dimension is directly determined based on the pre-trained bidirectional encoder model; b) the original high-dimensional vector is compressed to a specified dimension using a dimension compression algorithm (such as principal component analysis or autoencoder), and the compressed vector should retain at least 85% of the information content of the original vector.

[0031] By constructing user historical data into an asynchronous user event stream and performing semantic encoding on each individual event (such as a bullet comment or search term) within the stream, the method of this invention can capture high-temporal-precision points of interest that are ignored by traditional periodic aggregation methods (such as daily aggregation). This refined processing ensures that even transient or sparse expressions of interest can be effectively captured, providing richer and more accurate instantaneous semantic input for subsequent evolutionary analysis.

[0032] Further, the process of generating a history semantic sequence for each user and storing the history semantic sequence in the time series database comprises: for each user in the plurality of users, initializing a history state vector in the computer memory, obtaining the semantic vector and the timestamp, as an asynchronous user event stream, inputting one by one into the loop module in chronological order of the timestamp, the loop module retrieves the previous history state vector and the previous timestamp from the history semantic sequence when receiving the current semantic vector and the current timestamp, and calculates the time interval between the current timestamp and the previous timestamp; the loop module applies a trainable time decay function to decay the influence of the previous history state vector according to the time interval, generates a decayed history state; the loop module inputs the semantic vector and the decayed history state into the recursive update gate, calculates a new history state vector; the new history state vector and the timestamp are appended to the time series database as the latest state of the history semantic sequence, and the specific flow chart is as shown in Fig. 2 .

[0033] In the initialization phase, the embodiment initializes a history state vector in the computer memory for each user in the plurality of users, which vector (for example, a 768-dimensional zero vector) will serve as the starting point for recursive calculation, and in the asynchronous event stream processing phase, the semantic vector and its precise timestamp are obtained and input into the loop module as an asynchronous user event stream in chronological order of the precise timestamp.

[0034] The loop module receives the current instantaneous semantic vector and its current timestamp, and performs the following detailed process: First, the loop module retrieves the previous history state vector and its previous timestamp of the user from the time series database, then the module calculates the time interval between the current timestamp and the previous timestamp, i.e. the scalar time interval, in the processor. The scalar time interval represents the silent period or forgetting period between two interest expressions of the user in principle.

[0035] Then, the loop module applies a trainable time decay function, which decays the influence of the previous history state vector according to the calculated scalar time interval, to generate a decayed history state. This step is used to mathematically simulate the natural forgetting of user interest over time.

[0036] Subsequently, the cycle module inputs the semantic vector (representing the new interest event) and the decayed history state (representing the decayed old interest) into a recurrent update gate, which in this embodiment is a gated recurrent unit (GRU), the gating weights (learned in the subsequent model training phase) of the GRU dynamically calculate, through its update gate and reset gate, how much the new interest event should cover the decayed old interest, and output a new history state vector.

[0037] Finally, the new history state vector and its accurate timestamp are appended to the time series database, the new history state vector is the latest state of the history semantic state sequence, and waits for the arrival of the next asynchronous event.

[0038] By introducing time interval calculation and trainable time decay function in the cycle module, the "natural forgetting" process of user interest can be explicitly modeled, and this design can reasonably distinguish the different effects brought by long intervals (leading to interest decay) and short intervals (maintaining interest intensity) when processing asynchronous event streams with irregular time intervals, thereby improving the construction accuracy of the history semantic state sequence, and providing a more realistic time dynamic high-quality input for the subsequent evolution prediction model.

[0039] Further, the flow of the trainable time decay function includes: the trainable time decay function receives the calculated time interval as the input time; the trainable time decay function applies an internal trainable decay parameter vector, the dimension of the trainable decay parameter vector matches the dimension of the previous history state vector; each parameter value in the trainable decay parameter vector is learned and determined through the back propagation algorithm in the model training phase, and each parameter value is used to represent the interest forgetting rate of different dimensions in the history state vector; the trainable time decay function uses the trainable decay parameter vector as the forgetting rate to perform exponential decay operation on the input time to calculate a decay weight vector; the trainable time decay function performs element-wise multiplication between the decay weight vector and the previous history state vector to generate a decayed history state.

[0040] Specifically, in the definition and initialization phase of the model component, the embodiment internally defines and initializes a trainable decay parameter vector in the semantic evolution model, which is a trainable weight matrix, and the dimension of the vector is strictly set to match the dimension of the previous historical state vector. For example, if the historical state vector is 768-dimensional, the trainable decay parameter vector is also a vector containing 768 independent floating-point parameters. The specific numerical value of each parameter value in the trainable decay parameter vector is not a hyperparameter set by humans in advance, but is learned and determined through the back propagation algorithm in the model training phase as a trainable part of the semantic evolution model. The technical meaning of this design is that each parameter value is trained to represent the forgetting rate of a specific interest represented by a different dimension (e.g., the i-th dimension) in the historical state vector.

[0041] In the runtime computation phase, when the loop module receives the calculated time interval (a scalar representing the time elapsed between two events) as the input time, the trainable time decay function performs the following detailed process: first, the trainable time decay function applies its internal trainable decay parameter vector (as a forgetting rate) to perform an exponential decay operation on the input time, multiplies the input time (scalar) with the trainable decay parameter vector (a 768-dimensional vector) (through the broadcast mechanism, i.e., scalar multiplication with each element of the vector), and then takes the negative exponent. The output of this operation is a new 768-dimensional vector, which is the decay weight vector in this embodiment. Finally, the trainable time decay function performs element-wise multiplication of the decay weight vector and the previous historical state vector (a 768-dimensional vector). The calculation result of the element-wise multiplication (i.e., multiplication of corresponding dimensions of two 768-dimensional vectors) is the decayed historical state vector, which is then sent to the recursive update gate for the next step of fusion computation.

[0042] The implementation of the trainable time decay function can adopt one of the following ways: exponential decay method: suitable for scenarios where user interest forgetting follows an exponential law; piecewise linear decay method: the decay function can be defined in segments, for example: fast decay (slope 0.1 / day) within 0-7 days, and slow decay (slope 0.02 / day) within 7-30 days, to more finely simulate the forgetting process of interest; high-order polynomial decay method: use a second or third order polynomial function to adapt to more complex interest evolution laws. The selected decay method should be evaluated by the performance of the validation set in the model training phase to select the method that minimizes the validation loss.

[0043] To ensure the stability and effectiveness of model training, the parameter values of the trainable decay parameter vector are set as follows: (1) Initialization: each parameter value is randomly sampled from a uniform distribution in the interval [0.01, 0.1]; (2) Training constraint: during the model training process, the gradient clipping method is used to ensure that each parameter value is always located in the interval (0, 1) to ensure the mathematical validity of the exponential decay function; (3) Anti-overflow: exponential decay operation is used wherein is a time interval (unit: day), is a decay parameter, and is constrained to to prevent numerical underflow.

[0044] In a preferred embodiment, before the step of inputting the semantic vector and the decayed historical state into the recurrent update gate, the cycle module further comprises: when generating the semantic vector, additionally acquiring and storing a data source type, the data source type representing whether the historical unstructured text data is from a historical review, a bullet screen or a search query word; inputting the semantic vector and the corresponding data source type into a pre-trained source attention subnetwork; the source attention subnetwork is configured to calculate a semantic relevance weight between 0 and 1 according to the data source type, and the parameters of the source attention subnetwork are learned together with the cycle module in the model training stage; multiplying the semantic relevance weight and the semantic vector to generate a weighted semantic vector; and the step of inputting the semantic vector and the decayed historical state into the recurrent update gate is specifically implemented as: inputting the weighted semantic vector and the decayed historical state into the recurrent update gate to calculate a new historical state vector.

[0045] Specifically, the technical principle of the design is that not all text data can equally represent the real interests of users, a search query word may be only transient and low-relevance noise, while a long review represents high-relevance real interest. The embodiment solves this problem by introducing a source attention subnetwork. In the data acquisition and event stream construction stage, the method additionally acquires and stores a data source type. When processing a bullet screen, the transient semantic vector, the accurate timestamp and the data source type of the bullet screen are stored. When processing a search, the transient semantic vector, the accurate timestamp and the search query word are stored. In the asynchronous event stream processing stage, when the cycle module receives the current semantic vector and its data source type, after performing the decay step and before performing the recurrent update gate, the following source attention process is additionally performed.

[0046] The data source types include historical comments, bullet screens, and search query words; each category is encoded into an independent embedding vector, and the dimension is set to 64-256 dimensions, 128 dimensions in this embodiment; the source attention subnetwork is a multi-layer perceptron, and the structure is specifically: input layer (receiving the splicing of 768-dimensional semantic vector and 128-dimensional source embedding vector, a total of 896 dimensions) -> hidden layer 1 (256 units, ReLU activation) -> hidden layer 2 (64 units, ReLU activation) -> output layer (1 unit, Sigmoid activation); the semantic correlation weight (scalar) output by the Sigmoid is multiplied element by element with the semantic vector (768 dimensions) to generate a weighted semantic vector with a dimension of 768; at the beginning of model training, the initial weight corresponding to all data source types is set to 0.5 to avoid the suppression of some data sources due to improper initialization.

[0047] The semantic vector and the corresponding data source type are input into a source attention subnetwork, and in this embodiment, the source attention subnetwork is a multi-layer perceptron. The input layer of the multi-layer perceptron receives the splicing of the semantic vector and the embedding vector of the data source type. All parameters of the multi-layer perceptron and the embedding vectors of the data source types such as comments, bullet screens, and searches are learned and determined through the back propagation algorithm during the model training phase. During training, the model is forced to learn the contribution of different data sources to the prediction accuracy. The model learns that the input of the search query word type should be given a lower weight, and the input of the historical comment type should be given a higher weight.

[0048] The output layer of the multi-layer perceptron adopts a Sigmoid activation function to ensure that its output is a scalar between 0 and 1, which is the semantic correlation weight. Then, the semantic correlation weight scalar and the semantic vector are multiplied to generate a weighted semantic vector. Finally, the input received by the recursive update gate is no longer the original instantaneous semantic vector, but the weighted instantaneous semantic vector, which is fused with the decayed historical state to calculate a new historical state vector.

[0049] By introducing the source attention subnetwork, the importance of different data sources (such as comments and search words) can be automatically learned and distinguished. This design suppresses the interference of low correlation noise (such as search words) by weighting the input, improves the construction accuracy of the historical semantic state sequence, and provides more reliable input for subsequent evolution prediction.

[0050] By learning an independent trainable decay parameter for each dimension of the historical state vector, the method can more finely simulate the differential forgetting characteristics of different types of interests. This design allows the model to reasonably distinguish between long-term stable interests (slow forgetting) and short-term hot interests (fast forgetting) when processing asynchronous event streams with irregular time intervals, thereby improving the construction accuracy of the historical semantic state sequence and providing higher quality input for the subsequent evolution prediction model.

[0051] Further, the process of storing the future semantic vectors of multiple users in the semantic index comprises: in the model training phase, extracting a training sample pair from the time series database using a sliding window method, the training sample pair comprising a historical semantic state sequence as an input feature and an actual future semantic vector as a training label; inputting the input feature and the training label into the semantic evolution model for training, and obtaining the trained semantic evolution model by minimizing the vector distance loss between the future vector predicted by the model and the training label; in the offline batch processing phase, starting a periodic background computing task, traversing multiple users, for each user, retrieving the latest historical semantic sequence from the time series database, and applying the semantic evolution model to perform an inference calculation to calculate the future semantic vector corresponding to the user; the background computing task batch imports all the calculated future semantic vectors corresponding to the users into the high-dimensional vector index to construct and output the semantic index.

[0052] Specifically, in the model training phase, the following detailed process is performed in this embodiment: First, in the training sample construction step, training sample pairs are extracted from the time series database using a sliding window method, and the parameters of the sliding window method are configured as follows: the length of the historical window is set to 10-20 events if a window based on the number of events is used, and the span is set to 14-30 days if a window based on the time span is used; the future prediction horizon K is set to 7 days; when the total number of historical events of a user is less than the set length of the historical window, a padding strategy is used. Specifically, a historical window length (for example, N=10 events or N=14 days) and a future prediction horizon (for example, K=7 days) are set, and the window is slid on the user's historical semantic sequence to extract the sequence (including semantic vectors and accurate timestamps) within the historical window as input features, and the actual future semantic vector on the Kth day after the end of the window is extracted as the training label. Then, in the model training step, the input features and the training labels are batch input into the semantic evolution model, which is a sequence-to-vector neural network architecture. The model is trained by minimizing the vector distance loss between the predicted future vector of the model and the training label (actual future semantic vector). In this embodiment, the loss function is specifically the cosine similarity loss, which is used to maximize the directional consistency of two vectors in high-dimensional space in mathematics. After training, the trained semantic evolution model is obtained.

[0053] In the offline batch processing phase, the following detailed process is performed in this embodiment: First, in the task scheduling step, a periodic background computing task is started, which is configured by a task scheduling tool (such as Apache Airflow) to be automatically executed daily (for example, at 2 a.m. every day), and then in the batch inference step, after the background computing task is started, it will traverse all active users in the online streaming platform. For each user, the task retrieves its latest historical semantic state sequence from the time series database, then loads the trained general semantic evolution model and performs an inference calculation on the latest historical semantic state sequence to precompute the future semantic vector corresponding to the user. Finally, in the index construction step, the background computing task batch imports all precomputed future semantic vectors of users into a high-dimensional vector index, which is specifically a FAISS index in this embodiment, and can use the IndexIVFPQ (inverted file and product quantization) structure to balance search speed, accuracy and memory usage. After the FAISS index is constructed, it is loaded into the memory of the online query server as the predictive semantic index to wait for real-time calls in the online query phase.

[0054] In a preferred embodiment, the model training step applies a multi-task joint loss function, the calculation procedure of which includes: calculating the cosine similarity loss between the final predicted future semantic vector of the semantic evolution model and the actual future semantic vector of the training label as a first loss term; calculating the vector mutual information between the high-dimensional time interval vector and the final historical state vector as a second loss term; jointly optimizing and outputting the trained semantic evolution model by minimizing the first loss term while minimizing the second loss term.

[0055] In the present embodiment, when performing the model training step, the method extracts a training sample pair from the time series database, which is input into the semantic evolution model, which in one forward propagation calculates and outputs a high-dimensional time interval vector (output by the time interval embedding subnetwork) representing only the nonlinear representation of time passing, and a final historical state vector (output by the recursive state update layer) representing mainly the cumulative memory of semantic content over time.

[0056] At this time, the method applies a multi-task joint loss function composed of two loss terms: The first loss term is the prediction loss, the calculation procedure of which is: inputting the final historical state vector into the prediction output layer (MLP) to obtain the final predicted future semantic vector; then, calculating the cosine similarity loss between the predicted vector and the training label (actual future semantic vector). This loss term is the main optimization objective of the model, used to improve the prediction accuracy.

[0057] The second loss term is the decoupling loss, the calculation procedure of which is: obtaining the high-dimensional time interval vector and the final historical state vector, then the method calculates the vector mutual information (e.g., through a special mutual information estimator network such as MINE) between the two vectors, and the goal of this loss term is to minimize the mutual information between the two vectors, i.e., to force them to be orthogonal or statistically independent.

[0058] Finally, in the backpropagation step, all trainable parameters of the semantic evolution model are updated jointly by optimizing the prediction loss and the decoupling loss, and the calculation of the multi-task joint loss function adopts the weighted summation method, i.e., wherein Ltotal represents the total loss, Lpred represents the prediction loss, Ldec represents the decoupling loss, w represents the weighting weight, which is set to w = 1.0, w = 0.5, to eliminate the dimensional difference between the two loss terms, and The two corresponding loss terms are each normalized (e.g., divided by the respective mean in the current batch) before weighting. This joint optimization in principle forces the model to learn how to accurately predict the future while having to encode the temporal information and the content information into different, uncorrelated neuron subspaces, respectively.

[0059] By starting a periodic background computing task in the offline batch phase, the high computational overhead operation of calculating the future semantic vector for a large number of users is separated from the online query phase. This design of computing transfer avoids real-time reasoning for a large number of users during online query, ensuring the low-latency response and feasibility of the evolution prediction function in industrial deployment. At the same time, by using the sliding window method and cosine similarity loss for model training, an optimized embedding representation in the vector direction is provided for the subsequent nearest neighbor search task, which helps to improve the construction quality of the predictive semantic index and the final retrieval accuracy.

[0060] Further, the semantic evolution model comprises: a time interval embedding subnetwork that receives a current timestamp and a previous timestamp in a historical semantic sequence; calculates a scalar time interval; inputs the scalar time interval into a time feature extractor to convert the scalar into a high-dimensional time interval vector; the time feature extractor is a multilayer perceptron; a state fusion layer that receives a semantic vector, a previous historical state vector, and a high-dimensional time interval vector; fuses the three vectors into a fusion input vector through a vector splicing operation; a recursive state update layer that receives the fusion input vector; the recursive state update layer is a gated recurrent unit; the gated recurrent unit processes the fusion input vector through a gating weight to calculate a final historical state vector; a prediction output layer, the prediction output layer is a multilayer perceptron; receives the final historical state vector and outputs a predicted future semantic vector, the specific process is as shown in Fig. 3 .

[0061] Specifically, the internal computing structure of the semantic evolution model, the model is a time interval-aware recurrent architecture.

[0062] The first component of the architecture is the time interval embedding subnetwork, which is an input processing module specialized in handling non-uniform time intervals. The internal computation flow of this subnetwork includes: at runtime, when the current timestamp and the previous timestamp in the sequence of historical semantic states are received, the subnetwork performs a subtraction operation in the processor to calculate a scalar time interval. Subsequently, the scalar time interval is input into a time feature extractor, which in this embodiment is a multi-layer perceptron (MLP). The MLP performs a non-linear transformation on the scalar, mapping and outputting it as a high-dimensional time interval vector, which in principle represents the information brought by the passage of time itself.

[0063] The second component of the architecture is the state fusion layer, which is used to fuse content information (from the semantic vector) with time information (from the time interval vector). At runtime, it receives three inputs: the semantic vector (representing the content of the current event), the previous historical state vector (representing the past memory), and the high-dimensional time interval vector (representing the passage of time). The state fusion layer performs a vector concatenation operation in this embodiment, concatenating the three vectors in dimension to form a single, longer, time-aware fusion input vector. The dimensions of the fusion input vector are composed of three parts: the semantic vector dimension, the previous historical state vector dimension, and the high-dimensional time interval vector. To ensure model scale consistency, the dimensions of the high-dimensional time interval vector and the previous historical state vector are set to the same dimension as the semantic vector dimension. The number of hidden layer units of the gated recurrent unit is set to be no less than the same dimension as the semantic vector dimension, and its initial hidden state is initialized to a zero vector.

[0064] The third component of the architecture is the recurrent state update layer, which in this embodiment is a gated recurrent unit. The gated recurrent unit only receives the time-aware fusion input vector as its input. The gated recurrent unit applies its internal learned gating weights (i.e., update gate and reset gate) to process the time-aware fusion input vector, dynamically fusing new information and old memory, and calculating to output a new historical state vector.

[0065] The fourth component of the architecture is the prediction output layer, which in this embodiment is a multi-layer perceptron. When the last event of the input sequence is processed, the final historical state vector output by the recurrent state update core is input into the prediction output layer. The multi-layer perceptron performs a final non-linear transformation on it to output the final predicted future semantic vector, which is then used for loss calculation or offline reasoning.

[0066] By introducing a time interval embedding subnetwork, the irregular time intervals between user asynchronous and sparse events can be explicitly modeled within the model. This design enables the general semantic evolution model to distinguish between the different effects of long intervals (which may lead to interest forgetting) and short intervals (which may represent interest enhancement), thereby improving the accuracy of the historical semantic state sequence construction and providing higher-quality input that better fits real-time dynamics for subsequent evolution prediction.

[0067] Further, the process of generating the guest group centroid vector includes: in the online query stage, obtaining each seed user contained in the seed guest group; for each seed user, retrieving the corresponding historical semantic sequence from the time series database; inputting the historical semantic sequence of each seed user into the semantic evolution model respectively, performing an inference calculation, and generating a corresponding individual future semantic vector; after obtaining the individual future semantic vectors of all seed users in the seed guest group, performing vector averaging operation on multiple individual future semantic vectors, calculating and outputting the guest group centroid vector.

[0068] Specifically, in the online query request receiving stage, a seed guest group is received from an external application (for example, an advertisement placement platform or a content operation background), in this embodiment, the seed guest group is specifically a user ID list (for example, a list containing 1000 seed user IDs), in the historical data retrieval stage, the user ID list is traversed, for the ID of each seed user, the method initiates a concurrent retrieval to the time series database, the retrieval operation is used to obtain the latest historical semantic sequence (for example, the event sequence of the user in the last N days) of each seed user, the format of the sequence is consistent with the format of the input features used by the semantic evolution model during training.

[0069] In the batch inference calculation stage, all (for example, 1000) historical semantic sequences retrieved in the previous step are collected as a calculation batch, then the method inputs the calculation batch into the trained general semantic evolution model at one time, the semantic evolution model (loaded in the memory or video memory of the online inference server) performs an inference calculation on each historical semantic state sequence in the batch, and the output of the batch inference operation is multiple (for example, 1000) corresponding individual future semantic vectors.

[0070] In the centroid vector aggregation stage, the vector averaging operation is performed on the multiple individual future semantic vectors generated in the previous step, to overcome the representativeness problem that may be caused by simple averaging operation in high-dimensional space, the calculation of the guest group centroid vector should use the weighted average method, that is: wherein is the weight corresponding to the i-th seed user, for its individual future semantic vector. The weight The strategy should be activity-weighted: The weight should be proportional to the historical event frequency of the user, i.e. the weight of active users is higher. The calculated customer centroid vector should be subjected to an effectiveness test: the average cosine similarity of the vector with all individual vectors of the seed users should be not less than 0.85. If the value is lower than the threshold, it indicates that the interest difference in the group is too large, and the group should be considered to be divided into multiple sub-groups for separate processing. The step of dividing the group into multiple sub-groups for separate processing is specifically implemented as follows: starting clustering: when the average cosine similarity is lower than 0.85, the system starts the sub-group division process, and performs the K-Means clustering algorithm on the multiple future semantic vectors (i.e. the set) of the seed customer group; determining the K value (the number of sub-groups): to determine the optimal number of sub-groups K, the system uses the silhouette coefficient for evaluation. The system iterates within a preset K value range (for example, K = 2 to 5), and automatically selects the K value with the highest silhouette coefficient as the final number of sub-groups. After clustering is completed, the original seed customer group is divided into K sub-groups, and the system should regard each sub-group as an independent, new seed customer group, and perform the complete steps of calculating the centroid and performing the search for each sub-group, i.e. calculating the customer centroid vector of each of the K sub-groups, and performing the approximate neighbor search for the K centroid vectors, and finally outputting K sets of predicted evolved customer groups.

[0071] The generation of the customer centroid vector further includes: judging whether the user scale of the seed customer group is less than a preset minimum robustness threshold; if it is less than the threshold, starting a vector space enhancement process, and inputting multiple future semantic vectors into a pre-trained variational autoencoder; the decoder of the variational autoencoder samples from the latent distribution implied by the future semantic vector to generate an enhanced future semantic vector; merging the original future semantic vector and the enhanced future semantic vector to form an expanded vector set; and the execution of the vector average operation is specifically implemented as follows: performing the vector average operation on the expanded vector set to calculate and output the customer centroid vector.

[0072] In the online query request receiving stage, when a seed customer group (a user ID list) is obtained, the method of the present application first judges in the processor whether the user scale of the seed customer group is less than a preset minimum robustness threshold. In the present embodiment, the minimum robustness threshold is set to 100 users. If the user scale is greater than or equal to 100, the enhancement process of the present embodiment is skipped, and the standard vector average operation is directly performed.

[0073] If the user scale is less than 100 (e.g., only 50 seed users are acquired), a vector space augmentation procedure is initiated to address the biased and unstable centroid vector caused by the small sample size. In the vector space augmentation procedure, first the pre-sequenced steps are performed normally: retrieving the historical semantic state sequences of the 50 seed users from the time-series database, and applying the trained general semantic evolution model to perform inference computation to generate 50 corresponding future semantic vectors. Then, the 50 future semantic vectors (as a set) are input into a pre-trained variational autoencoder (VAE), which is pre-trained using all future semantic vectors stored in the predictive semantic index as unsupervised data in the offline phase.

[0074] The VAE includes an encoder and a decoder. The encoder is trained to compress an input future semantic vector (e.g., 768 dimensions) into a low-dimensional latent distribution (e.g., a 32-dimensional mean vector and a 32-dimensional variance vector), and the decoder is trained to sample from the latent distribution and reconstruct the original 768-dimensional vector; at the runtime of the augmentation procedure, the 50 future semantic vectors are fed into the encoder of the VAE to compute a population latent distribution that they collectively represent (e.g., by averaging the mean and variance of the 50 latent distributions).

[0075] The VAE is unsupervised trained using the future semantic vector samples stored in the predictive semantic index; the encoder structure of the VAE is specifically: input layer (768 dimensions) -> hidden layer 1 (512 units) -> hidden layer 2 (256 units) -> output layer (64 dimensions in total, i.e., 32-dimensional mean + 32-dimensional variance), and the decoder adopts a symmetric design in reverse order.

[0076] The training loss function of the VAE is: , represents the mean square error (MSE), represents the KL divergence, represents the KL divergence weight, is set to 0.001 to 0.01, and in this embodiment, it is 0.005; the VAE model is periodically retrained (e.g., once a week), and the online query sampling process samples N times from the population latent distribution, N is set to 50 to 500 times, and in this embodiment, it is 200 times, to generate sufficient augmented vectors.

[0077] Subsequently, the decoder of the variational autoencoder randomly samples (e.g., 200 times) from the population latent distribution to generate a set of augmented, synthetic future semantic vectors (a total of 200 new vectors that are similar to but not identical to the original vectors), merges the 50 original individual future semantic vectors with the 200 augmented, synthetic future semantic vectors to form an expanded vector set (a total of 250 vectors), and finally performs a vector averaging operation to calculate and output the customer cluster centroid vector; since the customer cluster centroid vector is calculated based on a larger and more robust vector set, its stability and representativeness are significantly improved.

[0078] By performing independent inference calculations for each user in the seed customer cluster and then averaging the output individual future vectors, the common prediction evolution center point of the seed customer cluster can be accurately calculated. This "prediction first, then average" process avoids information loss and noise interference caused by premature aggregation (averaging) of historical sequences, ensuring that the generated customer cluster centroid vector can more accurately reflect the common trend of group evolution, thereby providing a more accurate query target for the subsequent nearest neighbor search step.

[0079] Further, the process of identifying the predetermined number of users with the highest similarity to the customer cluster centroid vector as the predicted evolving customer cluster for output includes: inputting the customer cluster centroid vector as a query vector into a semantic index; the semantic index is an approximate nearest neighbor index; the approximate nearest neighbor index is configured to perform a candidate set retrieval operation; the candidate set retrieval operation returns a candidate user set with a size larger than the final output size of the predicted evolving customer cluster and the corresponding approximate distance of the candidate user set. For each candidate user in the candidate user set, extract the future semantic vector from the semantic index; calculate the vector distance between the future semantic vector of each candidate user and the customer cluster centroid vector in the computer processor; reorder the candidate user set according to the calculated vector distance, and select the top-ranked predetermined number of users from the reordered candidate user set as the approximate users for output.

[0080] Specifically, the process is implemented as a two-stage query and reordering architecture: In the first stage: candidate set retrieval (coarse recall), the method performs the following process: The predictive semantic index (i.e., semantic index) in this embodiment is a FAISS index, and its specific structure is IndexIVFPQ (inverted file and product quantization). In principle, this index structure compresses high-dimensional (e.g., 768-dimensional) future semantic vectors into low-dimensional codes through quantization technology to achieve extremely high retrieval speed, but at the cost of sacrificing the accuracy of distance. The guest cluster centroid vector is input into the FAISS index as a query vector. The index is configured to perform a candidate set retrieval operation, which performs approximate nearest neighbor search in the quantized space and returns a candidate user set (e.g., if 1000 users are finally needed, this step is configured to return 10000 candidate users) and its approximate distance, which is larger than the final output size of the predicted evolving guest cluster.

[0081] The quantization parameters of the approximate nearest neighbor index are configured as follows: the number of inverted files is set to the square root of the total number of users and is constrained between 100 and 100000; for example, for 1 million users, nlist is set to 1000. The number of quantization bits: the number of product quantization bytes is set to 4 to 16 bits, and in this embodiment, it is 8 bits. The size of the candidate set: the number of returned candidate users of the candidate set retrieval operation is set to 5 to 20 times the number of final output users, and in this embodiment, it is 10 times. To ensure retrieval quality, after the FAISS index is built, a query test is performed on known seed guest clusters to verify that the accuracy of Top-K retrieval should not be less than 0.8.

[0082] The configuration of the approximate nearest neighbor index should meet the following accuracy guarantee conditions: after the index is built, a comparison test with an exact index (IndexFlat) should be performed to verify that the recall rate of the coarse recall stage should not be less than 0.9 (i.e., the top K candidate users should contain at least 90% of the K nearest users). If the recall rate of the coarse recall stage is less than 0.9, the return size of the candidate set should be increased (e.g., from 10 times to 20 times) or the quantization parameter should be adjusted (e.g., the number of nbits is increased) until the recall is not less than 0.9. The system should regularly monitor the accuracy of coarse recall, and if there is a trend of decreasing recall rate, the index should be rebuilt.

[0083] In the second stage: accurate reordering (precision sorting), the method performs the following process: The method initiates a reordering process, and the technical principle of the process is that the approximate distance returned in the first stage is lossy, and the accurate calculation in this stage needs to use complete and uncompressed vectors, the method extracts the complete and pre-calculated future semantic vector of each candidate user in the candidate user set (for example, 10,000 users) from an independent vector database (for example, a key-value storage or an uncompressed FAISS IndexFlat) storing complete precision vectors, and then accurately calculates the accurate vector distance between the complete future semantic vector of each candidate user and the customer group centroid vector in a computer processor, in this embodiment, the distance measure is specifically cosine similarity, which is consistent with the loss function (cosine similarity loss) used in the training stage of the semantic evolution model, ensuring the consistency of training and reasoning, finally, the candidate user set (10,000 users) is reordered in descending order according to the calculated accurate cosine similarity, and the top-ranked predetermined number (for example, 1,000) of users in the reordered list are selected as the approximate users (i.e., the final predicted evolving customer group) and output.

[0084] By adopting the two-stage query architecture of approximate index coarse recall and accurate vector reordering, the speed of online query and the accuracy of the final result are considered, the method first uses approximate neighbor index to quickly filter out a larger candidate user set, then only extracts complete future semantic vectors for the candidate set, and performs accurate vector distance calculation, this reordering step corrects the accuracy loss that may be introduced by approximate search, and ensures that the approximate users output are the groups that are truly closest to the customer group centroid vector.

[0085] The innovation of offline pre-computation-online index query of the present application will transfer the task of calculating the future semantic vector of a large number of users in the whole station from the online query stage to the offline batch processing stage, this design of calculation transfer is a specific high-performance and low-latency system architecture implementation scheme for realizing complex evolution prediction function in the field of streaming media recommendation system, in addition, the present application does not simply apply FAISS, but specifically adopts the IndexIVFPQ structure for the specific technical scene of high-dimensional vectors and large-scale users, and gives specific configurations of key parameters such as nlist and nbits. Ensure that the present application considers speed, accuracy and memory occupation when deployed in an industrial level.

[0086] Embodiment two: In the data acquisition and event stream construction stage, the method first accesses the backend data stream of an online streaming media platform. The acquired historical unstructured text data specifically includes: (1) long-form reviews posted by users; (2) movie review articles published by users in the platform's dynamic area or column; (3) search query words related to specific analysis or in-depth interpretation entered by users in the platform's search box. The method extracts the precise timestamp of each piece of text data and constructs a timestamp-ordered, asynchronous user in-depth review event stream for each user.

[0087] In the generation stage of the instantaneous semantic vector, a single event (e.g., a 500-word movie review article) is processed in chronological order according to the user in-depth review event stream. The text is input into a pre-trained bidirectional encoder model (a BERT model), which performs semantic encoding and extracts the final hidden layer output vector (a 768-dimensional vector) corresponding to the [CLS] token as the instantaneous in-depth review semantic vector of the user at the precise timestamp.

[0088] In the recursive generation stage of the historical semantic state sequence, the instantaneous in-depth review semantic vector and its precise timestamp are input into a time interval-aware recurrent module. The module first retrieves the previous historical state vector and the previous timestamp of the user, calculates the scalar time interval between the two events, then applies a trainable time decay function (which includes a decay parameter vector learned during model training and matches the state vector dimension) to perform element-wise multiplication on the previous historical state vector based on the scalar time interval to generate a decayed historical state, and finally a recurrent update gate (a GRU) fuses the instantaneous in-depth review semantic vector with the decayed historical state, calculates and outputs a new historical state vector. The new vector and its timestamp are appended to the time series database.

[0089] In the model training and offline index construction stage, the method uses a time interval-aware recurrent architecture (such as a structure containing a time interval embedding subnetwork, a state fusion layer, and a recurrent state update core) as the semantic evolution model, uses a sliding window method to extract training sample pairs from the time series database (e.g., using a 90-day in-depth review sequence to predict the 180th day in-depth review interest vector), and uses cosine similarity loss to train the model. After training, a periodic background computing task is started, which iterates through all active users and applies the trained general semantic evolution model to pre-compute the future in-depth review semantic state vector of each user in the future (e.g., the 180th day in the future). All these pre-computed future vectors are batch imported into a FAISS (IndexIVFPQ) index to construct a predictive semantic index.

[0090] In the online query and generating fan centroid vector stage, when the platform wants to incubate high-value reviewers, a seed fan (for example, a list of user IDs of the platform's existing Top 500 film review KOLs) is obtained, each user in the seed fan is traversed, the historical deep review semantic sequence of each user is retrieved, and is respectively input into the trained semantic evolution model, inference calculation is performed to generate multiple individual future deep review semantic vectors, and then vector averaging operation is performed on the multiple individual vectors to calculate and output a fan centroid vector.

[0091] In the online matching and outputting predicted evolving fan stage, the method inputs the fan centroid vector generated in the previous step as a query vector into the predictive semantic index (FAISS index) to perform an approximate nearest neighbor search to quickly return a large-scale candidate user set (for example, Top 50,000 approximate users). Then, the method enters the accurate reordering stage: the complete and uncompressed future deep review semantic state vectors of the 50,000 candidate users are extracted, and the accurate cosine similarity between them and the fan centroid vector is calculated in the processor. The method reorders the candidate set according to the accurate cosine similarity, and selects the top predetermined number (for example, Top 1,000) of users as the predicted evolving fan (i.e., potential KOL) for output.

[0092] Although embodiments of the present application have been shown and described, it is to be understood that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A customer group evolution prediction method based on large-scale model semantic analysis, characterized in that, include: Acquire historical data from multiple users on a streaming platform. The historical data includes users' historical comments, bullet comments, and search queries. Semantic analysis is performed on historical data to generate semantic vectors for each user at multiple time points; the semantic vectors are processed in chronological order through a loop module to generate historical semantic sequences for each user, and the historical semantic sequences are stored in a time-series database. When the model training task is started, the historical semantic sequence is obtained, and the semantic evolution model is trained based on the historical semantic sequence. In the offline batch processing stage, the semantic evolution model is used to calculate the future semantic vector corresponding to each user, and the future semantic vectors of multiple users are stored in the semantic index. When a customer base expansion request is received, a seed customer base is obtained; Retrieve the corresponding historical semantic sequence of the seed customer group from the time series database; calculate the historical semantic sequence of the seed customer group using the semantic evolution model to generate the centroid vector of the customer group; The centroid vector of the customer group is used as the query vector and input into the semantic index. An approximate nearest neighbor search is performed in the semantic index to identify a predetermined number of users with the highest similarity to the centroid vector of the customer group, which are then output as the predicted evolved customer group.

2. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of generating semantic vectors for each user includes: acquiring historical data, including the user's historical comments, bullet comments, and search queries, and extracting the timestamp corresponding to each piece of historical data; for each user among multiple users, constructing the historical data and timestamps into an asynchronous user event stream ordered by timestamps in computer memory; processing individual events in the user event stream one by one according to the chronological order of the user event stream, where each individual event corresponds to a comment, a bullet comment, and / or a search query; inputting the text data of the individual event into a pre-trained bidirectional encoder model, where the bidirectional encoder model is a specific implementation of the larger model; and performing semantic encoding operations on the text data of the individual event, outputting the final hidden layer output vector of the labeled text data as the semantic vector corresponding to the user at the timestamp.

3. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of generating a historical semantic sequence for each user and storing the historical semantic sequence in a time-series database includes: for each of the multiple users, initializing a historical state vector in computer memory, obtaining the semantic vector and timestamp as an asynchronous user event stream, and inputting them one by one into a loop module in chronological order of timestamps; when the loop module receives the current semantic vector and current timestamp, retrieving the previous historical state vector and previous timestamp from the historical semantic sequence and calculating the time interval between the current timestamp and the previous timestamp; the loop module applies a trainable time decay function to decay the influence of the previous historical state vector according to the time interval, generating a decayed historical state; the loop module inputs the semantic vector and the decayed historical state together into a recursive update gate to calculate a new historical state vector; and appends and stores the new historical state vector and timestamp into the time-series database as the latest state of the historical semantic sequence.

4. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 3, characterized in that, The process of the trainable time decay function includes: the trainable time decay function receives the calculated time interval as the input time; the trainable time decay function applies an internal trainable decay parameter vector, the dimension of which matches the dimension of the previous historical state vector; each parameter value in the trainable decay parameter vector is learned and determined during the model training phase through backpropagation, and each parameter value is used to characterize the interest forgetting rate represented by different dimensions in the historical state vector; the trainable time decay function uses the trainable decay parameter vector as the forgetting rate to perform an exponential decay operation on the input time to calculate the decay weight vector; the trainable time decay function performs element-wise multiplication of the decay weight vector with the previous historical state vector to generate the decayed historical state.

5. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of storing the future semantic vectors of multiple users in a semantic index includes: during the model training phase, extracting training sample pairs from the time-series database using a sliding window method; the training sample pairs include historical semantic state sequences as input features and actual future semantic vectors as training labels; inputting the input features and training labels into the semantic evolution model for training, and obtaining the trained semantic evolution model by minimizing the vector distance loss between the future vectors predicted by the semantic evolution model and the training labels; during the offline batch processing phase, starting a periodic background computing task, traversing multiple users, retrieving the latest historical semantic sequence from the time-series database for each user, applying the semantic evolution model to perform an inference calculation, and calculating the future semantic vector corresponding to the user; the background computing task batch imports the calculated future semantic vectors corresponding to all users into a high-dimensional vector index, constructing and outputting the semantic index.

6. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 5, characterized in that, The semantic evolution model includes: The time interval embedding sub-network receives the current timestamp and the previous timestamp from the historical semantic sequence; calculates the scalar time interval; inputs the scalar time interval into the time feature extractor, and converts the scalar into a high-dimensional time interval vector; the time feature extractor is a multilayer perceptron. The state fusion layer receives a semantic vector, a previous historical state vector, and a high-dimensional time interval vector; it fuses the three vectors into a fusion input vector through vector concatenation. A recursive state update layer receives a fused input vector; the recursive state update layer is a gated loop unit; the gated loop unit processes the fused input vector through gate weights to calculate the final historical state vector; The prediction output layer, which is a multilayer perceptron, receives the final historical state vector and outputs a predicted future semantic vector.

7. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of generating the centroid vector of the customer group includes: in the online query stage, obtaining each seed user included in the seed customer group; for each seed user, retrieving the corresponding historical semantic sequence from the time series database; inputting the historical semantic sequence of each seed user into the semantic evolution model, performing an inference calculation, and generating a corresponding individual future semantic vector; after obtaining the individual future semantic vectors of all seed users in the seed customer group, performing a vector averaging operation on multiple individual future semantic vectors, calculating and outputting the centroid vector of the customer group.

8. The customer group evolution prediction method based on large-scale model semantic analysis according to claim 1, characterized in that, The process of identifying a predetermined number of users with the highest similarity to the centroid vector of the customer group and outputting them as the predicted evolved customer group includes: inputting the centroid vector of the customer group as a query vector into a semantic index; the semantic index is an approximate nearest neighbor index; the approximate nearest neighbor index is configured to perform a candidate set retrieval operation; the candidate set retrieval operation returns a candidate user set with a size larger than the final output size of the predicted evolved customer group and the approximate distances corresponding to the candidate user set; For each candidate user in the candidate user set, a future semantic vector is extracted from the semantic index; in the computer processor, the vector distance between the future semantic vector of each candidate user and the centroid vector of the customer group is calculated; based on the calculated vector distance, the candidate user set is reordered, and from the reordered candidate user set, a predetermined number of users with the highest ranking are selected as approximate users for output.

Images