High-dimensional sparse data set improvement method based on multi-dimensional query features
By constructing a multi-layer perceptron with high-dimensional sparse KD tree index and attention mechanism, the problem of feature distribution changes in high-dimensional sparse data sets is solved, and the rapid matching recall and recommendation are achieved, which improves processing efficiency and accuracy.
Patent Information
- Application Number
- CN202510570532.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-09-02
AI Technical Summary
The feature distribution of high-dimensional sparse data sets often changes, which is difficult for traditional machine learning models to adapt, and the distance calculation is complex, resulting in inefficient processing and high cost to build exclusive models.
Build a high-dimensional sparse KD tree index that integrates product review text, user behavior log and hotspot events. The most recent samples are obtained regularly through the high-dimensional sparse KD tree index, calculate the JS divergence to judge the distribution changes, use the reservoir random sampling and update the characteristics, and train it with a multi-layer perceptron of the attention mechanism to achieve rapid matching recall and recommendation.
It realizes efficient processing of fast matching recall and recommendation of high-dimensional sparse data sets, dynamically adapts to feature distribution changes, and improves processing efficiency and accuracy.
Smart Images
Figure CN120578701A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a method for improving a high-dimensional sparse data set based on multidimensional query features. Background Art
[0002] High-dimensional sparse datasets are a typical data structure generated by big data applications, appearing in scenarios such as e-commerce, social networks, cloud service optimization, online media, public transportation, and carrier services. Their frequent occurrence stems from the continuous expansion of application systems, which in turn leads to a continuous increase in the number of entities involved. This increases the dimensionality of entity relationships and a decrease in the proportion of the entity subset with which each entity can interact within the entire entity set. Consequently, data structures describing entity relationships are typically high-dimensional and sparse. The feature distribution of high-dimensional sparse datasets frequently changes, and traditional machine learning models, which fix the feature distribution during training, are unable to dynamically adapt to changes in this distribution. This results in a decline in the performance of traditional machine learning models when processing high-dimensional sparse feature sets. Furthermore, query matching and recall require fast nearest neighbor search capabilities, but distance calculations in high-dimensional data are complex, making traditional index structures difficult to support fast queries. However, building specialized models to handle these cases is costly and impractical. In summary, the frequently changing feature distribution of high-dimensional sparse datasets, the difficulty of traditional machine learning models in adapting to these changes, and the complexity of distance calculations in high-dimensional data present a series of challenges for data processing and model building. Therefore, there is a need for a method that can improve the processing efficiency of high-dimensional sparse data. Summary of the Invention
[0003] The present invention provides a method for improving high-dimensional sparse datasets based on multidimensional query features, which mainly includes:
[0004] Construct a high-dimensional sparse KD tree index that integrates three sources: product review text, user behavior logs, and hot events; regularly obtain the latest samples of the high-dimensional sparse dataset containing new reviews, new logs, and new events, and search through the high-dimensional sparse KD tree index to obtain the latest samples of the semantic feature subspace; calculate the JS divergence of the review text word vector, user operation sequence, and hot event bag-of-words vector, and determine whether the distribution has changed based on the JS divergence; if the JS divergence exceeds the preset threshold, it is determined that the high-dimensional dataset has a distribution drift in the three categories of reviews, user operation sequences, and hot events; the datasets of the drifted features are updated using the reservoir random sampling method; the three types of newly sampled features are merged into the review text word vector, user operation sequence, and hot event bag-of-words vector dataset using an accumulation method; based on the sampled and updated review, user operation, and hot event feature datasets, a multi-layer perceptron combined with an attention mechanism is trained; high-dimensional sparse KD tree index and multi-layer perceptron are used to achieve fast matching, recall, and recommendation based on multi-dimensional query features.
[0005] As a preferred solution, the construction of a high-dimensional sparse KD tree index integrating product review text, user behavior logs, and hot events includes:
[0006] The comment text is preprocessed by word segmentation and stop word removal, the user behavior log is cleaned and normalized, and the hot events are screened and denoised to obtain a high-dimensional sparse data set; the variance of the data in each dimension is calculated, and the dimension with the largest data variance is selected as the partition dimension; the data is sorted according to the value of the partition dimension, and the median is selected as the partition point; the high-dimensional sparse data set is recursively partitioned according to the partition dimension and partition point until all data is included in the KD tree; each recursive partitioning divides the data set of the current subtree into two subsets according to the partition point, which serve as the data sets of the left and right subtrees respectively, and then the left and right subtrees are recursively partitioned; the optimal KD tree construction algorithm is used to improve the tree construction efficiency, and the KD tree is optimized through pruning and reconstruction; a KD tree index structure is constructed for the constructed high-dimensional sparse KD tree, and the index file is saved.
[0007] As a preferred solution, the method of periodically obtaining the most recent samples of a high-dimensional sparse dataset containing new comments, new logs, and new events, and searching through a high-dimensional sparse KD tree index to obtain the most recent samples of the semantic feature subspace includes:
[0008] Regularly obtain the word vectors of new comments, the operation sequences of new user logs, and the bag-of-words vectors of new hot events to obtain new vectors; use the high-dimensional sparse KD tree index to search, set the search k value, and calculate the Euclidean distance between the new vector and each vector in the high-dimensional sparse KD tree index based on the high-dimensional sparse KD tree index to obtain the distance ranking; select the K samples with the smallest Euclidean distance as the nearest neighbor data of the new vector; and use the nearest neighbor data of the new vector as the closest sample in the semantic feature subspace.
[0009] As a preferred solution, the calculation of the JS divergence of the comment text word vector, the user operation sequence, and the hot event word bag vector, and judging whether the distribution has changed according to the JS divergence, includes:
[0010] Obtain the word vector representation obtained by converting the comment text to reflect the semantic information of the comment text; based on the word vectors of the comment text in different time periods, calculate the JS divergence between different time slices to determine the degree of change in the semantic distribution of the comment; when the JS divergence of the comment text word vector exceeds the preset threshold, the text semantic distribution changes; obtain the user's operation sequence in the APP or web page to represent the user's interest preference; calculate the JS divergence between the user operation sequences in different periods to determine the size of the change in the user's interest distribution; when the JS divergence of the user operation sequence exceeds the preset threshold, the user preference distribution changes; obtain the word bag vector corresponding to the current hot event to reflect the event topic distribution; for the word bag vectors constructed for different hot events, calculate the JS divergence size to determine the distribution difference between hot topics; when the hot time word bag vector exceeds the preset threshold, it means that the proportion of hot event topics has changed.
[0011] As a preferred solution, if the JS divergence exceeds the preset threshold, it is determined that the high-dimensional dataset has distribution drift in the three categories of comments, user operation sequences, and hot events, including:
[0012] When the JS divergence of the comment text word vector, user operation sequence and hot event word bag exceeds the preset threshold, it can be determined that the three have drifted in distribution at the same time; according to the drift situation, it is necessary to distinguish the sampling and update strategies for the comment text, user behavior and hot event datasets; feature processing is performed on the features that have drifted respectively; when the JS divergence of the comment text word vector, user operation sequence and hot event word bag does not exceed the preset threshold, it means that the comment semantic preference, user behavior and hot topic have not drifted, and there is no need to update this type of dataset.
[0013] As a preferred solution, the data sets of features that have drifted are updated using a reservoir random sampling method, including:
[0014] When the comment text drifts, word vectors are constructed based on the newly added comments, and the text features are updated through the reservoir random sampling method; when the user operation sequence drifts, the user operation sequence is sampled by the reservoir to obtain the most recent user operation sequence and update the user behavior features; when the hot event drifts, the latest hot event word frequency is extracted, and the event features are updated by reservoir sampling; the feature data sets of comment text, user logs and hot events are independently judged, and the drifted feature fields are effectively sampled by reservoir random sampling to obtain new data samples.
[0015] As a preferred solution, the three types of new sampled features are respectively merged into the comment text word vector, user operation sequence and hot event word bag vector dataset by using the accumulation method, including:
[0016] Set the sample size of the dataset. If the newly sampled features do not exceed the sample size, they are directly incorporated into the original comment text word vector, user operation sequence, and hot event word bag vector dataset. When the newly sampled features exceed the sample size, set the collection time window, extract samples within a period of time, and construct the samples within this period of time as feature vectors. For the feature vectors in the newly sampled features, accumulate them according to the feature values and incorporate them into the original comment text word vector, user operation sequence, and hot event word bag vector dataset.
[0017] As a preferred solution, the training of a multi-layer perceptron combined with an attention mechanism based on the sampled and updated comment, user operation, and hot event feature dataset includes:
[0018] Construct a multi-layer perceptron combined with an attention mechanism, set the input layer vector dimension and sample size and the number of hidden layer nodes of the comment word vector, user operation sequence and event word bag vector in the multi-layer perceptron; set the attention weight coefficient according to the comment semantics, user behavior and hot topics in different fields; input the comment, user operation and hot event feature data sets into the multi-layer perceptron, and iteratively train the multi-layer perceptron until the regression loss of the multi-layer perceptron meets the preset conditions or the number of iterative training exceeds the preset number, then end the training and obtain a trained multi-layer perceptron combined with an attention mechanism; input the updated feature data set into the trained multi-layer perceptron combined with an attention mechanism, and the multi-layer perceptron re-learns and outputs the prediction results of user behavior after re-learning.
[0019] As a preferred solution, the method of using high-dimensional sparse KD tree index and multi-layer perceptron to achieve fast matching, recall and recommendation based on multi-dimensional query features includes:
[0020] After receiving a query word vector containing multi-dimensional features of the user, the K samples closest to the feature space of the query word vector are searched based on the high-dimensional sparse KD tree index, and the searched samples are used as the retrieval result vector; the semantic similarity between the query vector and the retrieval result vector is calculated, and samples above the threshold are selected as the candidate set; the candidate set is learned using a multi-layer perceptron combined with an attention mechanism, and the user response prediction result is output; the user response score output by the multi-layer perceptron is sorted, and the product with the highest response is returned to the user to form a personalized recommendation result.
[0021] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:
[0022] The present invention discloses a method for improving high-dimensional sparse datasets based on multidimensional query features. The method can regularly obtain the most recent samples of a high-dimensional sparse dataset containing new comments, new logs, and new events, and construct a high-dimensional sparse KD tree based on the high-dimensional sparse dataset. The high-dimensional sparse KD tree is used to find the most recent samples in the semantic feature subspace. The method then determines the degree of change in the distribution of the three types of features. The method then updates the sampled and updated subsets of comments, behaviors, and event features, and trains a multi-layer perceptron combined with an attention mechanism. Finally, the updated high-dimensional sparse KD tree and the multi-layer perceptron combined with the attention mechanism are used to achieve rapid matching and recommendation based on multidimensional query features. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 The present invention is a flowchart of a method for improving high-dimensional sparse data sets based on multidimensional query features.
[0024] Figure 2 This is a schematic diagram of a method for improving high-dimensional sparse datasets based on multidimensional query features of the present invention. DETAILED DESCRIPTION
[0025] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0026] In this embodiment, a method for improving a high-dimensional sparse dataset based on multi-dimensional query features may specifically include:
[0027] Step S101: construct a high-dimensional sparse KD tree index that integrates three sources: product review text, user behavior logs, and hot events.
[0028] The comment text is preprocessed by word segmentation and stop word removal, the user behavior log is cleaned and normalized, and the hot events are screened and denoised to obtain a high-dimensional sparse data set; the variance of the data in each dimension is calculated, and the dimension with the largest data variance is selected as the partition dimension; the data is sorted according to the value of the partition dimension, and the median is selected as the partition point; the high-dimensional sparse data set is recursively partitioned according to the partition dimension and partition point until all data is included in the KD tree; each recursive partitioning divides the data set of the current subtree into two subsets according to the partition point, which serve as the data sets of the left and right subtrees respectively, and then the left and right subtrees are recursively partitioned; the optimal KD tree construction algorithm is used to improve the tree construction efficiency, and the KD tree is optimized through pruning and reconstruction; a KD tree index structure is constructed for the constructed high-dimensional sparse KD tree, and the index file is saved.
[0029] For example, the comment text submitted by the user "This product is very easy to use and has a high cost performance" can be constructed into a sparse vector by obtaining the sparse vector representation of each word through word vector technology.
[0030] [0.1, 0.3, 0, 0.5, 0, 0.7]. The sequence of user operations such as clicking and browsing in the APP can be extracted and constructed into a vector such as [0, 0, 0.2, 0, 0, 0, 0.5]. The current popular "Double Eleven" event can be represented as a bag-of-words vector [0.4, 0, 0.1, 0, 0, 0.3, 0, 0.9], where each element component represents the frequency of the word in the text document. There is a high-dimensional sparse dataset, where the variance of the first dimension is 10, the variance of the second dimension is 5, and the variance of the third dimension is 3. The first dimension with the largest variance is selected as the partitioning dimension. When recursively partitioning a high-dimensional sparse dataset, there are the following datasets: Dataset 1:
[0031] Dataset 2: [(1,2),(2,3),(3,4),(4,5)]: [(5,6),(6,7),(7,8),(8,9)] is recursively partitioned based on the partitioning dimension and partitioning point, where the partitioning dimension is the first dimension and the partitioning point is the median. An optimal KD tree construction algorithm is used to improve tree construction efficiency, optimizing the KD tree through pruning and reconstruction. A KD tree index structure is constructed based on the constructed KD tree, and the index file can be saved for subsequent queries.
[0032] Step S102: regularly obtain the most recent samples of the high-dimensional sparse dataset containing new comments, new logs, and new events, and search through the high-dimensional sparse KD tree index to obtain the most recent samples of the semantic feature subspace.
[0033] Regularly obtain the word vectors of new comments, the operation sequences of new user logs, and the bag-of-words vectors of new hot events to obtain new vectors; use the high-dimensional sparse KD tree index to search, set the search k value, and calculate the Euclidean distance between the new vector and each vector in the high-dimensional sparse KD tree index based on the high-dimensional sparse KD tree index to obtain the distance ranking; select the K samples with the smallest Euclidean distance as the nearest neighbor data of the new vector; and use the nearest neighbor data of the new vector as the closest sample in the semantic feature subspace.
[0034] For example, convert comment text into word vector representations, obtaining a 100-dimensional vector representing the word vector for the newly added comment. Convert the sequence of operations for new user registration into a numerical representation. Encode each operation as an integer, for example, 1 for browsing a product, 2 for adding to a shopping cart, and 3 for purchasing a product. If a user performs 10 operations in a day, the resulting sequence will be a sequence of 10 integers. Convert hot events into bag-of-words vector representations. There are 10 hot events, each consisting of multiple keywords. Count the number of occurrences of each keyword in each event to obtain a 10-dimensional vector representing the bag-of-words for the hot events. Concatenate the comment word vectors, the user operation sequence, and the bag-of-words vector for the hot event to obtain the newly added vector. Search will be performed using a high-dimensional sparse KD tree index. Setting the search k value to 3 indicates that the three most similar vectors to the newly added vector are to be found. Calculate the Euclidean distance between the newly added vector and each vector in the high-dimensional sparse KD tree index, and then sort by distance. The calculated Euclidean distances are 5 from vector 1, 2 from vector 2, 1 from vector 3, 9 from vector 4, and 14 from vector 5. The three samples with the smallest Euclidean distances are selected as the nearest neighbors of the newly added vector: vector 1, vector 2, and vector 3. Finally, the nearest neighbors of the newly added vector are used as the closest samples in the semantic feature subspace.
[0035] Step S103: Calculate the JS divergence of the comment text word vector, the user operation sequence, and the hot event word bag vector, and determine whether the distribution has changed based on the JS divergence.
[0036] Obtain the word vector representation obtained by converting the comment text to reflect the semantic information of the comment text; based on the word vectors of the comment text in different time periods, calculate the JS divergence between different time slices to determine the degree of change in the semantic distribution of the comment; when the JS divergence of the comment text word vector exceeds the preset threshold, the text semantic distribution changes; obtain the user's operation sequence in the APP or web page to represent the user's interest preference; calculate the JS divergence between the user operation sequences in different periods to determine the size of the change in the user's interest distribution; when the JS divergence of the user operation sequence exceeds the preset threshold, the user preference distribution changes; obtain the word bag vector corresponding to the current hot event to reflect the event topic distribution; for the word bag vectors constructed for different hot events, calculate the JS divergence size to determine the distribution difference between hot topics; when the hot time word bag vector exceeds the preset threshold, it means that the proportion of hot event topics has changed.
[0037] For example, the converted word vectors for a review text include: Comment 1 [2, 4, 1, 3], Comment 2 [1, 3, 2, 4], and Comment 3 [3, 5, 2, 0]. The JS divergence between the word vectors for the reviews in different time periods can be calculated to assess the degree of change in the semantic distribution of the reviews. The word vectors for the reviews in two time periods are: Comment 1 and Comment 2 in Time Period 1, and Comment 2 and Comment 3 in Time Period 2. The JS divergence between Time Period 1 and Time Period 2 is calculated to determine the change in the semantic distribution of the reviews. The calculated JS divergence, JS(F1||F2), is 0.15, with a preset threshold of 0.1, indicating a certain degree of drift in the semantic distribution of the reviews. The user's action sequence in an app or webpage represents user interests and preferences. The user's action sequence in two time periods is: Time Period 1 includes clicking on item A, adding item B to the shopping cart, and browsing item C; Time Period 2 includes clicking on item D, clicking on item E, and adding item F to favorites. The JS divergence between Time Period 1 and Time Period 2 is calculated to determine the change in the user's interest distribution. The calculated JS divergence (JS(F1||F2)) is 0.32, and the preset threshold is 0.25, indicating that the user preference distribution has shifted to some extent. The bag-of-words vectors corresponding to the current hot event reflect the event topic distribution. The bag-of-words vectors for two hot events are as follows: Hot Event 1 [2, 3, 4, 1] and Hot Event 2 [3, 2, 1, 4]. The JS divergence between Hot Event 1 and Hot Event 2 is calculated to determine the difference in the hot topic distribution. The calculated JS divergence is 0.8, and the preset threshold is 0.5, indicating that the proportion of hot event topics has shifted to some extent.
[0038] In step S104, if the JS divergence exceeds the preset threshold, it is determined that the high-dimensional dataset has distribution drift in the three categories of comments, user operation sequences, and hot events.
[0039] When the JS divergence of the comment text word vector, user operation sequence and hot event word bag exceeds the preset threshold, it can be determined that the three have drifted in distribution at the same time; according to the drift situation, it is necessary to distinguish the sampling and update strategies for the comment text, user behavior and hot event datasets; feature processing is performed on the features that have drifted respectively; when the JS divergence of the comment text word vector, user operation sequence and hot event word bag does not exceed the preset threshold, it means that the comment semantic preference, user behavior and hot topic have not drifted, and there is no need to update this type of dataset.
[0040] For example, when the JS divergence of the comment text word vectors, user action sequences, and hot event bag-of-words all exceed a preset threshold, it can be determined that the distributions of these three features have simultaneously drifted. Based on the drift, different sampling and update strategies are needed for the comment text, user behavior, and hot event datasets. For comment text, sampling of newly emerging vocabulary can be increased to reflect emerging language usage trends among commenters. For user behavior, the sampling ratios for different behavior types can be adjusted to accommodate changes in user behavior patterns. For hot events, the bag-of-words model can be updated to add keywords for emerging hot events and adjust word frequency weights to better capture new hot topics. Subsequent feature processing is also necessary for features that have drifted. Sentiment analysis can be performed on comment text to understand changes in commenters' attitudes towards specific topics; cluster analysis can be performed on user behavior to discover new user behavior patterns; and correlation analysis can be performed on hot events to identify new topic correlations. When the JS divergence of any of the three feature types does not exceed the preset threshold, it indicates that there has been no drift in comment semantic preferences, user behavior, or hot topics, and no update is required for this dataset.
[0041] In step S105 , the data sets of features that have drifted are updated using a reservoir random sampling method.
[0042] When the comment text drifts, word vectors are constructed based on the newly added comments, and the text features are updated through the reservoir random sampling method; when the user operation sequence drifts, the user operation sequence is sampled by the reservoir to obtain the most recent user operation sequence and update the user behavior features; when the hot event drifts, the latest hot event word frequency is extracted, and the event features are updated by reservoir sampling; the feature data sets of comment text, user logs and hot events are independently judged, and the drifted feature fields are effectively sampled by reservoir random sampling to obtain new data samples.
[0043] For example, when comment text drifts, there is a dataset containing 1,000 comments, each represented by a word vector. In the new comment data, there are 200 new comments. To construct word vectors and update text features, a reservoir random sampling method can be used. 10 comments are randomly selected as the reservoir, and their word vectors are averaged to obtain a new vector representing the features of the new comments. When user action sequences drift, there is a dataset containing 1,000 user action sequences, each with a set of action behavior features. In the new user action sequence data, the 200 most recent user action sequences need to have their user behavior features updated. To update the features, a reservoir sampling method can be used to randomly sample the 10 most recent user action sequences, average their features, and obtain a new vector representing the updated user behavior features. When hot events drift, there is a dataset containing 1,000 hot events, each represented by a word frequency representation. In the new hot event data, the 200 most recent hot events need to have their event features updated. To update the features, we can use the reservoir sampling method to randomly sample the 10 most recent hot events and average their bag-of-words vectors to obtain a new vector representing the updated event features. To calculate the average length of words in a comment, we can calculate the total length of words in all comments and divide it by the total number of comments to get the average length. If the total number of comments is 1000 and the total word length is 5000, the average length is 5.
[0044] In step S106, the three types of new sampled features are respectively merged into the comment text word vector, user operation sequence and hot event word bag vector datasets in an accumulation manner.
[0045] Set the sample size of the dataset. If the newly sampled features do not exceed the sample size, they are directly incorporated into the original comment text word vector, user operation sequence, and hot event word bag vector dataset. When the newly sampled features exceed the sample size, set the collection time window, extract samples within a period of time, and construct the samples within this period of time as feature vectors. For the feature vectors in the newly sampled features, accumulate them according to the feature values and incorporate them into the original comment text word vector, user operation sequence, and hot event word bag vector dataset.
[0046] For example, the original feature dataset has a sample capacity of 200 samples. The original feature dataset includes 100 feature samples, and now 50 feature samples are newly sampled. After adding the newly sampled features, the sample capacity is not exceeded, so the newly sampled features are directly incorporated into the original feature dataset. The final feature dataset contains 100 + 50 = 150 samples. The word frequency distribution of the original mobile phone product reviews is {"screen": 0.3, "performance": 0.2, "battery life": 0.4, "camera": 0.1}. For the newly collected review "This phone has a large screen and takes clear photos," a word frequency feature {"screen": 0.3, "camera": 0.2} is constructed. The word frequency distribution is then updated through accumulation as follows: {"screen": 0.3 + 0.3 = 0.6, "performance": 0.2, "battery life": 0.4, "camera": 0.1 + 0.2 = 0.3}. Meanwhile, the original statistical distribution of user browsing time on mobile phone product pages is [0.1, 0.2, 0.4, 0.3]. For the user behavior preferences extracted from the new logs ([0.2, 0.3, 0.1, 0.1]), this is updated to [0.3, 0.5, 0.5, 0.4] through vector superposition. Furthermore, the original event word frequency for topics like "5G" and "flagship" is [0.2, 0.3]. The word frequency of these two keywords in the new hotspot is [0.1, 0.2]. Vectors are directly accumulated to obtain the updated hot event feature vector [0.3, 0.5]. By sampling and updating the comment text, user logs, and hot event features separately, the most recent data distribution is obtained. And it is incorporated into the original statistical distribution in an additive manner to achieve the fusion and utilization of new and old features and ensure the continued effectiveness of feature expression.
[0047] Step S107: Based on the sampled and updated comment, user operation, and hot event feature dataset, a multi-layer perceptron combined with an attention mechanism is trained.
[0048] Construct a multi-layer perceptron combined with an attention mechanism, set the input layer vector dimension and sample size and the number of hidden layer nodes of the comment word vector, user operation sequence and event word bag vector in the multi-layer perceptron; set the attention weight coefficient according to the comment semantics, user behavior and hot topics in different fields; input the comment, user operation and hot event feature data sets into the multi-layer perceptron, and iteratively train the multi-layer perceptron until the regression loss of the multi-layer perceptron meets the preset conditions or the number of iterative training exceeds the preset number, then end the training and obtain a trained multi-layer perceptron combined with an attention mechanism; input the updated feature data set into the trained multi-layer perceptron combined with an attention mechanism, and the multi-layer perceptron re-learns and outputs the prediction results of user behavior after re-learning.
[0049] For example, let's build a multilayer perceptron (MLP) with an attention mechanism to predict a user's purchase intention for a product. The input features are review word vectors, user action sequences, and event bag-of-words vectors. In this MLP, the dimension of the review word vectors is 100, the dimension of the user action sequence is 50, and the dimension of the event bag-of-words vector is 100. The number of hidden layer nodes is 200. The attention weight coefficients should be determined based on the review semantics, user behavior, and hot topics in different domains. The influence of the review word vector on purchase intention is weighted as 0.4, the influence of the user action sequence is weighted as 0.3, and the influence of the event bag-of-words vector is weighted as 0.3. The training dataset is fed into the MLP for iterative training. The precondition is that the regression loss is less than 1, and the number of training iterations is set to 1000. After training, the performance of the MLP can be evaluated using a test dataset. Metrics such as prediction accuracy, precision, and recall are calculated on the test dataset to evaluate the model's performance. Using the above settings, a MLP with an attention mechanism is built and trained using 1000 training samples. At the 500th training iteration, the multilayer perceptron's calculation results converged, meeting the preset conditions, and training ended with a regression loss of 0.8. The training process lasted a total of 570 iterations. During the evaluation phase, using 500 test samples, the model's prediction accuracy for user behavior was 90%, exceeding the preset accuracy of 85%. This multilayer perceptron with an attention mechanism achieved the expected accuracy in predicting user purchase intentions and met the preset regression loss conditions.
[0050] Step S108, using high-dimensional sparse KD tree index and multi-layer perceptron to achieve fast matching recall and recommendation based on multi-dimensional query features.
[0051] After receiving a query word vector containing multi-dimensional features of the user, the K samples closest to the feature space of the query word vector are searched based on the high-dimensional sparse KD tree index, and the searched samples are used as the retrieval result vector; the semantic similarity between the query vector and the retrieval result vector is calculated, and samples above the threshold are selected as the candidate set; the candidate set is learned using a multi-layer perceptron combined with an attention mechanism, and the user response prediction result is output; the user response score output by the multi-layer perceptron is sorted, and the product with the highest response is returned to the user to form a personalized recommendation result.
[0052] For example, consider a query word vector with 10 dimensions, which contains the semantics of user comments, behavioral sequences, and related events. A high-dimensional sparse KD-tree index is used to search for the three closest examples in the feature space. The indexed dataset contains 10,000 examples, each with 10 dimensions. Searching through the high-dimensional sparse KD-tree index yields the indices of the three closest examples as [512, 1024, 281]. Next, the semantic similarity between the query vector and the retrieval result vector is calculated. Using cosine similarity as the similarity metric, a similarity score is obtained for each example with the query vector. For the first example, the similarity score is 85; for the second, 91; for the third, 78; and so on. If a similarity threshold is set at 80, examples with similarity scores above the threshold are selected as candidate sets. Since the first and second examples have similarity scores above the threshold, they are selected as candidate sets. Next, a multi-layer perceptron with an attention mechanism is used to perform feature fusion modeling on the candidate sets, outputting the user response prediction results. The multilayer perceptron outputs a user responsiveness score, ranging from 0 to 1. For the first candidate sample, the multilayer perceptron outputs a responsiveness score of 0.9, and for the second candidate sample, the multilayer perceptron outputs a responsiveness score of 0.75. Finally, the candidate samples are sorted according to the user responsiveness scores output by the multilayer perceptron, and products with high responsiveness are returned to the user, forming a precise and personalized recommendation result. Based on the responsiveness score sorting, the first candidate sample ranks first, and the second candidate sample ranks second. Therefore, the first candidate sample is returned to the user as the recommendation result. Finally, the product corresponding to sample 1 is returned.
[0053] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A method for improving high-dimensional sparse datasets based on multidimensional query features, characterized in that: The method comprises: Construct a high-dimensional sparse KD tree index that integrates three sources: product review text, user behavior logs, and hot events; regularly obtain the latest samples of the high-dimensional sparse dataset containing new reviews, new logs, and new events, and search through the high-dimensional sparse KD tree index to obtain the latest samples of the semantic feature subspace; calculate the JS divergence of the review text word vector, user operation sequence, and hot event bag-of-words vector, and determine whether the distribution has changed based on the JS divergence; if the JS divergence exceeds the preset threshold, it is determined that the high-dimensional dataset has a distribution drift in the three categories of reviews, user operation sequences, and hot events; the datasets of the drifted features are updated using the reservoir random sampling method; the three types of newly sampled features are merged into the review text word vector, user operation sequence, and hot event bag-of-words vector dataset using an accumulation method; based on the sampled and updated review, user operation, and hot event feature datasets, a multi-layer perceptron combined with an attention mechanism is trained; high-dimensional sparse KD tree index and multi-layer perceptron are used to achieve fast matching, recall, and recommendation based on multi-dimensional query features.
2. The method according to claim 1, wherein The construction of a high-dimensional sparse KD tree index integrating product review text, user behavior logs, and hot events includes: The review text is preprocessed by segmenting and removing stop words, the user behavior log is cleaned and normalized, and the hot events are filtered and denoised to obtain a high-dimensional sparse dataset. Calculate the variance of the data in each dimension and select the dimension with the largest data variance as the partitioning dimension; Sort the data according to the value of the partition dimension and choose the median as the partition point; Recursively partition high-dimensional sparse data sets according to partition dimensions and partition points until all data are included in the KD tree; Each recursive partitioning divides the data set of the current subtree into two subsets according to the partitioning point, which serve as the data sets of the left and right subtrees respectively, and then recursively partitions the left and right subtrees; uses the optimal KD tree construction algorithm to improve tree construction efficiency, and optimizes the KD tree through pruning and reconstruction; constructs a KD tree index structure for the constructed high-dimensional sparse KD tree, and saves the index file.
3. The method according to claim 1, wherein The method periodically obtains the most recent samples of a high-dimensional sparse dataset containing new comments, new logs, and new events, searches through a high-dimensional sparse KD tree index, and obtains the most recent samples of the semantic feature subspace, including: Regularly obtain the word vectors of new comments, the operation sequences of new user logs, and the bag-of-words vectors of new hot events to obtain new vectors; use the high-dimensional sparse KD tree index to search, set the search k value, and calculate the Euclidean distance between the new vector and each vector in the high-dimensional sparse KD tree index based on the high-dimensional sparse KD tree index to obtain the distance ranking; select the K samples with the smallest Euclidean distance as the nearest neighbor data of the new vector; and use the nearest neighbor data of the new vector as the closest sample in the semantic feature subspace.
4. The method according to claim 1, wherein The calculation of the JS divergence of the comment text word vector, the user operation sequence, and the hot event word bag vector, and judging whether the distribution has changed according to the JS divergence, includes: Obtain the word vector representation obtained by converting the comment text to reflect the semantic information of the comment text; based on the word vectors of the comment text in different time periods, calculate the JS divergence between different time slices to determine the degree of change in the semantic distribution of the comment; when the JS divergence of the comment text word vector exceeds the preset threshold, the text semantic distribution changes; obtain the user's operation sequence in the APP or web page to represent the user's interest preference; calculate the JS divergence between the user operation sequences in different periods to determine the size of the change in the user's interest distribution; when the JS divergence of the user operation sequence exceeds the preset threshold, the user preference distribution changes; obtain the word bag vector corresponding to the current hot event to reflect the event topic distribution; for the word bag vectors constructed for different hot events, calculate the JS divergence size to determine the distribution difference between hot topics; when the hot time word bag vector exceeds the preset threshold, it means that the proportion of hot event topics has changed.
5. The method according to claim 1, wherein If the JS divergence exceeds the preset threshold, it is determined that the high-dimensional dataset has distribution drift in the three categories of comments, user operation sequences, and hot events, including: When the JS divergence of the comment text word vector, user operation sequence, and hot event word bag exceeds the preset threshold, it can be determined that the three have simultaneously drifted in distribution. Based on the drift situation, it is necessary to distinguish the sampling and update strategies for the comment text, user behavior, and hot event datasets, and perform feature processing on the features that have drifted respectively. When the JS divergence of the comment text word vector, user operation sequence, and hot event word bag does not exceed the preset threshold, it means that the comment semantic preferences, user behavior, and hot topics have not drifted, and there is no need to update this type of dataset.
6. The method according to claim 1, wherein The data sets of the features that have drifted are updated using a reservoir random sampling method, including: When comment text drifts, word vectors are constructed based on newly added comments, and text features are updated through reservoir random sampling. When user operation sequences drift, reservoir sampling is performed on user operation sequences to obtain the most recent user operation sequences and update user behavior features. When hot events drift, the latest hot event word frequencies are extracted and event features are updated using reservoir sampling. Independent judgments are made on the feature data sets of comment texts, user logs, and hot events, and effective reservoir random sampling is performed on the feature fields that have drifted to obtain new data samples.
7. The method according to claim 1, wherein The three types of new sampled features are respectively merged into the comment text word vector, user operation sequence and hot event word bag vector dataset by using the accumulation method, including: Set the sample size of the dataset. If the newly sampled features do not exceed the sample size, they are directly incorporated into the original comment text word vector, user operation sequence, and hot event word bag vector dataset. When the newly sampled features exceed the sample size, set the collection time window, extract samples within a period of time, and construct the samples within this period of time as feature vectors. For the feature vectors in the newly sampled features, accumulate them according to the feature values and incorporate them into the original comment text word vector, user operation sequence, and hot event word bag vector dataset.
8. The method according to claim 1, wherein The training of a multi-layer perceptron combined with an attention mechanism based on the sampled and updated comment, user operation, and hot event feature dataset includes: Construct a multi-layer perceptron combined with an attention mechanism, set the input layer vector dimension and sample size of the comment word vector, user operation sequence, and event word bag vector in the multi-layer perceptron, as well as the number of hidden layer nodes; set the attention weight coefficient according to the comment semantics, user behavior, and hot topics in different fields; input the comment, user operation, and hot event feature data sets into the multi-layer perceptron, and iteratively train the multi-layer perceptron until the regression loss of the multi-layer perceptron meets the preset conditions or the number of iterative training exceeds the preset number, then terminate the training and obtain a trained multi-layer perceptron combined with an attention mechanism; input the updated feature data set into the trained multi-layer perceptron combined with an attention mechanism, and the multi-layer perceptron re-learns and outputs the prediction results of user behavior after re-learning.
9. The method according to claim 1, wherein The method utilizes high-dimensional sparse KD tree index and multi-layer perceptron to realize fast matching, recall and recommendation based on multi-dimensional query features, including: After receiving the query word vector containing the user's multi-dimensional features, the K samples closest to the feature space of the query word vector are searched based on the high-dimensional sparse KD tree index, and the searched samples are used as the retrieval result vector; Calculate the semantic similarity between the query vector and the retrieval result vector, and select samples above the threshold as the candidate set; use the multi-layer perceptron combined with the attention mechanism to learn the features of the candidate set and output the user response prediction results; sort the user response scores output by the multi-layer perceptron, and return the most responsive products to the user to form personalized recommendation results.