Data processing method, device, medium and electronic equipment
By using wandering sampling and industry matching knowledge graphs in the recommendation system, users' historical behavior data are processed, and new users and items are cold-started, homogeneous and data sparse, improving the accuracy and diversity of recommendations and improving the User experience.
Patent Information
- Application Number
- CN202111591947.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-23
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-12-23
AI Technical Summary
The existing recommendation system algorithms have problems such as inaccurate recommendations, serious homogeneity, lack of novelty and surprise when new users and items are cold-started, especially in scenarios with sparse data.
By obtaining user historical behavior data, building an undirected graph for roaming sampling, integrating product relevance and industry matching knowledge graphs, generating new training sequences to train models, and improving the diversity and relevance of recommendations.
In the data sparse scenario, the accuracy and diversity of recommendations are improved, the users' one-stop package purchase needs are met, the user experience is improved, and the interference of data sparseness and confusion in historical data is reduced.
Smart Images

Figure CN114254204B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to a data processing method, device, medium and electronic equipment. Background Art
[0002] At present, the mainstream recommendation system algorithms in the industry can be roughly summarized as follows:
[0003] 1. Content-based recommendation: Combine the information of the item itself, such as text, pictures, audio and video, to recommend similar products to users. The advantage of this algorithm is that it has strong explanatory power, and the recommended products are relatively accurate and in line with the interests of users. The disadvantage is that the recommended products are highly homogeneous, lack novelty and surprise, and cannot tap into other interests of users.
[0004] 2. Recommendation based on collaborative filtering: Utilize the user's historical behaviors, such as browsing, clicking, commenting, liking, sharing, forwarding and other data, to explore the similarities between users, users and items, and items, and then recommend items of potential interest to customers. This method can be further divided into user-based collaborative filtering recommendation, item-based collaborative filtering recommendation and latent semantic model recommendation, and the application scenarios of each method are also different. Since it is trained based on historical behavior data, the algorithm has a cold start problem for new users and items.
[0005] 3. Recommendation based on deep models: For example, neural network models such as DeepMatch and DSSM use data features such as items, users, and context (time, network, etc.) to fully fit the potential relationship between users and items to explore users' potential purchasing needs for items. This type of algorithm usually converts users and items into vectors, and obtains the closest items for recommendation through similarity calculation methods such as cosine similarity. Calculations based on dense vectors also lead to poor interpretability of deep model recommendation results. In addition, this type of algorithm usually requires a large amount of user historical data, and its performance in sparse data scenarios will be unsatisfactory.
[0006] 4. Algorithm based on hybrid recommendation: The current mainstream recommendation system framework is a multi-way recall combined with a ranking model, in which the multi-way recall source can include content-based recall methods, collaborative filtering-based recall methods, and deep model-based recall methods.
[0007] However, each of the above methods has its own shortcomings, so it is necessary to provide an improved data processing method to achieve improved product recommendations. Summary of the invention
[0008] Embodiments of the present application provide a data processing method, device, medium, and electronic device.
[0009] In a first aspect, an embodiment of the present application provides a data processing method for an electronic device, wherein the method comprises: an acquisition step of acquiring historical data within a predetermined time period, wherein the historical data comprises one or more behavior data of each of a plurality of users, wherein the one or more behavior data of each of the users constitutes a user sequence; a preprocessing step of excluding noise sequences from the plurality of user sequences to obtain a plurality of valid sequences, and dividing each of the valid sequences to obtain a plurality of valid subsequences; a construction step of constructing an undirected graph using one or more behavior data in each of the valid subsequences as nodes; a walking sampling step of performing multiple walking samplings in the undirected graph, wherein, in each walking sampling process, multiple sampling nodes are obtained based on the weights between two nodes in the undirected graph, and the multiple sampling nodes are combined into a sampling sequence; an adding step of adding auxiliary information to each of the sampling nodes in each of the sampling sequences to obtain a plurality of training sequences; and a training step of inputting the multiple training sequences into a training model to obtain training data for each of the sampling nodes.
[0010] In a possible implementation of the first aspect above, each of the behavior data includes a product code, and a predetermined level category of the corresponding product can be determined according to the product code, wherein each of the nodes has a corresponding predetermined level category.
[0011] In a possible implementation of the first aspect above, the wandering sampling step includes: a statistical step of counting the co-occurrence frequency of a pair of predetermined-level categories of two nodes belonging to the same valid subsequence to determine the category co-occurrence frequency between each node and other nodes belonging to the same valid subsequence as the weight; a threshold determination step of determining the weight threshold based on the maximum category co-occurrence frequency among multiple category co-occurrence frequencies; a sampling step of taking any node in the undirected graph as the starting node in each wandering sampling process, wandering to the next node with a weight between the starting node and the next node, and taking the starting node and the next node whose weight between the starting node and the next node is greater than or equal to the weight threshold as the sampling nodes.
[0012] In a possible implementation of the first aspect above, in the sampling step, if the weight between the starting node and the next node is less than the weight threshold, it is determined whether the predetermined level category of the starting node and the predetermined level category of the next node belong to the predetermined same industry knowledge graph, and the starting node and the next node belonging to the predetermined same industry knowledge graph are used as the sampling nodes.
[0013] In a possible implementation of the first aspect above, if the starting node and the next node do not belong to the same predetermined industry knowledge graph, return to the starting node and wander to another node having a weight between the starting node and the next node.
[0014] In a possible implementation of the first aspect above, in each walking sampling process, if the number of the sampling nodes reaches a predetermined number, or all nodes with weights between the starting node and the starting node are walked and sampled, the walking sampling is stopped.
[0015] In a possible implementation of the first aspect above, after multiple rounds of walking sampling, corresponding multiple sampling sequences are obtained.
[0016] In a possible implementation of the first aspect above, the co-occurrence frequencies of the same pair of predetermined-level categories in the same valid subsequence are not accumulated, and the co-occurrence frequencies of the same pair of predetermined-level categories in different valid subsequences are accumulated.
[0017] In a possible implementation of the first aspect above, in the threshold determination step, a frequency range of a predetermined percentage range of the maximum co-occurrence frequency is determined, and a co-occurrence frequency is taken from the frequency range as the weight threshold.
[0018] In a possible implementation of the first aspect above, user sequences whose amount of behavior data is greater than a first predetermined noise amount, or user sequences whose amount of behavior data within a predetermined time is greater than a second predetermined noise amount are excluded as the noise sequences.
[0019] In a possible implementation of the first aspect, each of the valid sequences is divided according to behavior data of a predetermined number and / or a predetermined time interval.
[0020] In a possible implementation of the first aspect above, a sorting step is further included, in which the training data of each sampling node is calculated to obtain the similarity of each sampling node, and the multiple sampling nodes are sorted according to the multiple similarities to obtain multiple recommendation nodes, so as to use the products corresponding to each recommendation node as recommended products.
[0021] In a possible implementation of the first aspect above, the auxiliary information is one or more of the category, brand, and price of the product.
[0022] In the second aspect, the embodiment of the present application provides a data processing device for electronic devices, characterized in that the device includes: an acquisition unit, which acquires historical data within a predetermined time period, wherein the historical data includes one or more behavior data of each of a plurality of users, and the one or more behavior data of each of the users constitute a user sequence; a preprocessing unit, which excludes noise sequences from the plurality of user sequences to obtain a plurality of valid sequences, and divides each of the valid sequences to obtain a plurality of valid subsequences; a construction unit, which uses one or more behavior data in each of the valid subsequences as nodes to construct an undirected graph; a walking sampling unit, which performs multiple walking samplings in the undirected graph, wherein in each walking sampling process, multiple sampling nodes are obtained based on the weights between the two nodes in the undirected graph, and the multiple sampling nodes are combined into a sampling sequence; an adding unit, which adds auxiliary information to each of the sampling nodes in each of the sampling sequences to obtain a plurality of training sequences; and a training unit, which inputs the plurality of training sequences into a training model to obtain training data for each of the sampling nodes. The above-mentioned acquisition unit, preprocessing unit, construction unit, walking sampling unit, adding unit, and training unit can be implemented by a processor having the functions of these modules or units in an electronic device.
[0023] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, causes the computer to execute the data processing method in the first aspect described above.
[0024] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; one or more memories; the one or more memories storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the data processing method in the above-mentioned first aspect.
[0025] The present invention incorporates prior knowledge (prior information) about product relevance and industry collocation knowledge graphs in the above-mentioned walking sampling process. On the one hand, it can solve the problem of inaccurate recommendations and insufficient relevance in data-sparse scenarios. On the other hand, the industry collocation knowledge graph can greatly improve the diversity of recommendations and meet users' one-stop package purchase needs. Therefore, the user experience can be greatly improved when data is sparse.
[0026] Furthermore, in the above-mentioned walking sampling process, the present invention fully explores the co-occurrence relationship of commodities in the user's historical behavior sequence and generates a new training sequence to train the model. This greatly reduces the shortcomings of sparse and chaotic data in historical data, reduces the interference caused by recommendations, and ultimately improves the relevance and diversity of recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 According to some embodiments of the present application, a flow chart of a data processing method is shown;
[0028] Figure 2 According to some embodiments of the present application, a specific flow diagram of the walking sampling step is shown;
[0029] Figure 3 According to some embodiments of the present application, a schematic diagram of multiple user sequences is shown;
[0030] Figure 4 According to some embodiments of the present application, a schematic diagram of an undirected graph is shown;
[0031] Figure 5 According to some embodiments of the present application, a schematic diagram of multiple sampling sequences is shown;
[0032] Figure 6 According to some embodiments of the present application, a schematic diagram of a data processing device is shown;
[0033] Figure 7 According to some embodiments of the present application, a schematic diagram of an electronic device is shown. DETAILED DESCRIPTION
[0034] The illustrative embodiments of the present application include, but are not limited to, data processing methods, apparatuses, media, and electronic devices.
[0035] The embodiments of the present application will be described in further detail below in conjunction with the accompanying drawings.
[0036] like Figure 1 , a flow chart of a data processing method according to an embodiment of the present application is shown, and the data processing method is used in an electronic device (not shown). The electronic device is, for example, a computer, a server, a mobile terminal, and the like.
[0037] like Figure 1 As shown, in the acquisition step S101, the electronic device acquires historical data within a predetermined time period, the historical data including one or more behavior data of a plurality of users respectively, and the one or more behavior data of each user constitutes a user sequence.
[0038] For example, the electronic device obtains the behavior data of multiple users in the past six months from a shopping website. Each user may have one or more behavior data. Each behavior data includes the product code of each product on the shopping website. It is understandable that each product has a unique product code.
[0039] Figure 3 FIG. 4 is a schematic diagram showing a user sequence according to some embodiments of the present application. Figure 3As shown, user sequence 1, user sequence 2 and user sequence 3 of user 1, user 2 and user 3 are shown. User sequence 1 includes behavior data D, A, B, etc., user sequence 2 includes behavior data E, C, B, B, A, etc., and user sequence 3 includes behavior data B, E, D, E, F, etc. It can be understood that Figure 3 The number of user sequences and the amount of behavior data in each user sequence shown in are for example only and can be any number without limitation.
[0040] It is understandable that the behavior data includes the product codes of the products clicked and / or purchased by the user on the shopping website.
[0041] Further, according to the product code, the electronic device can determine the predetermined level category of the product. For each product, there are multiple levels of categories, and each level of category represents a different level of classification of the product. For example, the product "3M mask" has a first-level category of "personal protection", a second-level category of "respiratory protection", a third-level category of "mask", and a fourth-level category of "disposable mask". For example, the product "DuPont protective clothing" has a first-level category of "personal protection", a second-level category of "body protection", a third-level category of "protective clothing", and a fourth-level category of "light protective clothing".
[0042] In this embodiment, for example, the third-level category of the corresponding commodity can be determined according to the commodity code, so each behavior data in each user sequence has a third-level category of the corresponding commodity.
[0043] In the preprocessing step S12, the electronic device excludes noise sequences from the multiple user sequences to obtain multiple valid sequences, and divides each valid sequence to obtain multiple valid subsequences.
[0044] Specifically, user sequences whose amount of behavior data is greater than a first predetermined noise amount, or user sequences whose amount of behavior data within a predetermined time is greater than a second predetermined noise amount are excluded as noise sequences.
[0045] For example, the first predetermined noise amount is 800, and the second predetermined noise amount is 300. If the amount of behavior data in user sequence 1 is greater than 800, or the amount of behavior data in user sequence 1 within 1 hour is greater than 300, then user sequence 1 is identified as a noise sequence, that is, user sequence 1 is identified as crawler behavior, and user sequence 1 is excluded.
[0046] It can be understood that, in this embodiment, user sequences 2 and 3 are valid sequences. Then, each valid sequence is divided according to the behavior data of a predetermined number and / or a predetermined time interval.
[0047] For example, if the predetermined number is 16, then every 16 behavior data in user sequence 2 are divided into a valid subsequence. In addition, for example, if the predetermined time interval is 30 minutes, then the behavior data within every 30 minutes in user sequence 2 are divided into a valid subsequence.
[0048] For example, if the length of sequence 2 is 20, the first 16 behavior sequences in sequence 2 are divided into valid subsequence 2-1, and the 4 behavior sequences after sequence 2 are divided into valid subsequence 2-2. In addition, for example, there are 21 behavior data in sequence 3. If the 1st to 10th behavior data in sequence 3 are all within the first 30 minutes, and the 11th to 21st behavior data are all within the last 30 minutes, then the 1st to 10th behavior data are divided into valid subsequence 3-1, and the 11th to 21st behavior data are divided into valid subsequence 3-2.
[0049] In addition, it is understandable that for the same user sequence, as long as the behavior data reaches a predetermined number or a predetermined time interval, it can be divided (i.e., truncated). In other words, for the same sequence, it can be divided according to both the predetermined number of behavior data and the predetermined time interval.
[0050] For example, a user purchased 80 items at one time, so the user sequence of this user includes 80 behavioral data, including masks, protective goggles, protective clothing, hand sanitizer, disposable gloves, etc. The first few items, masks, protective goggles, protective clothing, etc., are used for personal protection and are related combinations. The last pair of wrenches, pliers, and tape measures are purchased in connection with the user's business scenario. Therefore, this sequence should be split as much as possible so that related matching items are in the same sequence, and unrelated matching items are not in the same sequence. According to the above division method, the present invention can divide a large and complex user sequence into multiple valid subsequences with more relevant internal combinations.
[0051] See also Figure 3 , the dashed lines represent the division of user sequences. For example, in this embodiment, user sequence 2 is divided into valid subsequence 2-1 (including behavior data A, C, and B) and valid subsequence 2-2 (including behavior data B and A), and user sequence 3 is divided into valid subsequence 3-1 (including behavior data B, E, D, C, and F) and valid subsequence 3-2 (including behavior data D, E, and F).
[0052] In step S13, the electronic device constructs an undirected graph using one or more behavior data in each valid subsequence as nodes.
[0053] Figure 4is a schematic diagram of an undirected graph according to some embodiments of the present invention. In this embodiment, the behavior data A, B, C, D, E, and F in the valid subsequences 2-1, 2-2, 3-1, and 3-2 are used as nodes to construct Figure 4 The undirected graph shown. That is, Figure 4 The undirected graph shown contains nodes A, B, C, D, E, and F.
[0054] It can be understood that each of the nodes A, B, C, D, E, and F has a three-level category of corresponding products.
[0055] In the walking sampling step S14, Figure 4 In the undirected graph in , the electronic device performs multiple walk samplings, wherein in each walk sampling process, multiple sampling nodes are obtained based on the weights between the nodes in the undirected graph, and the multiple sampling nodes are combined into a sampling sequence.
[0056] Figure 2 yes Figure 1 For a detailed flowchart of the walking sampling step S14 in FIG. Figure 2 The walking sampling step S14 includes a statistical step S141, a threshold determination step S142, and a sampling step S143.
[0057] In the counting step S141, the electronic device counts the co-occurrence frequencies of a pair of predetermined level categories of two nodes in the same valid subsequence to determine the category co-occurrence frequencies between each node and other nodes in the same valid subsequence as weights. The co-occurrence frequencies of the same pair of predetermined level categories in the same valid subsequence are not accumulated, and the co-occurrence frequencies of the same pair of predetermined level categories in different valid subsequences are accumulated.
[0058] Specifically, for example, the third-level categories of nodes A, B, and F are all masks, the third-level categories of nodes C and D are all protective clothing, and the third-level category of node E is toys.
[0059] In the valid subsequence 2-1 (including nodes A, B, and C), the pair of third-level categories between node A and node B is "mask_mask", so the co-occurrence frequency of "mask_mask" is 1; the pair of third-level categories between node B and node C is "mask_protective clothing", so the co-occurrence frequency of "mask_protective clothing" is 1; the pair of third-level categories between node A and node C is "mask_protective clothing". Here, because "mask_protective clothing" has already appeared in the valid subsequence 2-1, it will not be accumulated, so the co-occurrence frequency of "mask_protective clothing" is still 1.
[0060] In this way, multiple pairs of third-level categories are obtained from the valid subsequence 2-1. Each pair of third-level categories can be called a co-occurrence category. Therefore, the co-occurrence categories and their co-occurrence frequencies obtained from the valid subsequence 2-1 are: {mask_mask: 1, mask_protective clothing: 1}.
[0061] Similarly, the co-occurrence categories and their co-occurrence frequencies obtained from the valid subsequence 3-1 are: {mask_mask: 1, mask_protective clothing: 1, mask_toy: 1, protective clothing_protective clothing: 1, protective clothing_toy: 1}.
[0062] The co-occurrence frequencies of the same co-occurrence categories obtained from the valid subsequences 2-1 and 3-1 are accumulated to obtain the co-occurrence categories and their accumulated co-occurrence frequencies: {mask_mask: 2, mask_protective clothing: 2, mask_toy: 1, protective clothing_protective clothing: 1, protective clothing_toy: 1}.
[0063] Here, for the convenience of description, only valid subsequences 2-1 and 3-1 are counted. It is understandable that similar statistics are performed on valid subsequences 2-2 and 3-2 and other valid subsequences, and then accumulated. For example, all co-occurrence categories and their final co-occurrence frequencies are finally obtained: {mask_mask: 340, mask_protective clothing: 550, mask_toy: 35, protective clothing_protective clothing: 270, protective clothing_toy: 20}.
[0064] Based on all the co-occurring categories and their final co-occurrence frequencies, the category co-occurrence frequency between each node and other nodes can be determined as a weight.
[0065] For example, the category co-occurrence frequency of the co-occurrence category "mask_mask" of node A and node B is 340, that is, the weight between node A and node B is 340; the category co-occurrence frequency of the co-occurrence category "mask_protective clothing" of node A and node C is 550, that is, the weight between node A and node C is 550. It can be understood that the weight between node A and node B is the weight of the direct edge (or, direct line) between node A and node B. In this way, the weight between each node and other nodes belonging to the same valid subsequence can be determined.
[0066] It is understandable that only the nodes in the same valid subsequence have weights. That is, the nodes in different valid subsequences may not have weights. In this embodiment, there is no weight between node A and node F, that is, there is no direct edge between node A and node F.
[0067] In the threshold determination step S142 , the electronic device determines a weight threshold based on the maximum category co-occurrence frequency among the plurality of category co-occurrence frequencies.
[0068] The maximum category co-occurrence frequency among the above-mentioned multiple category co-occurrence frequencies is taken out, a frequency range of a predetermined percentage range of the maximum co-occurrence frequency is determined, and a co-occurrence frequency is taken out from the frequency range as a weight threshold.
[0069] For example, the maximum category co-occurrence frequency is 550, and the predetermined percentage is 5%-15%. Thus, the frequency range of the predetermined percentage range of the maximum co-occurrence frequency is determined to be 27.5 to 82.5. After statistical analysis, 50, for example, is taken from the frequency range of 27.5 to 82.5 as the weight threshold. Here, the statistical analysis method is the same as that in the prior art, so it will not be described in detail here.
[0070] In sampling step S143, in each wandering sampling process, the electronic device takes any node in the undirected graph as the starting node, wanders to the next node with a weight between the starting node and the next node, and takes the starting node and the next node whose weight between the starting node and the next node is greater than or equal to the weight threshold as sampling nodes.
[0071] The following is a detailed description of the walking sampling process. Figure 4 In a walking sampling process, for example, starting from node A, walking to the next node with a weight between node A, for example, node B, and judging whether the weight between node A and node B is greater than or equal to the weight threshold. In this embodiment, as described above, the weight between node A and node B is 340, and the weight threshold is 50, for example. Therefore, the weight 340 between node A and node B is greater than the weight threshold 50, and node A and node B are used as sampling nodes.
[0072] Then, take node B as the starting node and walk to the next node with a weight between node B, for example, node C. The weight between node B and node C is 550, which is greater than the weight threshold of 50. Therefore, similarly, node B and node C are used as sampling nodes.
[0073] In addition, if the weight between the starting node and the next node is less than the weight threshold, it is determined whether the starting node and the next node belong to the predetermined knowledge graph of the same industry, and the starting node and the next node belonging to the predetermined knowledge graph of the same industry are used as sampling nodes.
[0074] Next, take node C as the starting node and walk to the next node with weight between node C, for example, node E. The weight between node C and node E is 20, which is less than the weight threshold of 50. That is, the weight between node C and node E is 20, which is less than the weight threshold of 50. At this time, determine whether the third-level category of node C and the third-level category of node E belong to the same predetermined industry knowledge graph.
[0075] Industry knowledge graphs are pre-defined knowledge graphs for different industries. For example, for the medical industry, masks, head covers, protective clothing, and gloves are often used together, and they belong to the medical industry knowledge graph; for the industrial industry, cutting machines, cutting discs, protective glasses, and anti-cut gloves are often used together, and they belong to the industrial industry knowledge graph. It is understandable that corresponding industry knowledge graphs can be pre-defined for different industries.
[0076] In this embodiment, the third-level category of node C is protective clothing, and the third-level category of node E is toys. For example, the third-level category of node C belongs to the medical industry knowledge graph, and the third-level category of node E belongs to the toy industry knowledge graph. Therefore, it is determined that the third-level category of node C and the third-level category of node E do not belong to the same industry knowledge graph. At this time, return to node C and walk to another node with a weight between node C, such as node F. Similar to the above, the weight between node C and node F is 550, which is greater than the weight threshold of 50, so node C and node F are used as sampling nodes.
[0077] Then, node F is used as the starting node, and for example, the node wanders to node D. Similar to the above, the weight between node F and node D is 550, which is greater than the weight threshold of 50, so node F and node D are used as sampling nodes.
[0078] It can be understood that if the third-level category of node C and the third-level category of node E belong to the same industry knowledge graph, then node C and node E are used as sampling nodes, and then node E is used as the starting node to walk to the next node, such as F, and it is determined whether the weight between node E and node F is greater than or equal to the weight threshold.
[0079] In each walking sampling process, if the number of sampled nodes reaches a predetermined number, or all nodes with weights between the starting node and the starting node are sampled, the walking sampling is stopped.
[0080] In this embodiment, for example, in the above-mentioned walking sampling process, the sampling nodes are A, B, C, F, and D, which are 5 sampling nodes in total, and the predetermined number is, for example, 16. It can be seen that the number of sampling nodes 5 has not yet reached the predetermined number 16. In addition, after sampling node D, walking is performed with node D as the starting node, but at this time, all nodes with weights between node D (i.e., B, C, E, and F) have been sampled by walking as described above, so this walking sampling is stopped.
[0081] It can be understood that in one walk sampling, the walk sampling will not be repeated for the same node.
[0082] It can be understood that for this walk sampling, sampling nodes A, B, C, F, and D are obtained, and the sampling nodes A, B, C, F, and D are combined into a sampling sequence 1, such as Figure 5 shown.
[0083] According to the above method, after the second, third, and fourth wandering sampling, we can get Figure 5 The sampling sequences shown are 2, 3, and 4. It can be understood that Figure 5 The four sampling sequences shown are only for example and not for limitation, and the number of sampling sequences is not limited.
[0084] It can be understood that the sampling nodes in each sampling sequence are not repeated, and the unified sampling nodes can appear in different sampling sequences.
[0085] In some embodiments, a walk sampling is started with each node in the undirected graph as the starting node. Each node can be used as the starting node of a walk sampling for, for example, 6 times. Then, a total of 6×N sampling sequences can be obtained, where N represents the number of nodes in the undirected graph.
[0086] It can be understood that the reconstructed multiple sampling sequences are combined according to different users, that is, the sampling sequence is independent of the user, and the reconstructed multiple sampling sequences are the result of interleaving and jointly sampling the user sequences of all users.
[0087] In the adding step S15, the electronic device adds auxiliary information to each sampling node in each sampling sequence to obtain multiple training sequences. The auxiliary information is one or more of the category, brand, price, etc. of the commodity.
[0088] In some embodiments, the auxiliary information is the category of the product. For example, for sampling nodes A, B, C, F, and D in sampling sequence 1, for example, three-level categories are added to each sampling node to obtain training sequence 1: {A+mask, B+mask, C+protective clothing, F+mask, D+protective clothing}.
[0089] Similarly, for each other sampling sequence, for example, three-level categories are added to each sampling node to obtain a corresponding training sequence.
[0090] It can be understood that the nodes described in this article include the above-mentioned commodity codes.
[0091] By adding auxiliary information, products in the same category can be made more relevant after training in subsequent models, which not only enriches the diversity of recommendations but also increases the interpretability of the model.
[0092] In the training step S16, the electronic device inputs a plurality of training sequences into the training model to obtain training data for each sampling node.
[0093] For example, multiple training sequences are input into a training model (e.g., a skip-gram model) for training, and the training data of each sampling node in each training sequence is output, that is, a 64-dimensional vector of each sampling node. The specific model training process is the same as the prior art and will not be described in detail here.
[0094] Preferably, the present invention further comprises a sorting step S16, for example, using a cosine similarity method to calculate the 64-dimensional vector of each sampling node to obtain the similarity of each sampling node.
[0095] Then, the multiple sampling nodes are sorted in descending order according to the similarity, and the top 1-10 sampling nodes, for example, are used as recommended nodes, and the products corresponding to each recommended node are used as recommended products. In this way, 10 recommended products can be obtained for recommendation to the user.
[0096] The present invention incorporates prior knowledge (prior information) about product relevance and industry collocation knowledge graphs in the above-mentioned walking sampling process. On the one hand, it can solve the problem of inaccurate recommendations and insufficient relevance in data-sparse scenarios. On the other hand, the industry collocation knowledge graph can greatly improve the diversity of recommendations and meet users' one-stop package purchase needs. Therefore, the user experience can be greatly improved when data is sparse.
[0097] Furthermore, in the above-mentioned walking sampling process, the present invention fully explores the co-occurrence relationship of commodities in the user's historical behavior sequence and generates a new training sequence to train the model. This greatly reduces the shortcomings of sparse and chaotic data in historical data, reduces the interference caused by recommendations, and ultimately improves the relevance and diversity of recommendation results.
[0098] The present invention also provides a data processing device 60, such as Figure 6As shown, the data processing device 60 includes: an acquisition unit 601, a preprocessing unit 602, a construction unit 603, a walk sampling unit 604, an adding unit 605, and a training unit 606. The acquisition unit 601 acquires historical data within a predetermined time period, and the historical data includes one or more behavior data of multiple users, and the one or more behavior data of each user constitutes a user sequence. The preprocessing unit 602 excludes noise sequences from the multiple user sequences to obtain multiple valid sequences, and divides each of the valid sequences to obtain multiple valid subsequences. The construction unit 603 uses one or more behavior data in each of the valid subsequences as nodes to construct an undirected graph. The walk sampling unit 604 performs multiple walk sampling in the undirected graph, wherein, in each walk sampling process, multiple sampling nodes are obtained based on the weights between the two nodes in the undirected graph, and the multiple sampling nodes are combined into a sampling sequence. The adding unit 605 adds auxiliary information to each sampling node in each sampling sequence to obtain multiple training sequences. The training unit 606 inputs the plurality of training sequences into a training model to obtain training data for each of the sampling nodes.
[0099] Preferably, the data processing device 60 also includes a sorting unit 607, which calculates the training data of each of the sampling nodes to obtain the similarity of each of the sampling nodes, and sorts the multiple sampling nodes according to the multiple similarities to obtain multiple recommendation nodes, thereby using the products corresponding to each of the recommendation nodes as recommended products.
[0100] It can be understood that the acquisition unit 601, the preprocessing unit 602, the construction unit 603, the wandering sampling unit 604, the adding unit 605, the training unit 606, and the recommendation can be implemented by a processor having the functions of these modules or units in the electronic device.
[0101] The present invention also provides a computer-readable storage medium, wherein the storage medium stores instructions, which, when executed on a computer, cause the computer to execute Figure 1 The method for recommending spare parts for equipment is shown in .
[0102] Reference now Figure 7 , Figure 7 An example electronic device 1400 according to an embodiment of the present invention is schematically shown. In one embodiment, the electronic device 1400 may include one or more processors 1404, a system control logic 1408 connected to at least one of the processors 1404, a system memory 1412 connected to the system control logic 1408, a non-volatile memory (NVM) 1416 connected to the system control logic 1408, and a network interface 1420 connected to the system control logic 1408.
[0103] In some embodiments, processor 1404 may include one or more single-core or multi-core processors. In some embodiments, processor 1404 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). Processor 1404 may be configured to execute various embodiments, such as Figure 1 The embodiment shown.
[0104] In some embodiments, system control logic 1408 may include any suitable interface controller to provide any suitable interface to at least one of processors 1404 and / or any suitable device or component in communication with system control logic 1408 .
[0105] In some embodiments, the system control logic 1408 may include one or more memory controllers to provide an interface to the system memory 1412. The system memory 1412 may be used to load and store data and / or instructions. In some embodiments, the memory 1412 of the electronic device 1400 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).
[0106] NVM / memory 1416 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM / memory 1416 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.
[0107] NVM / storage 1416 may include a portion of storage resources on a device on which electronic device 1400 is installed, or it may be accessible by the device but not necessarily a portion of the device. For example, NVM / storage 1416 may be accessed over a network via network interface 1420.
[0108] In particular, system memory 1412 and NVM / storage 1416 may include, respectively, a temporary copy and a permanent copy of instructions 1424. Instructions 1424 may include instructions that, when executed by at least one of processors 1404, cause electronic device 1400 to implement the following: Figure 2 In some embodiments, instructions 1424, hardware, firmware, and / or software components thereof may additionally / alternatively be placed in system control logic 1408, network interface 1420, and / or processor 1404.
[0109] The network interface 1420 may include a transceiver for providing a radio interface for the electronic device 1400, thereby communicating with any other suitable device (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 1420 may be integrated with other components of the electronic device 1400. For example, the network interface 1420 may be integrated with at least one of the processor 1404, the system memory 1412, the NVM / storage 1416, and a firmware device (not shown) having instructions. When at least one of the processors 1404 executes the instructions, the electronic device 1400 implements the following. Figure 1 The method shown.
[0110] The network interface 1420 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1420 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.
[0111] In one embodiment, at least one of the processors 1404 may be packaged together with logic for one or more controllers of the system control logic 1408 to form a system in package (SiP). In one embodiment, at least one of the processors 1404 may be integrated on the same die with logic for one or more controllers of the system control logic 1408 to form a system on a chip (SoC).
[0112] The electronic device 1400 may further include an input / output (I / O) device 1432. The I / O device 1432 may include a user interface to enable a user to interact with the electronic device 1400; the design of the peripheral component interface enables the peripheral components to interact with the electronic device 1400. In some embodiments, the electronic device 1400 further includes a sensor for determining at least one of an environmental condition and location information related to the electronic device 1400.
[0113] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.
[0114] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.
[0115] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.
[0116] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.
[0117] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, a floppy disk, an optical disk, an optical disk, a read-only memory (CD-ROM), a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier wave, infrared signal digital signal, etc.) using the Internet in an electrical, optical, acoustic or other form of propagation signal. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.
[0118] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.
[0119] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.
[0120] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0121] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.
Claims
1. A data processing method for an electronic device, characterized in that: The method comprises: An acquisition step of acquiring historical data within a predetermined time period, wherein the historical data includes one or more behavior data of each of a plurality of users, and the one or more behavior data of each of the users constitute a user sequence; A preprocessing step of excluding noise sequences from the plurality of user sequences to obtain a plurality of valid sequences, and dividing each of the valid sequences to obtain a plurality of valid subsequences; A construction step, using one or more of the behavior data in each of the valid subsequences as nodes to construct an undirected graph; A walking sampling step, performing walking sampling multiple times in the undirected graph, wherein in each walking sampling process, a plurality of sampling nodes are obtained based on the weights between two nodes in the undirected graph, and the plurality of sampling nodes are combined into a sampling sequence; An adding step of adding auxiliary information to each sampling node in each sampling sequence to obtain multiple training sequences; A training step, inputting a plurality of the training sequences into a training model to obtain training data for each of the sampling nodes, Each of the behavior data includes a product code, and a predetermined level category of the corresponding product is determined according to the product code. Each of the nodes has a corresponding predetermined level category. Wherein, the walking sampling step includes: A counting step of counting the co-occurrence frequencies of a pair of the predetermined level categories of two nodes in the same valid subsequence, so as to determine the category co-occurrence frequencies between each node and other nodes in the same valid subsequence as the weight; A threshold determination step, determining a weight threshold based on the maximum category co-occurrence frequency among the plurality of category co-occurrence frequencies; a sampling step, in each walking sampling process, taking any node in the undirected graph as a starting node, walking to the next node with a weight between the starting node and the next node, and taking the starting node and the next node whose weight between the starting node and the next node is greater than or equal to the weight threshold as the sampling nodes, Among them, in the sampling step, if the weight between the starting node and the next node is less than the weight threshold, it is determined whether the predetermined level category of the starting node and the predetermined level category of the next node belong to the predetermined same industry knowledge graph, and the starting node and the next node belonging to the predetermined same industry knowledge graph are used as the sampling nodes.
2. The method according to claim 1, characterized in that: If the starting node and the next node do not belong to the same predetermined industry knowledge graph, the starting node is returned to, and another node having a weight between the starting node and the next node is walked to.
3. The method according to claim 2, characterized in that In each walking sampling process, if the number of the sampling nodes reaches a predetermined number, or all nodes with weights between the starting node and the starting node are walked and sampled, the walking sampling is stopped.
4. The method according to claim 3, characterized in that After multiple walking samplings, multiple corresponding sampling sequences are obtained.
5. The method according to any one of claims 1 to 4, characterized in that The co-occurrence frequencies of the same pair of predetermined level categories in the same valid subsequence are not accumulated, and the co-occurrence frequencies of the same pair of predetermined level categories in different valid subsequences are accumulated.
6. The method according to claim 5, characterized in that In the threshold determination step, a frequency range of a predetermined percentage range of the maximum category co-occurrence frequency is determined, and a co-occurrence frequency is taken from the frequency range as the weight threshold.
7. The method according to claim 1, characterized in that A user sequence whose amount of behavior data is greater than a first predetermined noise amount, or a user sequence whose amount of behavior data within a predetermined time is greater than a second predetermined noise amount is excluded as the noise sequence.
8. The method according to claim 1, characterized in that Each of the valid sequences is divided into behavioral data of a predetermined number and / or predetermined time intervals.
9. The method according to claim 1, characterized in that: The method further includes a sorting step, calculating the training data of each sampling node to obtain the similarity of each sampling node, and sorting the multiple sampling nodes according to the multiple similarities to obtain multiple recommendation nodes, so as to use the products corresponding to each recommendation node as recommended products.
10. The method according to claim 1, characterized in that The auxiliary information is one or more of the category, brand, and price of the product.
11. A data processing device for use in electronic equipment, characterized in that: The device comprises: An acquisition unit, which acquires historical data within a predetermined time period, wherein the historical data includes one or more behavior data of each of a plurality of users, and the one or more behavior data of each of the users constitutes a user sequence; A preprocessing unit, excluding noise sequences from the plurality of user sequences to obtain a plurality of valid sequences, and dividing each of the valid sequences to obtain a plurality of valid subsequences; A construction unit, using one or more of the behavior data in each of the valid subsequences as nodes to construct an undirected graph; A walking sampling unit, performing multiple walking samplings in the undirected graph, wherein in each walking sampling process, multiple sampling nodes are obtained based on the weights between two nodes in the undirected graph, and the multiple sampling nodes are combined into a sampling sequence; An adding unit, adding auxiliary information to each sampling node in each sampling sequence to obtain multiple training sequences; A training unit inputs a plurality of the training sequences into a training model to obtain training data for each of the sampling nodes. Each of the behavior data includes a product code, and a predetermined level category of the corresponding product is determined according to the product code. Each of the nodes has a corresponding predetermined level category. Wherein, the walking sampling unit comprises: Counting the co-occurrence frequencies of a pair of the predetermined level categories of two nodes in the same valid subsequence, so as to determine the category co-occurrence frequencies between each node and other nodes in the same valid subsequence as the weight; Determining a weight threshold based on a maximum category co-occurrence frequency among the plurality of category co-occurrence frequencies; In each walking sampling process, any node in the undirected graph is used as the starting node, and walking is performed to the next node with a weight between the starting node and the next node, and the starting node and the next node whose weight between the starting node and the next node is greater than or equal to the weight threshold are used as the sampling nodes. Among them, if the weight between the starting node and the next node is less than the weight threshold, it is determined whether the predetermined level category of the starting node and the predetermined level category of the next node belong to the predetermined same industry knowledge graph, and the starting node and the next node belonging to the predetermined same industry knowledge graph are used as the sampling nodes.
12. A computer-readable storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, cause the computer to execute the data processing method according to any one of claims 1 to 10.
13. An electronic device, characterized in that: include: one or more processors; one or more memories; The one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the data processing method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Cooperative matrix decomposition method based on knowledge representation learning
CN108804565A
Recommendation method and device, terminal equipment and computer storage medium
CN111523010A
