An e-commerce user portrait construction method based on reinforcement increasing learning

Abstract

The application discloses an e-commerce user portrait construction method based on reinforcement learning, which comprises the following modules: a data acquisition module, which acquires information of users to provide a data source for comprehensive analysis of user behaviors; a data processing and analysis module, which performs data cleaning and preprocessing, and analyzes user behavior patterns and preferences according to the processed data; and a label generation and grading module, which automatically generates labels for users according to the results output by the data processing and analysis module and in combination with a preset label rule library, and calculates the comprehensive score of the user portrait. The application can accurately grasp user behavior patterns and interest points by calculating browsing behavior characteristics through a specific formula and analyzing interest preferences by using a text mining technology and a deep learning model. In addition, the application constructs a reward mechanism based on feedback, evaluates a push strategy by using a reinforcement learning algorithm, continuously selects an optimal strategy, and improves the push effect and the accuracy of the user portrait.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of user portrait construction, and in particular to an e-commerce user portrait construction method based on reinforcement-incremental learning. BACKGROUND

[0002] In today's digital age, e-commerce is booming, and e-commerce platforms are facing massive users and fierce market competition. In order to better meet user needs, improve user experience, increase user stickiness, and achieve precise marketing, e-commerce enterprises need to deeply understand user characteristics and behavior habits in order to provide personalized services and recommendations for users.

[0003] Traditional e-commerce user portrait construction methods often have some limitations. For example, in data collection, only part of the surface information may be focused on, resulting in an incomplete understanding of users; in data analysis, simple statistical analysis methods are often used, making it difficult to deeply mine potential behavior patterns and interest preferences of users; the label generation process may be rough, lacking a dynamic adjustment mechanism, and unable to reflect changes in users in a timely manner; the development of push strategies is often based on static rules, lacking real-time response to user feedback and optimization of strategies.

[0004] According to the search, the application scheme of Chinese patent application No. CN201510944668.6 discloses a user portrait establishment method and a user portrait management system based on big data, which establishes a temporary user portrait based on user behavior and / or content within an effective time limit, and makes the temporary user portrait inherit descriptive label attributes matching the user behavior and / or content within the effective time limit from the user portrait. When the user behavior and / or content within the effective time limit does not match the descriptive label attributes of the user portrait, a new descriptive label attribute is created in the temporary user portrait. The portrait establishment method in the above-mentioned patent has the following deficiencies: lack of feedback construction reward mechanism, and the accuracy of user portrait construction is insufficient, which needs to be improved. SUMMARY

[0005] The present application aims to solve the problems in the prior art and provides an e-commerce user portrait construction method based on reinforcement-incremental learning.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] An e-commerce user portrait construction method based on reinforcement-incremental learning, the system comprises:

[0008] A data acquisition module, the data acquisition module acquires user information to provide a data source for comprehensive analysis of user behavior;

[0009] The data processing and analysis module performs data cleaning and preprocessing, and analyzes the user's behavior patterns and preferences according to the processed data.

[0010] The label generation and classification module automatically generates labels for the user and calculates the comprehensive score of the user portrait according to the results output by the data processing and analysis module and the preset label rule library.

[0011] The push test module identifies users with ambiguous label classification by setting multi-dimensional filtering conditions according to the comprehensive score S of the user portrait and the matching degree of the label.

[0012] The reinforcement learning module builds a set of refined reward mechanisms based on user feedback information to comprehensively evaluate the effectiveness of the push strategy, compares and evaluates the cumulative reward values of different push strategies using reinforcement learning algorithms, and selects the strategy with the highest cumulative reward value as the current optimal strategy.

[0013] Preferably, the data processing and analysis module performs deep user behavior analysis and mines browsing behavior characteristics, specifically as follows:

[0014] Calculate the average browsing frequency F of the user avg , the formula is:

[0015]

[0016] where n represents the number of browsing times in the set time period, F i represents the frequency of the i-th browsing; at the same time, the fluctuation of the browsing frequency is analyzed, and the standard deviation σ F is calculated to measure the stability of the user's browsing behavior, the formula is:

[0017]

[0018] Calculate the average browsing time T of the user avg , the formula is:

[0019]

[0020] where m represents the number of browsing times in the statistical time, T j represents the time of the j-th browsing; the distribution rule of the user's browsing time and the browsing time change curve are analyzed to discover the user's behavior patterns and preferences.

[0021] Preferably, the data processing and analysis module performs user interest preference analysis, specifically as follows:

[0022] Adopting text mining technology, using deep learning model BERT to extract semantic information in the text, identify keywords, topics and sentiment orientation that users are interested in;

[0023] By calculating the TF-IDF value of the keyword in the user's browsing content, the formula is:

[0024]

[0025] Among them, TF is the word frequency, N is the total number of documents, DF is the number of documents containing the keyword, and the keyword with strong representation of user interest is screened out;

[0026] When constructing the user interest vector, the various attributes of the commodity and the behavior weight of the user are considered comprehensively; Specifically as follows:

[0027] For the commodity category c k , the interest degree of the user is calculated as follows:

[0028]

[0029] Among them, is the number of times the user browses the commodity of this category, N total is the total number of times the user browses, is the inherent weight of the commodity of this category, is the score of the user to the commodity of this category, and β is the adjustment coefficient, used to control the influence degree of the score on the interest degree; In this way, the interest degree of the user to different commodity categories is quantified.

[0030] Preferably, when the label generation and grading module automatically generates labels for the user, the result output by the data processing and analysis module is used, when the user meets the matching conditions of the corresponding label, the corresponding label is generated, and when the user does not meet the matching conditions of the corresponding label, the matching degree data of the label is obtained according to the proportion.

[0031] Preferably, when the label generation and grading module generates labels, according to the result output by the data processing and analysis module, in combination with the preset label rule library, rich labels are automatically generated for the user; The label weight is adjusted or the combined label is formed based on the label correlation degree.

[0032] Preferably, the specific way of the label generation and grading module for label grading and dynamic adjustment of weight is:

[0033] The user labels are divided into three levels of first-level labels, second-level labels and third-level labels;

[0034] Among them, the first label reflects the basic information and macro behavior characteristics of the user, including: age group, gender, regional distribution, and the initial weight W1 is set to 0.5-0.7;

[0035] Among them, the second label reflects the interest preference and common behavior mode characteristics of the user, including: interested commodity category, browsing peak period, and the initial weight W2 is between 0.2-0.3;

[0036] Among them, the third label reflects the detailed behavior characteristics and special preference characteristics of the user, including: promotion activity matching type, brand preference, and the initial weight W3 is 0.1-0.2;

[0037] The comprehensive score S of the user portrait is calculated according to the following formula:

[0038]

[0039] Among them, n1, n2, n3 are the number of first, second and third labels respectively, T 1i , T 2j , T 3k are the scores of the corresponding labels;

[0040] Periodically re-evaluate the weight and classification of the label according to the latest behavior data and feedback information of the user.

[0041] Preferably, the push test module screens the label classification ambiguous users, specifically as follows:

[0042] Based on the comprehensive score S of the user portrait and the matching degree of the label, a multi-dimensional screening condition is set to identify the label classification ambiguous users, specifically as follows:

[0043] For the user with comprehensive score S threshold , S threshold is a set threshold, and the difference degree D between multiple label scores is D threshold , D thresholf is a set threshold, and the user is included in the label classification ambiguous user category;

[0044] Analyze the browsing history and purchase record data of these users;

[0045] The push test module, for the label classification ambiguous user, according to its historical behavior and interest preference, combines the label correlation degree information, and customizes personalized push content combination for each user;

[0046] According to the frequently appearing commodity categories or keywords in the browsing history, the commodities with high correlation are screened out and combined for push, specifically as follows:

[0047] Calculate the weight W of each type of commodity or keyword in the user's browsing history hist , the formula is:

[0048]

[0049] Where N hist is the number of times the user browses a certain type of commodity or involves a certain keyword, N total_hist is the total number of user browsing history records, then, according to the weight from high to low, select the top K commodity categories or keywords, mix the popular commodities corresponding to these categories or keywords with the overall popular commodities of the platform, and form a personalized push content combination.

[0050] Preferably, the reinforcement learning module is based on user feedback information to build a reward mechanism to comprehensively evaluate the effect of the push strategy; Specifically as follows:

[0051] For the push strategy that the user clicks on the push content and generates a purchase behavior, a higher reward score R buy is given, and the calculation of the reward score is as follows:

[0052] R buy = α × P buy + β × M profit + γ × P inventroy

[0053] Where P buy , α is the weight coefficient of the purchase amount, M profit is the commodity profit, β is the weight coefficient of the profit, P inventroy is the inventory turnover rate improvement index, and γ is the weight coefficient of the inventory turnover rate.

[0054] For the push strategy that the user clicks but does not purchase, a lower reward score R clicl is given, and the reward score is based on the contribution of the user's click behavior to the platform traffic and user activity, and the formula is:

[0055] R ckick = δ × T stay + ∈ × P page

[0056] Where T stay is the user's stay time after clicking on the push content, δ is the weight coefficient of the stay time, P page is the number of pages browsed by the user after clicking, and ∈ is the weight coefficient of the page number.

[0057] For the push strategy that the user does not click, a certain penalty score R noclickThe penalty score is determined according to the expected effect of the push strategy and the actual non-clicking condition, is based on a set basic penalty value, and is adjusted in combination with the attraction of the pushed content and the accuracy of the target user.

[0058] According to the reward mechanism, the cumulative reward value Q of each push strategy is calculated, and the formula is:

[0059] Q = ∑ (R x γ t )

[0060] Wherein, R is the reward score of each push, t is the time step, γ is a discount factor, 0 < γ < 1, which is used to consider the uncertainty of future rewards and time value decay;

[0061] Using the reinforcement learning algorithm Q-learning, the cumulative reward values of different push strategies are compared and evaluated, and the strategy with the highest cumulative reward value is selected as the current optimal strategy.

[0062] Preferably, it comprises the following steps:

[0063] S1: The data acquisition module collects the user's browsing behavior data, transaction data and other related information in real time during the operation of the e-commerce platform, and performs preliminary sorting and packaging;

[0064] S2: The collected data is transmitted to the server-side data processing and analysis module in time;

[0065] S3: After the data processing and analysis module receives the data, it immediately starts the data cleaning program, eliminates invalid data, and performs normalization processing and feature extraction on the valid data;

[0066] S4: Analyze the product and user comment text using text mining technology, extract keywords and semantic information, and calculate user browsing behavior characteristic indicators using statistical analysis methods;

[0067] S5: According to the data processing results, combine the label generation rule library to generate an initial label for the user;

[0068] S6: Calculate the correlation between the labels and construct a label correlation network;

[0069] S7: According to the label classification rules and initial weight settings, the generated labels are classified and valued, the comprehensive score of the user portrait is calculated, and a preliminary user portrait is formed;

[0070] S8: Improve the user portrait.

[0071] Preferably, the e-commerce user portrait construction method, the specific way to improve the user portrait is:

[0072] S81: The push test module determines the user group with unclear label classification according to the comprehensive score of the user portrait and the degree of label determination;

[0073] S82: For these users, tailor-made push content is generated according to the personalized push content combination generation algorithm, and appropriate push time and channel are selected for push;

[0074] S83: During the push process, the feedback information of the user is tracked in real time; the feedback information is transmitted to the reinforcement learning module in time;

[0075] S84: The reinforcement learning module calculates the cumulative reward value of each push strategy according to the received feedback information according to the reward mechanism, and evaluates and optimizes the strategy by using the reinforcement learning algorithm;

[0076] S85: Select the strategy with the highest cumulative reward value as the optimal strategy, and adjust the weight and parameters of the push strategy according to the actual situation;

[0077] S86: Update the user portrait label according to the optimal strategy and the latest feedback information of the user; adjust the weight or delete the label that is verified to be inconsistent with the actual interest of the user in the push test; add new labels and perform hierarchical assignment in time for newly discovered user interest points or behavior characteristics, and perfect the user portrait.

[0078] The beneficial effects of the present application are:

[0079] 1. The present application can more accurately grasp the user behavior mode and interest point by calculating the browsing behavior characteristics through a specific formula, using text mining technology and deep learning model to analyze interest preferences; based on feedback, a reward mechanism is constructed, and a reinforcement learning algorithm is used to evaluate the push strategy, which can continuously select the optimal strategy to improve the push effect and the accuracy of the user portrait.

[0080] 2. The present application can automatically generate labels according to the processing and analysis results and the rule base, and can also adjust the weight or form a combined label based on the correlation degree, so that the labels are more rich and accurately reflect the user characteristics.

[0081] 3. The present application can identify users with unclear label classification, customize personalized push content combination for them, improve the accuracy and effectiveness of push; based on feedback, a reward mechanism is constructed, and a reinforcement learning algorithm is used to evaluate the push strategy, which can continuously select the optimal strategy to improve the push effect and the accuracy of the user portrait. BRIEF DESCRIPTION OF DRAWINGS

[0082] Figure 1 A flowchart of an e-commerce user portrait construction method based on reinforcement learning is proposed for the present application;

[0083] Figure 2A flowchart of a user portrait construction method based on reinforcement learning is provided for the present application. DETAILED DESCRIPTION

[0084] The technical solutions of the present application will be further described in detail below in combination with specific embodiments.

[0085] Embodiment 1

[0086] A user portrait construction method based on reinforcement learning for e-commerce, based on an e-commerce system, the system comprising:

[0087] A data acquisition module, the data acquisition module acquires information of the user, providing a data source for comprehensive analysis of user behavior;

[0088] A data processing and analysis module, the data processing and analysis module performs data cleaning and preprocessing, and analyzes the behavior patterns and preferences of the user according to the processed data;

[0089] A label generation and grading module, the label generation and grading module automatically generates labels for the user according to the results output by the data processing and analysis module, and calculates the comprehensive score of the user portrait according to the preset label rule library;

[0090] A push test module, the push test module identifies users with unclear label classification by setting multi-dimensional filtering conditions according to the comprehensive score S of the user portrait and the matching degree of the label;

[0091] A reinforcement learning module, based on the feedback information of the user, a set of refined reward mechanism is constructed to comprehensively evaluate the effect of the push strategy, and the reinforcement learning algorithm is used to compare and evaluate the cumulative reward values of different push strategies, and the strategy with the highest cumulative reward value is selected as the current optimal strategy.

[0092] The data acquisition module collects data from multiple channels, specifically as follows:

[0093] For the web end of the e-commerce platform, JavaScript code is embedded to capture the user's browsing behavior in real time, and detailed information such as the URL of the page browsed by the user, the product ID, the time stamp, the scroll depth, etc. is recorded to reflect the user's operation details on the webpage;

[0094] For mobile applications, the sensors and log systems of mobile devices are used to collect data such as touch operations, screen browsing tracks, and application usage time, and transmit them to the server through the network to ensure the complete collection of mobile user data;

[0095] Integrate the background transaction data of the e-commerce platform, including the user's purchase record, order amount, payment method, delivery address, etc.

[0096] The data processing and analysis module identifies and eliminates duplicate, incorrect or incomplete data records. For example, by comparing timestamps, checking the legality of page URL formats, etc., to ensure the accuracy and reliability of the data. Normalize different dimensions of data to make all types of data within a unified range of magnitude, facilitating subsequent analysis and calculation. For example, map browsing frequency and browsing time to the [0, 1] interval to eliminate the influence of dimensional differences on the analysis results.

[0097] Perform in-depth user behavior analysis and mine browsing behavior characteristics as follows:

[0098] Calculate the average browsing frequency F of the user avg , the formula is:

[0099]

[0100] where n represents the number of visits within a specified time period, F i represents the frequency of the i-th visit. At the same time, analyze the fluctuations in browsing frequency by calculating the standard deviation σ F to measure the stability of user browsing behavior, the formula is:

[0101]

[0102] Calculate the average browsing time T of the user avg , the formula is:

[0103]

[0104] where m represents the number of visits within the statistical time, T j represents the time of the j-th visit. Further analyze the distribution of user browsing time, such as drawing a distribution histogram of browsing time in different time periods (such as early morning, morning, noon, afternoon, evening) and a browsing time change curve on each day of the week to discover user behavior patterns and preferences.

[0105] User interest preference analysis: Use text mining techniques to deeply analyze text data such as product descriptions, category information, and user reviews. Use the deep learning model BERT to extract semantic information from the text and identify keywords, topics and sentiment trends that users are interested in.

[0106] For example, by calculating the TF-IDF value of keywords in user browsing content, the formula is:

[0107]

[0108] Wherein, TF is the word frequency, N is the total number of documents, DF is the number of documents containing the keyword, and the keyword representing the user's interest is screened out.

[0109] When constructing the user interest vector, the multiple attributes of the commodity and the behavior weight of the user are comprehensively considered. For the commodity category c k , the user's interest degree The calculation formula is:

[0110]

[0111] Wherein, is the number of times the user browses the category of commodities, N total is the total number of times the user browses, is the inherent weight of the category of commodities (determined according to commodity sales, heat and other factors), is the user's score for the category of commodities, and β is a regulation coefficient for controlling the influence degree of the score on the interest degree. In this way, the user's interest degree for different categories of commodities is more accurately quantified.

[0112] Wherein, when the label generation and grading module automatically generates a label for the user, when the user meets the matching conditions of the corresponding label, the corresponding label is generated, and when the user does not meet the matching conditions of the corresponding label, the matching degree data of the label is obtained according to the proportion; for example, when the condition for obtaining A label is to purchase 10 times of sneakers, if the user has only purchased 7 times of sneakers, the corresponding label cannot be generated because the matching conditions are not met, but the matching degree of A label is 0.7 or 70%.

[0113] Wherein, when the label generation and grading module generates a label, according to the output result of the data processing and analysis module, in combination with the preset label rule library, a rich label is automatically generated for the user. The label covers multiple dimensions of the user's basic information (such as age, gender, region, etc.), behavior characteristics (such as browsing frequency level, browsing time preference, etc.), interest preference (such as interested commodity category, brand, etc.), and consumption habit (such as purchase frequency, single price, etc.). For example, when the user's browsing frequency and purchase times of a certain brand exceed a certain threshold, the label of "brand loyalty-[brand name]" is added for the user;

[0114] Adjusting the label weight or forming a combined label based on the label correlation degree, specifically: by analyzing the user data, the potential correlation between the labels is mined. For example, it is found that there is a strong correlation between the "sports enthusiasts" label and the "sports shoes", "sports clothing", "fitness equipment" and other labels. When the user is labeled as "sports enthusiasts", the weight of these associated labels is increased or the corresponding combined label is generated, such as "sports equipment preference-shoes and clothing equipment".

[0115] The specific manner in which the label generation and grading module grades labels and dynamically adjusts weights is as follows:

[0116] The user labels are divided into three levels of first-level labels, second-level labels and third-level labels.

[0117] The first-level labels reflect the basic information and macro behavior characteristics of the user, such as "age group", "gender", "geographical distribution", etc., and the initial weight W1 is set to the range of 0.5-0.7, which is fine-tuned according to the universality and stability of the user group.

[0118] The second-level labels reflect the interest preferences and common behavior pattern characteristics of the user, such as "interests - major categories of goods", "browsing peak hours", etc., and the initial weight W2 is between 0.2-0.3, which is adjusted according to the dynamic changes of user behavior and the influence of label correlation.

[0119] The third-level labels reflect the detailed behavior characteristics and special preference characteristics of the user, such as "sensitive to specific promotional activities", "preference for niche brands", etc., and the initial weight W3 is 0.1-0.2, which is flexibly adjusted according to the personalized feedback and rare behaviors of the user.

[0120] The comprehensive score S of the user portrait is calculated according to the following formula:

[0121]

[0122] Where n1, n2, n3 are the number of first-level, second-level and third-level labels respectively, T 1i , T 2j , T 3k are the scores of the corresponding labels (calculated dynamically according to the importance of the label and the performance of the user).

[0123] The weights and levels of the labels are re-evaluated regularly according to the latest behavior data and feedback information of the user. For example, if a second-level label has a high match degree with the actual interest of the user in multiple push tests, it is upgraded to a first-level label and the weight is adjusted accordingly; on the contrary, if a label is inconsistent with the user's behavior for a long time, its weight is reduced or it is merged into other related labels.

[0124] The push test module screens users with ambiguous label classification as follows:

[0125] Based on the comprehensive score S of the user portrait and the matching degree of the label, a multi-dimensional screening condition is set to identify users with ambiguous label classification. Specifically as follows:

[0126] For users with a low comprehensive score (e.g. S threshold , where Sthreshold For the set threshold), and the multiple label scores are similar (by calculating the difference D between the label scores, if D < D threshold , the user is considered to have an ambiguous label classification. For example, set S threshold = 0.3, D threshold = 0.3, when the user's comprehensive score is lower than 0.3 and the difference between any two label scores is less than 0.1, the user is determined to have an ambiguous label classification.

[0127] Further analyze the browsing history, purchase records and other data of these users to deeply mine potential features and preference clues in their behavior patterns, and provide more accurate basis for subsequent push tests.

[0128] The push test module, for users with ambiguous label classification, according to their historical behavior and interest preferences, combines label relevance information to customize personalized push content combinations for each user.

[0129] For example, for users with some browsing history but no obvious interest preferences, in addition to pushing popular goods, they also filter out goods with high relevance according to the frequently appearing goods categories or keywords in their browsing history for combination and push. Specifically as follows:

[0130] Calculate the weight W hist of each category of goods or keyword in the user's browsing history, the formula is:

[0131]

[0132] Where N hist is the number of times the user browses a certain category of goods or involves a certain keyword, and N total_hist is the total number of user browsing history records. Then, select the top K categories or keywords according to the weight from high to low, mix the popular goods corresponding to these categories or keywords with the overall popular goods on the platform to form a personalized push content combination.

[0133] Consider the diversity and novelty of the push content, avoid excessive reliance on the user's historical behavior, and appropriately introduce some new goods or new categories that may have potential relevance to the user's current interest but have not been paid attention to by the user, to stimulate the user's interest in exploration. For example, according to the label relevance model, a certain percentage (such as 20%) of goods associated with the user's existing labels but not yet browsed by the user are recommended to be included in the push content combination.

[0134] Wherein, the push test module, when formulating and executing the push strategy, specifically as follows:

[0135] According to the user's active time, device type, geographic location, and other contextual information, select the appropriate push time and push channel. For example, for users who often use mobile devices to browse e-commerce platforms at night, send personalized push content through mobile push notifications during their evening leisure time; for users who are used to browsing on computers during working hours, push through email or web pop-up windows during lunch breaks or before leaving work.

[0136] Embed tracking codes or identifiers in the push content to accurately collect user feedback information, such as click-through rate, purchase conversion rate, dwell time, and other key indicators.

[0137] The reinforcement learning module builds a reward mechanism based on user feedback information to comprehensively evaluate the effectiveness of the push strategy; specifically as follows:

[0138] For push strategies that users click on push content and generate purchase behavior, give a higher reward score R buy The calculation of the reward score is as follows:

[0139] R buy = α × P buy + β × M profit + γ × P inventroy

[0140] Where P buy , α is the weight coefficient of purchase amount, M profit is the profit amount, β is the weight coefficient of profit amount, P inventroy is the inventory turnover rate improvement index (calculated according to inventory changes), and γ is the weight coefficient of inventory turnover rate. For push strategies that users click but do not purchase, give a lower reward score R click The reward score is mainly based on the contribution of user click behavior to platform traffic and user activity, and the formula is:

[0141] R click = δ × T stay + ∈ × P page

[0142] Where T stay is the dwell time after the user clicks on the push content, δ is the weight coefficient of dwell time, P page is the number of pages viewed after the user clicks, and ∈ is the weight coefficient of page number. For push strategies that users do not click on, give a certain penalty score R noclick The penalty score is determined according to the expected effect of the push strategy and the actual non-clicking situation, based on the set basic penalty value, and adjusted according to the attractiveness of the push content and the accuracy of the target user.

[0143] According to the reward mechanism, the cumulative reward value Q of each push strategy is calculated, and the formula is:

[0144] Q = ∑ (R x γ t )

[0145] Where R is the reward score of each push, t is the time step, and γ is the discount factor (0 < γ < 1) to consider the uncertainty of future rewards and time value decay.

[0146] Using the reinforcement learning algorithm Q-learning, the cumulative reward values of different push strategies are compared and evaluated, and the strategy with the highest cumulative reward value is selected as the current optimal strategy.

[0147] According to the optimal strategy and the latest feedback information of the user, the push strategy and the user portrait label are updated in real time. For example, if a certain push strategy performs well over a period of time, not only the weight of this strategy is increased, but also the reasons for the success of this strategy are analyzed, such as specific product combinations, push time or copy style, etc. These successful factors are integrated into similar push scenarios; if a certain label is proved to be inconsistent with the actual interest of the user after multiple push tests, in addition to adjusting the weight of the label or deleting the label, the reasons for the misjudgment are also analyzed in depth, such as data collection bias, inaccurate label definition or user behavior mutation, etc. The data collection method and label definition rules are improved in a targeted manner to improve the accuracy and stability of the user portrait.

[0148] The e-commerce user portrait construction method comprises the following steps:

[0149] S1: The data collection module collects the user's browsing behavior data, transaction data and other related information in real time during the operation of the e-commerce platform, and performs preliminary sorting and packaging;

[0150] S2: The collected data is transmitted to the server-side data processing and analysis module in a timely manner;

[0151] S3: After receiving the data, the data processing and analysis module immediately starts the data cleaning program, removes invalid data, and performs normalization processing and feature extraction on valid data;

[0152] S4: Analyze the product and user comment text using text mining technology, extract keywords and semantic information; calculate the user's browsing behavior characteristic index using statistical analysis method;

[0153] S5: According to the data processing result, combined with the label generation rule library, generate initial labels for users;

[0154] S6: Calculate the correlation between labels and construct a label correlation network;

[0155] S7: According to the label classification rule and the initial weight setting, the generated label is classified and valued, the comprehensive score of the user portrait is calculated, and the preliminary user portrait is formed;

[0156] S8: Perfect the user portrait.

[0157] The specific way of perfecting the user portrait in the e-commerce user portrait construction method is:

[0158] S81: The push test module screens out the user group with unclear label classification according to the comprehensive score of the user portrait and the label determination degree;

[0159] S82: For these users, the personalized push content combination generation algorithm is used to tailor the push content, and the appropriate push time and channel are selected for push;

[0160] S83: In the push process, the feedback information of the user is tracked in real time, including the click rate, the purchase conversion rate, the stay time, the page view number and other key indicators; the feedback information is transmitted to the reinforcement learning module in time;

[0161] S84: The reinforcement learning module calculates the cumulative reward value of each push strategy according to the received feedback information according to the reward mechanism, and uses the reinforcement learning algorithm to evaluate and optimize the strategy;

[0162] S85: The strategy with the highest cumulative reward value is selected as the optimal strategy, and the weight and parameter of the push strategy are adjusted according to the actual situation; for example, if a push strategy performs well in the test, the weight of the strategy is increased, and the use frequency of the strategy is appropriately increased in the subsequent push; if a strategy does not work well, the weight of the strategy is reduced or the strategy is modified and optimized and then retested;

[0163] S86: According to the optimal strategy and the latest feedback information of the user, the user portrait label is updated; for the label verified as not matching the actual interest of the user in the push test, the weight of the label is adjusted or the label is deleted; for the newly discovered user interest point or behavior feature, new labels are added in time and classified and valued, and the user portrait is perfected.

[0164] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A method for constructing an e-commerce user portrait based on reinforcement-incremental learning, characterized in that, The system is realized based on an e-commerce system, and the system comprises: A data acquisition module, which acquires information of users to provide a data source for comprehensive analysis of user behaviors; A data processing and analysis module, which performs data cleaning and preprocessing, and analyzes user behavior patterns and preferences according to the processed data; A label generation and grading module, which automatically generates labels for users according to the results output by the data processing and analysis module and in combination with a preset label rule library, and calculates a comprehensive score of a user portrait; A push test module, which identifies users with unclear label classification by setting multi-dimensional screening conditions according to the comprehensive score S of the user portrait and the matching degree of the label; A reinforcement learning module, which builds a set of refined reward mechanisms to comprehensively evaluate the effect of the push strategy based on feedback information of the user, compares and evaluates the cumulative reward values of different push strategies by using a reinforcement learning algorithm, and selects the strategy with the highest cumulative reward value as the current optimal strategy; The push test module screens users with unclear label classification, specifically as follows: Based on the matching degree of the comprehensive score S of the user portrait and the label, the user with unclear label classification is identified by setting multi-dimensional screening conditions, specifically as follows: for the comprehensive score S of the user portrait , is a set threshold value, and the difference degree between the plurality of label scores , is a set threshold value, and the user is included in the category of users with unclear label classification; Analyze the browsing history and purchase record data of these users; The push test module customizes personalized push content combinations for each user according to their historical behaviors and interest preferences in combination with label correlation information for users with unclear label classification; According to the frequently occurring product categories or keywords in the browsing history, high-correlation products are screened out and combined for push, specifically as follows: Computing weights of various commodities or keywords in a user's browsing history , the formula is: ; wherein, is the number of times a user browses a certain type of commodity or involves a certain keyword, is the total number of user browsing history records, then the top K commodity categories or keywords are selected from high to low according to the weight, the popular commodities corresponding to these categories or keywords are mixed and pushed with the overall popular commodities of the platform, forming a personalized push content combination; The method comprises the following steps: S1: The data acquisition module collects information of users in real time during the operation of the e-commerce platform, and performs preliminary arrangement and packaging; S2: The collected data is transmitted to the data processing and analysis module of the server side in a timely manner; S3: After receiving the data, the data processing and analysis module immediately starts a data cleaning program to eliminate invalid data, and performs normalization processing and feature extraction on valid data; S4: Analyze the product and user comment texts by using text mining technology to extract keywords and semantic information; calculate the browsing behavior characteristic indexes of users by using statistical analysis methods; S5: According to the data processing result, generate initial labels for users in combination with the label generation rule library; S6: Calculate the correlation between labels to construct a label correlation network; S7: According to the label grading rules and initial weight settings, grade and assign values to the generated labels, calculate the comprehensive score of the user portrait, and form a preliminary user portrait; S8: Improve the user portrait. 2.The e-commerce user portrait construction method based on reinforcement-incremental learning according to claim 1, characterized in that, The data processing and analysis module performs deep user behavior analysis and mines browsing behavior characteristics, specifically as follows: Calculating the average browsing frequency of users The formula is: ; wherein n represents the number of browsing times within a set time period, represents the frequency of the i-th browsing; meanwhile, the fluctuation of the browsing frequency is analyzed, and the stability of the user browsing behavior is measured by calculating the standard deviation , and the formula is: The average browsing time of the user is calculated , and the formula is: ; Wherein, m represents the number of visits within the statistical time, The time of the jth visit is represented; the distribution rule of the user's visit time and the visit time change curve are analyzed to find the user's behavior mode and preference. 3.The e-commerce user portrait construction method based on reinforcement-incremental learning according to claim 1, characterized in that, The data processing and analysis module performs user interest preference analysis, specifically as follows: Text mining technology is adopted to extract semantic information in the text by using a deep learning model BERT, and to identify keywords, themes and emotional tendencies that users are interested in; By calculating the TF-IDF value of the keyword in the user's browsing content, the formula is: Wherein, TF is the word frequency, N is the total number of documents, DF is the number of documents containing the keyword, and the keyword representing the user's interest is screened out; When constructing the user interest vector, the multiple attributes of the commodity and the behavior weight of the user are comprehensively considered; specifically as follows: for the commodity category User interest degree The calculation formula is: ; wherein, is the number of times the user browses the category of goods, is the total number of times the user browses, is the inherent weight of the category of goods, is the score of the user for the category of goods, and β is a regulation coefficient for controlling the degree of influence of the score on the interest degree; in this way, the interest degree of the user for different categories of goods is quantified. 4.The e-commerce user portrait construction method based on reinforcement-incremental learning according to claim 1, characterized in that, The label generation and grading module generates corresponding labels when the user meets the matching conditions of the corresponding labels according to the results output by the data processing and analysis module, and obtains the matching degree data of the labels in proportion when the user does not meet the matching conditions of the corresponding labels. 5.The e-commerce user portrait construction method based on reinforcement-incremental learning according to claim 4, characterized in that, The label generation and grading module generates rich labels for the user according to the results output by the data processing and analysis module and the preset label rule library when generating labels; and adjusts the label weight or forms a combined label based on the label correlation degree.

6. The e-commerce user portrait construction method based on reinforcement increasing learning according to claim 4, characterized in that, The specific manner of the label generation and grading module for label grading and dynamic adjustment of the weight is: The user labels are divided into three levels of first-level labels, second-level labels and third-level labels; Among them, the first label reflects the basic information and macro behavior characteristics of the user, including: age group, gender, regional distribution, and the initial weight of the first label is set to 0.5-0.7 is set to 0.5-0.7 Among them, the secondary label reflects the user's interest preferences and common behavior pattern characteristics, including: interested commodity categories, peak browsing period, initial weight between 0.2-0.3; Among them, the three-level label reflects the user's detailed behavior characteristics and special preference characteristics, including: promotion activity matching type, brand preference, initial weight is 0.1-0.2; The comprehensive score S of the user portrait is calculated according to the following formula: ; wherein, , , are the number of primary, secondary, and tertiary labels, respectively, , , are the scores for the corresponding labels, respectively. The weight and grade of the label are re-evaluated regularly according to the latest behavior data and feedback information of the user.

7. The e-commerce user portrait construction method based on reinforcement increasing learning according to claim 6, characterized in that, The reinforcement learning module constructs a reward mechanism to comprehensively evaluate the effect of the push strategy based on the feedback information of the user. Specific as follows: for the user to click on the push content and produce the push strategy of purchase behavior, give higher reward score , the calculation of reward score, formula is: ; wherein, , is a weight coefficient of the purchase amount, is a product profit amount, is a weight coefficient of the profit amount, is an inventory turnover rate improvement index, is a weight coefficient of the inventory turnover rate; For push strategies where the user clicks but does not purchase, give a lower reward score The reward score is based on the contribution of the user's click behavior to platform traffic and user engagement, formula is: ; wherein, is the dwell time after the user clicks the push content, is the weight coefficient of the dwell time, is the number of pages browsed after the user clicks, is the weight coefficient of the number of pages; For the push strategy that the user does not click, a certain penalty score is given The penalty score is determined according to the expected effect and the actual non-clicking situation of the push strategy, is based on the set basic penalty value, and is adjusted based on the attraction of the push content and the accuracy of the target user. According to the reward mechanism, the cumulative reward value Q of each push strategy is calculated, and the formula is: ; Wherein, R is the reward score of each push, t is the time step, γ is the discount factor, 0<γ<1, which is used to consider the uncertainty of future rewards and time value decay; The reinforcement learning algorithm Q-learning is used to compare and evaluate the cumulative reward values of different push strategies, and the strategy with the highest cumulative reward value is selected as the current optimal strategy. 8.The e-commerce user portrait construction method based on reinforcement-incremental learning according to claim 7, characterized in that, The specific manner of the e-commerce user portrait construction method for perfecting the user portrait is: S81: The push test module selects the user group with unclear label classification according to the comprehensive score of the user portrait and the label determination degree; S82: For these users, personalized push content combination generation algorithm is used to tailor the push content, and appropriate push time and channel are selected for push; S83: In the push process, the feedback information of the user is tracked in real time; The feedback information is transmitted to the reinforcement learning module in time; S84: The reinforcement learning module calculates the cumulative reward value of each push strategy according to the received feedback information according to the reward mechanism, and uses the reinforcement learning algorithm to evaluate and optimize the strategy; S85: The strategy with the highest cumulative reward value is selected as the optimal strategy, and the weight and parameters of the push strategy are adjusted according to the actual situation; S86: The user portrait label is updated according to the optimal strategy and the latest feedback information of the user; the weight of the label verified as not matching the actual interest of the user in the push test is adjusted or deleted; new labels are added in time for newly discovered user interest points or behavior characteristics, and the user portrait is perfected by grading and assigning values.

Images