User big data acquisition method and system applied to electronic commerce

Abstract

The invention relates to the technical field of data processing, in particular to a user big data collection method and system applied to e-commerce, and the method comprises the steps: obtaining the attention of a user to each type of commodities in each period according to the difference condition of commodity keyword clustering results of each period; according to the difference condition of the shopping demand change rate of each type of commodities in different periods, the shopping demand trend of the user for each type of commodities is obtained; according to the shopping demand trend and demand change consistency of each type of commodities, obtaining an overall attention trend of the user to each type of commodities; according to the difference between the overall attention trend of each type of commodities and the overall attention trend of other types of commodities, the collection rate of each type of commodities by the e-commerce platform is obtained; and carrying out acquisition optimization on the user behavior data based on the acquisition rate of the e-commerce platform for each type of commodities. According to the invention, the response capability of the e-commerce platform to the latest demand of the user is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a method and system for collecting user big data applied to e-commerce. Background Art

[0002] E-commerce refers to a new business model that uses the Internet, mobile communication, and related information technologies to realize commercial transactions, capital circulation, information exchange, and service delivery in a virtual environment. Against the backdrop of the booming development of e-commerce, user big data has become the core resource driving precision marketing and personalized recommendations. In recent years, a big data collection method and system based on a combination of client-side data logging and background rule management have been proposed. This system captures the real-time browsing, clicking, and purchasing behaviors of users and embeds dynamic data by pre-configuring collection rules, thereby providing more accurate, comprehensive, and secure data support for e-commerce platforms.

[0003] In the prior art, in an e-commerce platform, the purpose of collecting user big data is to provide accurate and personalized product recommendations. However, when a user browses and clicks on a certain product on a large scale and finally completes a purchase, the relevant data for this type of product may become irrelevant, and more relevant data on other associated products that the user is interested in needs to be collected. However, the e-commerce platform will continue to recommend the same type of products that the user has purchased until the proportion of the user's active browsing of other types of products exceeds the purchased products, and then a large amount of relevant data on other products will be recorded and uploaded. This often results in low-quality and information-redundant product data, thereby weakening the e-commerce platform's ability to respond to the latest needs of users. Summary of the Invention

[0004] The present invention provides a method and system for collecting user big data applied to e-commerce to solve the existing problem that the e-commerce platform will continuously recommend the same type of products that the user has purchased until the proportion of the user's active browsing of other types of products exceeds the purchased products, resulting in low-quality and information-redundant product data, thereby weakening the e-commerce platform's ability to respond to the latest needs of users.

[0005] A method and system for collecting user big data applied to e-commerce of the present invention adopt the following technical solutions: The present invention proposes a method for collecting user big data applied to e-commerce, and the method includes the following steps: Obtain all product keywords in each period in the user's historical behavior data; Cluster the product keywords in the user's historical behavior data to obtain the clustering results of product keywords in each period; according to the difference situation of the clustering results of product keywords in each period, obtain the attention degree of the user to each type of product in each period; By comparing the changes in the attention of each category of goods in adjacent periods, the change rate of the shopping demand of each category of goods in each period is obtained; according to the differences in the change rates of the shopping demands of each category of goods in different periods, the shopping demand trend of each category of goods is obtained; according to the overall change situation of the attention of each category of goods in different periods, the consistency of the demand changes of each category of goods is obtained; according to the shopping demand trend and the consistency of the demand changes of each category of goods, the overall attention trend of each category of goods is obtained; according to the differences in the overall attention trends of each category of goods and other categories of goods, the shopping willingness evaluation value of each category of goods is obtained; according to the shopping willingness evaluation value, the collection rate of each category of goods by the e-commerce platform is obtained; Based on the collection rate of each category of goods by the e-commerce platform, the user behavior data is optimized and collected.

[0006] Preferably, the specific method for obtaining the clustering result of the commodity keywords in each period by clustering the commodity keywords in the user's historical behavior data is as follows: For any period, the set composed of all the commodity keywords of all users in the any period and all the commodity keywords of all previous periods before the any period is denoted as the historical commodity keyword set of the any period; the co-occurrence probability between any two commodity keywords in the historical commodity keyword set of the any period is obtained by using natural language processing technology; According to the co-occurrence probability between any two commodity keywords, the DBSCAN clustering algorithm is used to cluster the historical commodity keyword set of the any period to obtain the clustering result of the commodity keywords in the any period.

[0007] Preferably, the specific method for obtaining the attention of each category of goods in each period according to the difference situation of the clustering results of the commodity keywords in each period is as follows: In the clustering result of the commodity keywords in the th period, the average value of the number of all users in all clustering clusters is denoted as the user average value in the th period; the difference between the number of all users in the clustering cluster corresponding to the th category of goods and the user average value in the th period is denoted as the first difference; the normalized value of the ratio between the first difference and the user average value in the th period is used as the user attention contrast ratio between the th category of goods and other categories of goods, and is denoted as the first ratio; The average value of the co-occurrence probabilities between all pairs of commodity keywords in the clustering cluster corresponding to the th category of goods is denoted as the The clustering density of the category of goods; the mean of the co-occurrence probabilities between all pairs of product keywords in all clustering clusters is denoted as the mean clustering density of goods in the th cycle; the ratio of the clustering density of the th category of goods to the mean clustering density of goods in the th cycle is used as the user focus contrast ratio between the th category of goods and other categories of goods, and is denoted as the second ratio; The product of the first ratio and the second ratio is used as the attention of users to the th category of goods in the

[0008] th cycle. Preferably, the specific method for obtaining the change rate of the shopping demand of users for each category of goods in each cycle by comparing the changes in the attention of each category of goods in adjacent cycles is as follows: The difference between the attention of users to the th category of goods in the th cycle and the attention of users to the th category of goods in the th cycle is denoted as the adjacent attention difference in the th cycle; the normalized value of the ratio of the adjacent attention difference in the th cycle to the attention of users to the th category of goods in the th cycle is used as the change rate of the shopping demand of users for the th category of goods in the

[0009] th cycle. Preferably, the specific method for obtaining the shopping demand trend of users for each category of goods according to the difference in the change rate of the shopping demand of each category of goods in different cycles is as follows: The normalized value of the difference between the change rate of the shopping demand of users for the th category of goods in the th cycle and the change rate of the shopping demand of users for the th category of goods in the th cycle is denoted as the shopping demand difference value of users for the th category of goods in the th cycle; the mean of the shopping demand difference values of users for the th category of goods in all cycles is used as the shopping demand trend of users for the

[0010] Preferably, the specific method for obtaining the consistency of the demand change of users for each category of goods according to the overall change of the attention of each category of goods in different cycles is as follows: Using the least squares method to fit curves and straight lines to the attention of users to the category of goods in all cycles, obtaining the fitted straight line of the attention of users to the category of goods and the fitted curve of the attention of users to the category of goods; normalizing the slope value of the fitted straight line of the attention of users to the category of goods, and denoting it as the overall attention trend of users to the category of goods; Denoting the slope of the data point corresponding to the category of goods on the fitted curve of the attention of users to the th cycle as the attention slope of users to the category of goods in the th cycle; taking the absolute value of the difference between the attention slope of users to the category of goods in the th cycle and the overall attention trend of users to the category of goods, and denoting it as the attention change factor of users to the category of goods in the th cycle; taking the inverse normalization value of the mean of the attention change factors of users to the category of goods in all cycles as the demand change consistency of users to the category of goods.

[0011] Preferably, the specific method for obtaining the overall attention trend of users to each category of goods according to the shopping demand trend and demand change consistency of each category of goods is: Taking the product of the demand change consistency of users to the category of goods and the shopping demand trend of users to the category of goods as the overall attention trend of users to the category of goods.

[0012] Preferably, the specific method for obtaining the shopping intention evaluation value of users to each category of goods according to the difference in the overall attention trend between each category of goods and other categories of goods is: Normalizing the value of the difference between the overall attention trend of users to the category of goods and the overall attention trend of users to the category of goods, and denoting it as the overall attention difference between the category of goods and the category of goods; taking the mean of the overall attention differences between the category of goods and all other categories of goods as the shopping intention evaluation value of users to the category of goods.

[0013] Preferably, the specific method for obtaining the collection rate of each type of commodity on the e-commerce platform according to the shopping intention evaluation value is as follows: The normalization value of the difference between the shopping intention evaluation value of the user for the th type of commodity and the attention degree of the user for the th type of commodity in the last cycle is denoted as the correction factor; the product of the correction factor and the shopping intention evaluation value of the user for the th type of commodity is used as the collection rate of the e-commerce platform for the th type of commodity.

[0014] The present invention also provides a user big data collection system applied to e-commerce, including a memory and a processor. The processor executes the computer program stored in the memory to implement the steps of the above-mentioned user big data collection method applied to e-commerce.

[0015] The beneficial effects of the technical solution of the present invention are as follows: according to the difference situation of the commodity keyword clustering results in each cycle, the attention degree of the user for each type of commodity in each cycle is obtained; according to the difference situation of the shopping demand change rate of each type of commodity in different cycles, the shopping demand trend of the user for each type of commodity is obtained; according to the shopping demand trend and demand change consistency of each type of commodity, the overall attention trend of the user for each type of commodity is obtained; according to the difference of the overall attention trend of each type of commodity and other types of commodities, the collection rate of the e-commerce platform for each type of commodity is obtained; the user behavior data is collected and optimized based on the collection rate of the e-commerce platform for each type of commodity. By optimizing the collection rate of the e-commerce platform for each type of commodity in this way, the key points of the user's shopping interest can be accurately identified, so as to ensure the quick response of the recommendation system in the e-commerce platform to the user's latest needs; by monitoring the real-time changes of the user's shopping needs, the collection rate of each type of commodity is automatically adjusted to ensure that the data collection always focuses on the user's latest needs; furthermore, the response ability of the e-commerce platform to the user's latest needs is strengthened. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 is the step flow chart of a user big data collection method applied to e-commerce of the present invention; Figure 2 is the characteristic relationship flow chart of a user big data collection method applied to e-commerce of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manner, structure, features and effects of a user big data collection method and system applied to e-commerce proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0020] The following specifically describes the specific solution of a user big data collection method and system applied to e-commerce provided by the present invention with reference to the accompanying drawings.

[0021] Please refer to Figure 1 , which shows a step flowchart of a user big data collection method applied to e-commerce provided by an embodiment of the present invention. The method includes the following steps: Step S001: Obtain all product keywords in each cycle of the user's historical behavior data.

[0022] In a specific implementation manner of the embodiment of the present invention, taking one day as a cycle within the past week as an example for illustration; The specific method of using client-side embedding and server logs to collect the behavior data of all users on the e-commerce platform in real time within the past week is as follows: By embedding JavaScript code in the e-commerce platform, record the content searched by users, the content of the pages browsed and the content clicked; at the same time, analyze the access logs and transaction logs generated by the server through the server logs, obtain the user access content, order content and interaction content, and use the API interface provided by the e-commerce platform to obtain the product information content and user comments; the content collected within the past week mentioned above is collectively referred to as the user's historical behavior data; Perform data cleaning on the collected user historical behavior data, remove invalid and duplicate data, and fill in missing values for preprocessing to obtain all text data in the user historical behavior data; perform word segmentation, stop word removal and punctuation removal preprocessing on all text data to obtain the preprocessed text data; use the TextRank algorithm to extract keywords from the preprocessed text data to obtain all product keywords in each cycle of the user historical behavior data.

[0023] Among them, data cleaning, removing invalid and duplicate data, and preprocessing for filling missing values; preprocessing all text data by tokenization, removing stop words and punctuation marks; and the TextRank algorithm are all existing technologies, and will not be elaborated here in this embodiment.

[0024] So far, all product keywords in each cycle of the user's historical behavior data are obtained through the above method.

[0025] Step S002: Cluster the product keywords in the user's historical behavior data to obtain the product keyword clustering results for each cycle; according to the difference situation of the product keyword clustering results for each cycle, obtain the user's attention to each category of products in each cycle.

[0026] It should be noted that when the user's shopping needs change, the e-commerce platform will still recommend similar products of the same category for a period of time based on the user's historical behavior data. At this time, the user's shopping intention decreases, and most of the browsing data and the like are still products of the same category, which will lead to the e-commerce platform frequently pushing products of the same category, possibly causing an information cocoon, reducing the user's shopping experience and resulting in user loss. To avoid the interference of the collection of low-quality redundant data on the user's shopping experience, it is necessary to analyze the user's historical behavior data to obtain the user's attention to each category of products in each cycle.

[0027] Preferably, in some implementation manners of the embodiments of the present invention, since the change in the quantity of a certain category of product keywords in the user's historical behavior data in different cycles can reflect the change in the user's shopping intention, the change in the user's shopping intention is obtained by clustering the product keywords in the user's historical behavior data; the specific method for clustering the product keywords in the user's historical behavior data to obtain the product keyword clustering results for each cycle is as follows: For any one cycle, the set composed of all product keywords of all users in the any one cycle and all product keywords of all cycles before the any one cycle is denoted as the historical product keyword set of the any one cycle; use natural language processing technology to obtain the co-occurrence probability between any two product keywords in the historical product keyword set of the any one cycle; According to the co-occurrence probability between any two product keywords, use the DBSCAN clustering algorithm to cluster the historical product keyword set of the any one cycle to obtain the product keyword clustering results of the any one cycle; It should be noted that the clustering results of commodity keywords in the cycle include several clustering clusters; each clustering cluster corresponds to a category of commodities, and the category of commodities corresponding to the largest clustering cluster may be the current shopping demand of the user; if the user has already purchased or does not need a certain category of commodities, the shopping demand shown in the overall clustering may be relatively low, and it can be found that the number of commodity keywords shows a downward trend in the change of the clustering results of each cycle; therefore, cluster the commodity keywords of the first cycle to obtain the clustering results of the commodity keywords of the first cycle, and then add the commodity keywords of the second cycle to the commodity keywords of the first cycle for joint clustering to obtain the clustering results of the commodity keywords of the second cycle, and obtain the clustering results of the commodity keywords of each cycle in turn.

[0028] Among them, natural language processing technology and DBSCAN clustering algorithm are existing technologies, and will not be elaborated here in this embodiment; the co-occurrence probability is a statistical index that measures the co-occurrence frequency of two words in a specific context through natural language processing (NLP) technology.

[0029] Preferably, in some implementation manners of the embodiments of the present invention, when the user wants to purchase a certain category of commodities, there will be a large number of users of this category of commodities in the user historical behavior data, that is, the more the number of users containing the keywords of this category of commodities in the clustering results, the more all users pay attention to this category of commodities. At the same time, the more focused and concentrated the keywords of this category of commodities are, and the smaller the clustering range is, it indicates that the user's shopping goal is more accurate; therefore, according to the difference situation of the clustering results of the commodity keywords of each cycle, the method for obtaining the attention degree of the user to each category of commodities in each cycle is as follows: In the clustering results of the commodity keywords of the th cycle, denote the mean value of the number of all users in all clustering clusters as the user mean value of the th cycle; denote the difference between the number of all users in the clustering cluster corresponding to the th category of commodities and the user mean value of the th cycle as the first difference; take the normalized value of the ratio between the first difference and the user mean value of the th cycle as the user attention contrast ratio between the th category of commodities and other categories of commodities, and denote it as the first ratio; Denote the mean value of the co-occurrence probability between all pairs of commodity keywords in the clustering cluster corresponding to the th category of commodities as the clustering density of the th category of commodities; denote the mean value of the co-occurrence probability between all pairs of commodity keywords in all clustering clusters as the mean value of the commodity clustering density of the th cycle; take the ratio of the clustering density of the th category of commodities to the mean value of the commodity clustering density of the The user focus contrast between the class of goods and other classes of goods, and is denoted as the second ratio; Take the product between the first ratio and the second ratio as the user's attention to the class of goods in the th cycle; ; In the formula, represents the user's attention to the class of goods in the th cycle; represents the number of all users in the clustering cluster corresponding to the class of goods in the commodity keyword clustering result in the th cycle; represents the average value of the number of all users in all clustering clusters in the commodity keyword clustering result in the th cycle; represents the clustering density of the class of goods in the commodity keyword clustering result in the th cycle; represents the average value of the clustering density of commodities in the th cycle; represents the linear normalization function.

[0030] It should be noted that represents the user attention contrast between the class of goods and other classes of goods. The larger this formula is, the more the user pays attention to the class of goods; represents the ratio between the clustering density of the class of goods and the average value of the clustering density of all classes of goods. The larger this formula is, the more clear and focused the user's attention to the class of goods is.

[0031] Thus, through the above method, the user's attention to each class of goods in each cycle is obtained.

[0032] Step S003: Obtain the shopping demand change rate of each category of goods in each period according to the change in the attention of each category of goods in adjacent periods; obtain the shopping demand trend of each category of goods according to the difference in the shopping demand change rate of each category of goods in different periods; obtain the demand change consistency of each category of goods according to the overall change in the attention of each category of goods in different periods; obtain the overall attention trend of each category of goods according to the shopping demand trend and demand change consistency of each category of goods; obtain the shopping willingness evaluation value of each category of goods according to the difference in the overall attention trend of each category of goods and other categories of goods; obtain the collection rate of each category of goods by the e-commerce platform according to the shopping willingness evaluation value.

[0033] It should be noted that when the user has completed the purchase of a certain category of goods or does not need this category of goods, or when the user's shopping demand changes, the platform's recommendation mechanism based on historical data will force the user to browse multiple pieces of behavioral data related to this category of goods, and too much similar content will greatly reduce the user experience. At the same time, it will cause a long delay in the time for the e-commerce platform to recommend the user's real shopping needs or potentially interesting goods, which may lead to an increase in the user churn rate. Therefore, it is necessary to analyze the change in the attention of each category of goods in different periods to obtain the shopping demand trend of each category of goods; and then discover the user's real shopping needs to obtain the shopping willingness evaluation value of each category of goods.

[0034] Preferably, in some implementation manners of the embodiments of the present invention, if the attention of a certain category of goods continuously rises in each period, it indicates that the user's demand for this category of goods is increasing. If the attention of a certain category of goods suddenly shows a downward trend in each period, it means that the user may no longer need this category of goods for some reason. Therefore, the specific method for obtaining the shopping demand change rate of each category of goods in each period according to the comparison of the change in the attention of each category of goods in adjacent periods is as follows: Let the attention of the user to the th category of goods in the th period be compared with the attention of the user to the th category of goods in the th period, and the difference is denoted as the adjacent attention difference in the th period; take the normalized value of the ratio of the adjacent attention difference in the th period to the attention of the user to the th category of goods in the th period as the shopping demand change rate of the user to the th category of goods in the th period; The specific formula is: ; Wherein, represents the change rate of the shopping demand of the user for the category of goods in the th cycle; represents the attention degree of the user to the category of goods in the th cycle; represents the attention degree of the user to the category of goods in the th cycle; represents a linear normalization function.

[0035] Preferably, in some implementation manners of the embodiments of the present invention, since when the user has purchased a certain category of goods, the behavioral data (such as clicks, browsing duration, etc.) of the user for this category of goods after shopping will become less and less, showing a large change and turning point compared with before shopping. When the shopping demand of the user changes, the behavioral data of the new shopping target will gradually increase, and the change rate of shopping demand will become larger and larger. Therefore, according to the difference in the change rate of shopping demand of the user for each category of goods in different cycles, the specific method for obtaining the shopping demand trend of the user for each category of goods is as follows: Normalize the difference between the change rate of the shopping demand of the user for the category of goods in the th cycle and the change rate of the shopping demand of the user for the category of goods in the th cycle, and denote it as the shopping demand difference value of the user for the category of goods in the th cycle; Take the average value of the shopping demand difference values of the user for the category of goods in all cycles as the shopping demand trend of the user for the category of goods; The specific formula is: ; Wherein, represents the shopping demand trend of the user for the category of goods; represents the number of all cycles; represents the change rate of the shopping demand of the user for the category of goods in the th cycle; represents the change rate of the shopping demand of the user for the category of goods in the th cycle; represents a linear normalization function.

[0036] Among them, the shopping demand difference value of the user for the category of goods in the last cycle and the shopping demand difference value of the user for the The shopping demand difference values of the

[0037] Preferably, in some implementation manners of the embodiments of the present invention, after the user purchases a commodity, a behavior of comparing the information of similar commodities may occur, resulting in a weak upward trend in the shopping demand trend of the user, but the change rate of demand before shopping will turn. Therefore, when the change rate of the user's attention to this type of commodity in each cycle is quite different from the overall trend of the user's attention, it indicates that the user's attitude towards this type of commodity has changed, and the smaller the consistency of the user's demand change for this type of commodity; then, according to the overall change situation of the user's attention to each type of commodity in different cycles, the specific method for obtaining the consistency of the user's demand change for each type of commodity is as follows: Using the least squares method to fit the curve and the straight line to the attention of the user to the type of commodity in all cycles, obtaining the attention fitting straight line of the user to the type of commodity and the attention fitting curve of the user to the type of commodity; normalizing the slope value of the attention fitting straight line of the user to the type of commodity, and denoting it as the overall attention trend of the user to the type of commodity; among them, the least squares method is the prior art, and no more details are described here in this embodiment; Denoting the slope of the data point corresponding to the type of commodity on the attention fitting curve of the user to the th cycle as the attention slope of the user to the type of commodity in the th cycle; denoting the absolute value of the difference between the attention slope of the user to the type of commodity in the th cycle and the overall attention trend of the user to the type of commodity as the attention change factor of the user to the type of commodity in the th cycle; taking the inverse normalization value of the mean value of the attention change factors of the user to the type of commodity in all cycles as the demand change consistency of the user to the type of commodity; The specific formula is: ; In the formula, represents the demand change consistency of the user to the type of commodity; represents the number of all cycles; represents the user's attention to the type of commodity in the The attention slope in a cycle; Indicates the slope of the fitting line of the user's attention to the category of goods; Indicates taking the absolute value; Indicates the linear normalization function; Indicates the exponential function with the natural constant as the base. In the embodiment, the model is used to present the inverse proportional relationship and normalization processing, is the input of the model. The implementer can select the inverse proportional function and normalization function according to the actual situation.

[0038] Preferably, in some implementation manners of the embodiment of the present invention, when the higher the consistency of the user's demand change for a certain category of goods, and the higher the shopping demand trend at the same time, it indicates that this category of goods may be the most suitable recommended target for the user currently; Therefore, according to the shopping demand trend and demand change consistency of the user for each category of goods, the specific method for obtaining the overall attention trend of the user for each category of goods is: Multiply the demand change consistency of the user for the category of goods by the shopping demand trend of the user for the category of goods, and use it as the overall attention trend of the user for the category of goods; The specific formula is: ; Indicates the overall attention trend of the user for the category of goods; Indicates the shopping demand trend of the user for the category of goods; Indicates the demand change consistency of the user for the category of goods.

[0039] Preferably, in some implementation manners of the embodiment of the present invention, when the user has purchased the required goods, the demand for a certain category of goods may increase, or there is no specific shopping target, and the user is interested in other categories of goods; Therefore, the proportion of the collected commodity-related data can be obtained by comparing the overall attention trend of the user for a certain category of goods with other categories of goods. Then, according to the difference in the overall attention trend of the user for each category of goods and other categories of goods, the calculation method for obtaining the shopping willingness evaluation value of the user for each category of goods is: Normalize the difference between the overall attention trend of the user for the category of goods and the overall attention trend of the user for the category of goods, and record it as the overall attention difference between the user for the category of goods and the category of goods; Normalize the difference between the overall attention trend of the user for the The mean of the overall attention difference between the goods of this class and the goods of all other classes is used as the evaluation value of the user's shopping willingness for the goods of class ; The specific formula is: ; represents the evaluation value of the user's shopping willingness for the goods of class ; represents the quantity of goods of all classes; represents the overall attention trend of the user for the goods of class ; represents the overall attention trend of the user for the goods of class ; represents the linear normalization function.

[0040] Preferably, in some implementation manners of the embodiments of the present invention, when the user's shopping demand changes, the e-commerce platform should quickly collect a large amount of relevant data on the user's current shopping willingness, so as to optimize the recommendation system and marketing decisions; that is, when the user's current shopping willingness for a certain class of goods is greater, the collection ratio of the relevant data of the user for this class of goods is greater; the specific method for obtaining the collection rate of each class of goods by the e-commerce platform according to the shopping willingness evaluation value is: Normalize the difference between the evaluation value of the user's shopping willingness for the goods of class and the attention degree of the user for the goods of class in the last period, and denote it as the correction factor; multiply the correction factor by the evaluation value of the user's shopping willingness for the goods of class as the collection rate of the e-commerce platform for the goods of class ; The specific formula is: ; represents the collection rate of the e-commerce platform for the goods of class ; represents the evaluation value of the user's shopping willingness for the goods of class ; represents the class of goods' attention degree in the last period; represents the linear normalization function.

[0041] Among them, represents the difference between the evaluation value of the user's shopping willingness for the goods of class and the attention degree of the goods of class in the last period; when this formula is larger, it means that the user's shopping demand has switched to class For Category products, the greater the shopping willingness, the lower the attention obtained from the user data in the past week; that is to say, the greater the conversion rate of shopping demand for Category products, the greater the collection rate of the e-commerce platform for Category

[0042] products should be.

[0043] Step S004: Optimize the collection of user behavior data based on the collection rate of the e-commerce platform for each category of products.

[0044] Preferably, in some implementation manners of the embodiments of the present invention, the collection rate of the e-commerce platform for each category of products is obtained according to the user historical behavior data, and the method for guiding the e-commerce platform to collect and store the subsequent user behavior data is that when the system detects that the collection rate of a certain category of products is continuously high, it indicates that the user is actively paying attention to or converting the shopping willingness, and the system will automatically adjust the data collection strategy for this category of products. The specific method is as follows: 1. Modify the parameters of the client-side buried point code to shorten the reporting time interval of the data related to this category of products (such as clicks, views, searches), ensuring that the real-time data collection is more intensive; in the backend data collection module, give priority to processing and storing the data related to this category of products according to the collection rate, ensuring that more detailed information is recorded; 2. Store through a distributed storage system (NoSQL database), and use a real-time stream processing platform (SparkStreaming) to dynamically analyze the data related to this category of products, construct user behavior data in real time, and use the analysis results to feedback the recommendation algorithm and marketing strategy to further verify the effectiveness of the collection rate adjustment.

[0045] 3. Continuously adjust and optimize the collection and buried point strategies of the data related to products according to the shopping willingness evaluation value of users for each category of products, forming an automatic feedback loop to ensure that the e-commerce platform can always focus on high-quality data reflecting the latest shopping intentions of users.

[0046] Please refer to Figure 2 , which shows a characteristic relationship flowchart of a user big data collection method applied to e-commerce; Through the above steps, a user big data collection method applied to e-commerce is completed.

[0047] Another embodiment of the present invention provides a user big data collection system applied to e-commerce. The system includes a memory and a processor. When the processor executes the computer program stored in the memory, it executes the above method steps S001 to S004.

[0048] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for collecting user big data applied to e-commerce, characterized in that, The method includes the following steps: Obtain all product keywords for each period in the user's historical behavior data; By clustering the product keywords in the user's historical behavior data, obtain the clustering results of product keywords for each period; according to the differences in the clustering results of product keywords for each period, obtain the attention of the user to each type of product in each period; By comparing the changes in the attention of each type of product in adjacent periods, obtain the shopping demand change rate of the user for each type of product in each period; according to the differences in the shopping demand change rates of each type of product in different periods, obtain the shopping demand trend of the user for each type of product; according to the overall change in the attention of each type of product in different periods, obtain the demand change consistency of the user for each type of product; according to the shopping demand trend and demand change consistency of each type of product, obtain the overall attention trend of the user for each type of product; according to the differences in the overall attention trends of each type of product and other types of products, obtain the shopping willingness evaluation value of the user for each type of product; according to the shopping willingness evaluation value, obtain the collection rate of each type of product by the e-commerce platform; Based on the collection rate of each type of product by the e-commerce platform, optimize the collection of user behavior data.

2. The method for collecting user big data applied to e-commerce according to claim 1, wherein The specific method for obtaining the clustering results of product keywords for each period by clustering the product keywords in the user's historical behavior data is as follows: For any period, denote the set composed of all product keywords of all users in the any period and all product keywords of all periods before the any period as the historical product keyword set of the any period; use natural language processing technology to obtain the co-occurrence probability between any two product keywords in the historical product keyword set of the any period; According to the co-occurrence probability between any two product keywords, use the DBSCAN clustering algorithm to cluster the historical product keyword set of the any period to obtain the clustering results of product keywords for the any period.

3. The method for collecting user big data applied to e-commerce according to claim 2, wherein, The specific method for obtaining the attention of the user to each type of product in each period according to the differences in the clustering results of product keywords for each period is as follows: In the commodity keyword clustering result of the th cycle, the mean value of the number of all users in all clustering clusters is denoted as the user mean value of the th cycle; the difference between the number of all users in the clustering cluster corresponding to the th category of commodities and the user mean value of the th cycle is denoted as the first difference. Normalize the ratio between the first difference and the user average value in the th cycle, and use it as the user attention contrast ratio between the type of goods and other types of goods, which is denoted as the first ratio; The average of the co-occurrence probabilities between every pair of product keywords in the clustering cluster corresponding to the category of products is denoted as the clustering density of the category of products; the average of the co-occurrence probabilities between every pair of product keywords in all clustering clusters is denoted as the average clustering density of products in the th cycle; the ratio of the clustering density of the category of products to the average clustering density of products in the th cycle is used as the user focus contrast between the category of products and other categories of products, and is denoted as the second ratio; Take the product between the first ratio and the second ratio as the user's attention to the category of goods in the th cycle.

4. The user big data collection method applied to e-commerce according to claim 1, characterized in that, The specific method for obtaining the shopping demand change rate of the user for each type of product in each period by comparing the changes in the attention of each type of product in adjacent periods is as follows: Denote the difference between the attention of users to the category of goods in the th cycle and the attention of users to the category of goods in the th cycle as the adjacent attention difference in the th cycle; Take the normalized value of the ratio between the adjacent attention difference in the th cycle and the attention of users to the category of goods in the th cycle as the shopping demand change rate of users for the category of goods in the th cycle.

5. The method for collecting user big data applied to e-commerce according to claim 1, wherein The specific method for obtaining the shopping demand trend of the user for each type of product according to the differences in the shopping demand change rates of each type of product in different periods is as follows: The normalized value of the difference between the change rate of the shopping demand of the user for the commodity of the nd category in the th cycle and the change rate of the shopping demand of the user for the commodity of the th category in the th cycle is denoted as the shopping demand difference value of the user for the commodity of the th category in the th cycle; the mean value of the shopping demand difference values of the user for the commodity of the th category in all cycles is taken as the shopping demand trend of the user for the commodity of the th category.

6. The user big data collection method applied to e-commerce according to claim 1, characterized in that, The specific method for obtaining the demand change consistency of the user for each type of product according to the overall change in the attention of each type of product in different periods is as follows: Using the least squares method, curve and straight line fitting are performed on the attention of users to the category of commodities in all cycles, and the fitting straight line of the attention of users to the category of commodities and the fitting curve of the attention of users to the category of commodities are obtained; the normalized value of the slope of the fitting straight line of the attention of users to the category of commodities is denoted as the overall attention trend of users to the category of commodities; The slope of the data point corresponding to the th cycle on the fitting curve of the user's attention to the th category of goods is denoted as the attention slope of the user to the th category of goods in the th cycle; the absolute value of the difference between the attention slope of the user to the th category of goods in the th cycle and the overall attention trend of the user to the th category of goods is denoted as the attention change factor of the user to the th category of goods in the th cycle; Take the inverse proportional normalization value of the mean of the attention change factors of the user for the goods of the category in all cycles as the demand change consistency of the user for the goods of the category.

7. The method for collecting user big data applied to e-commerce according to claim 1, wherein, The specific method for obtaining the overall attention trend of the user for each type of product according to the shopping demand trend and demand change consistency of each type of product is as follows: Multiply the consistency of the change in the user's demand for the category of goods by the shopping demand trend of the user for the category of goods, and use the result as the overall attention trend of the user for the category of goods.

8. The method for collecting user big data applied to e-commerce according to claim 1, characterized in that, The specific method for obtaining the shopping willingness evaluation value of the user for each type of product according to the differences in the overall attention trends of each type of product and other types of products is as follows: The normalized value of the difference between the overall attention trend of users towards the goods of the category and the overall attention trend of users towards the goods of the category is denoted as the overall attention difference between the goods of the category and the goods of the category; the mean value of the overall attention differences between the goods of the category and all other categories of goods is taken as the shopping willingness evaluation value of the goods of the category.

9. The user big data collection method applied to e-commerce according to claim 1, characterized in that, The specific method for obtaining the collection rate of each type of product by the e-commerce platform according to the shopping willingness evaluation value is as follows: The normalized value of the difference between the user's shopping willingness evaluation value for the category of goods and the user's attention to the category of goods in the last period is denoted as the correction factor; Multiply the correction factor by the user's evaluation value of the willingness to purchase goods of category as the collection rate of goods of category on the e-commerce platform.

10. A user big data collection system applied to e-commerce, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of a method for collecting user big data applied to e-commerce as described in any one of claims 1-9.

Images