A user big data collection method and system for e-commerce

Abstract

The present invention relates to the field of data processing technology, and specifically to a user big data collection method and system for e-commerce, comprising: obtaining the user's attention to each category of goods in each period based on the differences in the results of commodity keyword clustering in each period; obtaining the user's shopping demand trend for each category of goods based on the differences in the shopping demand change rate for each category of goods in different periods; obtaining the user's overall attention trend for each category of goods based on the shopping demand trend and demand change consistency for each category of goods; obtaining the e-commerce platform's collection rate for each category of goods based on the difference in the overall attention trend for each category of goods compared to other categories of goods; and optimizing the collection of user behavior data based on the e-commerce platform's collection rate for each category of goods. The present invention enhances the e-commerce platform's ability to respond to users' latest needs.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for collecting user big data applied to e-commerce. Background Art

[0002] E-commerce refers to a new business model that leverages the internet, mobile communications, and related technologies to facilitate commercial transactions, capital flows, information exchange, and service delivery in a virtual environment. Amidst the booming e-commerce landscape, user big data has become a core resource driving precision marketing and personalized recommendations. In recent years, a big data collection method and system has been proposed that combines client-side tracking with backend rule management. This system, through pre-configured collection rules, enables real-time capture and dynamic data embedding of user browsing, clicking, and purchasing behaviors, thereby providing more accurate, comprehensive, and secure data support for e-commerce platforms.

[0003] In the prior art, in e-commerce platforms, the purpose of collecting user big data is to provide accurate and personalized product recommendations; however, when users browse and click on a certain product on a large scale and ultimately complete the purchase, the relevant data for that type of product may become irrelevant, and it is necessary to collect more user data on other related products; but the e-commerce platform will continue to recommend the same type of products until the proportion of users actively browsing other types of products exceeds the proportion of purchased products, and then record and upload a large amount of relevant data for other products, which often results in low-quality, information-redundant product data, thereby weakening the e-commerce platform's ability to respond to users' latest needs. Summary of the Invention

[0004] The present invention provides a method and system for collecting user big data for e-commerce to solve the existing problem that e-commerce platforms will continue to recommend products of the same type as those already purchased by users until the proportion of users actively browsing other types of products exceeds the proportion of products already purchased, resulting in low-quality and redundant product data, thereby weakening the e-commerce platform's ability to respond to users' latest needs.

[0005] The present invention provides a method and system for collecting user big data for e-commerce using the following technical solutions:

[0006] The present invention proposes a method for collecting user big data applied to e-commerce, which comprises the following steps:

[0007] Obtain all product keywords for each period in the user's historical behavior data;

[0008] By clustering the product keywords in the user's historical behavior data, the product keyword clustering results for each period are obtained. Based on the differences in the product keyword clustering results for each period, the user's attention to each type of product in each period is obtained.

[0009] By comparing the changes in the attention of each category of goods in adjacent cycles, the change rate of users' shopping demand for each category of goods in each cycle is obtained; based on the differences in the change rates of shopping demand for each category of goods in different cycles, the shopping demand trend of users for each category of goods is obtained; based on the overall changes in the attention of each category of goods in different cycles, the consistency of users' demand changes for each category of goods is obtained; based on the shopping demand trend and demand change consistency of each category of goods, the overall attention trend of users for each category of goods is obtained; based on the differences in the overall attention trends of each category of goods and other categories of goods, the shopping intention evaluation value of users for each category of goods is obtained; based on the shopping intention evaluation value, the collection rate of each category of goods on the e-commerce platform is obtained;

[0010] Optimize the collection of user behavior data based on the e-commerce platform's collection rate for each category of goods.

[0011] Preferably, the specific method of obtaining the product keyword clustering result of each period by clustering the product keywords in the user historical behavior data is:

[0012] For any period, all product keywords of all users in the period, as well as all product keywords in all periods before the period, are recorded as the historical product keyword set of the period; natural language processing technology is used to obtain the co-occurrence probability between any two product keywords in the historical product keyword set of the period;

[0013] According to the co-occurrence probability between any two product keywords, the DBSCAN clustering algorithm is used to cluster the historical product keyword set of the arbitrary period to obtain the product keyword clustering result of the arbitrary period.

[0014] Preferably, the specific method for obtaining the user's attention to each category of products in each period according to the difference in the product keyword clustering results in each period is:

[0015] In the In the product keyword clustering results of the period, the mean of the number of all users in all clusters is recorded as The average value of users in the period; The number of all users in the cluster corresponding to the product category and the The difference between the user mean values ​​of the first period is recorded as the first difference; the first difference is added to the The normalized value of the ratio between the user means of the periods is used as the The user attention contrast between the product of this category and other products, which is recorded as the first ratio;

[0016] The first The mean of the co-occurrence probabilities between all product keywords in the cluster corresponding to the product category is recorded as The clustering density of the product category; the mean of the co-occurrence probability between all product keywords in all clusters is recorded as The average value of commodity clustering density in the period; The clustering density of the commodity category and the The ratio of the mean values ​​of commodity clustering density in the first period is used as the The user focus contrast between the product of this category and other products, which is recorded as the second ratio;

[0017] The product of the first ratio and the second ratio is used as the user's Category of products The attention in a cycle.

[0018] Preferably, the specific method for obtaining the change rate of users' shopping demand for each category of goods in each cycle by comparing the change in the attention of each category of goods in adjacent cycles is:

[0019] The user Category of products The attention degree in the first cycle and the user's Category of products The difference between the attention levels in the first cycle is recorded as The adjacent attention difference of the period; The adjacent attention difference of the cycle and the user's attention to the Category of products The normalized value of the ratio of the attention in the first cycle is used as the user's attention in the first Category of products The rate of change of shopping demand in a cycle.

[0020] Preferably, the specific method for obtaining the user's shopping demand trend for each category of goods based on the difference in the shopping demand change rate of each category of goods in different cycles is:

[0021] The user Category of products The change rate of shopping demand in the first cycle and the user's Category of products The normalized value of the difference between the shopping demand change rates in the first cycle is recorded as the user's Category of products The shopping demand difference value under the cycle; The average of the shopping demand difference values ​​of the category of goods in all cycles is used as the user's Shopping demand trends for similar products.

[0022] Preferably, the specific method for obtaining the consistency of the change in user demand for each category of goods based on the overall change in the attention of each category of goods in different cycles is:

[0023] Least square method is used to estimate the user The attention of the product category in all periods is fitted by curve and straight line to obtain the user's attention to the first The fitting straight line of attention of the product category and the user's attention to the Fitting curve of user's attention to the first The normalized value of the slope of the fitting line of the attention degree of the product category is recorded as the user's attention to the first The overall attention trend of similar products;

[0024] The user The attention fitting curve of the category product is The slope of the data point corresponding to the cycle is recorded as the user's Category of products The slope of attention in the first cycle; Category of products The slope of attention in the first cycle is related to the user's attention to the The absolute value of the difference between the overall attention trend of the category of goods is recorded as the user's attention to the Category of products The attention change factor under the cycle; The inverse normalized value of the mean of the attention change factor of the commodity in all periods is used as the user's attention to the first The consistency of demand changes for similar products.

[0025] Preferably, the specific method for obtaining the overall attention trend of users for each category of goods based on the shopping demand trend and demand change consistency of each category of goods is:

[0026] The user The consistency of demand changes for the following commodities is related to the users' The product of the shopping demand trends of the following categories of goods is used as the user's The overall attention trend of similar products.

[0027] Preferably, the specific method for obtaining the user's shopping intention evaluation value for each category of goods based on the difference in overall attention trends between each category of goods and other categories of goods is:

[0028] The user The overall attention trend of the category and the users' attention to the The normalized value of the difference between the overall attention trend of the category of goods is recorded as the user's attention to the Category of goods and The overall attention difference between the two categories of products; The average of the overall attention difference between the first category and all other categories of products is used as the user's attention to the The shopping intention evaluation value of this type of product.

[0029] Preferably, the specific method for obtaining the acquisition rate of each category of goods on the e-commerce platform according to the shopping intention evaluation value is:

[0030] The user The shopping intention evaluation value of the product category and the user's The normalized value of the difference between the attention of the category products in the last cycle is recorded as the correction factor; the correction factor is added to the user's attention to the first The product of the shopping intention evaluation values ​​of the second category of goods is used as the evaluation of the e-commerce platform for the The collection rate of this category of products.

[0031] The present invention also proposes a user big data collection system for e-commerce, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the steps of the above-mentioned user big data collection method for e-commerce.

[0032] The beneficial effects of the technical solution of the present invention are as follows: the present invention obtains the user's attention to each category of goods in each period according to the differences in the results of keyword clustering of goods in each period; obtains the user's shopping demand trend for each category of goods according to the differences in the shopping demand change rate of each category of goods in different periods; obtains the user's overall attention trend for each category of goods according to the shopping demand trend and demand change consistency of each category of goods; obtains the e-commerce platform's collection rate for each category of goods according to the difference in the overall attention trend between each category of goods and other categories of goods; and optimizes the collection of user behavior data based on the e-commerce platform's collection rate for each category of goods. In this way, by optimizing the e-commerce platform's collection rate for each category of goods, the user's shopping interests can be accurately identified, thereby ensuring that the recommendation system in the e-commerce platform responds quickly to the user's latest needs; by monitoring the real-time changes in the user's shopping needs, the collection rate of each category of goods is automatically adjusted to ensure that data collection always focuses on the user's latest needs; thereby enhancing the e-commerce platform's ability to respond to the user's latest needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 This is a flowchart of the steps of a method for collecting user big data applied to e-commerce according to the present invention;

[0035] Figure 2 This is a feature relationship flow chart of a method for collecting user big data applied to e-commerce according to the present invention. DETAILED DESCRIPTION

[0036] To further illustrate the technical means and effects employed by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a method and system for collecting user big data for e-commerce applications, including its specific implementation, structure, features, and effects. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0037] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0038] The following describes in detail a method and system for collecting user big data for e-commerce provided by the present invention with reference to the accompanying drawings.

[0039] See also Figure 1 , which shows a flowchart of a method for collecting user big data applied to e-commerce provided by one embodiment of the present invention, the method comprising the following steps:

[0040] Step S001: Obtain all product keywords in each period in the user's historical behavior data.

[0041] In a specific implementation of the embodiment of the present invention, the description is made by taking one day in the past week as an example;

[0042] Using client tracking and server logs, we can collect real-time behavioral data of all users on the e-commerce platform over the past week. The specific method is as follows:

[0043] By embedding JavaScript code in the e-commerce platform, users' searches, browsed pages, and clicks will be recorded. Meanwhile, the server logs will be analyzed to obtain user access content, order content, and interaction content. Furthermore, the API provided by the e-commerce platform will be used to obtain product information and user reviews. The above content collected in the past week is collectively referred to as user historical behavior data.

[0044] The collected user historical behavior data is cleaned to remove invalid and duplicate data, and preprocess missing values ​​to obtain all text data in the user historical behavior data; all text data is preprocessed by word segmentation, removal of stop words and punctuation marks to obtain preprocessed text data; the TextRank algorithm is used to extract keywords from the preprocessed text data to obtain all product keywords for each period in the user historical behavior data.

[0045] Among them, data cleaning, elimination of invalid and duplicate data, and preprocessing of filling missing values; preprocessing of word segmentation, removal of stop words and punctuation marks for all text data; and TextRank algorithm are all existing technologies and will not be described in detail in this embodiment.

[0046] At this point, all product keywords for each period in the user's historical behavior data are obtained through the above method.

[0047] Step S002: clustering the product keywords in the user's historical behavior data to obtain the product keyword clustering results for each period; and obtaining the user's attention to each type of product in each period based on the differences in the product keyword clustering results for each period.

[0048] It's important to note that when user shopping needs change, e-commerce platforms may continue to recommend similar products for a period of time based on historical user behavior data. This can reduce a user's willingness to purchase, while browsing data still primarily focuses on similar products. This can lead to frequent push notifications of similar products from e-commerce platforms, potentially creating information cocoons, degrading the user shopping experience, and potentially leading to user churn. To prevent the collection of low-quality, redundant data from disrupting the user's shopping experience, it's necessary to analyze historical user behavior data to determine the user's interest in each product category over each period.

[0049] Preferably, in some implementations of the embodiments of the present invention, since the change in the number of a certain type of product keywords in the user's historical behavior data in different periods can reflect the change in the user's shopping intention, the change in the user's shopping intention is obtained by clustering the product keywords in the user's historical behavior data. The specific method for obtaining the product keyword clustering results for each period by clustering the product keywords in the user's historical behavior data is as follows:

[0050] For any period, all product keywords of all users in the period, as well as all product keywords in all periods before the period, are recorded as the historical product keyword set of the period; natural language processing technology is used to obtain the co-occurrence probability between any two product keywords in the historical product keyword set of the period;

[0051] Clustering the historical product keyword set of any period using the DBSCAN clustering algorithm based on the co-occurrence probability between any two product keywords to obtain the product keyword clustering result of the any period;

[0052] It should be noted that the product keyword clustering results of the period include several clusters; each cluster corresponds to a category of products, and the category of products corresponding to the largest cluster may be the user's current shopping needs; if the user has purchased or does not need a certain category of products, the shopping needs expressed in the overall cluster may be low, and the changes in the clustering results of each period can be found that the number of product keywords shows a downward trend; therefore, the product keywords of the first period are clustered to obtain the product keyword clustering results of the first period, and then the product keywords of the second period are added to the product keywords of the first period for joint clustering to obtain the product keyword clustering results of the second period, and the product keyword clustering results of each period are obtained in turn.

[0053] Among them, natural language processing technology and DBSCAN clustering algorithm are existing technologies and will not be described in detail in this embodiment; co-occurrence probability is a statistical indicator that measures the frequency of co-occurrence of two words in a specific context through natural language processing (NLP) technology.

[0054] Preferably, in some implementations of the embodiments of the present invention, when a user wants to purchase a certain type of product, there will be a large number of users related to this type of product in the user's historical behavior data. That is, the more users contain keywords for this type of product in the clustering results, the more attention all users pay to this type of product. At the same time, the more focused the keywords for this type of product are and the smaller the clustering range is, the more accurate the user's shopping goal is. Therefore, based on the differences in the product keyword clustering results in each period, a method for obtaining the user's attention to each type of product in each period is as follows:

[0055] In the In the product keyword clustering results of the period, the mean of the number of all users in all clusters is recorded as The average value of users in the period; The number of all users in the cluster corresponding to the product category and the The difference between the user mean values ​​of the first period is recorded as the first difference; the first difference is added to the The normalized value of the ratio between the user means of the periods is used as the The user attention contrast between the product of this category and other products, which is recorded as the first ratio;

[0056] The first The mean of the co-occurrence probabilities between all product keywords in the cluster corresponding to the product category is recorded as The clustering density of the product category; the mean of the co-occurrence probability between all product keywords in all clusters is recorded as The average value of commodity clustering density in the period; The clustering density of the commodity category and the The ratio of the mean values ​​of commodity clustering density in the first period is used as the The user focus contrast between the product of this category and other products, which is recorded as the second ratio;

[0057] The product of the first ratio and the second ratio is used as the user's Category of products The degree of attention in each cycle;

[0058] The specific formula is:

[0059] ;

[0060] Where, Indicates the user's Category of products The degree of attention in each cycle; Indicates in In the product keyword clustering results of the period, The number of all users in the cluster corresponding to the product category; Indicates in The average number of all users in all clusters in the product keyword clustering results of the period; Indicates in In the product keyword clustering results of the period, Clustering density of similar products; Indicates the The mean value of commodity clustering density in each period; represents the linear normalization function.

[0061] It should be noted that Indicates the The user attention contrast between the first category of products and other categories of products. The larger the value of this formula is, the more users pay attention to the first category of products. Category of goods; Indicates the The ratio between the clustering density of the product category and the average clustering density of all product categories. The larger the ratio, the more users are interested in the product category. The more specific and focused the attention is on a category of products.

[0062] So far, the above method has been used to obtain the user's attention to each category of goods in each cycle.

[0063] Step S003: Based on the comparison of the changes in the attention of each category of goods in adjacent periods, the rate of change of users' shopping demand for each category of goods in each period is obtained; based on the differences in the rate of change of shopping demand for each category of goods in different periods, the shopping demand trend of users for each category of goods is obtained; based on the overall changes in the attention of each category of goods in different periods, the consistency of users' demand changes for each category of goods is obtained; based on the shopping demand trend and the consistency of demand changes for each category of goods, the overall attention trend of users for each category of goods is obtained; based on the differences in the overall attention trends of each category of goods and other categories of goods, the shopping intention evaluation value of users for each category of goods is obtained; based on the shopping intention evaluation value, the collection rate of each category of goods on the e-commerce platform is obtained.

[0064] It's important to note that when a user has already purchased a certain type of product or no longer needs it, or when their shopping needs change, the platform's recommendation mechanism based on historical data forces them to browse multiple pieces of behavioral data related to that type of product. This excessive amount of similar content significantly reduces the user experience and leads to longer delays before the e-commerce platform recommends products that the user actually needs or may be interested in, potentially increasing user churn. Therefore, it's necessary to analyze how user interest in each product category changes over time to understand the trends in user shopping demand for each category. This can then reveal users' true shopping needs and derive an assessment of their willingness to purchase each product category.

[0065] Preferably, in some implementations of the embodiments of the present invention, if the attention level of a certain type of product continues to rise in each cycle, it indicates that the user's demand for this type of product is increasing. If the attention level of a certain type of product suddenly decreases in each cycle, it indicates that the user may no longer need this type of product for some reason. Therefore, by comparing the changes in the user's attention level for each type of product in adjacent cycles, the specific method for obtaining the user's shopping demand change rate for each type of product in each cycle is as follows:

[0066] The user Category of products The attention degree in the first cycle and the user's Category of products The difference between the attention levels in the first cycle is recorded as The adjacent attention difference of the period; The adjacent attention difference of the cycle and the user's attention to the Category of products The normalized value of the ratio of the attention in the first cycle is used as the user's attention in the first Category of products The rate of change of shopping demand in a cycle;

[0067] The specific formula is:

[0068] ;

[0069] Where, Indicates the user's Category of products The rate of change of shopping demand in a cycle; Indicates the user's Category of products The degree of attention in each cycle; Indicates the user's Category of products The degree of attention in each cycle; represents the linear normalization function.

[0070] Preferably, in some implementations of the embodiments of the present invention, once a user has purchased a certain type of product, the behavioral data (clicks, browsing time, etc.) for that type of product after purchase will decrease, which is a significant change compared to before purchase. However, when the user's shopping needs change, the behavioral data for the new shopping target will gradually increase, and the shopping demand change rate will become increasingly larger. Therefore, based on the differences in the shopping demand change rate for each type of product in different cycles, the specific method for obtaining the user's shopping demand trend for each type of product is as follows:

[0071] The user Category of products The change rate of shopping demand in the first cycle and the user's Category of products The normalized value of the difference between the shopping demand change rates in the first cycle is recorded as the user's Category of products The shopping demand difference value under the cycle; The average of the shopping demand difference values ​​of the category of goods in all cycles is used as the user's Shopping demand trends for similar products;

[0072] The specific formula is:

[0073] ;

[0074] Where, Indicates the user's Shopping demand trends for similar products; Indicates the number of all cycles; Indicates the user's Category of products The rate of change of shopping demand in a cycle; Indicates the user's Category of products The rate of change of shopping demand in a cycle; represents the linear normalization function.

[0075] Among them, users The difference in shopping demand for the same product in the last cycle and the difference in the user's demand for the same product in the first cycle The shopping demand difference values ​​of similar products in the second to last cycle are the same.

[0076] Preferably, in some implementations of the embodiments of the present invention, after purchasing a product, the user may compare information about similar products, resulting in a slight upward trend in their shopping demand. However, the rate of change in demand will reverse compared to the rate before purchase. Therefore, when the rate of change in the user's attention to this type of product in each cycle differs significantly from the overall trend of their attention, it indicates that the user's attitude towards this type of product has changed, and the consistency of the change in the user's demand for this type of product is smaller. Based on the overall change in the user's attention to each type of product in different cycles, the specific method for obtaining the consistency of the change in the user's demand for each type of product is as follows:

[0077] Least square method is used to estimate the user The attention of the product category in all periods is fitted by curve and straight line to obtain the user's attention to the first The fitting straight line of attention of the product category and the user's attention to the Fitting curve of user's attention to the first The normalized value of the slope of the fitting line of the attention degree of the product category is recorded as the user's attention to the first The overall attention trend of the category of goods; wherein, the least square method is an existing technology, and this embodiment will not be described in detail here;

[0078] The user The attention fitting curve of the category product is The slope of the data point corresponding to the cycle is recorded as the user's Category of products The slope of attention in the first cycle; Category of products The slope of attention in the first cycle is related to the user's attention to the The absolute value of the difference between the overall attention trend of the category of goods is recorded as the user's attention to the Category of products The attention change factor under the cycle; The inverse normalized value of the mean of the attention change factor of the commodity in all periods is used as the user's attention to the first Consistency of demand changes for similar products;

[0079] The specific formula is:

[0080] ;

[0081] Where, Indicates the user's Consistency of demand changes for similar products; Indicates the number of all cycles; Indicates the user's Category of products The slope of attention in each cycle; Indicates the user's The slope of the fitted line of attention of similar products; Indicates taking the absolute value; represents the linear normalization function; Represents an exponential function with a natural constant as the base, and the embodiment adopts Model to present inverse proportional relationship and normalization processing, As the input of the model, the implementer can choose the inverse proportional function and normalization function according to the actual situation.

[0082] Preferably, in some implementations of the embodiments of the present invention, when the consistency of the change in user demand for a certain category of goods is higher and the shopping demand trend is higher, it indicates that the category of goods may be the most suitable recommendation target for the user at the moment; therefore, based on the shopping demand trend and demand change consistency of the user for each category of goods, the specific method for obtaining the user's overall attention trend for each category of goods is as follows:

[0083] The user The consistency of demand changes for the following commodities is related to the users' The product of the shopping demand trends of the following categories of goods is used as the user's The overall attention trend of similar products;

[0084] The specific formula is:

[0085] ;

[0086] Indicates the user's The overall attention trend of similar products; Indicates the user's Shopping demand trends for similar products; Indicates the user's The consistency of demand changes for similar products.

[0087] Preferably, in some implementations of the present invention, after a user has purchased a desired product, they may have an increased demand for a certain category of products, or may not have a specific shopping goal but be interested in other categories of products. Therefore, the user's overall interest trend in a certain category of products can be compared with that of other categories of products to obtain the proportion of collected product-related data. Based on the difference in the user's overall interest trend in each category of products compared with other categories of products, the calculation method for obtaining the user's shopping intention evaluation value for each category of products is as follows:

[0088] The user The overall attention trend of the category and the users' attention to the The normalized value of the difference between the overall attention trend of the category of goods is recorded as the user's attention to the Category of goods and The overall attention difference between the two categories of products; The average of the overall attention difference between the first category and all other categories of products is used as the user's attention to the The shopping intention evaluation value of the product category;

[0089] The specific formula is:

[0090] ;

[0091] Indicates the user's The shopping intention evaluation value of the product category; Indicates the quantity of all categories of goods; Indicates the user's The overall attention trend of similar products; Indicates the user's The overall attention trend of similar products; represents the linear normalization function.

[0092] Preferably, in some implementations of the embodiments of the present invention, when a user's shopping needs change, the e-commerce platform should quickly collect a large amount of relevant data about the user's current shopping intention, so as to optimize the recommendation system and marketing decisions; that is, the greater the user's current shopping intention for a certain category of goods, the greater the proportion of user-related data collected for that product; the specific method for obtaining the e-commerce platform's collection rate for each category of goods based on the shopping intention evaluation value is:

[0093] The user The shopping intention evaluation value of the product category and the user's The normalized value of the difference between the attention of the category products in the last cycle is recorded as the correction factor; the correction factor is added to the user's attention to the first The product of the shopping intention evaluation values ​​of the second category of goods is used as the evaluation of the e-commerce platform for the The collection rate of similar products;

[0094] The specific formula is:

[0095] ;

[0096] Indicates that the e-commerce platform The collection rate of similar products; Indicates the user's The shopping intention evaluation value of the product category; Indicates the The attention level of similar products in the last cycle; represents the linear normalization function.

[0097] in, Indicates the user's The shopping intention evaluation value of the first The difference between the attention of the same type of goods in the last cycle; when the formula is larger, it means that the user's shopping demand has shifted to the first The greater the purchase intention of the product, the lower the attention it receives from the user data of the past week; The greater the conversion rate of shopping demand for the following products, the more likely the e-commerce platform will The collection rate of this type of goods should be greater.

[0098] So far, the collection rate of each category of goods on the e-commerce platform has been obtained through the above method.

[0099] Step S004: Optimize the collection of user behavior data based on the collection rate of each category of goods on the e-commerce platform.

[0100] Preferably, in some implementations of the embodiments of the present invention, the e-commerce platform's collection rate for each category of goods is obtained based on historical user behavior data, and a method for guiding the e-commerce platform to collect and store subsequent user behavior data is as follows: when the system detects that the collection rate of a certain category of goods is continuously high, indicating that the user is actively paying attention to or changing their shopping intention, the system will automatically adjust the data collection strategy for this category of goods. The specific method is:

[0101] 1. Modify the client tracking code parameters to shorten the reporting interval for data related to this category of products (such as clicks, views, and searches), ensuring more intensive real-time data collection. In the back-end data collection module, prioritize the processing and storage of data related to this category of products based on the collection rate, ensuring that more detailed information is recorded.

[0102] 2. Storing data in a distributed storage system (NoSQL database) and utilizing a real-time stream processing platform (Spark Streaming) dynamically analyzes relevant data on this type of product, constructing user behavior data in real time. The analysis results are then used to feed back into recommendation algorithms and marketing strategies, further verifying the effectiveness of collection rate adjustments.

[0103] 3. Based on the user's shopping intention evaluation for each product category, the product-related data collection and tracking strategies are continuously adjusted and optimized to form an automatic feedback loop, ensuring that the e-commerce platform can always focus on high-quality data that reflects the user's latest shopping intentions.

[0104] See also Figure 2 , which shows a feature relationship flow chart of a method for collecting user big data applied to e-commerce;

[0105] Through the above steps, a method for collecting user big data applied to e-commerce is completed.

[0106] Another embodiment of the present invention provides a user big data collection system for e-commerce, the system comprising a memory and a processor, and when the processor executes the computer program stored in the memory, it performs steps S001 to S004 of the above method.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for collecting user big data applied to e-commerce, characterized in that: The method comprises the following steps: Obtain all product keywords for each period in the user's historical behavior data; By clustering the product keywords in the user's historical behavior data, the product keyword clustering results for each period are obtained. Based on the differences in the product keyword clustering results for each period, the user's attention to each type of product in each period is obtained. By comparing the changes in the attention of each category of goods in adjacent cycles, the change rate of users' shopping demand for each category of goods in each cycle is obtained; based on the differences in the change rates of shopping demand for each category of goods in different cycles, the shopping demand trend of users for each category of goods is obtained; based on the overall changes in the attention of each category of goods in different cycles, the consistency of users' demand changes for each category of goods is obtained; based on the shopping demand trend and demand change consistency of each category of goods, the overall attention trend of users for each category of goods is obtained; based on the differences in the overall attention trends of each category of goods and other categories of goods, the shopping intention evaluation value of users for each category of goods is obtained; based on the shopping intention evaluation value, the collection rate of each category of goods on the e-commerce platform is obtained; Optimize the collection of user behavior data based on the e-commerce platform's collection rate for each category of goods.

2. The method for collecting user big data for e-commerce according to claim 1, characterized in that: The specific method of obtaining the product keyword clustering results of each period by clustering the product keywords in the user historical behavior data is as follows: For any period, all product keywords of all users in the period, as well as all product keywords in all periods before the period, are recorded as the historical product keyword set of the period; natural language processing technology is used to obtain the co-occurrence probability between any two product keywords in the historical product keyword set of the period; According to the co-occurrence probability between any two product keywords, the DBSCAN clustering algorithm is used to cluster the historical product keyword set of the arbitrary period to obtain the product keyword clustering result of the arbitrary period.

3. The method for collecting user big data for e-commerce according to claim 2, characterized in that: The specific method for obtaining the user's attention to each category of products in each period based on the differences in the product keyword clustering results in each period is: In the In the product keyword clustering results of the period, the mean of the number of all users in all clusters is recorded as The average value of users in the period; The number of all users in the cluster corresponding to the product category and the The difference between the user means of the periods is recorded as the first difference; The first difference and the The normalized value of the ratio between the user means of the periods is used as the The user attention contrast between the product of this category and other products, which is recorded as the first ratio; The first The mean of the co-occurrence probabilities between all product keywords in the cluster corresponding to the product category is recorded as The clustering density of the product category; the mean of the co-occurrence probability between all product keywords in all clusters is recorded as The average value of commodity clustering density in the period; The clustering density of the commodity category and the The ratio of the mean values ​​of commodity clustering density in the first period is used as the The user focus contrast between the product of this category and other products, which is recorded as the second ratio; The product of the first ratio and the second ratio is used as the user's Category of products The attention in a cycle.

4. The method for collecting user big data for e-commerce according to claim 1, characterized in that: The specific method for obtaining the change rate of users' shopping demand for each category of goods in each cycle by comparing the changes in the attention of each category of goods in adjacent cycles is: The user Category of products The attention degree in the first cycle and the user's Category of products The difference between the attention levels in the first cycle is recorded as The adjacent attention difference of the period; The adjacent attention difference of the cycle and the user's attention to the Category of products The normalized value of the ratio of the attention in the first cycle is used as the user's attention in the first Category of products The rate of change of shopping demand in a cycle.

5. The method for collecting user big data applied to e-commerce according to claim 1, characterized in that: The specific method for obtaining the user's shopping demand trend for each category of goods based on the difference in the shopping demand change rate of each category of goods in different cycles is: The user Category of products The change rate of shopping demand in the first cycle and the user's Category of products The normalized value of the difference between the shopping demand change rates in the first cycle is recorded as the user's Category of products The shopping demand difference value under the cycle; The average of the shopping demand difference values ​​of the category of goods in all cycles is used as the user's Shopping demand trends for similar products.

6. The method for collecting user big data for e-commerce according to claim 1, characterized in that: The specific method for obtaining the consistency of user demand changes for each category of goods based on the overall changes in the attention of each category of goods in different cycles is: Least square method is used to estimate the user The attention of the product category in all periods is fitted by curve and straight line to obtain the user's attention to the first The fitting straight line of attention of the product category and the user's attention to the Fitting curve of user's attention to the first The normalized value of the slope of the fitting line of the attention degree of the product category is recorded as the user's attention to the first The overall attention trend of similar products; The user The attention fitting curve of the category product is The slope of the data point corresponding to the cycle is recorded as the user's Category of products The slope of attention in the first cycle; Category of products The slope of attention in the first cycle is related to the user's attention to the The absolute value of the difference between the overall attention trend of the category of goods is recorded as the user's attention to the Category of products Focus on the change factors under the cycle; The user The inverse normalized value of the mean of the attention change factor of the commodity in all periods is used as the user's attention to the first The consistency of demand changes for similar products.

7. The method for collecting user big data for e-commerce according to claim 1, characterized in that: The specific method for obtaining the overall attention trend of users for each category of goods based on the shopping demand trend and demand change consistency of each category of goods is: The user The consistency of demand changes for the following commodities is related to the users' The product of the shopping demand trends of the following categories of goods is used as the user's The overall attention trend of similar products.

8. The method for collecting user big data for e-commerce according to claim 1, characterized in that: The specific method for obtaining the user's shopping intention evaluation value for each category of goods based on the difference in overall attention trends between each category of goods and other categories of goods is as follows: The user The overall attention trend of the category and the users' attention to the The normalized value of the difference between the overall attention trend of the category of goods is recorded as the user's attention to the Category of goods and The overall attention difference between the two categories of products; The average of the overall attention difference between the first category and all other categories of products is used as the user's attention to the The shopping intention evaluation value of this type of product.

9. The method for collecting user big data for e-commerce according to claim 1, characterized in that: The specific method for obtaining the collection rate of each category of goods on the e-commerce platform based on the shopping intention evaluation value is as follows: The user The shopping intention evaluation value of the product category and the user's The normalized value of the difference between the attention levels of similar products in the last cycle is recorded as the correction factor; The correction factor is compared with the user's The product of the shopping intention evaluation values ​​of the second category of goods is used as the evaluation of the e-commerce platform for the The collection rate of this category of products.

10. A user big data collection system for e-commerce, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of a method for collecting user big data applied to e-commerce as described in any one of claims 1 to 9 are implemented.

Images