An e-commerce platform order analysis method and system based on data mining

Abstract

The present invention discloses an e-commerce platform order analysis method and system based on data mining, which relates to the technical field of e-commerce data analysis, including collecting e-commerce platform order data, constructing a multi-layer data lake architecture for data processing, combining the Lambda architecture to analyze the data, and constructing a model through an autoencoder to reconstruct the order data to identify data features. The method of the present invention can extract structured and semi-structured data from the original data area through the multi-layer data lake architecture in conjunction with Lambda, and conduct large-scale analysis. Through the training of the anomaly detection model of the autoencoder and the precise threshold setting, the system can efficiently identify and process abnormal order data. Through the combination of the product recommendation system and the inventory management system, from user demand prediction to inventory strategy formulation, sales and inventory turnover rates are significantly improved, and the cost-effectiveness is maximized. By continuously acquiring and analyzing order data, the effect of quickly adjusting product prices is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of e-commerce data analysis, and in particular to an e-commerce platform order analysis method and system based on data mining. Background Art

[0002] With the increase of users and shopping behaviors on e-commerce platforms, there is a large amount of order data. These data not only include users' shopping behaviors, but also contain rich market demand information. In order to mine user order data and improve user stickiness of e-commerce platforms, in the existing technology, the application of data mining in the e-commerce field mainly focuses on user behavior analysis and the construction of recommendation systems.

[0003] However, when processing large-scale order data, most existing e-commerce platforms use traditional data warehouses and data table models. Although some systems have introduced recommendation algorithms and inventory management models, they often lack in-depth mining of order data characteristics. In the process of user behavior analysis, it is difficult to optimize the balance between inventory costs and out-of-stock costs at the same time, resulting in insufficient accuracy of recommendations and precision of inventory management, making it difficult to meet users' personalized needs and rapidly changing market demands, as well as control inventory-related costs. Summary of the invention

[0004] In view of the problems existing in the above-mentioned existing e-commerce platform order analysis method and system based on data mining, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that although some systems have introduced recommendation algorithms and inventory management models, they often lack in-depth mining of order data features. In the process of user behavior analysis, it is difficult to simultaneously optimize the balance between inventory costs and out-of-stock costs, resulting in insufficient accuracy of recommendations and precision of inventory management, making it difficult to meet users' personalized needs and rapidly changing market demands, as well as control inventory-related costs.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an e-commerce platform order analysis method based on data mining, which comprises:

[0007] Collect order data from e-commerce platforms, build a multi-layer data lake architecture for data processing, analyze the data using the Lambda architecture, build a model through autoencoders, reconstruct order data and identify data features;

[0008] Analyze the correlation of order data from different users, predict product ratings to generate a recommendation list, set the objective function of minimum inventory cost and out-of-stock cost, determine the maximum sampling point data as the training data iteration objective function through the expected improvement algorithm, and cooperate with the conversion rate data of the generated recommendation list to determine the safety inventory value and replenishment frequency, forming a feedback closed loop;

[0009] Analyze the mutual influence of demand and price data through demand function based on order data to determine the optimal price;

[0010] Obtain social media data from e-commerce platforms, analyze user media reviews through time series, organize order data, encrypt and store them in a cloud storage server, and back up the data to an external hard drive.

[0011] As a preferred solution of the e-commerce platform order analysis method based on data mining described in the present invention, wherein: the e-commerce platform order data is collected, a multi-layer data lake architecture is constructed for data processing, and the data is analyzed in combination with the Lambda architecture, including:

[0012] Collect order data from e-commerce platforms and build a multi-layer data lake architecture based on cloud computing data centers, including raw data areas, refined data areas, data warehouse areas, public data areas, work areas, and sensitive data areas;

[0013] Import order data into the multi-layer data lake architecture through ETL tools, and divide it into structured data, semi-structured data, and unstructured data according to the data structure;

[0014] Parse semi-structured data into standard structured data formats;

[0015] Unify the formats of the timestamp and amount fields in the order data and summarize the order data;

[0016] Utilize the automatic expansion function of cloud computing data centers to dynamically adjust storage and computing resources according to changes in data volume;

[0017] Building Lambda architecture includes batch processing layer, real-time processing layer and service layer;

[0018] The batch processing layer stores data in a distributed manner based on the Hadoop distributed file system, uses the MapReduce framework for parallel computing, and extracts order data for processing and analysis;

[0019] The real-time processing layer processes new order data based on Apache Spark Streaming and transmits it to the processing system in real time;

[0020] The service layer performs data query and services through a distributed database.

[0021] As a preferred solution of the e-commerce platform order analysis method based on data mining described in the present invention, wherein: the model is constructed by the autoencoder to reconstruct the order data identification data features, including:

[0022] Extract cleaned and normalized order data from the multi-layer data lake architecture and build an anomaly detection model through deep learning autoencoder, including input layer, encoding layer, decoding layer and output layer;

[0023] The input layer inputs the number of features of the order data, the encoding layer reduces the number of neurons layer by layer, compresses the input data and extracts the key features of the order data, the decoding layer increases the number of neurons layer by layer, restores the compressed feature data to the original data, and the output layer outputs the reconstructed order feature data;

[0024] Use the training set and the mean square error as the loss function to calculate the reconstruction error MSE, select the Adam optimizer to optimize the model parameters, and use the gradient descent method to update the model parameters. When the model loss no longer decreases significantly during the continuous iteration process, stop iterating, output the model parameters, and update the model.

[0025] Use the data of the model training set and the validation set to input the model for reconstruction, and calculate the reconstruction error of the order data. The reconstruction error standard deviation multiplied by the 95th percentile significance level plus the mean of the reconstruction error is used as the reconstruction threshold;

[0026] New order data is input and reconstructed by the model to output order feature data. If the calculated reconstruction error is greater than the reconstruction threshold, it is judged as data anomaly and the abnormal order data is isolated to the specified abnormal data area.

[0027] As a preferred solution of the e-commerce platform order analysis method based on data mining described in the present invention, wherein: the correlation of order data of different users is analyzed, commodity ratings are predicted to generate a recommendation list, the objective function of minimum inventory cost and out-of-stock cost is set, the maximum sampling point data is determined as the training data iteration objective function through the expected improvement algorithm, the conversion rate data of the recommendation list is generated, the safety inventory value and the replenishment frequency are determined, and a feedback closed loop is formed, including:

[0028] Based on order data, distinguish the behavior data of different users, including browsing history, click history and purchase history;

[0029] Calculate the relevance of order data of different users based on collaborative filtering algorithm;

[0030] Based on the calculated similarity, similar users are selected based on the 95th percentile according to the similarity, and the rating data of the selected users on the products are summarized;

[0031] For products that users have not yet rated, the potential rating is predicted by similar users, expressed as:

[0032]

[0033] in represents the predicted rating of user u for product j, rv,j represents the rating of user v for product j, V represents the set of similar users v of user u, and S(u,v) represents the correlation between users u and v;

[0034] Based on user u's predicted rating for product j, a personalized recommendation list is generated by ranking from high to low.

[0035] Dynamically adjust the user's personalized recommendation list based on the user's order data collected in real time on the e-commerce platform;

[0036] The inventory objective function is set based on the inventory parameters, expressed as:

[0037]

[0038] Ck=h·g;

[0039] Cq=p·(Dg)+;

[0040] Where Ob represents the calculated value of the inventory objective function, Ck represents the inventory cost, Cq represents the out-of-stock cost, h represents the inventory holding cost per unit, g represents the inventory quantity, p represents the out-of-stock cost per unit, and D represents the demand quantity;

[0041] A Gaussian process regression model is constructed based on the radial basis function as the kernel function. The data combination sampled from the order data, including inventory quantity, demand quantity, corresponding inventory cost and out-of-stock cost, and inventory objective function value, is used as the sampling point data to train the Gaussian process regression model. The kernel function parameters are calculated by the maximum likelihood estimation method, and the predicted mean μ(x) and predicted variance σ(x) of the inventory objective function under different parameter combinations are calculated.

[0042] The next parameter sampling point is selected by the expected improvement algorithm, and the point with the greatest potential improvement is selected by comprehensively considering the predicted mean and predicted variance;

[0043] Compare the calculation of EI values ​​of all sampling points, select the sampling point data with the maximum EI calculation value, calculate the sum of the actual inventory cost and out-of-stock cost, add it to the training data set as new training data to update the Gaussian process regression model, and recalculate the predicted mean and predicted standard deviation;

[0044] Perform iterative calculations and select new parameter combinations to evaluate the inventory objective function. When the improvement of the inventory objective function is no longer obvious during the continuous iteration process, stop the iteration and use the minimum sampling point data of the inventory objective function value as the target data, and use the calculated value of the current inventory as the safety stock value.

[0045] The conversion rate data of the user's personalized recommendation list and the predicted scores of related users are counted to calculate the predicted demand, which is expressed as:

[0046]

[0047] Where f represents the replenishment frequency, L represents the time from order placement to actual receipt of replenishment, Dy represents the predicted demand, Zj represents the conversion rate of product j, and δ represents the total number of products;

[0048] The replenishment strategy for the next cycle is dynamically adjusted through the user's personalized recommendation list in conjunction with the inventory objective function. Based on the calculated safety inventory value and product out-of-stock information, feedback is fed back to the personalized recommendation list, the out-of-stock product recommendation is canceled, and the user's personalized recommendation list is optimized in real time to form a feedback closed loop.

[0049] As a preferred solution of the e-commerce platform order analysis method based on data mining described in the present invention, wherein: the analysis of the mutual influence of demand and price data based on order data through demand function to determine the optimal price includes:

[0050] Based on the demand function, the impact of price on demand is expressed. Based on the demand and price data of the goods in the order data, the demand parameters are determined, which are expressed as:

[0051] Q = α-βP;

[0052]

[0053] Where Q represents the calculated demand for the product, P represents the price of the product, α and β represent the parameters of the demand function, k represents the sample size, Pi and Qi represent the historical price of the i-th product and the demand for the i-th product, respectively. and are the averages of historical price and demand respectively;

[0054] Dynamically adjust the pricing strategy and set the profit function, expressed as:

[0055] Pr(P)=(PC)·(α-βP);

[0056] Where Pr(P) represents the profit of the product, and C represents the unit cost of the product;

[0057] Based on the derivation of the profit function, the optimal profit is calculated, which is expressed as:

[0058]

[0059] Where P* represents the optimal price.

[0060] As a preferred solution of the e-commerce platform order analysis method based on data mining described in the present invention, wherein: the social media data of the e-commerce platform is obtained, and user media evaluation is analyzed through time series, including:

[0061] Obtain social media data related to e-commerce platforms through API interfaces, including user comments, likes, and sharing behaviors;

[0062] Format the collected social media data into structured data tables;

[0063] Export product-related sales data from e-commerce platforms, including sales quantity, sales amount, visits and conversion rate;

[0064] Delete duplicate user comments, irrelevant likes, forwarded messages, and filter out advertisements and spam;

[0065] Use the existing sentiment dictionary to train a custom sentiment dictionary through machine learning algorithms, use the word segmentation tool NLTK to segment social media comments, use the visualization tool Matplotlib to draw word segmentation icons, and analyze user media evaluations through time series.

[0066] As a preferred solution of the e-commerce platform order analysis method based on data mining described in the present invention, wherein: the sorted order data is encrypted and stored in a cloud storage server, and the data is backed up to an external hard disk, including:

[0067] Export e-commerce platform data and social media data, and align the data timestamps;

[0068] Store the organized data on the local hard disk and encrypt it using the BitLocker encryption file system;

[0069] Use cloud storage servers, encrypt data transmission using SSL / TLS protocols, transmit and store data wirelessly, and regularly back up data to external hard drives.

[0070] Another object of the present invention is to provide a system for analyzing e-commerce platform orders based on data mining, which comprises:

[0071] The data processing module collects order-related data and performs batch analysis on large-scale historical order data;

[0072] The data analysis module builds an autoencoder model, uses the model to reconstruct order data and identify features in the data, detects abnormal orders through the reconstruction error of the model, and performs labeling processing;

[0073] User order correlation module, which analyzes the correlation of order data of different users, identifies similarities between users, and generates personalized product recommendation lists;

[0074] Inventory optimization module sets the objective function of minimum inventory cost and out-of-stock cost, and optimizes inventory by combining order data and conversion rate data output by the recommendation system;

[0075] The dynamic pricing module, based on order data, analyzes the mutual influence between demand and price data through demand functions, generates demand forecasting models, and determines the optimal pricing strategy under different circumstances;

[0076] The data security module encrypts the transmission and storage of the collated order data, and regularly backs up the data to multiple external hard drives.

[0077] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned e-commerce platform order analysis method based on data mining when executing the computer program.

[0078] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned e-commerce platform order analysis method based on data mining.

[0079] The beneficial effects of the present invention are as follows: through the multi-layer data lake architecture in conjunction with Lambda, structured and semi-structured data can be extracted from the original data area for large-scale analysis; through the training of the autoencoder's anomaly detection model and precise threshold setting, the system can efficiently identify and process abnormal order data; through the combination of the product recommendation system and the inventory management system, from user demand prediction to inventory strategy formulation, sales and inventory turnover rates are significantly improved, and cost-effectiveness is maximized; by continuously acquiring and analyzing order data, the effect of quickly adjusting product prices is achieved, ensuring that the price strategy always meets the current market environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0081] Figure 1 The figure is a flow chart of the e-commerce platform order analysis method based on data mining.

[0082] Figure 2 This is a structural diagram of the e-commerce platform order analysis system based on data mining. DETAILED DESCRIPTION

[0083] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0084] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0085] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0086] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and this embodiment provides an e-commerce platform order analysis method based on data mining. The e-commerce platform order analysis method based on data mining includes:

[0087] S1, collects order data from e-commerce platforms, builds a multi-layer data lake architecture for data processing, analyzes the data in combination with the Lambda architecture, builds a model through the autoencoder, reconstructs the order data and identifies data features;

[0088] Preferably, collect order data from the e-commerce platform, build a multi-layer data lake architecture for data processing, and analyze the data in combination with the Lambda architecture, including:

[0089] Collect order data from e-commerce platforms and build a multi-layer data lake architecture based on cloud computing data centers, including raw data areas, refined data areas, data warehouse areas, public data areas, work areas, and sensitive data areas;

[0090] The raw data area stores unprocessed raw data, which is stored in its most original form;

[0091] The refined data area stores preprocessed and cleaned data, including missing value filling, outlier removal, and other processing;

[0092] The data warehouse area stores structured data, usually a structured version of the refined data area, to facilitate efficient query;

[0093] The public data area stores cleansed, publicly accessible data for use by users and systems;

[0094] Workspaces create multiple "data pools" for different users or projects for data scientists or engineers to perform data analysis and processing;

[0095] The sensitive data area stores sensitive or private data and applies strict access control policies to ensure data security;

[0096] Import order data into the multi-layer data lake architecture through ETL tools, and divide it into structured data, such as order ID, user ID, product information, order amount, etc., semi-structured data, such as attached user comments or product attributes in JSON format, and unstructured data, such as invoice pictures associated with the order or evaluation videos uploaded by users;

[0097] Parse semi-structured data into standard structured data formats. For example, parse user reviews in JSON format into two fields: text content and rating.

[0098] Unify the format of fields such as timestamp and amount in order data, and summarize the order data;

[0099] Utilize the automatic expansion function of cloud computing data centers to dynamically adjust storage and computing resources according to changes in data volume;

[0100] Building Lambda architecture includes batch processing layer, real-time processing layer and service layer;

[0101] The batch processing layer stores data in a distributed manner based on the Hadoop Distributed File System (HDFS), uses the MapReduce framework for parallel computing, and extracts order data for processing and analysis;

[0102] The real-time processing layer processes new order data based on Apache Spark Streaming and transmits it to the processing system in real time, such as window calculation and state maintenance, to generate a short-term view of the system;

[0103] The service layer performs data query and services through a distributed database.

[0104] Through the regional division and flexible resource allocation of the multi-layer data lake architecture, it can efficiently adapt to various types of data (structured, semi-structured, and unstructured) and achieve dynamic expansion according to business needs. It not only improves the flexibility of data storage and processing, but also ensures that the system can maintain efficient and stable operation as the business grows through cloud computing and containerization technology. Compared with the traditional single-layer data storage method, it provides higher scalability and adaptability, and can meet the complex needs of modern e-commerce platforms for mass data management and processing;

[0105] Through the multi-layer data lake architecture, a unified data storage and management platform is provided for the Lambda architecture. The combination of the two allows the batch processing layer of the Lambda architecture to extract structured and semi-structured data from the original data area for large-scale analysis; the real-time processing layer can directly process streaming data and quickly store the results in the refined data area for the service layer to call, enhancing the operability and practical application value of the data.

[0106] Furthermore, a model is built through an autoencoder to reconstruct the order data and identify data features, including:

[0107] Extract cleaned and normalized order data from the multi-layer data lake architecture, including order ID, order amount, user behavior characteristics, order time, and payment method, and build an anomaly detection model through deep learning autoencoder, including input layer, encoding layer, decoding layer, and output layer;

[0108] The input layer inputs the number of features of the order data, the encoding layer reduces the number of neurons layer by layer, compresses the input data and extracts the key features of the order data, the decoding layer increases the number of neurons layer by layer, restores the compressed feature data to the original data, and the output layer outputs the reconstructed order feature data;

[0109] Using the training set and the mean square error as the loss function, the reconstruction error MSE is calculated, which is expressed as:

[0110]

[0111] Where xi represents the i-th order data input, represents the reconstructed output of the i-th order data, and n represents the total number of samples;

[0112] Select Adam optimizer to optimize model parameters, and use gradient descent method to update model parameters. When the model loss no longer decreases significantly during continuous iterations, stop iterating, output model parameters and update the model.

[0113] Use the data of the model training set and the validation set to input the model for reconstruction, and calculate the reconstruction error of the order data. The reconstruction error standard deviation multiplied by the 95th percentile significance level plus the mean of the reconstruction error is used as the reconstruction threshold;

[0114] New order data is input and reconstructed by the model to output order feature data. If the calculated reconstruction error is greater than the reconstruction threshold, it is judged as data anomaly and the abnormal order data is isolated to the specified abnormal data area.

[0115] Through the training of the autoencoder anomaly detection model and precise threshold setting, the system can efficiently identify and process abnormal order data, ensure the accuracy and real-time nature of order analysis on the e-commerce platform, learn the pattern of normal order data through training models, and identify abnormal orders that are inconsistent with the normal pattern when the model is reconstructed. Traditional anomaly detection methods usually rely on predefined rules or thresholds, which are difficult to adapt to dynamically changing order data. The autoencoder model can adaptively learn the complex patterns of data and automatically identify anomalies in the absence of clear rules. It performs statistical analysis based on the distribution of reconstruction errors, dynamically sets the threshold for anomaly detection, and uses the inverse cumulative distribution function (CDF) of the normal distribution to calculate the threshold, ensuring the accuracy and sensitivity of anomaly detection. It can flexibly adjust the threshold according to the actual distribution of order data, avoiding the problem of underreporting or false alarms that may be caused by static thresholds. Compared with the traditional fixed threshold setting method, the dynamic threshold setting based on the reconstruction error distribution is more flexible and accurate.

[0116] S2, analyzes the correlation of order data of different users, predicts product ratings to generate a recommendation list, sets the objective function of minimum inventory cost and out-of-stock cost, determines the maximum sampling point data as the training data iteration objective function through the expected improvement algorithm, and cooperates with the conversion rate data of the generated recommendation list to determine the safety inventory value and replenishment frequency, forming a feedback closed loop;

[0117] Preferably, the correlation of order data of different users is analyzed, the product ratings are predicted to generate a recommendation list, the objective function of minimum inventory cost and out-of-stock cost is set, the maximum sampling point data is determined as the training data iteration objective function through the expected improvement algorithm, and the conversion rate data of the recommendation list is generated to determine the safety inventory value and replenishment frequency, forming a feedback closed loop, including:

[0118] Based on order data, distinguish the behavior data of different users, including browsing history, click history and purchase history;

[0119] Based on the collaborative filtering algorithm, the correlation of order data of different users is calculated, which is expressed as:

[0120]

[0121] Where u and v represent two different users, ru,i and rv,i represent the ratings of user u and v on item i respectively, m represents the number of items, and S(u,v) represents the correlation between users u and v;

[0122] Based on the calculated similarity, similar users are selected based on the 95th percentile according to the similarity, and the rating data of the selected users on the products are summarized;

[0123] For products that users have not yet rated, the potential rating is predicted by similar users, expressed as:

[0124]

[0125] in represents the predicted rating of user u for product j, rv,j represents the rating of user v for product j, and V represents the set of similar users v of user u;

[0126] Based on user u's predicted rating for product j, a personalized recommendation list is generated by ranking from high to low.

[0127] Dynamically adjust the user's personalized recommendation list based on the user's order data collected in real time on the e-commerce platform;

[0128] The inventory objective function is set based on the inventory parameters, expressed as:

[0129]

[0130] Ck=h·g;

[0131] Cq=p·(Dg)+;

[0132] Where Ob represents the calculated value of the inventory objective function, Ck represents the inventory cost, including holding cost and maintenance cost, Cq represents the out-of-stock cost, including the sales loss caused by out-of-stock, and may also include the potential cost of decreased customer satisfaction, h represents the inventory holding cost per unit, g represents the inventory quantity, p represents the out-of-stock cost per unit, D represents the demand, and (Dg)+ represents the out-of-stock quantity when the demand exceeds the inventory quantity;

[0133] A Gaussian process regression model is constructed based on the radial basis function as the kernel function. The data combination sampled from the order data, including inventory quantity, demand quantity, corresponding inventory cost and out-of-stock cost, and inventory objective function value, is used as the sampling point data to train the Gaussian process regression model. The kernel function parameters are calculated by the maximum likelihood estimation method, and the predicted mean μ(x) and predicted variance σ(x) of the inventory objective function under different parameter combinations are calculated.

[0134] The next parameter sampling point is selected by the Expected Improvement (EI) algorithm, and the point with the greatest potential improvement is selected by comprehensively considering the predicted mean and predicted variance, expressed as:

[0135]

[0136]

[0137] Where EI(x) represents the expected improvement value, f(x+) represents the currently known optimal inventory objective function value, Φ and φ represent the cumulative distribution function and probability density function of the standard normal distribution respectively;

[0138] Where z represents the variable that needs to be input, which represents the standardized difference between the predicted mean and the current optimal inventory objective function in the EI calculation, and t represents the intermediate variable used in the integration process;

[0139] Compare the calculation of EI values ​​of all sampling points, select the sampling point data with the maximum EI calculation value, calculate the sum of the actual inventory cost and out-of-stock cost, add it to the training data set as new training data to update the Gaussian process regression model, and recalculate the predicted mean and predicted standard deviation;

[0140] Perform iterative calculations and select new parameter combinations to evaluate the inventory objective function. When the improvement of the inventory objective function is no longer obvious during the continuous iteration process, stop the iteration and use the minimum sampling point data of the inventory objective function value as the target data, and use the calculated value of the current inventory as the safety stock value.

[0141] The conversion rate data of the user's personalized recommendation list and the predicted scores of related users are counted to calculate the predicted demand, which is expressed as:

[0142]

[0143] Where f represents the replenishment frequency, L represents the time from order placement to actual receipt of replenishment, Dy represents the predicted demand, Zj represents the conversion rate of product j, and δ represents the total number of products;

[0144] The replenishment strategy for the next cycle is dynamically adjusted through the user's personalized recommendation list in conjunction with the inventory objective function. Based on the calculated safety inventory value and product out-of-stock information, feedback is fed back to the personalized recommendation list, the out-of-stock product recommendation is canceled, and the user's personalized recommendation list is optimized in real time to form a feedback closed loop.

[0145] By integrating the behavioral data of similar users into the recommendation algorithm, the recommendation content is highly personalized, which improves the user's click-through rate and purchase rate. The product recommendation list is dynamically updated according to the user's real-time interaction data, ensuring that the recommendation system can quickly adapt to changes in user interests and avoid the problem of invalid or outdated recommendations that may be caused by static recommendation lists. By optimizing the calculated replenishment frequency f and safety stock S, the system can effectively balance the inventory holding cost and out-of-stock risk, ensuring that the inventory level matches the actual demand. Combined with the Gaussian process regression model (GPR), the system can accurately calculate the replenishment frequency and safety stock to ensure that efficient supply chain management can be maintained when demand fluctuates. Accurate inventory management reduces the occurrence of out-of-stock situations, thereby improving customer satisfaction and sales. The inventory management strategy is based on a clear mathematical model and optimization algorithm, ensuring that each decision is fully calculated and verified.

[0146] The combination of the product recommendation system and the inventory management system forms a complete closed loop from user demand prediction to inventory strategy formulation. The output of the recommendation system directly affects the inventory strategy, which enables inventory management to meet actual needs more accurately and avoids the problem of disconnection from actual demand in traditional inventory management. By directly introducing the demand prediction of the product recommendation system into the inventory management system, the system can more accurately predict and prepare inventory, thereby avoiding out-of-stock during peak demand periods and avoiding inventory backlogs during low demand periods. This optimization of supply and demand matching significantly improves sales and inventory turnover rates. The combined system achieves the effect of maximizing cost-effectiveness by optimizing recommendations and inventory strategies. The output of the user recommendation system is directly used for the optimization of the inventory management system. This is a cross-system collaborative optimization that is not available in traditional inventory management methods. This combination not only innovatively utilizes user behavior data, but also introduces a real-time closed-loop feedback mechanism, which enables the system to adjust inventory strategies and recommendation content in real time based on user behavior and market feedback.

[0147] S3, based on the order data, analyzes the mutual influence of demand and price data through the demand function to determine the optimal price;

[0148] Preferably, the interaction between demand and price data is analyzed based on the order data through a demand function to determine the optimal price, including:

[0149] Based on the demand function, the impact of price on demand is expressed. Based on the demand and price data of the goods in the order data, the demand parameters are determined, which are expressed as:

[0150] Q = α-βP;

[0151]

[0152] Where Q represents the calculated demand for the product, P represents the price of the product, α and β represent the parameters of the demand function, k represents the sample size, Pi and Qi represent the historical price of the i-th product and the demand for the i-th product, respectively. and are the averages of historical price and demand respectively;

[0153] Dynamically adjust the pricing strategy and set the profit function, expressed as:

[0154] Pr(P)=(PC)·(α-βP);

[0155] Where Pr(P) represents the profit of the product, and C represents the unit cost of the product;

[0156] Based on the derivation of the profit function, the optimal profit is calculated, which is expressed as:

[0157]

[0158] Where P* represents the optimal price.

[0159] By continuously acquiring and analyzing data such as market demand, user behavior, inventory levels, and competitor prices, the effect of quickly adjusting product prices can be achieved. In combination with user demand, flexible pricing can be performed to ensure that the pricing strategy always meets the current market environment. Through the application of dynamic pricing models, the system can increase prices during peak demand periods to maximize profits, and speed up inventory turnover by lowering prices during low demand periods, further reducing the cost increase caused by inventory occupancy and optimizing the overall efficiency of inventory management. By verifying and adjusting the demand function model through actual sales data, the optimized pricing strategy can better adapt to market dynamics.

[0160] S4, obtains social media data from the e-commerce platform, analyzes user media reviews through time series, organizes order data, encrypts and stores it in a cloud storage server, and backs up the data to an external hard drive;

[0161] Preferably, social media data of the e-commerce platform is obtained, and user media evaluations are analyzed through time series, including:

[0162] Obtain social media data related to e-commerce platforms through API interfaces, including user comments, likes, and sharing behaviors;

[0163] Format the collected social media data into structured data tables;

[0164] Export product-related sales data from e-commerce platforms, including sales volume, sales revenue, visits, conversion rate and other indicators;

[0165] Delete duplicate user comments, irrelevant likes, forwarded messages, and filter out advertisements and spam;

[0166] Use the existing sentiment dictionary to train a custom sentiment dictionary through machine learning algorithms, use the word segmentation tool NLTK to segment social media comments, use the visualization tool Matplotlib to draw word segmentation icons, and analyze user media evaluations through time series.

[0167] By obtaining social media data through the API interface and formatting it into a structured data table, the originally scattered and unstructured social media data can be managed in a unified manner with the e-commerce platform data. By exporting e-commerce platform data such as sales quantity, sales volume, visits, conversion rate, etc., the sales trend and market performance of the product can be comprehensively analyzed. Through in-depth analysis of sales data, e-commerce platforms can formulate more accurate marketing strategies, optimize product pricing, promotional activities and advertising, thereby increasing sales and market share. By deleting duplicate comments, irrelevant likes, forwarded information, and filtering advertisements and spam, the purity and relevance of the data are ensured. By using Matplotlib for data visualization, complex data analysis results can be displayed in the form of intuitive charts, making it easier for decision makers to understand and interpret the data.

[0168] Furthermore, the order data is encrypted and stored in a cloud storage server, and the data is backed up to an external hard drive, including:

[0169] Export e-commerce platform data and social media data, and align the data timestamps;

[0170] Store the organized data on the local hard disk and encrypt it using the BitLocker encryption file system;

[0171] Use cloud storage servers, encrypt data transmission using SSL / TLS protocols, transmit and store data wirelessly, and regularly back up data to external hard drives.

[0172] By aligning the timestamps of e-commerce platform data and social media data, we can ensure that data from different sources are analyzed within the same time window. The full-disk encryption function provided by BitLocker can effectively protect locally stored data. Using the SSL / TLS protocol to encrypt data transmission can prevent it from being intercepted or tampered with by middlemen during wireless transmission, ensuring the confidentiality and integrity of data during transmission. Regularly backing up data to multiple external hard drives can ensure the recoverability of data after emergencies (such as hard drive damage, system failure, and network attacks).

[0173] Example 2, reference Figure 2, which is the second embodiment of the present invention, and which is different from the previous embodiment, provides a system for analyzing an e-commerce platform order based on data mining, including:

[0174] The data processing module collects order-related data and performs batch analysis on large-scale historical order data;

[0175] The data analysis module builds an autoencoder model, uses the model to reconstruct order data and identify features in the data, detects abnormal orders through the reconstruction error of the model, and performs labeling processing;

[0176] User order correlation module, which analyzes the correlation of order data of different users, identifies similarities between users, and generates personalized product recommendation lists;

[0177] Inventory optimization module sets the objective function of minimum inventory cost and out-of-stock cost, and optimizes inventory by combining order data and conversion rate data output by the recommendation system;

[0178] The dynamic pricing module, based on order data, analyzes the mutual influence between demand and price data through demand functions, generates demand forecasting models, and determines the optimal pricing strategy under different circumstances;

[0179] The data security module encrypts the transmission and storage of the collated order data, and regularly backs up the data to multiple external hard drives.

[0180] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0181] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0182] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0183] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0184] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An e-commerce platform order analysis method based on data mining, characterized by: include, Collect order data from e-commerce platforms, build a multi-layer data lake architecture for data processing, analyze the data using the Lambda architecture, build a model through autoencoders, reconstruct order data and identify data features; Analyze the correlation of order data from different users, predict product ratings to generate a recommendation list, set the objective function of minimum inventory cost and out-of-stock cost, determine the maximum sampling point data as the training data iteration objective function through the expected improvement algorithm, and cooperate with the conversion rate data of the generated recommendation list to determine the safety inventory value and replenishment frequency, forming a feedback closed loop; Analyze the mutual influence of demand and price data through demand function based on order data to determine the optimal price; Obtain social media data from e-commerce platforms, analyze user media reviews through time series, organize order data, encrypt and store them in a cloud storage server, and back up the data to an external hard drive; The analysis of the correlation of different user order data, the prediction of product ratings to generate a recommendation list, the setting of the objective function of minimum inventory cost and out-of-stock cost, the determination of the maximum sampling point data as the training data iteration objective function through the expected improvement algorithm, the conversion rate data of the generated recommendation list, the determination of the safety inventory value and the replenishment frequency, and the formation of a feedback closed loop include: Based on order data, distinguish the behavior data of different users, including browsing history, click history and purchase history; Calculate the relevance of order data of different users based on collaborative filtering algorithm; Based on the calculated similarity, similar users are selected based on the 95th percentile according to the similarity, and the rating data of the selected users on the products are summarized; For products that users have not yet rated, the potential rating is predicted by similar users, expressed as: in represents the predicted rating of user u for product j, rv,j represents the rating of user v for product j, V represents the set of similar users v of user u, and S(u,v) represents the correlation between users u and v; Based on user u's predicted rating for product j, a personalized recommendation list is generated by ranking from high to low. Dynamically adjust the user's personalized recommendation list based on the user's order data collected in real time on the e-commerce platform; The inventory objective function is set based on the inventory parameters, expressed as: Ck=h·g; Cq=p·(Dg)+; Where Ob represents the calculated value of the inventory objective function, Ck represents the inventory cost, Cq represents the out-of-stock cost, h represents the inventory holding cost per unit, g represents the inventory quantity, p represents the out-of-stock cost per unit, and D represents the demand quantity; A Gaussian process regression model is constructed based on the radial basis function as the kernel function. The data combination sampled from the order data, including inventory quantity, demand quantity, corresponding inventory cost and out-of-stock cost, and inventory objective function value, is used as the sampling point data to train the Gaussian process regression model. The kernel function parameters are calculated by the maximum likelihood estimation method, and the predicted mean μ(x) and predicted variance σ(x) of the inventory objective function under different parameter combinations are calculated. The next parameter sampling point is selected by the expected improvement algorithm, and the point with the greatest potential improvement is selected by comprehensively considering the predicted mean and predicted variance; Compare the calculation of EI values ​​of all sampling points, select the sampling point data with the maximum EI calculation value, calculate the sum of the actual inventory cost and out-of-stock cost, add it to the training data set as new training data to update the Gaussian process regression model, and recalculate the predicted mean and predicted standard deviation; Perform iterative calculations and select new parameter combinations to evaluate the inventory objective function. When the improvement of the inventory objective function is no longer obvious during the continuous iteration process, stop the iteration and use the minimum sampling point data of the inventory objective function value as the target data, and use the calculated value of the current inventory as the safety stock value. The conversion rate data of the user's personalized recommendation list and the predicted scores of related users are counted to calculate the predicted demand, which is expressed as: Where f represents the replenishment frequency, L represents the time from order placement to actual receipt of replenishment, Dy represents the predicted demand, Zj represents the conversion rate of product j, and δ represents the total number of products; Dynamically adjust the replenishment strategy for the next cycle through the user's personalized recommendation list in conjunction with the inventory objective function. Feedback to the personalized recommendation list based on the calculated safety inventory value and product out-of-stock information, cancel the out-of-stock product recommendation, and optimize the user's personalized recommendation list in real time, forming a feedback closed loop. The method of analyzing the mutual influence of demand quantity and price data through the demand function based on the order data and determining the optimal price includes: Based on the demand function, the impact of price on demand is expressed. Based on the demand and price data of the goods in the order data, the demand parameters are determined, which are expressed as: Q = α-βP; Where Q represents the calculated demand for the product, P represents the price of the product, α and β represent the parameters of the demand function, k represents the sample size, Pi and Qi represent the historical price of the i-th product and the demand for the i-th product, respectively. and are the averages of historical price and demand respectively; Dynamically adjust the pricing strategy and set the profit function, expressed as: Pr(P)=(PC)·(α-βP); Where Pr(P) represents the profit of the product, and C represents the unit cost of the product; Based on the derivation of the profit function, the optimal profit is calculated, which is expressed as: Where P* represents the optimal price.

2. The e-commerce platform order analysis method based on data mining according to claim 1, characterized in that: The method of collecting order data from the e-commerce platform, building a multi-layer data lake architecture for data processing, and combining the Lambda architecture to analyze the data includes: Collect order data from e-commerce platforms and build a multi-layer data lake architecture based on cloud computing data centers, including raw data areas, refined data areas, data warehouse areas, public data areas, work areas, and sensitive data areas; Import order data into the multi-layer data lake architecture through ETL tools, and divide it into structured data, semi-structured data, and unstructured data according to the data structure; Parse semi-structured data into standard structured data formats; Unify the formats of the timestamp and amount fields in the order data and summarize the order data; Utilize the automatic expansion function of cloud computing data centers to dynamically adjust storage and computing resources according to changes in data volume; Building Lambda architecture includes batch processing layer, real-time processing layer and service layer; The batch processing layer stores data in a distributed manner based on the Hadoop distributed file system, uses the MapReduce framework for parallel computing, and extracts order data for processing and analysis; The real-time processing layer processes new order data based on Apache Spark Streaming and transmits it to the processing system in real time; The service layer performs data query and services through a distributed database.

3. The e-commerce platform order analysis method based on data mining according to claim 2, characterized in that: The model is constructed by the autoencoder to reconstruct the order data and identify the data features, including: Extract cleaned and normalized order data from the multi-layer data lake architecture and build an anomaly detection model through deep learning autoencoder, including input layer, encoding layer, decoding layer and output layer; The input layer inputs the number of features of the order data, the encoding layer reduces the number of neurons layer by layer, compresses the input data and extracts the key features of the order data, the decoding layer increases the number of neurons layer by layer, restores the compressed feature data to the original data, and the output layer outputs the reconstructed order feature data; Use the training set and the mean square error as the loss function to calculate the reconstruction error MSE, select the Adam optimizer to optimize the model parameters, and use the gradient descent method to update the model parameters. When the model loss no longer decreases significantly during the continuous iteration process, stop iterating, output the model parameters, and update the model. Use the data of the model training set and the validation set to input the model for reconstruction, and calculate the reconstruction error of the order data. The reconstruction error standard deviation multiplied by the 95th percentile significance level plus the mean of the reconstruction error is used as the reconstruction threshold; New order data is input and reconstructed by the model to output order feature data. If the calculated reconstruction error is greater than the reconstruction threshold, it is judged as data anomaly and the abnormal order data is isolated to the specified abnormal data area.

4. The e-commerce platform order analysis method based on data mining according to claim 3, characterized in that: The social media data of the e-commerce platform is obtained, and the user media evaluation is analyzed through time series, including: Obtain social media data related to e-commerce platforms through API interfaces, including user comments, likes, and sharing behaviors; Format the collected social media data into structured data tables; Export product-related sales data from e-commerce platforms, including sales quantity, sales amount, visits and conversion rate; Delete duplicate user comments, irrelevant likes, forwarded messages, and filter out advertisements and spam; Use the existing sentiment dictionary to train a custom sentiment dictionary through machine learning algorithms, use the word segmentation tool NLTK to segment social media comments, use the visualization tool Matplotlib to draw word segmentation icons, and analyze user media evaluations through time series.

5. The e-commerce platform order analysis method based on data mining according to claim 4, characterized in that: The sorting order data is encrypted and stored in a cloud storage server, and the data is backed up to an external hard disk, including: Export e-commerce platform data and social media data, and align the data timestamps; Store the organized data on the local hard disk and encrypt it using the BitLocker encryption file system; Use cloud storage servers, encrypt data transmission using SSL / TLS protocols, transmit and store data wirelessly, and regularly back up data to external hard drives.

6. A system based on the e-commerce platform order analysis method based on data mining according to any one of claims 1 to 5, characterized in that: include, The data processing module collects order-related data and performs batch analysis on large-scale historical order data; The data analysis module builds an autoencoder model, uses the model to reconstruct order data and identify features in the data, detects abnormal orders through the reconstruction error of the model, and performs labeling processing; User order correlation module, which analyzes the correlation of order data of different users, identifies similarities between users, and generates personalized product recommendation lists; Inventory optimization module sets the objective function of minimum inventory cost and out-of-stock cost, and optimizes inventory by combining order data and conversion rate data output by the recommendation system; The dynamic pricing module, based on order data, analyzes the mutual influence between demand and price data through demand functions, generates demand forecasting models, and determines the optimal pricing strategy under different circumstances; The data security module encrypts the transmission and storage of the collated order data, and regularly backs up the data to multiple external hard drives.

7. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the e-commerce platform order analysis method based on data mining described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the e-commerce platform order analysis method based on data mining described in any one of claims 1 to 5 are implemented.

Images