Commodity recommendation method and system based on big data and artificial intelligence

Abstract

The invention belongs to the technical field of commodity recommendation, and particularly relates to a commodity recommendation method and system based on big data and artificial intelligence, and the method comprises the following specific steps: S1, collecting user behavior data and commodity data through multiple channels through a data collection module, and meanwhile, accessing external data to provide richer data sources; s2, constructing a data warehouse through a data storage module by adopting a distributed storage technology; and S3, identifying and deleting duplicated data by using a data processing tool or writing codes through a duplicate removal module, and then processing missing values in the data through a missing value processing module. According to the system, multi-dimensional data including various behavior data of the user on the e-commerce platform, external social media data and the like can be acquired through the data acquisition module, and the interests and preferences of the user can be deeply analyzed in combination with an artificial intelligence algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of commodity recommendation, in particular to a commodity recommendation method and system based on big data and artificial intelligence. BACKGROUND

[0002] In today's booming Internet economy, the e-commerce market continues to expand. According to relevant data, the number of global e-commerce platform goods is in the hundreds of millions, and the number of users is also growing explosively. The massive amount of goods and user information makes it difficult for users to quickly find goods that meet their needs during the shopping process, and businesses need to accurately target customers in the fierce competition to improve the exposure rate and sales conversion rate of goods.

[0003] Traditional commodity recommendation systems, such as rule-based recommendations and simple collaborative filtering recommendations, have limited data processing capabilities and relatively simple algorithms, making it difficult to meet the increasingly diverse and personalized shopping needs of users and adapt to complex and changing market environments. Based on this, the traditional commodity recommendation system has the following problems:

[0004] 1. User individualized needs are difficult to meet: Traditional recommendation systems often rely on basic user information and limited historical behavior data for recommendations, making it difficult to deeply mine user individualized needs. For example, after a photography enthusiast purchases a camera on an e-commerce platform, most of the subsequent recommendations are for similar cameras or accessories, ignoring the user's need for photography courses, photography books, and other related goods. This "one-size-fits-all" approach cannot accurately match users' diverse interests and needs, resulting in low user satisfaction with the recommendations and a poor shopping experience.

[0005] 2. Poor timeliness of recommendations: Users' interests and needs change dynamically with time and context, but traditional recommendation systems have slow data updates and calculation speeds, making it difficult to capture these changes in a timely manner. For example, in the summer, users may frequently browse and purchase summer goods such as sandals and short-sleeved shirts. However, when the season changes to autumn, traditional recommendation systems may still recommend summer goods to users, failing to timely recommend new products such as windbreakers and boots suitable for autumn, causing the recommended content to be out of sync with users' current needs and reducing the effectiveness of the recommendations.

[0006] 3. Data noise and redundancy interfere with recommendation accuracy: In actual e-commerce data, there is a large amount of data noise and redundant information. For example, data generated by user misoperation behavior (such as accidental clicks and accidental purchases), as well as repetitive and irrelevant information in product descriptions, can interfere with the judgment of the user's true needs and the characteristics of the goods by the recommendation system. Traditional recommendation systems have difficulty effectively handling these noise and redundant data, resulting in biased recommendations and reducing the accuracy and reliability of the recommendations.

[0007] Recommended goods lack comparison with similar goods: the traditional recommendation system will recommend similar goods in subsequent recommendations after the user searches for a commodity once, and will not compare the goods needed by the user with similar goods in price and style, resulting in the user purchasing the recommended goods and then discovering other similar goods and returning the goods for purchase again.

[0008] Recommended goods are only the goods themselves: the traditional recommendation system only recommends the goods themselves and does not recommend extended products or products that match the goods, for example, if the user searches for a barbecue grill, only various styles of barbecue grills are recommended, and no matching products such as barbecue sauce or barbecue charcoal are recommended.

[0009] Based on the above, therefore, the application provides a commodity recommendation method and system based on big data and artificial intelligence. SUMMARY

[0010] To solve the above technical problems, according to one aspect of the application, the application provides the following technical solutions:

[0011] A commodity recommendation method based on big data and artificial intelligence, comprising the following specific steps:

[0012] S1: The data acquisition module collects user behavior data and commodity data through multiple channels, and also accesses external data to provide more abundant data sources;

[0013] S2: The data storage module uses distributed storage technology to build a data warehouse;

[0014] S3: First, the duplicate removal module uses data processing tools or writes code to identify and delete duplicate data, then the missing value processing module processes missing values in the data, and then the error correction module checks whether there are incorrect formats, logical errors or abnormal values in the data, and corrects or deletes them according to business rules and data characteristics;

[0015] S4: The feature extraction module extracts information reflecting the characteristics of users and goods from the data, and then the feature selection module selects features that have an impact on the recommendation results from the features, and removes redundant or irrelevant features;

[0016] S5: The user behavior data and commodity data are acquired by the data acquisition module, after acquisition, the corresponding recommendation algorithm is selected according to the demand and data characteristics by the algorithm selection module, after selection, the corresponding model is constructed according to the selected recommendation algorithm by the model construction module, after construction, the acquired data is divided into training set, verification set and test set by the data set division module, after division, the model is trained using the training set by the model training module, after training, the hyperparameters of the model are adjusted using the verification set by the hyperparameter adjustment module, so that the model reaches the best performance on the verification set, after adjustment, the trained model is evaluated using the test set by the evaluation index calculation module, and various evaluation indexes are calculated to reflect the performance of the model from different angles, after evaluation, the evaluation indexes of different models are compared by the model comparison and selection module, and the model with the best performance is selected as the final recommendation model, after comparison and selection, the characteristics of the user and the commodity are input into the finally selected recommendation model by the recommendation generation module, so that the model generates a personalized commodity recommendation list according to the learned knowledge and patterns;

[0017] S6: The target of A / B test is determined by the explicit test module first, then the quantifiable evaluation index is determined according to the test target, after determination, at least one variant version is designed based on the original version by the test scheme design module, each variant has only one key factor different, other factors remain unchanged, then the required sample size is calculated using statistical methods to ensure that the test results have sufficient statistical significance, then how to allocate users to different versions for testing is determined to formulate the shunting rule, after formulation, the test environment is established by the test environment building module first, the behavior data of users on different versions is recorded, then pre-testing is carried out in a small range to check whether the test environment is correctly built, whether the data collection is normal, and whether there are obvious problems or vulnerabilities in the variant version, and necessary adjustment and optimization are carried out according to the results of pre-testing, after checking, the user traffic is guided to different versions according to the predetermined shunting rule by the test execution module, and the formal A / B test is started, then the progress of the test is checked regularly to discover and solve possible problems in time, after regular checking, the collected data is cleaned first by the test analysis module, and the abnormal data and error data are removed, then the data is arranged and summarized for statistical analysis, then the index data of different versions is analyzed using statistical methods to compare whether there are significant differences between each other, then according to the results of statistical analysis, it is judged which version performs better on the target index, at the same time, considering the actual significance and business impact of the test results, it is determined whether it is worth popularizing the version with better performance to the whole user group;

[0018] The feature extraction module extracts information reflecting the characteristics of users and goods from the data, searches for the product and similar products, highlights the differences between the product and similar products, and provides the user with a comparison for selection when purchasing goods;

[0019] The feature extraction module extracts information reflecting the characteristics of users and goods from the data, searches for the product and similar products, highlights the differences between the product and similar products, and provides the user with a comparison for selection when purchasing goods;

[0020] S7: The recommendation result display module displays the recommendation result to the user in a friendly and intuitive manner.

[0021] A product recommendation system based on big data and artificial intelligence, comprising:

[0022] The data acquisition module is used for acquiring user behavior data and product data through multiple channels, and also accesses external data to provide more abundant data sources;

[0023] The data storage module is used for constructing a data warehouse by using a distributed storage technology; for structured data, an HBase database is used for storage; for unstructured data, a MongoDB non-relational database is used for storage; at the same time, a Redis cache database is used for caching frequently accessed data, improving data reading speed and reducing system response time;

[0024] The data processing module is used for processing the stored data;

[0025] The feature engineering module is used for extracting user features and product features from the processed data;

[0026] The recommendation algorithm module is used for using multiple recommendation algorithms to construct a hybrid recommendation model to generate a personalized product recommendation list;

[0027] The A / B test module is used for randomly dividing users into multiple experimental groups and applying different recommendation algorithms or recommendation strategies to different groups of users, and then comparing and analyzing the user behavior data of different groups to evaluate the pros and cons of each recommendation scheme;

[0028] The recommendation result display module is used for displaying the recommendation result to the user in a friendly and intuitive manner.

[0029] As a preferred scheme of the product recommendation system based on big data and artificial intelligence, the data processing module comprises:

[0030] The duplicate removal module is used for identifying and deleting duplicate data by using a data processing tool or writing code;

[0031] a missing value processing module configured to process missing values in the data;

[0032] a correction error module configured to check whether there are errors in the data, such as format errors, logical errors or abnormal values, and correct or delete them according to business rules and data characteristics.

[0033] As a preferred scheme of the commodity recommendation system based on big data and artificial intelligence, the feature engineering module comprises:

[0034] a feature extraction module configured to extract information reflecting user and commodity characteristics from the data;

[0035] a feature selection module configured to select features that have an impact on the recommendation result from the features, and remove redundant or irrelevant features.

[0036] As a preferred scheme of the commodity recommendation system based on big data and artificial intelligence, the recommendation algorithm module comprises:

[0037] a data acquisition module configured to acquire user behavior data and commodity data;

[0038] an algorithm selection module configured to select a corresponding recommendation algorithm according to requirements and data characteristics, wherein the recommendation algorithm comprises a content-based recommendation algorithm, a collaborative filtering algorithm and a deep learning algorithm;

[0039] a model construction module configured to construct a corresponding model according to the selected recommendation algorithm.

[0040] As a preferred scheme of the commodity recommendation system based on big data and artificial intelligence, the recommendation algorithm module further comprises:

[0041] a data set division module configured to divide the acquired data into a training set, a validation set and a test set, wherein the training set is used to train the model, the validation set is used to adjust the hyperparameters of the model, and the test set is used to evaluate the performance of the model;

[0042] a model training module configured to train the model using the training set;

[0043] a hyperparameter adjustment module configured to adjust the hyperparameters of the model through the validation set, so that the model achieves the best performance on the validation set.

[0044] As a preferred scheme of the commodity recommendation system based on big data and artificial intelligence, the recommendation algorithm module further comprises:

[0045] The evaluation index calculation module is configured to evaluate the trained model using the test set and calculate various evaluation indexes to reflect the performance of the model from different angles.

[0046] The model comparison and selection module is configured to compare the evaluation indexes of different models and select the model with the optimal performance as the final recommended model.

[0047] The recommendation generation module is configured to input the features of the user and the goods into the finally selected recommendation model to enable the model to generate an individualized goods recommendation list according to the learned knowledge and patterns.

[0048] As a preferred scheme of the goods recommendation system based on big data and artificial intelligence, the A / B test module comprises:

[0049] The explicit test module is configured to first determine the target of the A / B test and then determine the quantifiable evaluation indexes according to the test target.

[0050] The test scheme design module is configured to first design at least one variant version based on the original version, each variant having only one key factor different from the original version and other factors remaining unchanged, then calculate the required sample size using a statistical method to ensure that the test result has sufficient statistical significance, and then determine how to allocate users to different versions for testing to formulate a shunting rule.

[0051] The test environment building module is configured to first establish a data collection mechanism to record the behavior data of users on different versions, then perform pre-testing in a small range to check whether the test environment is correctly built, whether the data collection is normal, and whether the variant version has obvious problems or vulnerabilities, and make necessary adjustments and optimizations according to the results of the pre-testing.

[0052] The test execution module is configured to first direct user traffic to different versions according to the predetermined shunting rule to start formal A / B testing, and then regularly check the progress of the test to enable timely discovery and solution of possible problems.

[0053] The test analysis module is configured to first clean the collected data to remove abnormal and erroneous data, then arrange and summarize the data for statistical analysis, then analyze the index data of different versions using a statistical method to compare whether there is a significant difference between them, and then determine which version performs better in the target index according to the results of the statistical analysis, while considering the actual significance and business impact of the test results to determine whether it is worth promoting the version with better performance to the entire user group.

[0054] Compared with the prior art, the present application has the following advantages:

[0055] 1. For the problem of difficult to meet the personalized needs of users: the application can collect multi-dimensional data through the data acquisition module, including various behavior data of users on e-commerce platforms, external social media data, etc., and combined with artificial intelligence algorithms, it can deeply analyze user interests and preferences. For example, through the analysis of the behavior data and browsing records of a photography enthusiast after purchasing a camera, not only camera accessories are recommended, but also photography courses, photography books and other related goods are accurately recommended. At the same time, with the recommendation algorithm module, it can realize personalized recommendation from "one size fits all" to "thousand faces", significantly improving the user's satisfaction with the recommendation results and shopping experience.

[0056] 2. For the problem of poor timeliness of recommendation: the application has the functions of rapid data collection, cleaning and extraction through the data processing module and feature engineering module, and combined with corresponding computing technology, it can timely capture user behavior changes. When the season changes, the system can quickly adjust the recommendation strategy according to the user's recent browsing and purchasing data, and timely recommend new products of the season to the user, such as wind clothes and boots in autumn, etc., to ensure that the recommended content always meets the user's current needs and improves the effectiveness and timeliness of the recommendation.

[0057] 3. For the problem of data noise and redundancy interfering with the accuracy of recommendation: the application can deeply clean the data through the data processing module to remove invalid information in user misoperation data and product descriptions; combined with the feature engineering module, it can extract accurate user and product features; and can effectively avoid data noise and redundancy interference, improve the accuracy of the recommendation system in judging user's real needs and product features, and make the recommended results more accurate and reliable.

[0058] Through the comparison of similar goods of the recommended goods: compare the price and style of the goods needed by the user with similar goods, so that the user can compare and select when purchasing goods, and purchase the goods with higher cost performance.

[0059] Through the recommended goods and their extension products or products matched with them, the user can directly purchase the matched goods on the recommended page after purchasing the needed goods. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The figure is a schematic diagram of the overall framework of the application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the application clearer, the embodiments of the application will be described in detail below with reference to the drawings.

[0062] The application provides a commodity recommendation method based on big data and artificial intelligence, please refer to Figure 1 , including the following specific steps:

[0063] S1: Collect user behavior data and commodity data through the data acquisition module using multiple channels, and access external data to provide more abundant data sources;

[0064] S2: Use distributed storage technology to build a data warehouse through the data storage module;

[0065] S3: First, use data processing tools or write code to identify and delete duplicate data through the duplicate removal module, then process missing values in the data through the missing value processing module, and then check for incorrect formats, logical errors or outliers in the data through the error correction module, and correct or delete them according to business rules and data characteristics;

[0066] S4: Extract information reflecting user and commodity characteristics from the data through the feature extraction module, then select features that have an impact on the recommendation result from the features through the feature selection module, and remove redundant or irrelevant features;

[0067] S5: Obtain user behavior data and commodity data through the data acquisition module, then select the corresponding recommendation algorithm according to the requirements and data characteristics through the algorithm selection module, then build the corresponding model according to the selected recommendation algorithm through the model construction module, then divide the obtained data into training set, validation set and test set through the data set division module, then train the model using the training set through the model training module, then adjust the hyperparameters of the model using the validation set through the hyperparameter adjustment module to make the model achieve the best performance on the validation set, then evaluate the trained model using the test set through the evaluation index calculation module, and calculate various evaluation indexes to reflect the performance of the model from different angles, then compare the evaluation indexes of different models through the model comparison and selection module, and select the model with the best performance as the final recommendation model, then input the features of the user and the commodity into the finally selected recommendation model through the recommendation generation module, so that the model can generate a personalized commodity recommendation list according to the learned knowledge and patterns;

[0068] S6: Through the explicit test module, first determine the target of A / B testing, then determine the quantifiable evaluation index according to the test target, after determination, then through the design test scheme module, first based on the original version, design at least one variant version, each variant has only one key factor different, other factors remain unchanged, then use statistical method to calculate the required sample size, to ensure that the test result has enough statistical significance, then determine how to allocate users to different versions for testing, to formulate the shunt rule, after formulating, then through the test environment module, first establish the data collection mechanism, record the user behavior data on different versions, then pretest in a small range, to check whether the test environment is built correctly, whether the data collection is normal, and whether the variant version has obvious problems or vulnerabilities, and according to the results of pretest, make necessary adjustment and optimization, after checking, then through the test module, first according to the predetermined shunt rule, guide the user traffic to different versions, start formal A / B test, then regularly check the progress of the test, to be able to discover and solve possible problems in time, after regular checking, then through the test analysis module, first clean the collected data, remove abnormal data and error data, then arrange and summarize the data, in order to carry on the statistical analysis, then use statistical method to analyze the index data of different versions, compare whether there is significant difference between each other, then according to the results of statistical analysis, judge which version performs better in the target index, at the same time, consider the actual meaning and business influence of the test result, determine whether it is worth promoting the version with better performance to all user groups;

[0069] Through the feature extraction module, information reflecting the characteristics of users and goods is extracted from the data, the product and similar products are searched, and the differences between the product and similar products are highlighted for users to compare and select when purchasing goods;

[0070] Through the feature extraction module, information reflecting the characteristics of goods is extracted from the data, and according to the goods information, extension products or products matched with it are recommended, so that users can directly purchase the matched products on the recommended page after purchasing the required goods;

[0071] S7: Through the recommendation result display module, the recommendation result is displayed to the user in a friendly and intuitive way.

[0072] A commodity recommendation system based on big data and artificial intelligence, comprising: a data acquisition module for acquiring user behavior data and commodity data through multiple channels, and also accessing external data to provide more abundant data sources; a data storage module for using distributed storage technology to build a data warehouse; for structured data, using an HBase database for storage; for unstructured data, using a MongoDB non-relational database for storage; at the same time, using a Redis cache database to cache frequently accessed data, improve data reading speed, and reduce system response time; a data processing module for processing stored data; a feature engineering module for extracting user features and commodity features from processed data; a recommendation algorithm module for using multiple recommendation algorithms to build a hybrid recommendation model to generate a personalized commodity recommendation list; an A / B test module for randomly dividing users into multiple experimental groups and applying different recommendation algorithms or recommendation strategies to different groups, then comparing and analyzing user behavior data of different groups to evaluate the pros and cons of each recommendation scheme; a recommendation result display module for displaying the recommendation result to the user in a friendly and intuitive way.

[0073] The data processing module includes: a duplicate removal module for identifying and deleting duplicate data using data processing tools or writing code; a missing value processing module for processing missing values in the data; a correction error module for checking whether there are incorrect formats, logical errors or abnormal values in the data, and correcting or deleting them according to business rules and data characteristics.

[0074] The feature engineering module includes: a feature extraction module for extracting information reflecting user and commodity features from the data; a feature selection module for selecting features that have an impact on the recommendation result from the features, and removing redundant or irrelevant features.

[0075] The recommendation algorithm module comprises: a data acquisition module for acquiring user behavior data and commodity data; an algorithm selection module for selecting a corresponding recommendation algorithm according to requirements and data characteristics, the recommendation algorithm comprising a content-based recommendation algorithm, a collaborative filtering algorithm and a deep learning algorithm; a model construction module for constructing a corresponding model according to the selected recommendation algorithm; a data set division module for dividing the acquired data into a training set, a validation set and a test set, the training set being used for training the model, the validation set being used for adjusting hyperparameters of the model, and the test set being used for evaluating performance of the model; a model training module for training the model using the training set; a hyperparameter adjustment module for adjusting the hyperparameters of the model through the validation set, so that the model reaches optimal performance on the validation set; an evaluation index calculation module for evaluating the trained model using the test set and calculating various evaluation indexes to reflect performance of the model from different angles; a model comparison and selection module for comparing evaluation indexes of different models and selecting a model with optimal performance as a final recommendation model; and a recommendation generation module for inputting features of a user and commodities into the finally selected recommendation model, so that the model generates an individualized commodity recommendation list according to learned knowledge and patterns.

[0076] The A / B test module comprises: an explicit test module for first determining a target of A / B test, and then determining quantifiable evaluation indexes according to the test target; a test scheme design module for first designing at least one variant version based on an original version, each variant having only one key factor different and other factors consistent, then calculating a required sample size using a statistical method to ensure that a test result has sufficient statistical significance, and then determining how to allocate users into different versions for testing to formulate a shunting rule; a test environment building module for first establishing a data collection mechanism to record behavior data of users on different versions, then performing pre-testing in a small range to check whether a test environment is correctly built, data collection is normal, and variant versions have no obvious problems or vulnerabilities, and making necessary adjustments and optimizations according to a result of pre-testing; an execution test module for first guiding user traffic to different versions according to a predetermined shunting rule to start formal A / B test, then regularly checking a progress of the test to be able to timely discover and solve possible problems; and a test analysis module for first cleaning collected data to remove abnormal data and error data, then arranging and summarizing the data to facilitate statistical analysis, then analyzing index data of different versions using a statistical method to compare whether there are significant differences between each other, and then judging which version performs better on a target index according to a result of statistical analysis, and simultaneously determining whether it is worth popularizing the version performing better to an entire user group considering actual significance and business impact of a test result.

[0077] By setting the A / B test module, the system administrator can timely find the problems of the recommendation algorithm, optimize the recommendation strategy, find the most suitable recommendation scheme for different user groups and business scenarios, and continuously improve the overall performance of the recommendation system.

[0078] Although the present application has been described with reference to the embodiments above, various changes and modifications can be made without departing from the scope of the present application. In particular, the features of the disclosed embodiments can be used in any combination without structural conflict, and the combinations are not exhaustively described in the specification only for the purpose of omitting the length and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A commodity recommendation method based on big data and artificial intelligence, characterized in that: The specific steps are as follows: S1: The data collection module collects user behavior data and product data from multiple channels. It also integrates external data to provide a richer data source. S2: Build a data warehouse using distributed storage technology through the data storage module; S3: First, use data processing tools or code to identify and delete duplicate data through the Deduplication Module. Then, use the Missing Value Processing Module to process missing values. Finally, use the Error Correction Module to check for incorrect formats, logical errors, or outliers in the data, and correct or delete them based on business rules and data characteristics. S4: The feature extraction module extracts information that reflects user and product characteristics from the data. After extraction, the feature selection module selects features that influence the recommendation results and removes redundant or irrelevant features. S5: User behavior data and product data are acquired through the data acquisition module. After acquisition, the corresponding recommendation algorithm will be selected according to the needs and data characteristics through the algorithm selection module. After selection, the corresponding model will be built according to the selected recommendation algorithm through the model construction module. After construction, the acquired data will be divided into training set, validation set and test set through the data set division module. After division, the model will be trained using the training set through the model training module. After training, the hyperparameters of the model will be adjusted using the validation set through the hyperparameter adjustment module to make the model achieve the best performance on the validation set. After adjustment, the trained model will be evaluated using the test set through the evaluation indicator calculation module, and various evaluation indicators will be calculated to reflect the performance of the model from different angles. After evaluation, the evaluation indicators of different models will be compared through the model comparison and selection module, and the model with the best performance will be selected as the final recommendation model. After comparison and selection, the user and product features will be input into the finally selected recommendation model through the recommendation generation module, so that the model can generate a personalized product recommendation list based on the learned knowledge and patterns; S6: First determine the goal of the A / B test through the clear test module, and then determine the quantifiable evaluation indicators based on the test goal. After the determination, the test plan module will be used to design at least one variant version based on the original version. Each variant has only one key factor that is different, and the other factors remain the same. Then use statistical methods to calculate the required sample size to ensure that the test results have sufficient statistical significance. Then determine how to assign users to different versions for testing in order to formulate diversion rules. After formulation, the data collection mechanism will be established through the test environment module to record the user's behavior data on different versions. Then, a pre-test will be conducted on a small scale to check whether the test environment is built correctly, whether the data collection is normal, and whether there are obvious problems or loopholes in the variant version, and according to the pre-test results, the data collection mechanism will be established. The results will be adjusted and optimized as necessary. After the inspection, the test module will be used to guide user traffic to different versions according to the predetermined diversion rules, and the formal A / B test will begin. The progress of the test will then be checked regularly to promptly discover and resolve possible problems. After regular inspections, the collected data will be cleaned and abnormal and erroneous data will be removed through the test analysis module. The data will then be sorted and summarized for statistical analysis. Statistical methods will then be used to analyze the indicator data of different versions to compare whether there are significant differences between them. Based on the results of the statistical analysis, it will be determined which version performs better in terms of the target indicators. At the same time, considering the practical significance and business impact of the test results, it will be determined whether it is worth promoting the better-performing version to the entire user group. The feature extraction module extracts information that reflects the characteristics of users and products from the data, searches for the product and similar products, and highlights the differences between the product and similar products for users to compare and choose when purchasing products; The feature extraction module extracts information about product features from the data and recommends extended products or complementary products based on the product information, so that users can directly purchase complementary products on the recommendation page after purchasing the desired product. S7: The recommendation result display module displays the recommendation results to the user in a friendly and intuitive manner.

2. A product recommendation system based on big data and artificial intelligence, characterized by: include: The data collection module is used to collect user behavior data and product data through multiple channels. It also accesses external data to provide a richer data source. The data storage module is used to build a data warehouse using distributed storage technology. For structured data, the HBase database is used for storage; for unstructured data, the MongoDB non-relational database is used for storage. At the same time, the Redis cache database is used to cache frequently accessed data, improving data reading speed and reducing system response time. A data processing module, used for processing stored data; Feature engineering module, used to extract features from processed data to extract user features and product features; The recommendation algorithm module is used to apply multiple recommendation algorithms to build a hybrid recommendation model to generate a personalized product recommendation list; The A / B testing module is used to randomly divide users into multiple experimental groups and apply different recommendation algorithms or strategies to users in different groups. Then, the advantages and disadvantages of each recommendation scheme are evaluated by comparing and analyzing the user behavior data of different groups. The recommendation result display module is used to display the recommendation results to users in a friendly and intuitive manner.

3. The product recommendation system based on big data and artificial intelligence according to claim 2, characterized in that: The data processing module includes: Deduplication modules, which use data processing tools or code to identify and remove duplicate data; Missing value processing module, used to process missing values ​​in data; The error correction module is used to check whether there are incorrect formats, logical errors or abnormal values ​​in the data, and correct or delete them according to business rules and data characteristics.

4. The product recommendation system based on big data and artificial intelligence according to claim 2, characterized in that: The feature engineering module includes: Feature extraction module, used to extract information that can reflect user and product characteristics from the data; The feature selection module is used to select features that have an impact on the recommendation results and remove redundant or irrelevant features.

5. The product recommendation system based on big data and artificial intelligence according to claim 2, characterized in that: The recommendation algorithm module includes: Data acquisition module, used to acquire user behavior data and product data; An algorithm selection module is used to select a corresponding recommendation algorithm based on demand and data characteristics. The recommendation algorithm includes a content-based recommendation algorithm, a collaborative filtering algorithm, and a deep learning algorithm; The model building module is used to build the corresponding model according to the selected recommendation algorithm.

6. The product recommendation system based on big data and artificial intelligence according to claim 5, characterized in that: The recommendation algorithm module also includes: A data set partitioning module is used to divide the acquired data into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the hyperparameters of the model, and the test set is used to evaluate the performance of the model. Model training module, used to train the model using the training set; The hyperparameter adjustment module is used to adjust the hyperparameters of the model through the validation set so that the model achieves the best performance on the validation set.

7. The product recommendation system based on big data and artificial intelligence according to claim 6, characterized in that: The recommendation algorithm module also includes: The evaluation index calculation module is used to evaluate the trained model using the test set and calculate various evaluation indicators to reflect the performance of the model from different perspectives; Model comparison and selection module, used to compare the evaluation indicators of different models and select the model with the best performance as the final recommendation model; The recommendation generation module is used to input user and product features into the final selected recommendation model so that the model can generate a personalized product recommendation list based on the learned knowledge and patterns.

8. The product recommendation system based on big data and artificial intelligence according to claim 2, characterized in that: The A / B testing module includes: Define the testing module to first determine the goals of the A / B test and then determine quantifiable evaluation indicators based on the test goals; The test plan design module is used to first design at least one variant based on the original version. Each variant differs in only one key factor, while other factors remain the same. Statistical methods are then used to calculate the required sample size to ensure sufficient statistical significance for the test results. The module then determines how to allocate users to different versions for testing and formulates diversion rules. Build a test environment module to first establish a data collection mechanism and record user behavior data on different versions. Then conduct pre-tests on a small scale to check whether the test environment is set up correctly, whether data collection is normal, and whether there are obvious problems or vulnerabilities in the variant versions. Make necessary adjustments and optimizations based on the results of the pre-tests. The test execution module is used to first direct user traffic to different versions according to the predetermined diversion rules, start the formal A / B test, and then regularly check the progress of the test to promptly identify and resolve any potential problems; The test analysis module is used to first clean the collected data, remove abnormal and erroneous data, and then organize and summarize the data for statistical analysis. Statistical methods are then used to analyze the indicator data of different versions to compare whether there are significant differences between them. Based on the results of the statistical analysis, it is determined which version performs better in the target indicators. At the same time, considering the practical significance and business impact of the test results, it is determined whether it is worth promoting the better-performing version to the entire user group.

Images