An internet data intelligent collection method and system

Abstract

The application discloses an internet data intelligent collection method and system, relates to the field of data intelligent collection, and comprises a historical data acquisition module, a data analysis module and an intelligent collection module. The internet historical data set of a user in a website is collected, and the internet historical data set of the user in the website is analyzed, so that the internet data collection control set of each commodity type corresponding to the user in the website is obtained. The data basis is provided for subsequent commodity recommendation corresponding to the user, the accuracy and effectiveness of the internet recommended commodity data collection and acquisition corresponding to the user are indirectly improved, and the timeliness and accuracy of the recommendation decision are further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data intelligent collection, in particular to an internet data intelligent collection method and system. BACKGROUND

[0002] Intelligent collection technology can collect a large amount of data from the Internet extensively and deeply, including areas that traditional data collection cannot cover. This comprehensiveness enables decision makers to have a more comprehensive understanding of market, users, competitors and other information, providing a more solid data foundation for decision making. Intelligent collection technology can collect data from the Internet in real time and update data in a timely manner to ensure that decision makers can obtain the latest information. This real-time nature enables decision makers to respond more quickly to market changes, improving the timeliness and accuracy of decision making. However, the existing technology has the following shortcomings:

[0003] The existing technology lacks intelligent collection and analysis of user demand and user willingness product data when recommending products, resulting in a lack of effectiveness and pertinence in product recommendation, which reduces the accuracy of product recommendation and is not conducive to stimulating users' shopping desire and reducing product sales. SUMMARY

[0004] The present application aims to provide an internet data intelligent collection method and system to solve the problems raised in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: an internet data intelligent collection system, comprising:

[0006] A historical data acquisition module is configured to acquire the historical internet data collection status of a website to obtain an internet historical data set corresponding to the website.

[0007] A data analysis module is configured to analyze the internet historical data set corresponding to the website to obtain an internet data collection control set.

[0008] An intelligent collection module is configured to collect data according to the corresponding internet data collection control set to obtain an internet data collection result.

[0009] In a preferred embodiment of the present application, the historical data acquisition module is implemented as follows:

[0010] The data call log corresponding to the API port is acquired, and the internet historical data set corresponding to the website is acquired through the data call log, wherein the internet historical data set includes time node data and user behavior data, and the user behavior data includes browsing behavior data, search behavior data, social behavior data and purchase behavior data corresponding to each user.

[0011] The time node data refers to a collection time point corresponding to the user behavior data;

[0012] The browsing behavior data includes each commodity page browsed by the user, a starting browsing time point of each commodity page, and a browsing duration corresponding to each commodity page;

[0013] The search behavior data includes a keyword input by the user at each search and a search time point of each search;

[0014] The social behavior data includes a number of evaluation readings corresponding to each browsed commodity page read by the user;

[0015] The purchase behavior data includes each purchased commodity corresponding to the user, a commodity price corresponding to each purchased commodity, a purchase time point corresponding to each purchased commodity, a commodity display image of each purchased commodity, and a commodity name.

[0016] In a preferred embodiment of the present application, the data analysis module is specifically implemented as follows:

[0017] A data extraction relationship between the data analysis module and the database is established, and each commodity corresponding to a commodity type, a commodity display image, a commodity name, and a commodity price stored in the database is extracted;

[0018] A time period between a last purchase commodity time point and a current purchase commodity time point is recorded as a comparison time period of the current purchase commodity by dividing a time period corresponding to adjacent two purchased commodities of the user, and a comparison time period of each purchased commodity corresponding to the user is obtained by statistics, and a duration of the comparison time period of each purchased commodity corresponding to the user is obtained;

[0019] The browsing behavior data, the search behavior data, and the social behavior data corresponding to the target user are divided by a time point of the comparison time period of each purchased commodity corresponding to the user, and a comparison data set corresponding to the comparison time period of each purchased commodity corresponding to the user is obtained;

[0020] A commodity display image, a commodity name, and a commodity price corresponding to each commodity page browsed by each user are obtained by information screening of each commodity page browsed by each user, a commodity type and a commodity price corresponding to each commodity page browsed by each user are obtained by matching the commodity type, the commodity display image, and the commodity name corresponding to each commodity stored in the database, and each commodity type browsing times, a total commodity type browsing duration, a commodity type browsing frequency, a total commodity type browsing duration ratio, and a commodity type browsing price interval within the comparison time period of each purchased commodity corresponding to the user are obtained by data analysis matching in combination with the comparison time period of each purchased commodity corresponding to the user and the duration of the comparison time period;

[0021] The data of each commodity category in the comparison time period of each purchase commodity of the user is obtained by comparing the commodity category corresponding to each commodity page browsed by the user and the comparison time period of each purchase commodity of the user.

[0022] The commodity category corresponding to each keyword stored in the database is extracted.

[0023] The commodity category corresponding to each search of the user is obtained by matching the keyword input by the user in each search with the commodity category corresponding to each keyword stored in the database, and the search times of each commodity category and the search proportion of each commodity category in the comparison time period of each purchase commodity of the user are obtained by matching the comparison time period of each purchase commodity of the user and the length of the comparison time period.

[0024] The browsing times of each commodity category in the comparison time period of each purchase commodity of the user, the total browsing time of each commodity category, the browsing frequency of each commodity category, the total browsing time proportion of each commodity category, the browsing price interval of each commodity category, the search times of each commodity category, the search proportion of each commodity category, and the total evaluation reading times of each commodity category are recorded as the to-be-analyzed data set of the comparison time period of each purchase commodity of the user.

[0025] The commodity category of each purchase commodity of each user is obtained by matching the commodity display image and the commodity name of each purchase commodity of each user with the commodity category, the commodity display image, and the commodity name corresponding to each commodity stored in the database.

[0026] The effective data set in the analysis data set is obtained by matching the to-be-analyzed data set of the comparison time period of each purchase commodity with the commodity category of each purchase commodity of each user, and the effective data set corresponding to each purchase commodity is recorded.

[0027] The effective data set refers to the browsing times of the commodity category, the total browsing time of the commodity category, the browsing frequency of the commodity category, the total browsing time proportion of the commodity category, the browsing price interval of the commodity category, the search times of the commodity category, the search proportion of the commodity category, and the total evaluation reading times of the commodity category corresponding to each purchase commodity.

[0028] The average product price and the product price standard deviation of each product category corresponding to the user are calculated according to the product category and the product price corresponding to each purchase of the user, and the data matching is performed according to the product price, the product category, the product price standard deviation and the product price average of each product category corresponding to each purchase of the product, when the product price corresponding to each purchase of the product is in the standard price interval corresponding to the corresponding product category, it is indicated that the purchase of the product belongs to the effective purchase, and the effective data set of each product category corresponding to each effective purchase is matched and obtained;

[0029] The standard interval refers to the interval that the product price is less than or equal to the product price standard deviation of the product category corresponding to the purchased product + the product price average, and greater than or equal to the product price average of the product category corresponding to the purchased product - the product price standard deviation, and the standard price interval corresponding to each product category is obtained by statistics;

[0030] The effective data set of each product category corresponding to each effective purchase is extracted to obtain the browsing price interval of each product category corresponding to each effective purchase;

[0031] The browsing price interval of each product category corresponding to each effective purchase is compared with the standard price interval corresponding to each product category to obtain the coincidence degree of the browsing price interval of each product category corresponding to each effective purchase and the standard price interval corresponding to each product category, which is recorded as the price coincidence degree of each product category corresponding to each effective purchase;

[0032] The price coincidence degree of each product category corresponding to each effective purchase is compared with the preset price coincidence degree threshold value, if the price coincidence degree is less than the preset price coincidence degree threshold value, it is indicated that the browsing price interval of the product category corresponding to the effective purchase lacks reference significance, if the price coincidence degree is greater than or equal to the preset price coincidence degree threshold value, it is indicated that the browsing price interval of the product category corresponding to the effective purchase has reference significance, and the effective data set corresponding to the purchase is recorded as the reference data set, and each reference data set corresponding to each product category is obtained by statistics;

[0033] The data statistics and data calculation are performed on each reference data set corresponding to each product category to obtain the standard deviation of the product category browsing times, the standard deviation of the product category browsing total time, the standard deviation of the product category browsing frequency, the standard deviation of the product category browsing total time ratio, the standard deviation of the product category browsing price interval, the standard deviation of the search times of the product category, the standard deviation of the search ratio of the product category and the standard deviation of the total number of evaluation reading of the product category;

[0034] perform data statistics on each reference data set corresponding to each commodity category to obtain an interval of commodity category browsing times, an interval of commodity category total browsing time, an interval of commodity category browsing frequency, an interval of commodity category total browsing time proportion, an interval of commodity category search times, an interval of commodity category search proportion, and an interval of total evaluation reading number of commodity category corresponding to each commodity category;

[0035] The standard price interval corresponding to each commodity category, the interval of commodity category browsing times, the interval of commodity category total browsing time, the interval of commodity category browsing frequency, the interval of commodity category total browsing time proportion, the interval of commodity category search times, the interval of commodity category search proportion, and the interval of total evaluation reading number of commodity category corresponding to each commodity category are denoted as an Internet data collection control set.

[0036] In a preferred embodiment of the present scheme, the specific execution mode of the intelligent collection module is as follows:

[0037] extracting the recommendation values corresponding to the commodity category browsing times, the commodity category total browsing time, the commodity category browsing frequency, the commodity category total browsing time proportion, the commodity category search times, the commodity category search proportion, and the total evaluation reading number of commodity category stored in the database;

[0038] obtaining browsing behavior data, search behavior data, and social behavior data after the latest commodity purchase time point, performing data analysis on the browsing behavior data, search behavior data, and social behavior data after the latest commodity purchase time point, and obtaining the commodity category browsing times, the commodity category total browsing time, the commodity category browsing frequency, the commodity category total browsing time proportion, the commodity category search times, the commodity category search proportion, and the total evaluation reading number of commodity category;

[0039] If the commodity category browsing times, the commodity category total browsing time, the commodity category browsing frequency, the commodity category total browsing time proportion, the commodity category search times, the commodity category search proportion, and the total evaluation reading number of commodity category are in the interval of commodity category browsing times, the interval of commodity category total browsing time, the interval of commodity category browsing frequency, the interval of commodity category total browsing time proportion, the interval of commodity category search times, the interval of commodity category search proportion, and the interval of total evaluation reading number of commodity category corresponding to each commodity category, the recommendation values corresponding to the commodity category browsing times, the commodity category total browsing time, the commodity category browsing frequency, the commodity category total browsing time proportion, the commodity category search times, the commodity category search proportion, and the total evaluation reading number of commodity category are obtained;

[0040] The total recommendation value of each commodity category is obtained after the latest commodity purchase time point, and the total recommendation value of each commodity category is compared with a preset total recommendation value threshold, if the total recommendation value of a certain commodity category is less than or equal to the total recommendation value threshold, the commodity category is not recommended, if the total recommendation value of a certain commodity category is greater than the total recommendation value threshold, the commodity category is recommended, the commodity price of each commodity corresponding to the commodity category after the latest commodity purchase time point is compared with the standard price interval corresponding to each commodity category, each commodity meeting the standard price interval corresponding to each commodity category is screened and recorded as the recommended commodity corresponding to each commodity category, and the recommended commodity corresponding to each commodity category is data collected, and the recommended commodity corresponding to each commodity category collected is recorded as the internet data collection result.

[0041] To achieve the above object, the present application further provides the following technical scheme: an internet data intelligent collection method, comprising the following steps:

[0042] S1: obtaining the historical internet data collection status of a website to obtain the internet historical data set corresponding to the website;

[0043] S2: analyzing the internet historical data set corresponding to the website to obtain the internet data collection control set;

[0044] S2: collecting data according to the corresponding internet data collection control set to obtain the internet data collection result.

[0045] Compared with the prior art, the present application has the following beneficial effects:

[0046] The present application collects the internet historical data set of users in a website, and systematically analyzes the internet historical data set of users in the website to obtain the internet data collection control set of each commodity category corresponding to the users in the website, thereby providing a data basis for subsequent commodity recommendation of the users, indirectly improving the accuracy and effectiveness of the internet recommended commodity data collection of the users, and further improving the timeliness and accuracy of the recommendation decision. BRIEF DESCRIPTION OF DRAWINGS

[0047] The present application is further described by using the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 The present application is further described by using the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 2 The present application is further described by using the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present application, and other drawings can be obtained by those skilled in the art without creative labor. DETAILED DESCRIPTION

[0050] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0051] Please refer to Figure 1 The present application provides an intelligent internet data collection system, which comprises a historical data acquisition module, a data analysis module and an intelligent collection module.

[0052] The historical data acquisition module is connected with the data analysis module, and the data analysis module is connected with the intelligent collection module.

[0053] The historical data acquisition module is used for acquiring the historical internet data collection status of a website, and obtaining an internet historical data set corresponding to the website.

[0054] Further, the specific execution mode of the historical data acquisition module is as follows:

[0055] The data calling log corresponding to the API port is acquired, and the internet historical data set corresponding to the website is acquired through the data calling log, wherein the internet historical data set comprises time node data and user behavior data, and the user behavior data comprises browsing behavior data, search behavior data, social behavior data and purchase behavior data corresponding to each user.

[0056] The time node data refers to the collection time point corresponding to the user behavior data.

[0057] The browsing behavior data comprises each commodity page browsed by the user, the starting browsing time point of each commodity page and the browsing time length corresponding to each commodity page.

[0058] The search behavior data comprises the keyword input by the user at each search and the search time point of each search.

[0059] The social behavior data comprises the number of comments read by the user for each browsing commodity page.

[0060] The purchase behavior data comprises each purchased commodity corresponding to the user, the commodity price corresponding to each purchased commodity, the purchase time point corresponding to each purchased commodity, the commodity display picture and the commodity name of each purchased commodity.

[0061] The data analysis module is used for analyzing the internet historical data set corresponding to the website, and obtaining an internet data collection control set.

[0062] Further, the specific execution mode of the data analysis module is as follows:

[0063] establishing a data extraction relationship between the data analysis module and the database, extracting the corresponding commodity categories, commodity display pictures, commodity names and commodity prices of each commodity stored in the database;

[0064] By dividing the time period of the user corresponding to the adjacent two purchase commodities, the time period between the last purchase commodity time point and the current purchase commodity time point is recorded as the comparison time period of the current purchase commodity, and the comparison time period of the user corresponding to each purchase commodity is obtained by statistical analysis, and the duration of the comparison time period of the user corresponding to each purchase commodity is obtained.

[0065] By dividing the time point of the comparison time period of the user corresponding to each purchase commodity, the browsing behavior data, search behavior data and social behavior data corresponding to the target user are obtained, and the comparison data set corresponding to the comparison time period of the user corresponding to each purchase commodity is obtained.

[0066] By filtering the information of each commodity page browsed by each user, the corresponding commodity display pictures, commodity names and commodity prices of each commodity page browsed by each user are obtained, and the corresponding commodity categories and commodity prices of each commodity page browsed by each user are obtained by matching the corresponding commodity categories, commodity display pictures and commodity names stored in the database, and the comparison time period of the user corresponding to each purchase commodity and the duration of the comparison time period are matched by data analysis, and the number of times of browsing each commodity category, the total duration of browsing each commodity category, the browsing frequency of each commodity category, the total duration of browsing each commodity category and the browsing price interval of each commodity category in the comparison time period of the user corresponding to each purchase commodity are obtained.

[0067] By data statistics of the commodity categories corresponding to each commodity page browsed by the user and the comparison time period of each purchase commodity of the user, the total number of evaluation readings of each commodity category in the comparison time period of each purchase commodity of the user is obtained.

[0068] Extracting the corresponding commodity categories of each keyword stored in the database;

[0069] By data matching of the keywords input by the user in each search with the commodity categories corresponding to each keyword stored in the database, the corresponding commodity categories of each search of the user are obtained, and the comparison time period of the user corresponding to each purchase commodity and the duration of the comparison time period are matched by data matching, and the search times of each commodity category and the search proportion of each commodity category in the comparison time period of the user corresponding to each purchase commodity are obtained.

[0070] The browsing times of each product category, the total browsing time of each product category, the browsing frequency of each product category, the proportion of the total browsing time of each product category, the browsing price interval of each product category, the search times of each product category, the search proportion of each product category, and the total number of evaluation reading of each product category of the user in the comparison time period of each purchase product are recorded as the to-be-analyzed data set of the comparison time period of each purchase product corresponding to the user;

[0071] The product display image and the product name of each purchase product corresponding to each user are matched with the product category, the product display image, and the product name corresponding to each product stored in the database to obtain the product category of each purchase product corresponding to each user;

[0072] The to-be-analyzed data set of the comparison time period of each purchase product is matched with the product category of each purchase product corresponding to each user to obtain the effective data set in the analysis data set that is consistent with the product category of each purchase product, which is recorded as the effective data set corresponding to each purchase product;

[0073] The effective data set refers to the browsing times of the product category, the total browsing time of the product category, the browsing frequency of the product category, the proportion of the total browsing time of the product category, the browsing price interval of the product category, the search times of the product category, the search proportion of the product category, and the total number of evaluation reading of the product category corresponding to each purchase product;

[0074] The average product price and the standard deviation of the product price of each product category corresponding to the user are calculated according to the product category and the product price of each purchase product, and the product price, the product category, the standard deviation of the product price, and the average product price of each product category corresponding to each purchase product are matched, when the product price of each purchase product is in the standard price interval corresponding to the corresponding product category, it indicates that the purchase product belongs to effective purchase, and the effective purchase corresponding to each product category is obtained by statistics, and the effective data set of the effective purchase corresponding to each product category is obtained by matching;

[0075] The standard interval refers to the interval in which the product price is less than or equal to the standard deviation of the product price of the product category corresponding to the purchase product plus the average product price, and greater than or equal to the average product price of the product category corresponding to the purchase product minus the standard deviation of the product price, and the standard price interval corresponding to each product category is obtained by statistics;

[0076] The effective data set of each effective purchase corresponding to each product category is extracted to obtain the browsing price interval of each effective purchase corresponding to each product category;

[0077] The data of the browsing price interval corresponding to each valid purchase of each commodity category is compared with the standard price interval corresponding to each commodity category to obtain the coincidence degree of the browsing price interval corresponding to each valid purchase of each commodity category and the standard price interval corresponding to each commodity category, which is recorded as the price coincidence degree of each commodity category corresponding to each valid purchase;

[0078] The price coincidence degree of each commodity category corresponding to each valid purchase is compared with the preset price coincidence degree threshold value. If the price coincidence degree is less than the preset price coincidence degree threshold value, it indicates that the browsing price interval of the corresponding commodity category of the valid purchase lacks reference significance. If the price coincidence degree is greater than or equal to the preset price coincidence degree threshold value, it indicates that the browsing price interval of the corresponding commodity category of the valid purchase has reference significance, and the valid data set corresponding to the purchase is recorded as a reference data set. The reference data set corresponding to each commodity category is obtained by statistical acquisition.

[0079] The reference data set corresponding to each commodity category is subjected to data statistics and data calculation to obtain the standard deviation of the commodity category browsing times, the standard deviation of the commodity category total browsing time, the standard deviation of the commodity category browsing frequency, the standard deviation of the commodity category total browsing time ratio, the standard deviation of the commodity category browsing price interval, the standard deviation of the commodity category search times, the standard deviation of the commodity category search ratio, and the standard deviation of the total number of commodity category evaluation readings.

[0080] The reference data set corresponding to each commodity category is subjected to data statistics to obtain the interval of the commodity category browsing times, the interval of the commodity category total browsing time, the interval of the commodity category browsing frequency, the interval of the commodity category total browsing time ratio, the interval of the commodity category search times, the interval of the commodity category search ratio, and the interval of the total number of commodity category evaluation readings.

[0081] The standard price interval corresponding to each commodity category, the interval of the commodity category browsing times, the interval of the commodity category total browsing time, the interval of the commodity category browsing frequency, the interval of the commodity category total browsing time ratio, the interval of the commodity category search times, the interval of the commodity category search ratio, and the interval of the total number of commodity category evaluation readings are recorded as an internet data acquisition control set.

[0082] The intelligent acquisition module is configured to perform data acquisition according to the corresponding internet data acquisition control set to obtain an internet data acquisition result.

[0083] Further, the specific execution mode of the intelligent acquisition module is as follows:

[0084] extract the recommendation value corresponding to the browse times of the commodity category, the total browse time of the commodity category, the browse frequency of the commodity category, the total browse time proportion of the commodity category, the search times of the commodity category, the search proportion of the commodity category and the total number of evaluation reading of the commodity category from the database;

[0085] obtain the browse behavior data, the search behavior data and the social behavior data after the latest commodity purchase time point, perform data analysis on the browse behavior data, the search behavior data and the social behavior data after the latest commodity purchase time point, and obtain the browse times of each commodity category, the total browse time of each commodity category, the browse frequency of each commodity category, the total browse time proportion of each commodity category, the search times of each commodity category, the search proportion of each commodity category and the total number of evaluation reading of each commodity category;

[0086] if the browse times of each commodity category, the total browse time of each commodity category, the browse frequency of each commodity category, the total browse time proportion of each commodity category, the search times of each commodity category, the search proportion of each commodity category and the total number of evaluation reading of each commodity category are in the interval of the browse times of each commodity category, the interval of the total browse time of each commodity category, the interval of the browse frequency of each commodity category, the interval of the total browse time proportion of each commodity category, the interval of the search times of each commodity category, the interval of the search proportion of each commodity category and the interval of the total number of evaluation reading of each commodity category, the recommendation value corresponding to the browse times of each commodity category, the total browse time of each commodity category, the browse frequency of each commodity category, the total browse time proportion of each commodity category, the search times of each commodity category, the search proportion of each commodity category and the total number of evaluation reading of each commodity category is obtained;

[0087] compare the total recommendation value of each commodity category with a preset total recommendation value threshold, if the total recommendation value of a certain commodity category is less than or equal to the total recommendation value threshold, the commodity category is not recommended, if the total recommendation value of a certain commodity category is greater than the total recommendation value threshold, the commodity category is recommended, compare the commodity prices of each commodity corresponding to the commodity category after the latest commodity purchase time point with the standard price interval corresponding to each commodity category, filter each commodity meeting the standard price interval corresponding to each commodity category and mark as the recommended commodity corresponding to each commodity category, perform data collection on the recommended commodity corresponding to each commodity category, and mark the collected recommended commodity corresponding to each commodity category as the internet data collection result.

[0088] To achieve the above object, the present application further provides the following technical scheme: an internet data intelligent collection method, comprising the following steps:

[0089] S1: obtain the historical internet data collection status of a website, and obtain the internet historical data set corresponding to the website;

[0090] S2: Analyzing the internet history data set corresponding to the website to obtain an internet data collection control set;

[0091] S2: Collecting data according to the corresponding internet data collection control set to obtain an internet data collection result.

[0092] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, so that: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. An intelligent internet data acquisition system, characterized in that: include: Historical data acquisition module: used to acquire the historical Internet data collection status of the website and obtain the corresponding Internet historical dataset of the website; Data Analysis Module: Used to analyze the historical internet dataset corresponding to the website to obtain the internet data collection control set; The average price and standard deviation of each product category for each user's purchase are calculated by taking into account the product category and price of each purchase. Data matching is then performed based on the product price, product category, standard deviation of product price for each product category, and average price of each purchase. If the product price of each purchase falls within the standard price range for the corresponding product category, it is considered a valid purchase. Valid purchases for each product category are statistically obtained, and a valid dataset of valid purchases for each product category is obtained through matching. Data is extracted from the valid datasets corresponding to each valid purchase for each product category to obtain the browsing price range for each valid purchase for each product category; The browsing price range for each valid purchase corresponding to each product category is compared with the standard price range corresponding to each product category to obtain the overlap between the browsing price range for each valid purchase corresponding to each product category and the standard price range corresponding to each product category. This overlap is recorded as the price overlap between each valid purchase corresponding to each product category. The price overlap of each valid purchase for each product category is compared with the preset price overlap threshold. If the price overlap is greater than or equal to the preset price overlap threshold, it means that the price range of the product category for that valid purchase is meaningful. The valid dataset corresponding to that valid purchase is recorded as the reference dataset. The reference datasets corresponding to each product category are statistically obtained. Data statistics were performed on each reference dataset corresponding to each product category to obtain the range of product category browsing times, product category total browsing time, product category browsing frequency, product category total browsing time ratio, product category search times, product category search ratio, and product category total review read count. The standard price range corresponding to each product category, the range of product category browsing times, the range of total browsing time for each product category, the range of product category browsing frequency, the range of product category total browsing time ratio, the range of product category search times, the range of product category search ratio, and the range of total number of product category reviews read are denoted as the Internet data collection control set. Intelligent data acquisition module: Used to acquire data according to the corresponding Internet data acquisition control set and obtain Internet data acquisition results.

2. The Internet data intelligent acquisition system according to claim 1, characterized in that: The specific execution method of the historical data acquisition module is as follows: Obtain the data call logs corresponding to the API port, and obtain the Internet historical dataset corresponding to the website through the data call logs. The Internet historical dataset includes time node data and user behavior data. The user behavior data includes browsing behavior data, search behavior data, social behavior data and purchase behavior data for each user. The time point data refers to the collection time point corresponding to the user behavior data. The browsing behavior data includes each product page browsed by the user, the start time of browsing each product page, and the corresponding browsing duration for each product page; The search behavior data includes the keywords entered by the user in each search and the search time of each search; The social behavior data includes the number of reviews read by users for each product page they browse; The purchase behavior data includes each product purchased by the user, the price of each product purchased, the purchase time of each product purchased, the product display image and product name of each product purchased.

3. The Internet data intelligent acquisition system according to claim 2, characterized in that: The specific execution method of the data analysis module is as follows: Establish the data extraction relationship between the data analysis module and the database, and extract the product categories, product display images, product names and product prices corresponding to each product stored in the database; By dividing the time period into two consecutive purchases by a user, the time period between the previous purchase time and the current purchase time is recorded as the comparison time period for the current purchase. The comparison time periods for each purchase by the user are statistically obtained, and the duration of the comparison time periods for each purchase by the user is obtained. By dividing the browsing behavior data, search behavior data, and social behavior data of the target user into data segments based on the time points of the comparison period of each purchase of goods by the user, a comparison dataset corresponding to the comparison period of each purchase of goods by the user is obtained. By filtering information from each product page viewed by each user, the product display image, product name, and product price corresponding to each product page viewed by each user are obtained. By matching with the product categories, product display images, and product names corresponding to each product stored in the database, the product categories and product prices corresponding to each product page viewed by each user are obtained. Combined with the comparison time period and duration of each user's purchase, data analysis and matching are performed to obtain the number of times each product category was viewed, the total viewing time of each product category, the viewing frequency of each product category, the proportion of the total viewing time of each product category, and the viewing price range of each product category within the comparison time period of each user's purchase. By statistically analyzing the product categories corresponding to each product page browsed by the user and the time period of each purchase, the total number of reviews read for each product category within the comparison time period of each purchase is obtained. Extract the product categories corresponding to each keyword stored in the database; By matching the keywords entered by users during each search with the corresponding product categories stored in the database, the product categories corresponding to each user's search are obtained. By combining the comparison time period and duration of each purchase by the user, the number of searches for each product category and the search ratio of each product category within the comparison time period of each purchase are obtained. The dataset to be analyzed for each user's purchase is recorded as follows: the number of times each product category was viewed, the total viewing time for each product category, the viewing frequency for each product category, the percentage of total viewing time for each product category, the price range for each product category, the number of searches for each product category, the search percentage for each product category, and the total number of reviews read for each product category within the comparison time period. The product display images and product names for each user's purchases are matched with the product categories, product display images, and product names stored in the database to obtain the product categories for each user's purchases. By matching the product categories of each user's purchases with the data of the comparison time period of each purchase, the effective dataset that matches the product categories of each purchase in the analysis dataset is obtained and recorded as the effective dataset corresponding to each purchase. The effective dataset refers to the number of times each product category was viewed, the total viewing time of each product category, the viewing frequency of each product category, the proportion of the total viewing time of each product category, the price range of each product category, the number of searches for each product category, the search ratio of each product category, and the total number of reviews read for each product category. The average price and standard deviation of each product category for each user's purchase are calculated by taking into account the product category and price of each purchase. Data matching is then performed based on the product price, product category, standard deviation of product price for each product category, and average price of each purchase. If the product price of each purchase falls within the standard price range for the corresponding product category, it is considered a valid purchase. Valid purchases for each product category are statistically obtained, and a valid dataset of valid purchases for each product category is obtained through matching. The standard price range refers to the range where the price of a commodity is less than or equal to the standard deviation of commodity prices plus the average price of the commodity corresponding to the purchased commodity category, and is greater than or equal to the average price of the commodity corresponding to the purchased commodity category minus the standard deviation of commodity prices. The standard price range corresponding to each commodity category is obtained by statistical analysis. Data is extracted from the valid datasets corresponding to each valid purchase for each product category to obtain the browsing price range for each valid purchase for each product category; The browsing price range for each valid purchase corresponding to each product category is compared with the standard price range corresponding to each product category to obtain the overlap between the browsing price range for each valid purchase corresponding to each product category and the standard price range corresponding to each product category. This overlap is recorded as the price overlap between each valid purchase corresponding to each product category. The price overlap of each valid purchase for each product category is compared with a preset price overlap threshold. If the price overlap is less than the preset price overlap threshold, it means that the price range of the product category for that valid purchase has no reference value. If the price overlap is greater than or equal to the preset price overlap threshold, it means that the price range of the product category for that valid purchase has reference value. The valid dataset corresponding to that valid purchase is recorded as the reference dataset. The reference datasets corresponding to each product category are statistically obtained. Data statistics and calculations were performed on each reference dataset corresponding to each product category to obtain the standard deviation of the number of times the product category was viewed, the standard deviation of the total viewing time of the product category, the standard deviation of the viewing frequency of the product category, the standard deviation of the proportion of the total viewing time of the product category, the standard deviation of the price range of the product category, the standard deviation of the number of searches for the product category, the standard deviation of the search proportion of the product category, and the standard deviation of the total number of reviews read for the product category. Data statistics were performed on each reference dataset corresponding to each product category to obtain the range of product category browsing times, product category total browsing time, product category browsing frequency, product category total browsing time ratio, product category search times, product category search ratio, and product category total review read count. The standard price range corresponding to each product category, the range of product category browsing times, the range of total browsing time for each product category, the range of product category browsing frequency, the range of product category total browsing time percentage, the range of product category search times, the range of product category search percentage, and the range of total number of product category reviews read are denoted as the Internet data collection control set.

4. The Internet data intelligent acquisition system according to claim 3, characterized in that: The specific execution method of the intelligent data acquisition module is as follows: Extract the recommendation values ​​corresponding to the number of times each product category is viewed, the total viewing time for each product category, the viewing frequency of each product category, the percentage of total viewing time for each product category, the number of searches for each product category, the search percentage for each product category, and the total number of reviews read for each product category stored in the database. We acquire browsing behavior data, search behavior data, and social behavior data since the last purchase time. We then analyze this data to obtain the number of views, total browsing time, browsing frequency, percentage of total browsing time, number of searches, search percentage, and total number of reviews read for each product category. If the number of views, total viewing time, frequency of views, percentage of total viewing time, number of searches, percentage of searches, and total number of reviews for each product category fall within the corresponding ranges for each product category, then the recommended value corresponding to the number of views, total viewing time, frequency of views, percentage of total viewing time, number of searches, percentage of searches, and total number of reviews for each product category is obtained. The system calculates the total recommendation value for each product category after the latest purchase time. This total recommendation value is then compared to a preset threshold. If the total recommendation value for a product category is less than or equal to the threshold, that product category is not recommended. If the total recommendation value is greater than the threshold, that product category is recommended. The system also compares the prices of each product within that category after the latest purchase time with the corresponding standard price range. Products that match the standard price range for each product category are selected and recorded as recommended products for that category. Data is then collected for these recommended products, and the collected data is recorded as the internet data collection results.

5. An intelligent internet data acquisition method, applied to the intelligent internet data acquisition system described in any one of claims 1-4, characterized in that: include: S1: Obtain the historical Internet data collection status of the website to obtain the corresponding historical Internet dataset of the website; S2: Analyze the historical internet dataset corresponding to the website to obtain the internet data collection control set; S2: Collect data according to the corresponding Internet data collection control set to obtain Internet data collection results.

Images