A method and system for automatically collecting Internet data streams based on demand analysis

Abstract

The present invention discloses an automatic collection method and system for Internet data streams based on demand analysis, and relates to the technical field of information acquisition. It includes: information acquisition: obtaining the user's search interaction records to obtain an interaction count set, the interaction count set contains at least one interaction count item, and based on the interaction count set, respectively obtain the browsing time of the interaction count item to obtain a browsing time set; range division: divide the interaction count set and the browsing time set into at least two ranges, and classify the data corresponding to the interaction count set and the browsing time set into the target range based on the classification method. The present invention combines the user's interaction count range and the browsing time range, sets the comparison relationship between the search interaction and the browsing time according to the priority of the result division, and correspondingly collects the data stream that matches the division result of the comparison relationship between the search interaction and the browsing time, thereby realizing accurate analysis of user interests.

Description

Technical Field

[0001] The present invention relates to the technical field of information acquisition, and in particular to a method and system for automatically collecting Internet data streams based on demand analysis. Background Art

[0002] Internet data flow refers to the flow of information, data or content transmitted through the Internet platform. These data can be in the form of text, pictures, videos, audio and other forms of information. The data generated by users browsing, searching, publishing, sharing and other activities on the Internet can be regarded as Internet data flow. On the Internet, data flow is usually real-time, continuous and highly dynamic. By analyzing Internet data flow, we can understand user needs and behaviors, discover trends and patterns, and perform personalized recommendations and real-time monitoring. The processing and use of Internet data flow are of great significance for user experience optimization, business decision-making, marketing and other aspects. The purpose of collecting Internet data flow is to gain insight into user behavior and needs. By collecting Internet data flow, we can deeply understand users' browsing, searching, interaction and other behaviors, so as to better understand user needs and preferences, and provide decision-making basis for product design and service optimization. In addition, through the collection and analysis of Internet data flow, we can obtain market dynamics and competitive intelligence in a timely manner, discover industry trends and competitors' movements, and provide support for enterprises to formulate strategies. Even the collection and analysis of Internet data flow can also achieve more accurate personalized recommendations and marketing strategies, improve user experience and marketing effects, so it is necessary to collect and analyze data flow.

[0003] The Internet information acquisition method and device with patent publication number CN111209463A describes the Internet information acquisition logic to an acyclic graph, generates the code of the web crawler through the process engine after the process is defined, and realizes Internet information acquisition by executing the web crawler code through the client. Users do not need to perform Python programming. The crawler definition process is code-free and the interface is more friendly. It can lower the threshold for Internet information acquisition and expand the scope of application of the Internet information acquisition method, making it simpler and more convenient.

[0004] In the process of collecting data streams in the above-mentioned and similar technical solutions, a large amount of data will be generated when users browse, search, interact, etc. At this time, the workload of collecting and analyzing all the data will be large, and since the amount of data is large and relatively messy, it is impossible to accurately analyze the user's preferences. In addition, when users browse, search, interact, etc., there will be a correspondence between the search interaction and browsing time of the interactive operation data. If the data of interest to the user is collected and the needs analysis is performed at this time, the user's needs analysis results can be completed more accurately. However, when all interactive operations are collected, the user may not browse after the interaction due to temporary interest. At this time, collecting such data will also affect the user's preference analysis. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for automatically collecting Internet data streams based on demand analysis to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a method and system for automatically collecting Internet data streams based on demand analysis, comprising:

[0007] Information acquisition: obtaining the user's search interaction records to obtain an interaction count set, wherein the interaction count set contains at least one interaction count item, and obtaining the browsing time of the interaction count items based on the interaction count set to obtain a browsing time set;

[0008] Range division: divide the interaction count set and the browsing time set into at least two ranges respectively, and classify the data corresponding to the interaction count set and the browsing time set into the target ranges based on the classification method, so as to obtain at least two count range sets and at least two time range sets;

[0009] Data extraction: Based on the frequency range set and time range set, the corresponding data is collected through extraction to obtain the required data;

[0010] The extraction method includes:

[0011] Step 1: Prioritization, prioritize and sort the frequency range set and the time range set respectively, and obtain a frequency sorting set and a time sorting set respectively. The frequency sorting set contains at least two frequency range sets, and the time sorting set contains at least two time range sets. Based on the frequency sorting set and the time sorting set, the frequency range set and the time range set are prioritized respectively.

[0012] Step 2: Range combination, combining the frequency sorting set with the time sorting set to obtain at least one result merged set, the result merged set at least includes the combination results of two frequency range sets and two time range sets, and sorting the combination results based on the priority number to obtain a result sorted set;

[0013] Step three: Result extraction. Based on the result sorting set, determine the existence mode of the data in the combined results. When the data only exists in the frequency sorting set but not in the time sorting set, extract the target threshold data in the result sorting as the required data. When the data exists in both the frequency sorting set and the time sorting set, preferentially extract the target data as the first data, and extract the target threshold data of the remaining data in the result sorting as the second data. The first data and the second data are combined into the required data. When the data only exists in the time sorting set but not in the frequency sorting set, also extract the target threshold data in the result sorting as the required data.

[0014] Furthermore, the method of extracting the target threshold data in the result sorting includes:

[0015] S1: Target threshold determination, setting the target ratio as the target threshold;

[0016] S2: Corresponding data determination: based on the set target ratio and result sorting set, the data in the result sorting set that is within the target ratio range is used as the target threshold data;

[0017] S3: Data extraction, extracting target threshold data as demand data.

[0018] Furthermore, the method of extracting the target threshold data of the remaining data in the result sorting includes:

[0019] S4: determining proportion data, determining proportion information of the target data in the result sorting set based on the target data, and obtaining a first data proportion;

[0020] S5: determining the remaining data, removing the first data ratio from the target ratio to obtain the remaining data ratio, and determining the proportion of the remaining data in the result sorting based on the remaining data ratio to obtain the second data ratio;

[0021] S6: Data combination, the proportion information corresponding to the combination of the first data ratio and the second data ratio is used as demand data.

[0022] Furthermore, the classification method includes:

[0023] M1: range division, setting at least two interaction count thresholds and at least two browsing time thresholds, and dividing at least two interaction count ranges and two browsing time ranges based on the interaction count thresholds and the browsing time thresholds respectively;

[0024] M2: Data partitioning, based on the interaction times range and browsing time range, the data whose interaction times and browsing time reach the interaction times range and browsing time range respectively are partitioned into the interaction times range and browsing time range respectively, to obtain at least two times partitioning items and at least two time partitioning items, and further obtain at least two times range sets and at least two time range sets.

[0025] Furthermore, the number of interactions and browsing time include invalid number of interactions and invalid time, respectively, and the classification methods include:

[0026] M3: Invalidation judgment: set judgment thresholds respectively, and set the data whose interaction times and browsing time do not reach the judgment thresholds as invalid data;

[0027] M4: Data optimization, based on the frequency partition items and the time partition items, the invalid data is eliminated to obtain the eliminated frequency partition items and the eliminated time partition items, and then the frequency range set and the time range set are optimized.

[0028] Furthermore, the method of prioritizing and sorting includes:

[0029] N1: Basic sorting, based on the interaction count set and browsing time set, the data is initially prioritized according to the interaction count and browsing time respectively;

[0030] N2: Data matching: Based on the data sorting results of the interaction count set and the browsing time set, the sorting results are matched with the data in the count range set and the time range set;

[0031] N3: Priority sorting. Based on the data matching results, the data in the frequency range set and the time range set corresponding to the initial priority sorting are used as the priority sorting standard, and the data with high initial priority in the initial priority sorting is used as the priority data to obtain the frequency sorting set and the time sorting set.

[0032] Furthermore, the method of prioritizing and sorting includes:

[0033] N4: In the process of N1, the demand segment is set, and the data of the interaction number set and the browsing time set in the demand segment are prioritized according to the interaction number and browsing time;

[0034] N5: Repeat the process of N2-N3, and use the result of the priority sorting of the demand segments as the sorting criteria corresponding to the initial priority sorting in the number range set and the time range set to obtain the number sorting set and the time sorting set.

[0035] Furthermore, the method of combining the number sorting set with the time sorting set includes:

[0036] Z1: Feature extraction: Based on the interaction count set and the browsing time set, the features of the interaction count item and the browsing time corresponding to the interaction count item are extracted respectively to obtain two sets of interaction feature items;

[0037] Z2: Feature combination, combine the two sets of interactive feature items respectively to obtain the result merge set.

[0038] Furthermore, a system for automatically collecting Internet data streams based on demand analysis uses the above-mentioned method for automatically collecting Internet data streams based on demand analysis.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The method and system for automatically collecting Internet data streams based on demand analysis respectively set at least two interaction frequency ranges and two browsing time ranges based on the user's search interaction frequency ranges and browsing time, classify the user's search interaction and browsing data based on the interaction frequency ranges and browsing time ranges, and prioritize the classified results. The user's interaction frequency ranges and browsing time ranges are combined, and the comparison relationship between the search interaction and the browsing time is set according to the priority of the result classification. According to the division of the comparison relationship between the search interaction and the browsing time, the corresponding matching data stream is collected, thereby realizing accurate analysis of the user's interests.

[0041] At the same time, when the user's interaction times are at the highest level and the browsing time is at the lowest level, or when the interaction times are at the lowest level and the browsing time is at the highest level, the comparison relationship between the search interaction and the browsing time is divided for the above situations respectively, and the comparison relationship between the search interaction and the browsing time is divided into different levels. According to the level division results, the data flow within the target level is collected, so as to more accurately analyze the user's needs.

[0042] Moreover, when users are performing interactive search and browsing, three situations may occur: first, interactive search but no browsing; second, interactive search and browsing; third, browsing but no interactive search. According to these three situations, the correspondence between the user's search interaction for data and the browsing time is divided respectively, and the corresponding data is extracted according to these three situations, thereby making the data extraction more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the relationship between information acquisition, scope division and data extraction of the present invention;

[0044] Figure 2 It is a schematic diagram of the scope division of the present invention;

[0045] Figure 3 It is a schematic diagram of the extraction method of the present invention;

[0046] Figure 4 It is a schematic diagram of the scope combination relationship of the present invention;

[0047] Figure 5 A schematic diagram of the relationship between the data of the present invention existing in the number of times sorted set;

[0048] Figure 6 A schematic diagram of the relationship between the data of the present invention existing in a time-ordered set;

[0049] Figure 7 The figure is a schematic diagram of the relationship that the data of the present invention exists in both the frequency sorting set and the time sorting set. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] When users browse, search, interact, etc., they may browse or interact with data that they are not interested in due to misoperation. At this time, when such data is collected, it will have a great impact on the results of the user's demand analysis, and may even cause a misjudgment of the user's preferences, resulting in the push of content that the user is not interested in. When the technical solution provided by the present application is used, based on the user's search interaction times and browsing time, at least two interaction frequency ranges and two browsing time ranges are set respectively, and based on the interaction frequency range and the browsing time range, the user's search interaction and browsing data are classified, and the classification results are prioritized, and the user's interaction frequency range is set. And combined with the browsing time range, the comparison relationship between the search interaction and the browsing time is set according to the priority of the result division; according to the division of the comparison relationship between the search interaction and the browsing time, the corresponding data stream is collected; when the user's interaction times range is at the highest level and the browsing time range is at the lowest level, or when the interaction times range is at the lowest level and the browsing time range is at the highest level, the comparison relationship between the search interaction and the browsing time is divided for the above situations respectively, and the comparison relationship between the search interaction and the browsing time is divided into different levels; according to the level division results, the data stream within the target level is collected, so as to more accurately analyze the user's needs.

[0052] like Figure 1-Figure 7As shown, the present invention provides a technical solution: a method and system for automatically collecting Internet data streams based on demand analysis, including: information acquisition: obtaining the user's search interaction records to obtain an interaction frequency set, the interaction frequency set contains at least one interaction frequency item, and based on the interaction frequency set, respectively obtain the browsing time of the interaction frequency item to obtain a browsing time set; range division: dividing the interaction frequency set and the browsing time set into at least two ranges, and classifying the data corresponding to the interaction frequency set and the browsing time set into the target range based on the classification method, respectively, to obtain at least two frequency range sets and at least two time range sets; data extraction: based on the frequency range set and the time range set, collecting the corresponding data by extraction method to obtain the demand data; the extraction method includes: step one: priority division, respectively performing priority division and sorting on the frequency range set and the time range set, respectively obtaining a frequency sorting set and a time sorting set, the frequency sorting set contains at least two frequency range sets, and the time sorting set contains at least two time range sets, And based on the number sorting set and the time sorting set, the number range set and the time range set are prioritized respectively; Step 2: Range combination, combining the number sorting set with the time sorting set to obtain at least one result merged set, the result merged set at least contains the mutual combination results of two number range sets and two time range sets, and the combined results are sorted based on the priority labels to obtain the result sorting set; Step 3: Result extraction, based on the result sorting set, determine the existence mode of the data in the mutual combination results, when the data only exists in the number sorting set but not in the time sorting set, extract the target threshold data in the result sorting as the required data, when the data exists in both the number sorting set and the time sorting set, preferentially extract the target data as the first data, and extract the target threshold data of the remaining data in the result sorting as the second data, the first data and the second data are combined into the required data, when the data only exists in the time sorting set but not in the number sorting set, also extract the target threshold data in the result sorting as the required data.

[0053] It should be noted that by obtaining the user's search interaction records, the interaction count item includes the keywords of the search interaction records and the search interaction record counts of the related keywords, and the browsing time of the user for a single interaction count item is obtained based on the interaction count item. When the user is performing a search interaction, the browsing time is recorded according to the keyword of the search interaction, thereby obtaining a browsing time set, and the interaction count set and the browsing time set are divided into three ranges, wherein the interaction count set is divided into frequent, normal and few, and the browsing time set is divided into long time, medium time and short time, respectively. Three thresholds are set according to the ranges divided by the interaction count set and the browsing time set, wherein the threshold units of frequent, normal and few are times, which are more than five times, three to five times and less than three times respectively, and the threshold units of long time, medium time and short time are time, which are more than five minutes, three to five minutes and less than three minutes respectively. According to the threshold division results of the number set and the browsing time set, the relevant data are divided into ranges, thereby obtaining three number range sets and three time range sets.

[0054] It should be noted that when acquiring information, the number range sets and time range sets are first prioritized and sorted, and the sorted number range sets and time range sets are prioritized. Since there are three number range sets and three time range sets, the priority numbers are also one, two, and three, with one being the highest priority, two being the second, and three being the last. The three number range sets and the three time range sets are marked separately, and the number range sets and the time range sets are combined according to the marking results, so that nine sets of combined results can be obtained, and data is collected based on the combined results.

[0055] It should be noted that when users conduct interactive search and browsing, three situations may occur: first, interactive search but no browsing; second, interactive search and browsing; third, browsing but no interactive search. Therefore, according to these three situations, the corresponding relationship between the user's search interaction for data and the browsing time is divided, among which the second situation is the most interesting, the first situation is the second, and the third situation is the last. At this time, data is extracted according to these three situations. When the data only exists in the frequency sorting set but not in the time sorting set, it means that this is the first situation. At this time, the target threshold data of the three groups of frequency range sets are extracted as the required data, among which the target threshold is the first 50% of the three groups of frequency range sets, which is the first two after rounding. The combination result of the group data, when the data exists in both the frequency sorting set and the time sorting set, this is the second case, and the data in the second case is preferentially extracted as the first data, and the target threshold data of the remaining data in the nine combined results is extracted as the second data. Because in the second case, the data may be mixed in the nine combined results respectively, the target threshold is 50% of the remaining data at this time, and after rounding, it is the combination result of the first six groups of data. The first data and the second data are combined into the required data. When the data only exists in the time sorting set but not in the frequency sorting set, the target threshold data in the three time range sets are extracted as the required data, where the target threshold is the first 50% of the three time range sets, and after rounding, it is the combination result of the first two groups of data.

[0056] like Figure 5 and Figure 6 As shown, the method for extracting the target threshold data in the result sorting includes: S1: target threshold determination, setting the target ratio as the target threshold; S2: corresponding data determination, based on the set target ratio and the result sorting set, the data in the result sorting set that is within the target ratio range is used as the target threshold data; S3: data extraction, extracting the target threshold data as the demand data.

[0057] It should be noted that the set target ratio is the first 50% of the total data. When the data only exists in the frequency sorting set but not in the time sorting set, it means that this is the first case. At this time, the target threshold data in the three groups of frequency range sets are extracted as the required data. When the data only exists in the time sorting set but not in the frequency sorting set, the target threshold data in the three groups of time range sets are extracted as the required data. Therefore, the first 50% of the three groups of frequency range sets and the first 50% of the three groups of time range sets are taken respectively, and after rounding, they are the first two groups respectively.

[0058] like Figure 7As shown, the method for extracting the target threshold data of the remaining data in the result sorting includes: S4: determining the proportion data, determining the proportion information of the target data in the result sorting set based on the target data, and obtaining the first data proportion; S5: determining the remaining data, eliminating the first data proportion in the target proportion, obtaining the remaining data proportion, and determining the proportion information of the remaining data in the result sorting based on the remaining data proportion to obtain the second data proportion; S6: combining data, using the proportion information corresponding to the combination of the first data proportion and the second data proportion as the demand data.

[0059] It should be noted that when data exists in both the frequency sorting set and the time sorting set, the data in the second case is preferentially extracted as the first data, and the target threshold data of the remaining data in the nine groups of combined results are extracted as the second data. Because in the second case, the data may be mixed in the nine groups of combined results respectively, the target threshold is 50% of the remaining data at this time, and after rounding, it is the combined result of the first six groups of data. The first data and the second data are combined into the required data.

[0060] The classification methods include: M1: range division, setting at least two interaction count thresholds and at least two browsing time thresholds, and dividing at least two interaction count ranges and two browsing time ranges based on the interaction count thresholds and the browsing time thresholds; M2: data division, based on the interaction count range and the browsing time range, dividing the data whose interaction count and browsing time reach the interaction count range and the browsing time range respectively into the interaction count range and the browsing time range, obtaining at least two count division items and at least two time division items, and then obtaining at least two count range sets and at least two time range sets.

[0061] It should be noted that when a user is performing a search interaction, the browsing time is recorded according to the keywords of the search interaction, thereby obtaining a browsing time set, and dividing the interaction number set and the browsing time set into three ranges, wherein the interaction number set is divided into frequent, normal and few, and the browsing time set is divided into long, medium and short time, respectively. Three thresholds are set according to the ranges divided by the interaction number set and the browsing time set, wherein the threshold units of frequent, normal and few are times, which are more than five times, three to five times and less than three times respectively, and the threshold units of long, medium and short time are time, which are more than five minutes, three to five minutes and less than three minutes respectively. According to the threshold division results of the interaction number set and the browsing time set, the relevant data are divided into ranges, thereby obtaining three number range sets and three time range sets.

[0062] The number of interactions and browsing time include invalid times and invalid time respectively, and the classification methods include: M3: invalid judgment, respectively setting judgment thresholds, and setting the data whose number of interactions and browsing time do not reach the judgment thresholds as invalid data; M4: data optimization, based on the number division items and the time division items, the invalid data is eliminated to obtain the eliminated number division items and the eliminated time division items, and then the number range set and the time range set are optimized.

[0063] It should be noted that in the process of dividing the interaction count set and the browsing time set range, there will be invalid times and invalid times. For example, when the number of interactions is one, it is set as an invalid number. When the browsing time is less than one minute, it is set as an invalid time. After eliminating the invalid times and invalid time, the number range set and the time range set are optimized, so that the data can be obtained more accurately.

[0064] The priority division and sorting methods include: N1: basic sorting, based on the interaction number set and the browsing time set, the data is initially prioritized according to the interaction number and the browsing time respectively; N2: data matching, based on the data sorting results of the interaction number set and the browsing time set, the sorting results are matched with the data in the number range set and the time range set; N3: priority sorting, based on the data matching results, the data in the number range set and the time range set corresponding to the initial priority sorting is used as the priority sorting standard, and the data with high initial priority in the initial priority sorting is used as the priority data, so as to obtain the number sorting set and the time sorting set.

[0065] It should be noted that the data is first prioritized according to the number of interactions and browsing time, and the data in the interaction number set and the browsing time set are sorted according to the results of the initial priority sorting. Among the three interaction number sets and the three browsing time sets, each interaction number set and the browsing time set are composed of the data that have been initially prioritized, and the initial priority sorting has a priority, so the three interaction number sets and the three browsing time sets will also have a priority, and then the number sorting set and the time sorting set are obtained, which is convenient for more effective data extraction and improves the analysis results of users.

[0066] The priority sorting method includes: N4: in the process of N1, setting the demand segment, and prioritizing the data of the interaction frequency set and the browsing time set within the demand segment according to the interaction frequency and the browsing time; N5: repeating the processes of N2-N3, and using the result of the demand segment priority sorting as the sorting standard corresponding to the initial priority sorting in the frequency range set and the time range set, to obtain the frequency sorting set and the time sorting set.

[0067] It should be noted that the demand segment is a time period. For example, if the demand segment is set to the past six months, the data of the interaction count and browsing time set within the past six months will be obtained, and the demand segments will be prioritized according to the interaction count and browsing time. Compared with counting the total interaction count and browsing time, a more accurate analysis of the data within the demand segment can be performed.

[0068] The method of combining the frequency sorting set with the time sorting set includes: Z1: feature extraction, based on the interaction frequency set and the browsing time set, respectively extracting the features of the interaction frequency items and the browsing time corresponding to the interaction frequency items to obtain two sets of interaction feature items; Z2: feature combination, combining the two sets of interaction feature items respectively to obtain a result merged set.

[0069] A system for automatically collecting Internet data streams based on demand analysis uses the above-mentioned method for automatically collecting Internet data streams based on demand analysis.

[0070] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited by the attached embodiments and their equivalents.

Claims

1. A method for automatically collecting Internet data streams based on demand analysis, characterized in that: include: Information acquisition: obtaining the user's search interaction records to obtain an interaction count set, which includes at least one interaction count item, and obtaining the browsing time of the interaction count items based on the interaction count set to obtain a browsing time set. By obtaining the user's search interaction records, the interaction count items include keywords of the search interaction records and the number of search interaction records of related keywords; Range division: divide the interaction count set and the browsing time set into at least two ranges respectively, and classify the data corresponding to the interaction count set and the browsing time set into the target ranges based on the classification method, so as to obtain at least two count range sets and at least two time range sets; Data extraction: Based on the frequency range set and time range set, the corresponding data is collected through extraction to obtain the required data; The extraction method includes: Step 1: Prioritization, prioritize and sort the frequency range set and the time range set respectively, and obtain a frequency sorting set and a time sorting set respectively. The frequency sorting set contains at least two frequency range sets, and the time sorting set contains at least two time range sets. Based on the frequency sorting set and the time sorting set, the frequency range set and the time range set are prioritized respectively. Step 2: Range combination, combining the frequency sorting set with the time sorting set to obtain at least one result merged set, the result merged set at least includes the combination results of two frequency range sets and two time range sets, and sorting the combination results based on the priority number to obtain a result sorted set; Step three: Result extraction. Based on the result sorting set, determine the existence mode of the data in the combined results. When the data only exists in the frequency sorting set but not in the time sorting set, extract the target threshold data in the result sorting as the required data. When the data exists in both the frequency sorting set and the time sorting set, preferentially extract the target data as the first data, and extract the target threshold data of the remaining data in the result sorting as the second data. The first data and the second data are combined into the required data. When the data only exists in the time sorting set but not in the frequency sorting set, also extract the target threshold data in the result sorting as the required data.

2. The method for automatically collecting Internet data streams based on demand analysis according to claim 1 is characterized in that: The method for extracting the target threshold data in the result sorting includes: S1: Target threshold determination, setting the target ratio as the target threshold; S2: Corresponding data determination: based on the set target ratio and result sorting set, the data in the result sorting set that is within the target ratio range is used as the target threshold data; S3: Data extraction, extracting target threshold data as demand data.

3. The method for automatically collecting Internet data streams based on demand analysis according to claim 2 is characterized in that: The method for extracting the target threshold data of the remaining data in the result sorting includes: S4: determining proportion data, determining proportion information of the target data in the result sorting set based on the target data, and obtaining a first data proportion; S5: determining the remaining data, removing the first data ratio from the target ratio to obtain the remaining data ratio, and determining the proportion of the remaining data in the result sorting based on the remaining data ratio to obtain the second data ratio; S6: Data combination, the proportion information corresponding to the combination of the first data ratio and the second data ratio is used as demand data.

4. The method for automatically collecting Internet data streams based on demand analysis according to claim 1 is characterized in that: The classification methods include: M1: range division, setting at least two interaction count thresholds and at least two browsing time thresholds, and dividing at least two interaction count ranges and two browsing time ranges based on the interaction count thresholds and the browsing time thresholds respectively; M2: Data partitioning, based on the interaction times range and browsing time range, the data whose interaction times and browsing time reach the interaction times range and browsing time range respectively are partitioned into the interaction times range and browsing time range respectively, to obtain at least two times partitioning items and at least two time partitioning items, and further obtain at least two times range sets and at least two time range sets.

5. The method for automatically collecting Internet data streams based on demand analysis according to claim 4 is characterized in that: The number of interactions and browsing time include invalid number of interactions and invalid time, respectively, and the classification methods include: M3: Invalidation judgment: set judgment thresholds respectively, and set the data whose interaction times and browsing time do not reach the judgment thresholds as invalid data; M4: Data optimization, based on the frequency partition items and the time partition items, the invalid data is eliminated to obtain the eliminated frequency partition items and the eliminated time partition items, and then the frequency range set and the time range set are optimized.

6. The method for automatically collecting Internet data streams based on demand analysis according to claim 1 is characterized in that: The method of prioritization and sorting includes: N1: Basic sorting, based on the interaction count set and browsing time set, the data is initially prioritized according to the interaction count and browsing time respectively; N2: Data matching: Based on the data sorting results of the interaction count set and the browsing time set, the sorting results are matched with the data in the count range set and the time range set; N3: Priority sorting. Based on the data matching results, the data in the frequency range set and the time range set corresponding to the initial priority sorting are used as the priority sorting standard, and the data with high initial priority in the initial priority sorting is used as the priority data to obtain the frequency sorting set and the time sorting set.

7. The method for automatically collecting Internet data streams based on demand analysis according to claim 6 is characterized in that: The method of prioritization and sorting includes: N4: In the process of N1, the demand segment is set, and the data of the interaction number set and the browsing time set in the demand segment are prioritized according to the interaction number and browsing time; N5: Repeat the process of N2-N3, and use the result of the priority sorting of the demand segments as the sorting criteria corresponding to the initial priority sorting in the number range set and the time range set to obtain the number sorting set and the time sorting set.

8. The method for automatically collecting Internet data streams based on demand analysis according to claim 1 is characterized in that: The method of combining the frequency sorting set with the time sorting set includes: Z1: Feature extraction: Based on the interaction count set and the browsing time set, the features of the interaction count item and the browsing time corresponding to the interaction count item are extracted respectively to obtain two sets of interaction feature items; Z2: Feature combination, combine the two sets of interactive feature items respectively to obtain the result merge set.

9. An Internet data stream automatic collection system based on demand analysis, characterized in that: A method for automatically collecting Internet data streams based on demand analysis as described in any one of claims 1-8 is used.

Images