Big data-based digital economy data collection method and system

Abstract

The application provides a big data-based digital economy data collection method and system, wherein the method comprises: performing edge processing on the to-be-collected digital economy data stored in the big data nodes according to data acquisition requirements; generating global data features according to the correspondence between the node processing network and the processed data of the edge processing; and collecting the processed data and the analysis result corresponding to the processed data into a database based on the global data feature analysis of the processed data. The big data-based digital economy data collection method and system perform edge processing on the to-be-collected digital economy data stored in the big data nodes, realize the rapid processing of the to-be-collected digital economy data, and have stronger real-time performance; the cloud network performs global analysis on the processed data based on the global data features, can mine the implicit knowledge of the collected data when the user has a digital economy data acquisition requirement, and stores the collected data and the implicit knowledge in the database correspondingly, thereby assisting the user in understanding the collected data and being more user-friendly.

Description

Technical Field

[0001] This invention relates to the field of big data resource service technology, and in particular to a method and system for collecting digital economy data based on big data. Background Technology

[0002] Digital economy data is a core production factor of the digital economy. It refers to digitized information generated, collected, processed, and utilized in digital economic activities, such as user identity information when registering on portal websites, behavioral data generated by users browsing websites, transaction data between enterprises, and enterprise user profiles. The collection of digital economy data is a crucial link in promoting the development of the digital economy. However, the data collection process faces many practical problems. For example, when processing heterogeneous data across platforms in batches, targeted processing is required, resulting in low collection efficiency. Furthermore, users cannot immediately access the implicit information in the collected data, making it less user-friendly.

[0003] In view of this, there is an urgent need for data collection methods and systems for the digital economy based on big data, in order to at least address the above-mentioned shortcomings. Summary of the Invention

[0004] One of the objectives of this invention is to provide a method and system for collecting digital economy data based on big data. First, based on data acquisition needs, edge processing is performed on the digital economy data to be collected stored in big data nodes, enabling rapid processing of the data with enhanced real-time performance. Then, global data features are generated based on the correspondence between the node processing network and the edge processing data. Global analysis of the processed data is then performed using the cloud network based on these global data features. When users have a need to acquire digital economy data, the implicit knowledge in the collected data can be mined and stored together with the collected data in a database, assisting users in understanding the collected data and making the process more user-friendly.

[0005] The digital economy data collection method based on big data provided in this invention includes:

[0006] Based on data acquisition needs, edge processing is performed on the digital economy data to be collected stored in big data nodes;

[0007] Generate global data features based on the correspondence between the node processing network and the edge processing data;

[0008] Data is processed based on global data feature analysis, and the processed data and the corresponding analysis results are collected and stored in the database.

[0009] Preferably, based on data acquisition needs, edge processing is performed on the digital economy data to be collected stored in big data nodes, including:

[0010] Based on the historical data acquisition needs of the demanders, perform pre-response of the node processing network;

[0011] The pre-response was revised based on data acquisition needs, and then connected to the formal response network.

[0012] Edge processing of digital economy data to be collected is based on the formal response network.

[0013] Preferably, based on the historical data acquisition needs of the demanders, the node processing network performs a pre-response, including:

[0014] Step 101: Based on the historical data acquisition requirements, determine the historical processing network sequence within the preset target time period. The historical processing networks in the historical processing network sequence are sorted in order from the closest to the current time according to their processing time.

[0015] Step 102: Based on the overlap of the first n historical processing networks and the overlap of the first n+1 historical processing networks in the historical processing network sequence, determine the change in overlap, where 1≤n≤N-1;

[0016] Step 103 includes:

[0017] If there are T consecutive overlapping cases that remain unchanged, a pre-response is performed based on the historical processing network for the corresponding overlapping cases, where T is greater than... And closest Integers;

[0018] If there are no consecutive T overlapping cases that have not changed, obtain the updated historical processing network sequence, and execute steps 101 to 102 again based on the updated historical processing network sequence. If there are still no consecutive T overlapping cases that have not changed, perform a pre-response based on the historical processing network of the historical processing time closest to the current time.

[0019] Preferably, obtaining updated historical processing network sequences includes:

[0020] Traverse the historical processing networks in the historical processing network sequence in order;

[0021] If the processing time intervals of the historical processing network being traversed and its adjacent historical processing networks are both greater than a preset first threshold, then the historical processing network being traversed will be taken as the target historical processing network.

[0022] After traversal is complete, the target historical processing network is removed from the historical processing network sequence to obtain the updated historical processing network sequence.

[0023] Preferably, data processing based on global data feature analysis includes:

[0024] Based on global data features and cloud-based heterogeneous data understanding templates, data understanding and processing are performed.

[0025] Once the understanding is complete, cloud-based data economy data analysis AI, based on cloud user data and the understood data, predicts the data collection intentions of the demand side.

[0026] Based on the data collection intent, output analysis results to assist in the intent.

[0027] The preferred method for constructing cloud-based heterogeneous data understanding templates is as follows:

[0028] Based on understanding expert nodes from heterogeneous data, a set of rules for parsing and semantic mapping of heterogeneous data from different sources is collected;

[0029] Based on the rule set, construct a template for understanding heterogeneous data in the cloud.

[0030] Preferably, data processing based on global data feature analysis also includes:

[0031] Data economy data analytics AI uses real-time cloud user data for training and updates.

[0032] Preferably, cloud-based data economy data analysis AI predicts the data collection intent of demanders based on cloud user data and understood data, including:

[0033] Identify the intent set of cloud users that matches the user profiles of the demand side;

[0034] Calculate the first data similarity between the first source data for analysis and the understanding data corresponding to the preselected intents in the intent set;

[0035] Based on the preset intent selection rules, the selection intent in the pre-selected intents is determined according to the first data similarity.

[0036] If the selection intent is not unique, the selection intent corresponding to the highest similarity between the second data of the auxiliary analysis data required for the selection intent and the historical analysis data of the demand side shall be used as the data collection intent.

[0037] Preferably, the process of acquiring historical analysis data includes:

[0038] Based on the second source data and the understood data of the pre-selected historical analysis data, extract the analysis dependencies;

[0039] If the dependency extraction is successful, the corresponding pre-selected historical analysis data will be used as the historical analysis data.

[0040] The digital economy data acquisition system based on big data provided in this embodiment of the invention includes:

[0041] The edge processing module is used to perform edge processing on the digital economy data to be collected stored in big data nodes according to data acquisition needs.

[0042] The feature generation module is used to generate global data features based on the correspondence between the processing data of the node processing network and the edge processing.

[0043] The data entry module is used to analyze and process data based on global data features, and to collect the processed data and the corresponding analysis results into the database.

[0044] The beneficial effects of this invention are as follows:

[0045] This invention first performs edge processing on the digital economy data to be collected stored in big data nodes based on data acquisition needs, achieving rapid processing of the digital economy data to be collected with stronger real-time performance; it generates global data features based on the correspondence between the node processing network and the processed data of edge processing, and uses the cloud network to perform global analysis of the processed data based on the global data features. When users have a need to acquire digital economy data, it can mine the implicit knowledge of the collected data and store it in the database along with the collected data, helping users to understand the collected data and making it more user-friendly.

[0046] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0047] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0049] Figure 1 This is a schematic diagram of a digital economy data collection method based on big data in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of a big data-based digital economy data acquisition system in an embodiment of the present invention. Detailed Implementation

[0051] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0052] This invention provides a method for collecting digital economy data based on big data, such as... Figure 1 As shown, it includes:

[0053] Step 1: Based on the data acquisition requirements, perform edge processing on the digital economy data to be collected stored in the big data nodes;

[0054] The data acquisition needs are as follows: the need to acquire digital economy data, such as the need to acquire supply chain information of a certain industry in region A; big data nodes are distributed physical or virtual server units that store digital economy data, each node having independent computing and storage capabilities, such as edge servers; the digital economy data to be collected is the data corresponding to the data acquisition needs from all the digital economy data stored in the big data nodes (such as the industrial chain data of various industries in region A); edge processing refers to: when the big data nodes upload the corresponding required data to the data acquisition demander, the data to be uploaded is first processed at the edge server corresponding to the big data nodes (such as filtering invalid supplier information);

[0055] Step 2: Generate global data features based on the correspondence between the node processing network and the edge processing data;

[0056] Among them, the node processing network is a node network composed of edge server nodes that perform edge processing; the global data features are the results obtained by characterizing the data after edge processing of the digital economy data to be collected by the node processing network, such as data type, data structure, and the edge server corresponding to the data.

[0057] Step 3: Analyze and process the data based on global data features, and collect the processed data and the corresponding analysis results into the database.

[0058] In the process of analyzing and processing data based on global data features, cloud-based AI is used to integrate multi-source data according to global data features, and the processed data and its analysis results are collected and stored in the database.

[0059] The working principle and beneficial effects of the above technical solution are as follows:

[0060] This invention first performs edge processing on the digital economy data to be collected stored in big data nodes based on data acquisition needs, achieving rapid processing of the digital economy data to be collected with stronger real-time performance; it generates global data features based on the correspondence between the node processing network and the processed data of edge processing, and uses the cloud network to perform global analysis of the processed data based on the global data features. When users have a need to acquire digital economy data, it can mine the implicit knowledge of the collected data and store it in the database along with the collected data, helping users to understand the collected data and making it more user-friendly.

[0061] In one embodiment, edge processing is performed on the digital economy data to be collected stored in big data nodes according to data acquisition needs, including:

[0062] Based on the historical data acquisition needs of the demanders, perform pre-response of the node processing network;

[0063] Among them, the demand side for data acquisition is: the subject that needs to acquire data, such as: a data analyst or a company manager; the historical data acquisition demand is: the record of the demand side's historical acquisition of digital economy data, and the cutoff time of the historical data is set manually according to the demand, such as: within one month before the current time; the pre-response of the node processing network refers to: predicting the current data acquisition demand based on the historical data acquisition demand, and matching the node processing network according to the predicted current data acquisition demand, and generating the communication queue of the edge server node corresponding to the matching node processing network;

[0064] The pre-response was revised based on data acquisition needs, and then connected to the formal response network.

[0065] Among them, modifying the pre-response based on data acquisition needs refers to adjusting the pre-response according to the actual data acquisition needs. For example, locally adjusting the communication instructions in the communication queue of the corresponding edge server node in the matching node processing network; the formal response network is the network jointly formed by the edge server nodes that respond after the modified pre-response is issued.

[0066] Edge processing of digital economy data to be collected is based on the formal response network.

[0067] The working principle and beneficial effects of the above technical solution are as follows:

[0068] This invention introduces a pre-response mechanism for the node processing network, based on the historical data acquisition needs of the data acquisition requesting party. The pre-response refers to the predicted communication mechanism between edge server nodes and the system that may require interfacing. Possible communication mechanisms are generated first, and then adjusted based on the data acquisition needs to achieve communication with the formal response network, thus improving communication efficiency.

[0069] In one embodiment, based on the historical data acquisition needs of the demanding party, the node processing network performs a pre-response, including:

[0070] Step 101: Based on the historical data acquisition requirements, determine the historical processing network sequence within the preset target time period. The historical processing networks in the historical processing network sequence are sorted in order from the closest to the current time according to their processing time.

[0071] The preset target duration is manually set, for example, within one month prior to the current time.

[0072] Step 102: Based on the overlap of the first n historical processing networks and the overlap of the first n+1 historical processing networks in the historical processing network sequence, determine the change in overlap, where 1≤n≤N-1;

[0073] Among them, the overlapping cases are: overlapping edge server nodes;

[0074] Step 103 includes:

[0075] If there are T consecutive overlapping cases that remain unchanged, a pre-response is performed based on the historical processing network for the corresponding overlapping cases, where T is greater than... And closest Integers;

[0076] If there are T consecutive overlapping cases that have not changed, it means that the access demand of the demand side has been relatively stable in the recent period of time. The communication instructions of the corresponding edge server nodes of the network are cached for the historical overlapping cases.

[0077] If there are no consecutive T overlapping cases that have not changed, obtain the updated historical processing network sequence, and execute steps 101 to 102 again based on the updated historical processing network sequence. If there are still no consecutive T overlapping cases that have not changed, perform a pre-response based on the historical processing network of the historical processing time closest to the current time.

[0078] If there are no consecutive T overlapping cases that remain unchanged, there are two scenarios:

[0079] The first type is where there are specific historical processing networks in the historical processing network sequence. In this case, it is necessary to remove them to eliminate interference and obtain the updated historical processing network. Specific historical processing networks refer to those historical processing networks with low similarity to other historical processing networks (e.g., network node similarity is less than 20%).

[0080] The second scenario involves situations where the demand from the client changes significantly (and the stable demand cannot be determined based on the updated historical processing network). In such cases, a pre-response is performed directly based on the historical processing network of the most recent historical processing moment.

[0081] Among these, obtaining the updated historical processing network sequence includes:

[0082] Traverse the historical processing networks in the historical processing network sequence in order;

[0083] Here, "in sequence" refers to the process from the beginning to the end of the historical network sequence.

[0084] If the processing time intervals of the historical processing network being traversed and its adjacent historical processing networks are both greater than a preset first threshold, then the historical processing network being traversed will be taken as the target historical processing network.

[0085] The preset first threshold is the average time interval between the processing times of two adjacent historical processing networks in the historical processing network sequence.

[0086] After traversal is complete, the target historical processing network is removed from the historical processing network sequence to obtain the updated historical processing network sequence.

[0087] The working principle and beneficial effects of the above technical solution are as follows:

[0088] Based on the need for historical data acquisition, this invention determines the historical processing network sequence, and the overlap of historical processing networks in the historical processing network sequence represents the network situation of repeated access in history.

[0089] Generally, the more historical processing networks selected, the more uncertainties arise. Therefore, in this embodiment of the invention, historical processing networks are selected in order of proximity to the current time, and the overlap is determined based on the overlap of the first n historical processing networks and the overlap of the first n+1 historical processing networks. If there are no changes in T consecutive overlaps, and T is greater than... And closest If the integer N is the number of historical processing networks in the historical processing network sequence, then there are enough consecutive unchanging overlapping cases within the selected historical time length, which improves the accuracy of the pre-response basis extraction.

[0090] Additionally, if there are no consecutive T instances of unchanged overlap, consider two scenarios:

[0091] The first approach involves identifying historical processing networks with specific characteristics within the historical processing network sequence. These networks need to be removed to eliminate interference and obtain a newer historical processing network. During removal, the historical processing networks in the sequence are traversed sequentially. If the processing time interval between the currently traversed historical processing network and its adjacent historical processing networks is greater than a first threshold, it indicates that the corresponding demand for that historical processing network is temporally isolated (the historical data collection demand of the corresponding historical processing network is not continuous with its surrounding demands in the time dimension), and its specificity is high, thus it is removed. After all historical processing networks have been identified, the remaining historical processing networks are sorted according to their processing time from latest to earliest to obtain a newer historical processing network sequence. This update process for the historical processing network sequence is more reasonable.

[0092] The second scenario involves situations where the demand from the client changes significantly (and the updated historical processing network still cannot determine stable access demands). In such cases, a pre-response is performed directly based on the historical processing network of the most recent historical processing moment.

[0093] This invention provides a reasonable basis for determining the pre-response based on the adaptability of different overlapping situations, thereby improving the suitability of the pre-response.

[0094] In one embodiment, data processing based on global data feature analysis includes:

[0095] Based on global data features and cloud-based heterogeneous data understanding templates, data understanding and processing are performed.

[0096] Once the understanding is complete, cloud-based data economy data analysis AI, based on cloud user data and the understood data, predicts the data collection intentions of the demand side.

[0097] Based on the data collection intent, output analysis results to assist in the intent.

[0098] Among them, the cloud refers to a cloud service platform that provides data services for the data economy;

[0099] Among them, the cloud-based heterogeneous data understanding template is a set of rules pre-configured by heterogeneous data understanding experts for parsing and semantic mapping of heterogeneous data from different sources. Its function is to standardize the processing of data with different structures and sources.

[0100] Among them, cloud user data refers to user data uploaded by cloud users that is accessible to developers (such as user cookies);

[0101] Among them, understanding the data refers to the semantics of the data processed by each edge device based on the global data features, such as: the production capacity and delivery cycle of upstream auto parts suppliers in the automotive industry in region A, and the sales speed of downstream dealers in the automotive industry in region A.

[0102] Among them, the data economy data analysis AI is: a cloud-based AI large model, which is trained and acquired based on the analysis and records of artificial data economy data. In addition, the data economy data analysis AI will also learn from real-time cloud user data to achieve self-learning of the model.

[0103] When predicting the data collection intent of the demand side, AI compares the user profile in the cloud with the user profile of the demand side, and also combines the historical data of the demand side for prediction.

[0104] The working principle and beneficial effects of the above technical solution are as follows:

[0105] This invention introduces a cloud-based heterogeneous data understanding template to understand the processed data. After understanding, the understood data is input into the cloud-based data economy data analysis AI to predict the data acquisition needs of the demand side and output analysis results to assist in the intent, making it more intelligent.

[0106] In one embodiment, a cloud-based data economy data analysis AI predicts the data collection intent of demanders based on cloud-based user data and understood data, including:

[0107] Identify the intent set of cloud users that matches the user profiles of the demand side;

[0108] Among them, profile matching refers to the similarity between the user profiles of the demand side and the cloud user in the cloud being greater than or equal to a preset second threshold; the intent set is the collection of historical intents of the corresponding matched cloud user;

[0109] Calculate the first data similarity between the first source data for analysis and the understanding data corresponding to the preselected intents in the intent set;

[0110] Among them, the first source data for analysis corresponding to the preselected intent is: the digital economy data input by the corresponding matched cloud user when performing cloud analysis of the corresponding historical intent; the first data similarity is: the degree of similarity between the digital economy data input by the corresponding matched cloud user when performing cloud analysis of the corresponding historical intent and the digital economy data understood by the understanding data.

[0111] Based on the preset intent selection rules, the selection intent in the pre-selected intents is determined according to the first data similarity.

[0112] The preset intent selection rule is as follows: if the similarity of the first data is greater than the preset third threshold, then the corresponding pre-selected intent will be selected as the selected intent.

[0113] If the selection intent is not unique, the selection intent corresponding to the highest similarity between the second data of the auxiliary analysis data required for the selection intent and the historical analysis data of the demand side shall be used as the data collection intent;

[0114] Among them, the auxiliary analysis data required for selecting intent includes: process analysis data for assisting in the analysis of the selected intent, such as: the thinking process data shown by the AI ​​model when outputting its final answer result; the historical analysis data of the demand side includes: process analysis data of the AI ​​model used by the demand side in the past to conduct data economy data analysis; the second data similarity is the similarity of the semantic understanding of the analysis data.

[0115] The process of acquiring historical analysis data includes:

[0116] Based on the second source data and the understood data of the pre-selected historical analysis data, extract the analysis dependencies;

[0117] The pre-selected historical analysis data consists of all analysis process data stored in the cloud by the demand side; the second source analysis data consists of digital economy data analyzed from the pre-selected historical analysis data; the analysis dependency relationship is: which type of digital economy data is analyzed first, and which type of digital economy data is analyzed next, for example: first calculate the digital payment penetration rate in a certain region, and then further analyze whether the influencing factors of digital payment usage are reasonable;

[0118] If the dependency extraction is successful, the corresponding pre-selected historical analysis data will be used as the historical analysis data.

[0119] The working principle and beneficial effects of the above technical solution are as follows:

[0120] This invention matches cloud users, determines the intent set of cloud users matched with profiles, and determines the selection intent based on the similarity between the data economic data from the pre-selected intent analysis source and the data economic data required by the demand side, as well as intent selection rules. When the selection intent is unique, it is directly used as the data collection intent; otherwise, the similarity of the semantic understanding between the process analysis data that assists in the selection intent analysis and the process analysis data of the AI ​​model that historically conducted data economic data analysis on the demand side is calculated. Since the second analysis source data of the historical analysis data is dependent on the understanding data, the greater the similarity of the semantic part of the model's thinking content, the more likely the final data collection intent of the current analysis process that depends on the corresponding historical analysis data is the corresponding selection intent. This invention introduces profile matching, the first analysis source data of the pre-selected intent, the auxiliary analysis data required for the selection intent, and the historical analysis data with analysis dependencies to collaboratively predict the data collection intent, resulting in more accurate intent prediction.

[0121] This invention provides a digital economy data acquisition system based on big data, such as... Figure 2 As shown, it includes:

[0122] Edge processing module 1 is used to perform edge processing on the digital economy data to be collected stored in big data nodes according to data acquisition needs.

[0123] Feature generation module 2 is used to generate global data features based on the correspondence between the node processing network and the edge processing data;

[0124] The data entry module 3 is used to analyze and process data based on global data features, and to collect and enter the processed data and the corresponding analysis results into the database.

[0125] The edge processing module performs edge processing on the digital economy data to be collected stored in the big data nodes according to data acquisition needs, including:

[0126] Based on the historical data acquisition needs of the demanders, perform pre-response of the node processing network;

[0127] The pre-response was revised based on data acquisition needs, and then connected to the formal response network.

[0128] Edge processing of digital economy data to be collected is based on the formal response network.

[0129] This includes, based on the historical data acquisition needs of the demanders, performing pre-response for the node processing network, including:

[0130] Step 101: Based on the historical data acquisition requirements, determine the historical processing network sequence within the preset target time period. The historical processing networks in the historical processing network sequence are sorted in order from the closest to the current time according to their processing time.

[0131] Step 102: Based on the overlap of the first n historical processing networks and the overlap of the first n+1 historical processing networks in the historical processing network sequence, determine the change in overlap, where 1≤n≤N-1;

[0132] Step 103 includes:

[0133] If there are T consecutive overlapping cases that remain unchanged, a pre-response is performed based on the historical processing network for the corresponding overlapping cases, where T is greater than... And closest Integers;

[0134] If there are no consecutive T overlapping cases that have not changed, obtain the updated historical processing network sequence, and execute steps 101 to 102 again based on the updated historical processing network sequence. If there are still no consecutive T overlapping cases that have not changed, perform a pre-response based on the historical processing network of the historical processing time closest to the current time.

[0135] Among these, obtaining the updated historical processing network sequence includes:

[0136] Traverse the historical processing networks in the historical processing network sequence in order;

[0137] If the processing time intervals of the historical processing network being traversed and its adjacent historical processing networks are both greater than a preset first threshold, then the historical processing network being traversed will be taken as the target historical processing network.

[0138] After traversal is complete, the target historical processing network is removed from the historical processing network sequence to obtain the updated historical processing network sequence.

[0139] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A big data-based digital economy data collection method, characterized in that, include: Based on data acquisition needs, edge processing is performed on the digital economy data to be collected stored in big data nodes; Generate global data features based on the correspondence between the node processing network and the edge processing data; Based on global data feature analysis and processing, the processed data and the corresponding analysis results are collected and stored in the database. Among them, data processing based on global data feature analysis includes: Based on global data features and cloud-based heterogeneous data understanding templates, data understanding and processing are performed. Once the understanding is complete, cloud-based data economy data analysis AI, based on cloud user data and the understood data, predicts the data collection intentions of the demand side. Based on the data collection intent, output analysis results to assist in the intent; Among them, cloud-based data economy data analysis AI, based on cloud user data and data understanding, predicts the data collection intentions of demanders, including: Identify the intent set of cloud users that matches the user profiles of the demand side; Calculate the first data similarity between the first source data for analysis and the understanding data corresponding to the preselected intents in the intent set; Based on the preset intent selection rules, the selection intent in the pre-selected intents is determined according to the first data similarity. If the selection intent is not unique, the selection intent corresponding to the highest similarity between the second data of the auxiliary analysis data required for the selection intent and the historical analysis data of the demand side shall be used as the data collection intent; The process of acquiring historical analysis data includes: Based on the second source data and the understood data of the pre-selected historical analysis data, extract the analysis dependencies; If the dependency extraction is successful, the corresponding pre-selected historical analysis data will be used as the historical analysis data.

2. The big data based digital economy data collection method of claim 1, wherein, Based on data acquisition needs, edge processing is performed on the digital economy data to be collected stored in big data nodes, including: Based on the historical data acquisition needs of the demanders, perform pre-response of the node processing network; The pre-response was revised based on data acquisition needs, and then connected to the formal response network. Edge processing of digital economy data to be collected is based on the formal response network.

3. The big data based digital economy data collection method of claim 2, wherein, Based on the historical data acquisition needs of the demanders, a pre-response mechanism is implemented for the node processing network, including: Step 101: Based on the historical data acquisition requirements, determine the historical processing network sequence within the preset target time period. The historical processing networks in the historical processing network sequence are sorted in order from the closest to the current time according to their processing time. Step 102: Based on the overlap of the first n historical processing networks and the overlap of the first n+1 historical processing networks in the historical processing network sequence, determine the change in overlap, where 1≤n≤N-1; where N is the number of historical processing networks in the historical processing network sequence. Step 103 includes: If there is no change in the consecutive T coincidences, then pre-respond based on the history of the network processing coincidences for the corresponding coincidence, where T is an integer greater than and closest to ​ If there are no consecutive T overlapping cases that have not changed, obtain the updated historical processing network sequence, and execute steps 101 to 102 again based on the updated historical processing network sequence. If there are still no consecutive T overlapping cases that have not changed, perform a pre-response based on the historical processing network of the historical processing time closest to the current time.

4. The big data based digital economy data collection method of claim 3, wherein, Obtain updated historical processing network sequences, including: Traverse the historical processing networks in the historical processing network sequence in order; If the processing time intervals of the historical processing network being traversed and its adjacent historical processing networks are both greater than a preset first threshold, then the historical processing network being traversed will be taken as the target historical processing network. After traversal is complete, the target historical processing network is removed from the historical processing network sequence to obtain the updated historical processing network sequence.

5. The big data based digital economy data collection method of claim 1, wherein, The construction method for cloud-based heterogeneous data understanding templates is as follows: Based on understanding expert nodes from heterogeneous data, a set of rules for parsing and semantic mapping of heterogeneous data from different sources is collected; Based on the rule set, construct a template for understanding heterogeneous data in the cloud.

6. The big data based digital economy data collection method of claim 1, wherein, Also includes: Data economy data analytics AI uses real-time cloud user data for training and updates.

7. A big data based digital economy data collection system characterized by, include: The edge processing module is used to perform edge processing on the digital economy data to be collected stored in the big data nodes according to the data acquisition requirements. The feature generation module is used to generate global data features based on the correspondence between the processing data of the node processing network and the edge processing. The data entry module is used to analyze and process data based on global data features, and to collect and enter the processed data and the corresponding analysis results into the database. Among them, data processing based on global data feature analysis includes: Based on global data features and cloud-based heterogeneous data understanding templates, data understanding and processing are performed. Once the understanding is complete, cloud-based data economy data analysis AI, based on cloud user data and the understood data, predicts the data collection intentions of the demand side. Based on the data collection intent, output analysis results to assist in the intent; Among them, cloud-based data economy data analysis AI, based on cloud user data and data understanding, predicts the data collection intentions of demanders, including: Identify the intent set of cloud users that matches the user profiles of the demand side; Calculate the first data similarity between the first source data for analysis and the understanding data corresponding to the preselected intents in the intent set; Based on the preset intent selection rules, the selection intent in the pre-selected intents is determined according to the first data similarity. If the selection intent is not unique, the selection intent corresponding to the highest similarity between the second data of the auxiliary analysis data required for the selection intent and the historical analysis data of the demand side shall be used as the data collection intent; The process of acquiring historical analysis data includes: Based on the second source data and the understood data of the pre-selected historical analysis data, extract the analysis dependencies; If the dependency extraction is successful, the corresponding pre-selected historical analysis data will be used as the historical analysis data.

Images