A data collection method and system based on edge computing

Abstract

The present invention relates to the field of data acquisition technology, and specifically discloses a data acquisition method and system based on edge computing, which is applied to edge computing of the Internet of Things, including: obtaining multi-source target information according to the Internet of Things, and classifying the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data; establishing a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and obtaining a data acquisition frequency set according to the multi-source data index. The present invention ensures the comprehensiveness and accuracy of data quality evaluation by obtaining the first data evaluation value, the second data evaluation value, the third data evaluation value, and the fourth data evaluation value of structured data, unstructured data, time series data, and cross-domain data, and solves the problems of incomplete evaluation and unmonitored quality changes.

Description

Technical Field

[0001] The present invention relates to the field of data acquisition technology, and in particular to a data acquisition method and system based on edge computing. Background Art

[0002] Edge computing is a new computing model that moves data processing from remote cloud servers to where the data is generated, that is, on so-called "edge" devices. These edge devices can be various smart devices such as smartphones, smart home devices, industrial sensors, cameras, etc. Summary of the invention

[0003] The purpose of the present invention is to provide a data acquisition method and system based on edge computing to solve the technical problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A data collection method based on edge computing, applied to edge computing of the Internet of Things, is characterized by comprising:

[0006] Acquire multi-source target information according to the Internet of Things, and classify the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data;

[0007] Establishing a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquiring a data collection frequency set according to the multi-source data index;

[0008] Acquire initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data collection frequency set, and perform multi-sequence collection on each of the initial data source information to obtain a multi-sequence data set;

[0009] Acquire multiple network collection information according to the multi-sequence data set, and acquire data evaluation values ​​corresponding to each initial data source information according to the multiple network collection information;

[0010] Obtain data quality assessment values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data based on the data assessment values, and obtain data quality deviation values ​​based on the data quality assessment values;

[0011] Determine whether the data quality deviation value is greater than the preset value;

[0012] If it is greater, the corresponding initial data source information is entered into the database to complete the data collection;

[0013] If it is less than that, the corresponding initial data source information will be re-collected in multiple sequences.

[0014] Preferably, the step of classifying the multi-source target information to obtain multi-source classified data includes:

[0015] Acquire multiple source target information according to multiple source target information;

[0016] Obtain source target format data according to each source target information;

[0017] Performing content recognition on each source target format data in turn, wherein the content recognition methods include text recognition method, timestamp recognition method and data physical location recognition method;

[0018] If the source target format data is identified by text recognition, it is determined that the source target format data is structured data;

[0019] If the source target format data is identified by text recognition and requires additional processing to be extracted, the source target format data is determined to be unstructured data;

[0020] If the source target format data is identified by a timestamp identification method, it is determined that the source target format data is time series data;

[0021] If the source target format data is identified by using the data physical location identification method, it is determined that the source target format data is cross-domain data.

[0022] Preferably, the step of establishing a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquiring a data collection frequency set according to the multi-source data index includes:

[0023] The structured data is saved in a text database table, and multiple field information is obtained according to the text database table;

[0024] Get the query frequency and quantity of each field information;

[0025] Obtain the field information with the highest query frequency and the highest number, and use it as the first index information;

[0026] The unstructured data is saved in a file database table, and multiple reference path information and multiple name information are obtained from the file system;

[0027] Generate search path information according to each reference path information and corresponding name information;

[0028] Using the search path information as second index information;

[0029] Save the time series data to the time series database table and obtain the data timestamp according to the time series database table;

[0030] Obtain multiple data time points according to the data timestamp, and generate a marked time point according to each data time point;

[0031] Using the marked time point as the third index information;

[0032] The cross-domain data is stored in a distributed file database table, and multiple region information is obtained according to the distributed file database table;

[0033] Compare each region information to obtain cross-region information, and use the cross-region information as fourth index information;

[0034] Collecting the first index information, the second index information, the third index information and the fourth index information to obtain a multi-source data index;

[0035] Acquire a corresponding first acquisition frequency, a second acquisition frequency, a third acquisition frequency, and a fourth acquisition frequency according to the first index information, the second index information, the third index information, and the fourth index information;

[0036] The first collection frequency, the second collection frequency, the third collection frequency and the fourth collection frequency are concentrated to obtain a data collection frequency set.

[0037] Preferably, the step of acquiring initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data collection frequency set, and performing multi-sequence collection on each of the initial data source information to obtain a multi-sequence data set includes:

[0038] The first acquisition frequency, the second acquisition frequency, the third acquisition frequency and the fourth acquisition frequency are acquired and concentrated to obtain a first data acquisition task, a second data acquisition task, a third data acquisition task and a fourth data acquisition task;

[0039] Acquire corresponding first configuration results, second configuration results, third configuration results, and fourth configuration results according to the first data collection task, the second data collection task, the third data collection task, and the fourth data collection task;

[0040] Obtain corresponding initial data source information according to the first configuration result, the second configuration result, the third configuration result, and the fourth configuration result;

[0041] Acquire a preset acquisition sequence according to each initial data source information;

[0042] The corresponding initial data source information is stored in a corresponding data storage list according to a preset acquisition sequence;

[0043] The data storage lists corresponding to the first configuration result, the second configuration result, the third configuration result and the fourth configuration result are concentrated to obtain a multi-sequence data set.

[0044] Preferably, the step of acquiring multiple network collection information according to the multi-sequence data set, and acquiring the data evaluation value corresponding to each initial data source information according to the multiple network collection information includes:

[0045] Obtain data result files corresponding to structured data, unstructured data, time series data, and cross-domain data based on multi-series data sets;

[0046] Read the data result file to obtain the acquisition sequence result data;

[0047] Extract information from the collected sequence result data to obtain key result information;

[0048] Based on key result information, obtain the information completeness, accuracy, timeliness, clarity, relevance and continuity of structured data, unstructured data, time series data and cross-domain data;

[0049] Obtain data evaluation values ​​based on information completeness, information accuracy, information timeliness, information clarity, information relevance and information continuity.

[0050] Preferably, the step of obtaining data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data evaluation values, and obtaining data quality deviation values ​​according to the data quality evaluation values ​​includes:

[0051] Obtaining a first data evaluation value, a second data evaluation value, a third data evaluation value, and a fourth data evaluation value corresponding to structured data, unstructured data, time series data, and cross-domain data;

[0052] Obtaining a first quality weight factor, a second quality weight factor, a third quality weight factor, and a fourth quality weight factor corresponding to the first data evaluation value, the second data evaluation value, the third data evaluation value, and the fourth data evaluation value;

[0053] Obtaining a first data volume, a second data volume, a third data volume, and a fourth data volume corresponding to structured data, unstructured data, time series data, and cross-domain data;

[0054] Acquire a first data quality evaluation value corresponding to the structured data according to the first data evaluation value, the first quality weight factor and the first data amount;

[0055] Acquire a second data quality evaluation value corresponding to the unstructured data according to the second data evaluation value, the second quality weight factor and the second data amount;

[0056] Acquire a third data quality evaluation value corresponding to the time series data according to the third data evaluation value, the third quality weight factor and the third data amount;

[0057] Acquire a fourth data quality evaluation value corresponding to the cross-domain data according to the fourth data evaluation value, the fourth quality weight factor and the fourth data volume;

[0058] Acquire a corresponding preset correction data amount according to the first data quality assessment value, the second data quality assessment value, the third data quality assessment value, and the fourth data quality assessment value;

[0059] The data quality deviation value is obtained by obtaining the difference of the corresponding preset correction data amount according to the first data quality evaluation value, the second data quality evaluation value, the third data quality evaluation value, and the fourth data quality evaluation value.

[0060] The present invention also provides a data acquisition system based on edge computing, which is applied to edge computing of the Internet of Things, and is characterized by comprising:

[0061] A first acquisition module is used to acquire multi-source target information according to the Internet of Things, and classify the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data;

[0062] A second acquisition module is used to establish a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquire a data collection frequency set according to the multi-source data index;

[0063] The third acquisition module is used to acquire initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data acquisition frequency set, and perform multi-sequence acquisition on each of the initial data source information to obtain a multi-sequence data set;

[0064] A fourth acquisition module is used to acquire multiple network acquisition information according to the multi-sequence data set, and acquire a data evaluation value corresponding to each initial data source information according to the multiple network acquisition information;

[0065] A fifth acquisition module is used to obtain data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data evaluation values, and to obtain data quality deviation values ​​according to the data quality evaluation values;

[0066] A judgment module is used to judge whether the data quality deviation value is greater than a preset value;

[0067] If it is greater, the corresponding initial data source information is entered into the database to complete the data collection;

[0068] If it is less than that, the corresponding initial data source information will be re-collected in multiple sequences.

[0069] Preferably, the first acquisition module includes:

[0070] A first acquisition unit, configured to acquire multiple source target information according to multiple source target information;

[0071] A second acquisition unit, used for acquiring source target format data according to each source target information;

[0072] The third acquisition unit is used to perform content recognition on each source target format data in turn, wherein the content recognition method includes a text recognition method, a timestamp recognition method and a data physical location recognition method;

[0073] A first judging unit is used to identify the source target format data by text recognition, and judge that the source target format data is structured data;

[0074] A second judgment unit is used to identify the source target format data through text recognition and require additional processing to extract, and judge that the source target format data is unstructured data;

[0075] A third determination unit is used to identify the source target format data by a timestamp identification method, and determine that the source target format data is time series data;

[0076] The fourth determination unit is used to identify the source target format data by using a data physical location identification method, and determine that the source target format data is cross-domain data.

[0077] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned edge computing-based data acquisition method when executing the computer program.

[0078] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned edge computing-based data acquisition method are implemented.

[0079] The beneficial effects of the present application are as follows: the present invention ensures the comprehensiveness and accuracy of data quality assessment by obtaining the first data assessment value, the second data assessment value, the third data assessment value and the fourth data assessment value of structured data, unstructured data, time series data and cross-domain data, and solves the problems of incomplete assessment and unmonitored quality changes; ensures the rationality and scientificity of the assessment result by obtaining the corresponding first quality weight factor, the second quality weight factor, the third quality weight factor and the fourth quality weight factor, and solves the problems of unreasonable and inflexible assessment; ensures the accuracy and completeness of data quantity statistics by obtaining the first data volume, the second data volume, the third data volume and the fourth data volume of structured data, unstructured data, time series data and cross-domain data, and solves the problems of unclear data volume and difficult resource management. The problem of data quality is solved by comprehensively considering the data evaluation value, quality weight factor and data volume, respectively obtaining the first data quality evaluation value of structured data, the second data quality evaluation value of unstructured data, the third data quality evaluation value of time series data, and the fourth data quality evaluation value of cross-domain data, to ensure the comprehensiveness and accuracy of the evaluation results and solve the problems of incomplete evaluation and insufficient decision support; by obtaining the preset correction data volume, optimizing the data collection and processing strategy, ensuring the rationality and accuracy of the data volume, and solving the problems of unreasonable data volume and waste of resources; finally, by calculating the difference of the preset correction data volume, obtaining the data quality deviation value, monitoring the changes in data quality, timely discovering and solving problems, ensuring the stability and consistency of data quality, and solving the problems of undiscovered quality problems and unmonitored quality changes. These steps effectively improve the efficiency and quality of data management and processing, provide a more scientific and efficient method for data collection and analysis, ensure the reliability and availability of data, support data-driven decision-making and optimization, and improve the overall data governance level. Specifically, these steps ensure the comprehensiveness and accuracy of data quality through multi-dimensional evaluation, ensure the rationality and scientificity of evaluation results through quality weight factors, ensure the accuracy and completeness of data volume through data volume statistics, ensure the comprehensiveness and accuracy of evaluation results through comprehensive evaluation, optimize data collection and processing strategies through preset correction data volume, monitor changes in data quality through data quality deviation values, discover and resolve data quality issues in a timely manner, ensure the stability and consistency of data quality, and ultimately improve the efficiency and quality of data management and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 The figure is a schematic diagram of a method flow of an embodiment of the present application.

[0081] Figure 2 A schematic diagram of the system structure of an embodiment of the present application.

[0082] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0083] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0084] like Figure 1 As shown, the present application provides a data collection method based on edge computing, which is applied to edge computing of the Internet of Things, including:

[0085] S1. Acquire multi-source target information according to the Internet of Things, and classify the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data;

[0086] S2. Establish a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and obtain a data collection frequency set according to the multi-source data index;

[0087] S3. Acquire initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data collection frequency set, and perform multi-sequence collection on each of the initial data source information to obtain a multi-sequence data set;

[0088] S4. Acquire multiple network collection information according to the multi-sequence data set, and acquire data evaluation values ​​corresponding to each initial data source information according to the multiple network collection information;

[0089] S5. Obtain data quality assessment values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data assessment values, and obtain data quality deviation values ​​according to the data quality assessment values;

[0090] S6. Determine whether the data quality deviation value is greater than a preset value;

[0091] If it is greater, the corresponding initial data source information is entered into the database to complete the data collection;

[0092] If it is less than that, the corresponding initial data source information will be re-collected in multiple sequences.

[0093] As described in the above steps S1-S6, edge computing significantly improves the performance and efficiency of the system during data collection through advantages such as low latency, bandwidth saving and cost reduction. However, in practical applications, it still faces defects such as limited resources, inconsistent data, difficult fault recovery and increased system complexity. For example, a factory is equipped with a large number of sensors, which generate a large amount of data per second. Edge devices (such as small embedded computers) may not be able to process this data in real time, resulting in data loss or slower processing speed. For example, a temperature sensor generates 1,000 data points per second, and the edge device's processing capacity can only reach 500 data points per second. Then the remaining 500 data points may be discarded, affecting subsequent data analysis and decision-making;

[0094] The present invention significantly improves the efficiency, accuracy and reliability of data collection through the classification of multi-source target information, the establishment of multi-source data indexes, the generation of data collection frequency sets, multi-sequence collection, the acquisition of data evaluation values, the calculation of data quality evaluation values ​​and data quality deviation values, and the automatic decision-making mechanism. Specifically, by classifying multi-source target information, the processing methods of different types of data are clarified, the inefficiency caused by mixed processing is avoided, and the efficiency and accuracy of data processing are improved. At the same time, the classified data can be targetedly processed and stored according to its characteristics, which simplifies data management, reduces data redundancy and repeated processing, and improves the efficiency of data management; by establishing a multi-source data index, it can quickly locate and retrieve It can improve the data access speed, optimize the data collection strategy, obtain the data collection frequency set according to the data index, formulate a more reasonable data collection strategy, improve the efficiency of data collection, and at the same time, through multi-source data index, it can better manage and utilize different types of data, and improve the utilization rate of data; obtain the initial data source information according to the data collection frequency set, and perform multi-sequence collection on each initial data source information, which ensures the accuracy and comprehensiveness of data collection, improves the coverage and completeness of data collection, reduces unnecessary data collection, avoids data redundancy, and improves storage and processing efficiency; obtain multiple network collection information through multi-sequence data sets, and obtain multiple network collection information based on multiple network collection information. The data evaluation value corresponding to each initial data source information is taken to provide a comprehensive data quality evaluation, refine the management of data sources, ensure the quality and reliability of data, and at the same time, timely feedback on data collection can be provided through the data evaluation value, helping managers to find problems in time and take measures, thereby improving the timeliness and accuracy of data collection; data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data are obtained according to the data evaluation value, and data quality deviation values ​​are obtained according to the data quality evaluation value, which quantifies the quality of the data, timely finds data quality problems, takes corresponding measures, continuously optimizes the data collection strategy, and improves the overall quality of the data. At the same time, through the data quality evaluation value and data quality deviation value, the data quality of the data can be improved. By detecting deviations from the default value, data collection strategies can be continuously optimized to improve the overall quality of data. By judging whether the data quality deviation value is greater than the preset value, a decision is automatically made whether to enter the data into the database, thereby improving the degree of automation in data collection. For initial data source information whose data quality does not meet the standards, multiple sequences are collected again to ensure continuous optimization and improvement of the data. At the same time, automatic decision-making and re-collection mechanisms are used to ensure that the quality of data entered into the database meets the standards, thereby improving the overall quality and reliability of the data. In short, these steps effectively solve defects such as data mixing, low retrieval efficiency, unreasonable collection strategies, inaccurate collection, insufficient coverage, incomplete evaluation, extensive management, subjective evaluation, delayed problem discovery, frequent manual intervention, and unstable data quality.

[0095] In one embodiment, the step of classifying the multi-source target information to obtain multi-source classified data includes:

[0096] S101, acquiring multiple source target information according to multiple source target information;

[0097] S102, acquiring source target format data according to each source target information;

[0098] S103, performing content recognition on each source target format data in turn, wherein the content recognition method includes a text recognition method, a timestamp recognition method, and a data physical location recognition method;

[0099] S104, if the source target format data is identified by text recognition, determining that the source target format data is structured data;

[0100] S105, if the source target format data is identified by text recognition and requires additional processing to be extracted, it is determined that the source target format data is unstructured data;

[0101] S106, if the source target format data is identified by the timestamp identification method, it is determined that the source target format data is time series data;

[0102] S107: If the source target format data is identified by the data physical location identification method, it is determined that the source target format data is cross-domain data.

[0103] As described in the above steps S101-S107, the present invention ensures the comprehensiveness and diversity of data collection by obtaining multiple source target information, covers more data sources, improves the richness and representativeness of data, and compares data from different data sources to verify the accuracy and consistency of data, improves the overall quality of data, flexibly responds to different data needs, selects appropriate data sources, improves the flexibility of data collection, and avoids the problems of single data source, low data quality and inflexible response; then, by obtaining each source target format data, ensures the uniformity and standardization of data format, facilitates subsequent data processing and analysis, avoids processing difficulties caused by inconsistent data formats, improves data processing efficiency, reduces errors and anomalies in data processing, improves the accuracy and reliability of data processing, and solves the problems of inconsistent data formats, low processing efficiency, errors and anomalies; then, through multiple content recognition methods (text recognition, timestamp recognition and data physical location recognition), accurately identifies different types of source target format data, ensures the comprehensiveness and diversity of data classification, improves the accuracy of data classification, flexibly responds to different types of data, improves the flexibility and adaptability of data processing, and solves inaccurate recognition. The problems of inaccurate classification, incomplete coverage and inflexible processing can be solved by using text recognition to accurately determine whether the source target format data is structured data, ensure the accuracy of data classification, facilitate the direct storage of structured data in relational databases, improve the efficiency and accuracy of data processing, and solve the problems of inaccurate classification, difficult processing and low efficiency; through text recognition and combined with additional processing requirements, accurately determine whether the source target format data is unstructured data, ensure the accuracy of data classification, flexibly respond to different processing requirements, improve the flexibility of data processing, ensure high data quality, and solve the problems of inaccurate classification, inflexible processing and low quality; through timestamp recognition, accurately determine whether the source target format data is time series data, ensure the accuracy of data classification, facilitate subsequent trend analysis and prediction, improve the efficiency of data processing, and solve the problems of inaccurate classification, difficult analysis and low efficiency; through data physical location recognition, accurately determine whether the source target format data is cross-domain data, ensure the accuracy of data classification, facilitate subsequent data management and synchronization, improve the reliability and consistency of data processing, and solve the problems of inaccurate classification, difficult management and low reliability;The above steps significantly improve the accuracy and comprehensiveness of data classification by obtaining multiple source target information, obtaining source target format data, multiple content recognition methods, text recognition methods, timestamp recognition methods and data physical location recognition methods, ensuring the comprehensiveness and diversity of data collection, the uniformity and standardization of data formats, the accuracy and comprehensiveness of data classification, the accurate classification of structured data, the flexible processing of unstructured data, the efficient analysis of time series data, and the reliable management of cross-domain data, solving the defects of single data source, inconsistent format, inaccurate recognition, incomplete coverage, inflexible processing, inaccurate classification, difficult analysis, low efficiency, difficult management and low reliability. ;

[0104] In one embodiment, the step of establishing a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquiring a data collection frequency set according to the multi-source data index includes:

[0105] S201, saving structured data into a text database table, and obtaining multiple field information according to the text database table;

[0106] S202, obtaining the query frequency and quantity of each field information;

[0107] S203, obtaining the field information with the highest query frequency and the highest number, and using it as the first index information;

[0108] S204, saving the unstructured data into a file database table, and obtaining multiple reference path information and multiple name information from the file system;

[0109] S205, generating search path information according to each reference path information and corresponding name information;

[0110] S206, using the search path information as second index information;

[0111] S207, saving the time series data into a time series database table, and obtaining a data timestamp according to the time series database table;

[0112] S208, acquiring multiple data time points according to the data timestamp, and generating a marked time point according to each data time point;

[0113] S209, using the marked time point as third index information;

[0114] S210, storing the cross-domain data in a distributed file database table, and obtaining multiple region information according to the distributed file database table;

[0115] S211, comparing each region information to obtain cross-region information, and using the cross-region information as fourth index information;

[0116] S212, collect the first index information, the second index information, the third index information and the fourth index information to obtain a multi-source data index;

[0117] S213. Acquire a corresponding first acquisition frequency, a second acquisition frequency, a third acquisition frequency, and a fourth acquisition frequency according to the first index information, the second index information, the third index information, and the fourth index information;

[0118] S214: Collect the first collection frequency, the second collection frequency, the third collection frequency, and the fourth collection frequency to obtain a data collection frequency set.

[0119] As described in the above steps S201-S214, the present invention ensures the uniformity and standardization of data storage by saving structured data into a text database table and obtaining multiple field information, which facilitates subsequent data query and analysis and solves the problems of inconsistent data storage and missing field information; by obtaining the query frequency and quantity of each field information, the usage and distribution of data are understood, data storage and indexing are optimized, query efficiency is improved, and the problems of low query efficiency and uneven data distribution are solved; by obtaining the field information with the highest query frequency and the highest quantity and using it as the first index information, data query speed is optimized, data processing efficiency is improved, and the problems of unoptimized indexes and low query efficiency are solved; by Structured data is saved in the file database table and multiple reference path information and name information are obtained to ensure the uniformity and standardization of data storage, facilitate subsequent data management and query, and solve the problems of inconsistent data storage and missing path and name information; by generating search path information, unstructured data can be quickly located and accessed, the query and access efficiency of data can be improved, and the problems of difficult data location and low access efficiency can be solved; by using the search path information as the second index information, the query speed of unstructured data can be optimized, the data processing efficiency can be improved, and the problems of unoptimized index and low access efficiency can be solved; by saving time series data in the time series database table and obtaining the data timestamp, the uniformity and standardization of data storage can be ensured , which is convenient for subsequent time series analysis and query, and solves the problems of inconsistent data storage and missing timestamps; by generating marked time points, time series data can be quickly located and accessed, and the query and access efficiency of data can be improved, solving the problems of difficult data location and low access efficiency; by using marked time points as the third index information, the query speed of time series data can be optimized, the data processing efficiency can be improved, and the problems of unoptimized indexes and low access efficiency can be solved; by saving cross-domain data in a distributed file database table and obtaining information on multiple regions, the uniformity and standardization of data storage can be ensured, which is convenient for subsequent data management and synchronization, and solves the problems of inconsistent data storage and missing regional information; by comparing regional information, cross-regional information can be obtained And use it as the fourth index information to optimize the management and query speed of cross-domain data, and solve the problems of difficult data management and unoptimized indexing; by centralizing various types of index information to form a multi-source data index, improve the comprehensive query and access efficiency of data, and solve the problems of low query efficiency and decentralized management; by obtaining the corresponding collection frequency based on various types of index information, optimize the data collection strategy, improve the efficiency and accuracy of data collection, and solve the problems of unreasonable collection strategy and inaccurate collection; by centralizing various types of collection frequencies to form a data collection frequency set, unified management of different types of data collection strategies, improve the efficiency and accuracy of data collection, and solve the problems of low collection efficiency and decentralized management.These steps effectively solve problems such as inconsistent data storage, missing field information, low query efficiency, uneven data distribution, difficult positioning, low access efficiency, difficult management, unoptimized indexing, unreasonable collection strategies, and inaccurate collection.

[0120] In one embodiment, the step of acquiring initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data collection frequency set, and performing multi-sequence collection on each of the initial data source information to obtain a multi-sequence data set includes:

[0121] S301, obtaining a first data collection task, a second data collection task, a third data collection task, and a fourth data collection task by correspondingly collecting the first collection frequency, the second collection frequency, the third collection frequency, and the fourth collection frequency;

[0122] S302, acquiring corresponding first configuration results, second configuration results, third configuration results, and fourth configuration results according to the first data collection task, the second data collection task, the third data collection task, and the fourth data collection task;

[0123] S303, obtaining corresponding initial data source information according to the first configuration result, the second configuration result, the third configuration result and the fourth configuration result;

[0124] S304, obtaining a preset acquisition sequence according to each initial data source information;

[0125] S305, storing the corresponding initial data source information into a corresponding data storage list according to a preset acquisition sequence;

[0126] S306. Collect the data storage lists corresponding to the first configuration result, the second configuration result, the third configuration result, and the fourth configuration result to obtain a multi-sequence data set.

[0127] As described in the above steps S301-S306, the present invention centrally manages different types of collection frequencies by centrally generating corresponding first data collection tasks, second data collection tasks, third data collection tasks and fourth data collection tasks through the first collection frequency, the second collection frequency, the third collection frequency and the fourth collection frequency, and generates corresponding data collection tasks, thereby ensuring the rationality and efficiency of task allocation and solving the problems of unreasonable task allocation and difficult task management; generates corresponding first configuration results, second configuration results, third configuration results and fourth configuration results according to the first data collection task, the second data collection task, the third data collection task and the fourth data collection task, thereby ensuring the accuracy and rationality of configuration and solving the problems of inaccurate configuration and difficult configuration ... configuration results, the second data collection task, the third data collection task and the fourth data collection task, thereby ensuring the accuracy and rationality of configuration and solving the problems of inaccurate configuration and difficult configuration management; generates corresponding first configuration results, second configuration results, third configuration results and fourth configuration results according to the first configuration results, the second data collection task, the third data collection task and the fourth data collection task, thereby ensuring the accuracy and rationality of configuration and solving the problems of inaccurate configuration and difficult configuration management. The configuration results, the third configuration results and the fourth configuration results obtain the corresponding initial data source information to ensure the accuracy and completeness of information acquisition, and solve the problems of inaccurate information acquisition and difficult information management; generate a preset acquisition sequence according to each initial data source information to ensure the orderliness and efficiency of data acquisition, and solve the problems of disorderly sequence generation and difficult sequence management; store the corresponding initial data source information in the corresponding data storage list according to the preset acquisition sequence to ensure the orderliness and standardization of data storage, and solve the problems of disordered storage and difficult storage management; finally, the data storage lists corresponding to the first configuration result, the second configuration result, the third configuration result and the fourth configuration result are concentrated to form a multi-sequence data set, improve the comprehensive management and query efficiency of data, and solve the problems of decentralized management and inconsistent storage.

[0128] In one embodiment, the step of acquiring multiple network acquisition information according to the multi-sequence data set, and acquiring the data evaluation value corresponding to each initial data source information according to the multiple network acquisition information includes:

[0129] S401. Obtain data result files corresponding to structured data, unstructured data, time series data, and cross-domain data according to the multi-sequence data set;

[0130] S402, reading the data result file to obtain the acquisition sequence result data;

[0131] S403, extracting information from the collected sequence result data to obtain key result information;

[0132] S404. Based on the key result information, obtain the information completeness, accuracy, timeliness, clarity, relevance and continuity of structured data, unstructured data, time series data and cross-domain data;

[0133] S405. Obtain data evaluation values ​​according to information completeness, information accuracy, information timeliness, information clarity, information relevance and information continuity.

[0134] As described in the above steps S401-S405, the present invention obtains result files of different types of data through a multi-sequence data set to ensure the integrity and consistency of the data. This step helps to integrate scattered data sources into a unified data set, which is convenient for subsequent data processing and analysis. For example, structured data and unstructured data can be stored in a relational database and a NoSQL database, respectively, time series data can be stored in a time series database, and cross-domain data can be stored in a distributed file system. These data can be easily managed and accessed through a unified result file, solving the problem that there is no unified data result file in the prior art, which may cause data dispersion and affect the efficiency of data management and query. For example, if the data is scattered and stored in different systems, it is necessary to switch between multiple systems during query and management, which increases the complexity and time cost of the operation, and then obtain the acquisition sequence result data by reading the data result file to ensure the integrity and accuracy of the data. This step helps to ensure that subsequent data processing and analysis have a reliable data basis. For example, reading the data result file can verify the integrity and format of the data, ensure that the data is not damaged or lost during transmission, and solve the problem that there is no reading of the data result file in the prior art, which may cause data loss and affect the integrity and accuracy of the data. For example, if the data result file is not read correctly, part of the data may not be able to enter the subsequent data processing process, resulting in incomplete data. Then, by extracting the key information in the acquisition sequence result data, the efficiency and accuracy of data processing can be ensured. This step helps to quickly locate and extract important data information and improve the efficiency of data processing. For example, the extracted key information can be the user's purchase record, the status report of the equipment, the measurement value of the sensor, etc. The quality indicators of different types of data are obtained through multi-dimensional evaluation (information completeness, information accuracy, information timeliness, information clarity, information relevance and information continuity) to ensure the comprehensiveness and accuracy of data quality. For example, information completeness can evaluate whether the data is missing, information accuracy can evaluate whether the data is correct, information timeliness can evaluate whether the data is timely, information clarity can evaluate whether the data is easy to understand, information relevance can evaluate whether the data is related to business needs, and information continuity can evaluate whether the data is coherent. Then, the data evaluation value is obtained by quantifying the data quality indicators (information completeness, information accuracy, information timeliness, information clarity, information relevance and information continuity) to ensure the objectivity and comparability of data quality evaluation. For example, data evaluation values ​​can be expressed as percentages, making it easier to compare the quality of different data sources.

[0135] In one embodiment, the step of obtaining data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data evaluation values, and obtaining data quality deviation values ​​according to the data quality evaluation values ​​includes:

[0136] S501, obtaining a first data evaluation value, a second data evaluation value, a third data evaluation value, and a fourth data evaluation value corresponding to structured data, unstructured data, time series data, and cross-domain data;

[0137] S502, obtaining a first quality weight factor, a second quality weight factor, a third quality weight factor, and a fourth quality weight factor corresponding to the first data evaluation value, the second data evaluation value, the third data evaluation value, and the fourth data evaluation value;

[0138] S503, obtaining a first data volume, a second data volume, a third data volume, and a fourth data volume corresponding to structured data, unstructured data, time series data, and cross-domain data;

[0139] S504, obtaining a first data quality evaluation value corresponding to the structured data according to the first data evaluation value, the first quality weight factor and the first data volume;

[0140] S505, obtaining a second data quality evaluation value corresponding to the unstructured data according to the second data evaluation value, the second quality weight factor and the second data volume;

[0141] S506, acquiring a third data quality evaluation value corresponding to the time series data according to the third data evaluation value, the third quality weight factor and the third data volume;

[0142] S507, obtaining a fourth data quality evaluation value corresponding to the cross-domain data according to the fourth data evaluation value, the fourth quality weight factor and the fourth data volume;

[0143] S508, acquiring a corresponding preset correction data amount according to the first data quality assessment value, the second data quality assessment value, the third data quality assessment value, and the fourth data quality assessment value;

[0144] S509: Obtain a data quality deviation value by obtaining a difference of corresponding preset correction data amounts according to the first data quality assessment value, the second data quality assessment value, the third data quality assessment value, and the fourth data quality assessment value.

[0145] As described in the above steps S501-S509, the present invention ensures the comprehensiveness and accuracy of data quality assessment by obtaining the first data assessment value, the second data assessment value, the third data assessment value, and the fourth data assessment value of structured data, unstructured data, time series data, and cross-domain data, and solves the problems of incomplete assessment and unmonitored quality changes; by obtaining the corresponding first quality weight factor, the second quality weight factor, the third quality weight factor, and the fourth quality weight factor, the rationality and scientificity of the assessment result are ensured, and the problems of unreasonable and inflexible assessment are solved. There may be multiple data that need to be collected in the structured data, so there will be multiple first data assessment values.

[0146] The first data quality evaluation value is calculated by the following formula:

[0147] ,in, is the first data quality assessment value, is the first data volume, is the first quality weight factor, The calculation process of the first data evaluation value, the second data quality evaluation value, the third data quality evaluation value, and the fourth data quality evaluation value is the same as that of the first data quality evaluation value; by obtaining the first data volume, the second data volume, the third data volume, and the fourth data volume of structured data, unstructured data, time series data, and cross-domain data, the accuracy and completeness of data volume statistics are ensured, and the problems of unclear data volume and difficult resource management are solved; by comprehensively considering the data evaluation value, the quality weight factor, and the data volume, the first data quality evaluation value of structured data, the second data quality evaluation value of unstructured data, the third data quality evaluation value of time series data, and the fourth data quality evaluation value of cross-domain data are respectively obtained to ensure the comprehensiveness and accuracy of the evaluation results, and solve the problems of incomplete evaluation and insufficient decision support; by obtaining the preset correction data volume, the data collection and processing strategy is optimized to ensure the rationality and accuracy of the data volume, and solve the problems of unreasonable data volume and waste of resources; finally, by calculating the difference of the preset correction data volume, the data quality deviation value is obtained, the changes in data quality are monitored, and problems are discovered and solved in a timely manner, to ensure the stability and consistency of data quality, and solve the problems of quality problems not being discovered and quality changes not being monitored. These steps effectively improve the efficiency and quality of data management and processing, provide a more scientific and efficient method for data collection and analysis, ensure the reliability and availability of data, support data-driven decision-making and optimization, and improve the overall data governance level; Specifically, obtaining the evaluation values ​​of different types of data ensures a comprehensive evaluation of data quality from multiple dimensions, avoids the limitations of a single evaluation indicator, and thus provides a comprehensive view of data quality. Obtaining quality weight factors makes the evaluation results more reasonable and scientific, and can be flexibly adjusted according to the importance and application scenarios of different data, improving the flexibility and applicability of the evaluation. Obtaining data volume ensures the accuracy and completeness of data volume statistics, helps to understand the scale and distribution of different types of data, optimizes data storage and processing strategies, and improves resource utilization. Obtaining data quality evaluation values ​​by comprehensively considering evaluation values, weight factors, and data volume ensures the comprehensiveness and accuracy of evaluation results, provides strong support for the management and decision-making of different types of data, helps managers to discover and solve problems in a timely manner, and optimizes data management and processing strategies. Obtaining preset correction data volume optimizes data collection and processing strategies, ensures the rationality and accuracy of data volume, avoids resource waste, and improves data processing efficiency and resource utilization. Finally, by calculating the difference between the preset correction data volume and obtaining the data quality deviation value, the changes in data quality were monitored, data quality problems were discovered and solved in a timely manner, and the stability and consistency of data quality were ensured.

[0148] like Figure 2 As shown, the present invention also provides a data acquisition system based on edge computing, which is applied to edge computing of the Internet of Things, and is characterized by comprising:

[0149] The first acquisition module 1 is used to acquire multi-source target information according to the Internet of Things, and classify the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data;

[0150] A second acquisition module 2 is used to establish a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and to acquire a data collection frequency set according to the multi-source data index;

[0151] The third acquisition module 3 is used to acquire initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data acquisition frequency set, and perform multi-sequence acquisition on each of the initial data source information to obtain a multi-sequence data set;

[0152] A fourth acquisition module 4 is used to acquire multiple network acquisition information according to the multi-sequence data set, and acquire a data evaluation value corresponding to each initial data source information according to the multiple network acquisition information;

[0153] A fifth acquisition module 5 is used to acquire data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data evaluation values, and to acquire data quality deviation values ​​according to the data quality evaluation values;

[0154] A judgment module 6 is used to judge whether the data quality deviation value is greater than a preset value;

[0155] If it is greater, the corresponding initial data source information is entered into the database to complete the data collection;

[0156] If it is less than that, the corresponding initial data source information will be re-collected in multiple sequences.

[0157] Preferably, the first acquisition module 1 includes:

[0158] A first acquisition unit, configured to acquire multiple source target information according to multiple source target information;

[0159] A second acquisition unit, used for acquiring source target format data according to each source target information;

[0160] The third acquisition unit is used to perform content recognition on each source target format data in turn, wherein the content recognition method includes a text recognition method, a timestamp recognition method and a data physical location recognition method;

[0161] A first judging unit is used to identify the source target format data by text recognition, and judge that the source target format data is structured data;

[0162] A second judgment unit is used to identify the source target format data through text recognition and require additional processing to extract, and judge that the source target format data is unstructured data;

[0163] A third determination unit is used to identify the source target format data by a timestamp identification method, and determine that the source target format data is time series data;

[0164] The fourth determination unit is used to identify the source target format data by using a data physical location identification method, and determine that the source target format data is cross-domain data.

[0165] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned edge computing-based data acquisition method when executing the computer program.

[0166] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned edge computing-based data acquisition method are implemented.

[0167] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, value library or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0169] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent results or equivalent process changes made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data collection method based on edge computing, applied to edge computing of the Internet of Things, characterized in that: include: Acquire multi-source target information according to the Internet of Things, and classify the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data; Establishing a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquiring a data collection frequency set according to the multi-source data index; Acquire initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data collection frequency set, and perform multi-sequence collection on each of the initial data source information to obtain a multi-sequence data set; Acquire multiple network collection information based on the multi-sequence data set, and acquire data evaluation values ​​corresponding to each initial data source information based on the multiple network collection information, wherein information completeness, information accuracy, information timeliness, information clarity, information relevance, and information continuity corresponding to structured data, unstructured data, time series data, and cross-domain data are acquired, and data evaluation values ​​are acquired based on information completeness, information accuracy, information timeliness, information clarity, information relevance, and information continuity; Obtain data quality assessment values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data based on the data assessment values, and obtain data quality deviation values ​​based on the data quality assessment values; Determine whether the data quality deviation value is greater than the preset value; If it is greater, the corresponding initial data source information is entered into the database to complete the data collection; If it is less than that, the corresponding initial data source information will be re-collected in multiple sequences.

2. The data collection method based on edge computing according to claim 1 is characterized in that: The step of classifying the multi-source target information to obtain multi-source classified data includes: Acquire multiple source target information according to multiple source target information; Obtain source target format data according to each source target information; Performing content recognition on each source target format data in turn, wherein the content recognition methods include text recognition method, timestamp recognition method and data physical location recognition method; If the source target format data is identified by text recognition, it is determined that the source target format data is structured data; If the source target format data is identified by text recognition and requires additional processing to be extracted, the source target format data is determined to be unstructured data; If the source target format data is identified by a timestamp identification method, it is determined that the source target format data is time series data; If the source target format data is identified by using the data physical location identification method, it is determined that the source target format data is cross-domain data.

3. The data collection method based on edge computing according to claim 2 is characterized in that: The step of establishing a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquiring a data collection frequency set according to the multi-source data index includes: The structured data is saved in a text database table, and multiple field information is obtained according to the text database table; Get the query frequency and quantity of each field information; Obtain the field information with the highest query frequency and the highest number, and use it as the first index information; The unstructured data is saved in a file database table, and multiple reference path information and multiple name information are obtained from the file system; Generate search path information according to each reference path information and corresponding name information; Using the search path information as second index information; Save the time series data to the time series database table and obtain the data timestamp according to the time series database table; Obtain multiple data time points according to the data timestamp, and generate a marked time point according to each data time point; Using the marked time point as the third index information; The cross-domain data is stored in a distributed file database table, and multiple region information is obtained according to the distributed file database table; Compare each region information to obtain cross-region information, and use the cross-region information as fourth index information; Collecting the first index information, the second index information, the third index information and the fourth index information to obtain a multi-source data index; Acquire a corresponding first acquisition frequency, a second acquisition frequency, a third acquisition frequency, and a fourth acquisition frequency according to the first index information, the second index information, the third index information, and the fourth index information; The first collection frequency, the second collection frequency, the third collection frequency and the fourth collection frequency are concentrated to obtain a data collection frequency set.

4. The data collection method based on edge computing according to claim 3 is characterized in that: The step of acquiring initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data collection frequency set, and performing multi-sequence collection on each of the initial data source information to obtain a multi-sequence data set includes: The first acquisition frequency, the second acquisition frequency, the third acquisition frequency and the fourth acquisition frequency are acquired and concentrated to obtain a first data acquisition task, a second data acquisition task, a third data acquisition task and a fourth data acquisition task; Acquire corresponding first configuration results, second configuration results, third configuration results, and fourth configuration results according to the first data collection task, the second data collection task, the third data collection task, and the fourth data collection task; Obtain corresponding initial data source information according to the first configuration result, the second configuration result, the third configuration result, and the fourth configuration result; Acquire a preset acquisition sequence according to each initial data source information; The corresponding initial data source information is stored in the corresponding data storage list according to the preset acquisition sequence; The data storage lists corresponding to the first configuration result, the second configuration result, the third configuration result and the fourth configuration result are concentrated to obtain a multi-sequence data set.

5. The data collection method based on edge computing according to claim 1 is characterized in that: The step of acquiring multiple network collection information according to the multi-sequence data set, and acquiring the data evaluation value corresponding to each initial data source information according to the multiple network collection information, comprises: Obtain data result files corresponding to structured data, unstructured data, time series data, and cross-domain data based on multi-series data sets; Read the data result file to obtain the acquisition sequence result data; Extract information from the collected sequence result data to obtain key result information; Based on key result information, obtain the information completeness, accuracy, timeliness, clarity, relevance and continuity corresponding to structured data, unstructured data, time series data and cross-domain data.

6. The data collection method based on edge computing according to claim 5 is characterized in that: The step of obtaining data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data evaluation values, and obtaining data quality deviation values ​​according to the data quality evaluation values ​​includes: Obtaining a first data evaluation value, a second data evaluation value, a third data evaluation value, and a fourth data evaluation value corresponding to structured data, unstructured data, time series data, and cross-domain data; Obtaining a first quality weight factor, a second quality weight factor, a third quality weight factor, and a fourth quality weight factor corresponding to the first data evaluation value, the second data evaluation value, the third data evaluation value, and the fourth data evaluation value; Obtaining a first data volume, a second data volume, a third data volume, and a fourth data volume corresponding to structured data, unstructured data, time series data, and cross-domain data; Acquire a first data quality evaluation value corresponding to the structured data according to the first data evaluation value, the first quality weight factor and the first data amount; Acquire a second data quality evaluation value corresponding to the unstructured data according to the second data evaluation value, the second quality weight factor and the second data amount; Acquire a third data quality evaluation value corresponding to the time series data according to the third data evaluation value, the third quality weight factor and the third data amount; Acquire a fourth data quality evaluation value corresponding to the cross-domain data according to the fourth data evaluation value, the fourth quality weight factor and the fourth data volume; Acquire a corresponding preset correction data amount according to the first data quality assessment value, the second data quality assessment value, the third data quality assessment value, and the fourth data quality assessment value; The data quality deviation value is obtained by obtaining the difference of the corresponding preset correction data amount according to the first data quality evaluation value, the second data quality evaluation value, the third data quality evaluation value, and the fourth data quality evaluation value.

7. A data acquisition system based on edge computing, applied to edge computing of the Internet of Things, characterized in that: include: A first acquisition module is used to acquire multi-source target information according to the Internet of Things, and classify the multi-source target information to obtain multi-source classified data, wherein the multi-source classified data includes structured data, unstructured data, time series data, and cross-domain data; A second acquisition module is used to establish a multi-source data index according to the structured data, unstructured data, time series data, and cross-domain data, and acquire a data collection frequency set according to the multi-source data index; The third acquisition module is used to acquire initial data source information corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data acquisition frequency set, and perform multi-sequence acquisition on each of the initial data source information to obtain a multi-sequence data set; A fourth acquisition module is used to acquire multiple network collection information according to the multi-sequence data set, and acquire data evaluation values ​​corresponding to each initial data source information according to the multiple network collection information, wherein information completeness, information accuracy, information timeliness, information clarity, information relevance, and information continuity corresponding to structured data, unstructured data, time series data, and cross-domain data are acquired, and data evaluation values ​​are acquired according to information completeness, information accuracy, information timeliness, information clarity, information relevance, and information continuity; A fifth acquisition module is used to obtain data quality evaluation values ​​corresponding to structured data, unstructured data, time series data, and cross-domain data according to the data evaluation values, and to obtain data quality deviation values ​​according to the data quality evaluation values; A judgment module is used to judge whether the data quality deviation value is greater than a preset value; If it is greater, the corresponding initial data source information is entered into the database to complete the data collection; If it is less than that, the corresponding initial data source information will be re-collected in multiple sequences.

8. The data acquisition system based on edge computing according to claim 7, characterized in that: The first acquisition module includes: A first acquisition unit, configured to acquire multiple source target information according to multiple source target information; A second acquisition unit, used for acquiring source target format data according to each source target information; The third acquisition unit is used to perform content recognition on each source target format data in turn, wherein the content recognition method includes a text recognition method, a timestamp recognition method and a data physical location recognition method; A first judging unit is used to identify the source target format data by text recognition, and judge that the source target format data is structured data; A second judgment unit is used to identify the source target format data through text recognition and require additional processing to extract, and judge that the source target format data is unstructured data; A third determination unit is used to identify the source target format data by a timestamp identification method, and determine that the source target format data is time series data; The fourth determination unit is used to identify the source target format data by using a data physical location identification method, and determine that the source target format data is cross-domain data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images