Multi-data fusion method and system based on edge calculation

Abstract

The invention provides a multi-data fusion method and system based on edge computing. Belongs to the technical field of edge computing. The method comprises the following steps: receiving heterogeneous resource data from different sensors, equipment and sources through an edge task allocation unit; according to the encryption level corresponding to each task in the heterogeneous resource data, dividing each task data into data groups with identifiers according to a preset segmentation strategy; according to the operation condition of the edge task processor, performing distribution calculation on the data set according to a preset distribution strategy; after the distribution is completed, obtaining an identification sequence of the distribution mode; the distributed data set is calculated and processed through the edge task processor, and after data calculation and processing are completed, the result in the data set is sent back to the edge task distribution unit. Through intelligent scheduling of the edge task distribution unit, data from different sensors and equipment can be received and processed in real time, so that the data processing efficiency and response speed are improved.

Description

Technical Field

[0001] The present invention proposes a multi-data fusion method and system based on edge computing, which belongs to the technical field of edge computing. Background Art

[0002] With the rapid development of the Internet of Things and big data technology, various intelligent devices have generated massive amounts of multimodal data. Traditional cloud computing models are difficult to process this data in real time due to network latency and bandwidth limitations. Edge computing reduces the latency and cost of data transmission by migrating computing tasks and data storage to the edge of the network. However, in the edge computing environment, how to effectively integrate multi-source heterogeneous data remains a problem that needs to be solved. Summary of the invention

[0003] The present invention provides a multi-data fusion method and system based on edge computing to solve the problems mentioned in the above background technology:

[0004] The present invention proposes a multi-data fusion method based on edge computing, the method comprising:

[0005] S1. Receive heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divide each task data into data groups with identification according to the encryption level corresponding to each task in the heterogeneous resource data according to the preset segmentation strategy;

[0006] S2. According to the running status of the edge task processor, the data group is allocated and calculated according to the preset allocation strategy; after the allocation is completed, an identification sequence of the allocation method is obtained;

[0007] S3, the edge task processor performs calculation processing on the assigned data group, and after the data calculation processing is completed, the result in the data group is transmitted back to the edge task allocation unit;

[0008] S4. The edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge device.

[0009] Furthermore, the S1 includes:

[0010] S11, the edge task allocation unit receives heterogeneous resource data from various sensors, smart devices and external data sources through high-speed network interfaces and Internet of Things protocols;

[0011] S12. In addition, during the heterogeneous data reception process, the edge task allocation unit performs a preliminary check on the integrity of the data;

[0012] S13, based on each received task data, the edge task allocation unit evaluates its encryption level according to the sensitivity and privacy requirements of the data;

[0013] S14. Based on the evaluation results, different processing strategies and storage methods are adopted for different levels of data; among which, the encryption level is divided into three levels: high, medium and low;

[0014] S15. According to a preset segmentation strategy, each task data is divided into multiple data groups with unique identifiers, and a metadata tag is generated for each data group.

[0015] Furthermore, the S14 includes:

[0016] For data assessed as having a high encryption level, the edge task allocation unit immediately uses a strong encryption algorithm to encrypt the data and directs the data to be stored in a specially configured high-security storage area. Each data access is recorded in detail.

[0017] According to the specific usage scenarios of the data, a dynamic encryption strategy is implemented for data with a medium encryption level, that is, data is encrypted during data transmission and decrypted as needed during internal processing. The data is stored in a partition with a medium security level. For some sensitive fields that do not need to be fully encrypted, a data desensitization strategy is implemented;

[0018] Encrypt low-level encryption levels through basic encryption algorithms, and ensure data integrity through checksums or hash values, and store such data through distributed file systems or object storage services;

[0019] Implement data lifecycle management strategies to automatically archive data that is no longer frequently accessed to low-cost storage media, while setting data retention periods and automatic deletion rules;

[0020] Through machine learning algorithms, it intelligently identifies and classifies continuously inflowing data and dynamically adjusts its encryption level and processing strategy;

[0021] Regularly evaluate the effectiveness of existing processing strategies and automatically adjust encryption levels and storage strategies based on changes in data usage patterns, new trends in security threats, or more.

[0022] Furthermore, the S15 includes:

[0023] S151. Dynamically adjust the preset segmentation strategy according to the historical data processing efficiency and system load;

[0024] S152, cutting the continuous data stream into independent segments or blocks according to a preset time window or data size, and classifying the data according to data type;

[0025] S153. Assign a globally unique identifier to each data group. For frequently updated data, introduce a version control mechanism to record the UUID and change time of each data update.

[0026] S154, generating metadata tags for each data group, annotating the data group according to the needs of subsequent data processing and analysis; and recording the association relationship between the data groups;

[0027] S155. Store the data group in a distributed file system or database based on the importance and access frequency of the data, and build an index for the data group and metadata tags.

[0028] Furthermore, the S151 includes:

[0029] Collect the completion time, resource consumption, and success rate of various data processing tasks in the past period of time, and use machine learning algorithms to predict the processing efficiency trend in the future under different segmentation strategies;

[0030] Integrated monitoring tools continuously track system resource usage and use deep learning models to predict changes in system load in future periods.

[0031] Based on the above historical efficiency analysis and system load prediction results, the granularity, frequency and data type classification strategy of data segmentation are dynamically adjusted through intelligent algorithms;

[0032] After implementing a new segmentation strategy, continuously monitor the execution efficiency of data processing tasks and changes in system load, and evaluate the effectiveness of the strategy through A / B testing or progressive deployment strategies; based on the evaluation results, continuously adjust and optimize algorithm parameters to form a closed-loop feedback mechanism;

[0033] Furthermore, the S2 includes:

[0034] S21. The edge task allocation unit monitors the key indicators of the edge task processor in real time, wherein the key indicators include load, computing power, memory usage, and energy consumption;

[0035] S22. Based on the monitoring data, evaluate the real-time processing capability and remaining resources of the edge task processor;

[0036] S23. Formulate a dynamic allocation strategy based on load balancing based on task priority, data size, transmission delay requirements and data encryption level;

[0037] S24, according to the allocation strategy, the edge task allocation unit allocates the data group to the most suitable edge task processor for processing;

[0038] S25, and generate an identification sequence including data group allocation information and edge task processor identifier, and use the identification sequence to recycle subsequent results and track errors.

[0039] Furthermore, the S3 includes:

[0040] S31, the edge task processor fuses the data in each data group based on a weighted average algorithm to generate a preliminary fusion result;

[0041] S32. For data groups that require in-depth analysis, they are integrated through machine learning algorithms;

[0042] S33. For data in a specific field, logical reasoning and judgment are performed based on a predefined set of rules;

[0043] S34. After completing the data fusion, the processing results of each data group are encapsulated into a standardized data packet, which contains the processing results, processing time, ETP identifier and the identifier of the original data group, and is sent back to the edge task allocation unit.

[0044] Furthermore, the S34 includes:

[0045] Before packaging, the data fusion results are verified. After passing the verification, the processing results are converted into a unified and structured data format according to the preset data format standard;

[0046] The timestamp of the whole process of receiving, processing and generating fusion results of each data group is recorded, and this time information is used as part of the data packet;

[0047] Assign a unique ETP identifier to each data group and append the ETP identifier to the data packet;

[0048] The identification information of the original data group is retained, and a mapping relationship between the original data group identification and the processing result is established in the data packet; according to the sensitivity and security requirements of the data, the data packet is encrypted and encapsulated using a secure encapsulation protocol;

[0049] Compress data packets and dynamically adjust the parameters of the compression algorithm according to network bandwidth and latency. Set priorities for data packets based on the importance and urgency of the data. Use routing algorithms in edge computing networks to plan the optimal transmission path for data packets.

[0050] The encapsulated data packet is sent back to the edge task allocation unit through the edge computing network, and a confirmation message from the edge task allocation unit is received. If no confirmation message is received, retransmission is performed according to the retransmission strategy.

[0051] Furthermore, the S4 includes:

[0052] S41, the edge task allocation unit integrates the processing results of each received data group according to the identification sequence, and during the integration process, checks the consistency of the results and resolves conflicts;

[0053] S42, the integrated results are first uploaded to the cloud platform, and further analyzed, stored and backed up through the cloud platform;

[0054] S43. At the same time, the edge task allocation unit distributes key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism to make real-time decisions and responses.

[0055] The present invention proposes a multi-data fusion system based on edge computing, the system comprising:

[0056] Data receiving unit: receives heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divides each task data into identified data groups according to the preset segmentation strategy according to the encryption level corresponding to each task in the heterogeneous resource data;

[0057] Allocation calculation unit: according to the running status of the edge task processor, the data group is allocated and calculated according to the preset allocation strategy; after the allocation is completed, the identification sequence of the allocation method is obtained;

[0058] Computation processing unit: performs computation on the assigned data group through the edge task processor. After the data computation is completed, the result in the data group is transmitted back to the edge task assignment unit;

[0059] Result return unit: The edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge devices.

[0060] Beneficial effects of the present invention: Through the intelligent scheduling of the edge task allocation unit, data from different sensors and devices can be received and processed in real time, improving the efficiency and response speed of data processing; by adopting corresponding processing and storage strategies for data of different encryption levels, the security and privacy of data can be effectively guaranteed. Strong encryption algorithms and high-security storage are used for highly sensitive data, while dynamic encryption and data desensitization strategies are implemented for medium and low sensitivity data to ensure the security of data in different life cycles; the edge task allocation unit can monitor the key performance indicators of the task processor in real time, and dynamically allocate resources according to the system load and task priority, thereby improving the resource utilization of the system; through algorithms such as machine learning and logical reasoning, in-depth analysis and fusion of data can effectively improve the accuracy and reliability of data processing. Moreover, when processing data in specific fields, the application of rule sets can make more accurate logical judgments; the implementation of data lifecycle management strategies can automatically archive and clean up data that is no longer frequently accessed, reduce storage costs, and improve system management efficiency; through dynamic adjustment of data segmentation strategies, it can adapt to different data flow characteristics and system load conditions, thereby optimizing data processing efficiency; the processing results are distributed to the cloud platform and edge devices in a timely manner through an efficient messaging mechanism to ensure real-time decision-making and response, and enhance the real-time and agility of the system; a unique identifier and its metadata tag are generated for each data group, which can easily track the entire process of data processing and improve the transparency and traceability of data management; by processing and analyzing data in an edge computing environment, the time required to transmit data to the cloud can be reduced, the overall system latency can be reduced, and the user experience can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a step diagram of the method of the present invention;

[0062] Figure 2 This is a system module diagram of the present invention. DETAILED DESCRIPTION

[0063] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0064] One embodiment of the present invention, as Figure 1 As shown, a multi-data fusion method based on edge computing, the method includes:

[0065] S1. Receive heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divide each task data into data groups with identification according to the encryption level corresponding to each task in the heterogeneous resource data according to the preset segmentation strategy;

[0066] S2. According to the running status of the edge task processor, the data group is allocated and calculated according to the preset allocation strategy; after the allocation is completed, an identification sequence of the allocation method is obtained;

[0067] S3, the edge task processor performs calculation processing on the assigned data group, and after the data calculation processing is completed, the result in the data group is transmitted back to the edge task allocation unit;

[0068] S4. The edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge device.

[0069] The working principle of the above technical solution is as follows: the edge task allocation unit, as the core component of the system, is responsible for receiving heterogeneous resource data from different sensors, devices and sources. The data includes environmental parameters such as temperature, humidity, light, as well as device status, location information, etc. According to the encryption level corresponding to each task in the heterogeneous resource data, the edge task allocation unit will perform security processing on each task data. Then, according to the preset segmentation strategy, each task data is divided into a data group with a unique identifier. The edge task allocation unit formulates a data allocation strategy based on the current operating status of the edge task processor (such as computing power, load status, etc.), as well as factors such as task priority, data size and transmission delay. According to the allocation strategy, the preprocessed data group is allocated to the corresponding edge task processor for calculation. At the same time, the identification sequence of the allocation method is recorded. After receiving the allocated data group, the edge task processor will perform a series of calculations. The data is weighted according to the reliability and importance of the data. The data is integrated and analyzed using mathematical models and algorithms to obtain more comprehensive and accurate information. The data is logically inferred and judged according to the pre-set rules to achieve comprehensive analysis of the data and deduction of conclusions. After the calculation is completed, the result data group is transmitted back to the edge task allocation unit. The edge task allocation unit obtains the calculation results of each task according to the previously recorded identification sequence. These results are summarized and sorted to form the final calculation result set. The final calculation result set is transmitted to the cloud platform, and at the same time, the relevant calculation results are transmitted to the corresponding edge devices to support real-time decision-making and response.

[0070] The effect of the above technical solution is: through edge computing, data can be processed at a location closer to the data source, significantly reducing the time delay of data transmission to the cloud, enabling the system to respond to data changes faster, especially in application scenarios with high real-time requirements (such as autonomous driving, intelligent manufacturing, etc.), which can significantly improve the response speed and performance of the system. The edge task allocation unit can intelligently allocate data groups for calculation according to the operating status of the edge task processor. It improves the utilization rate of computing resources and avoids computing bottlenecks caused by insufficient resources. By receiving heterogeneous resource data from different sensors, devices and sources and effectively fusing them, the system can obtain more comprehensive and accurate information and provide a more reliable basis for decision-making. Through weighted fusion, the system can weight the data according to the reliability and importance of the data, thereby improving the accuracy of the overall fusion result. Model fusion uses mathematical models and algorithms to integrate data, further improving the comprehensiveness and accuracy of the data. According to the encryption level corresponding to each task in the heterogeneous resource data, the data is graded, which not only protects the security of sensitive data, but also avoids the risk of data leakage. Some sensitive data can be processed locally, reducing the risk of transmission on the network, further improving the security of the system and protecting the privacy of users. The edge task allocation unit can dynamically adjust the allocation strategy according to the operating status and data characteristics of the edge task processor, so that the system can flexibly respond to different application scenarios and data requirements. With the continuous increase and upgrade of edge devices, this method can easily expand the processing capacity of the system; it provides convenience for future expansion and upgrade, and reduces the maintenance cost of the system. In the field of intelligent manufacturing, the above technical solution can monitor the operating status of the production line in real time, and make intelligent decisions based on the data fusion results. This helps to optimize the production process, improve production efficiency and reduce production costs. In application scenarios such as smart homes and smart cities, this method can process and analyze user data in real time. This helps to provide a more personalized and intelligent service experience and improve user satisfaction and loyalty.

[0071] In one embodiment of the present invention, the S1 includes:

[0072] S11, the edge task allocation unit receives heterogeneous resource data from various sensors, smart devices and external data sources through high-speed network interfaces and Internet of Things protocols;

[0073] S12. In addition, during the heterogeneous data reception process, the edge task allocation unit performs a preliminary check on the integrity of the data;

[0074] S13, based on each received task data, the edge task allocation unit evaluates its encryption level according to the sensitivity and privacy requirements of the data;

[0075] S14. Based on the evaluation results, different processing strategies and storage methods are adopted for different levels of data; among which, the encryption level is divided into three levels: high, medium and low;

[0076] S15. According to the preset segmentation strategy, each task data is divided into multiple data groups with unique identifiers, and a metadata tag including the source, timestamp, encryption level and data processing requirements is generated for each data group.

[0077] The working principle of the above technical solution is as follows: the edge task allocation unit, as the core component of the system, is responsible for receiving heterogeneous resource data from various sensors, smart devices and external data sources (such as cloud services, remote databases); the edge task allocation unit communicates efficiently and reliably with various data sources through high-speed network interfaces (such as Ethernet, Wi-Fi, etc.) and Internet of Things protocols (such as MQTT, CoAP, etc.). During the data reception process, the edge task allocation unit will perform a preliminary verification of the integrity of the data; the verification includes checking the format, length, checksum, etc. of the data to ensure that the data is not damaged or lost during transmission. The edge task allocation unit will evaluate the encryption level of each received task data based on its sensitivity and privacy requirements. The encryption level is divided into three levels: high, medium and low to reflect the sensitivity of the data and the need for privacy protection. According to the evaluation results of the encryption level, the edge task allocation unit will adopt different processing strategies and storage methods for data of different levels.

[0078] High-encryption-level data: requires the use of more complex encryption algorithms for encryption and is stored in storage media with higher security performance.

[0079] Medium encryption level data: uses a medium-strength encryption algorithm, is stored in ordinary storage media, and requires additional access control mechanisms.

[0080] Low-encryption-level data: uses a simpler encryption algorithm or is not encrypted and is stored in easily accessible storage media.

[0081] According to the preset segmentation strategy (such as based on time window, data type, data size, etc.), the edge task allocation unit will divide each task data into multiple data groups with unique identifiers; and generate a metadata label for each data group containing the source, timestamp, encryption level and data processing requirements.

[0082] The effect of the above technical solution is: the edge task allocation unit receives heterogeneous resource data from various sensors, smart devices and external data sources (such as cloud services, remote databases) through high-speed network interfaces and Internet of Things protocols (such as MQTT, CoAP), which can be compatible with multiple data sources and data formats, and improve the efficiency of data reception and processing. In the process of receiving heterogeneous data, the edge task allocation unit will perform a preliminary check on the integrity of the data to ensure the accuracy and reliability of the data and reduce errors and losses caused by data damage or loss. The edge task allocation unit evaluates the encryption level of each received task data according to the sensitivity and privacy requirements of the data; it helps to determine the protection level of the data and provide guidance for subsequent data processing and storage. Based on the evaluation results of the encryption level, different processing strategies and storage methods are adopted for data of different levels; it ensures that data with high sensitivity and high privacy requirements are more strictly protected, while reducing the processing cost of data with low sensitivity and low privacy requirements. Metadata tags containing source, timestamp, encryption level and data processing requirements are generated for each data group. The tags help track and manage data during the data life cycle to ensure data traceability and security. According to the preset segmentation strategy (such as based on time window, data type, data size, etc.), each task data is divided into multiple data groups with unique identifiers, which helps to decompose the data into smaller and easier to process parts, and improve the efficiency and flexibility of data processing. By dividing the data into multiple data groups, data processing and fusion can be performed in parallel. The data processing time can be shortened and the response speed and performance of the system can be improved. The information in the metadata tag (such as source, timestamp, encryption level and data processing requirements) is crucial for subsequent data fusion and analysis. It can determine the source, time and processing requirements of the data, so as to more accurately fuse and analyze the data. By generating a unique identifier and metadata tag for each data group, the coherence and consistency of the data processing process can be ensured, data confusion and errors can be avoided, and the accuracy and reliability of data processing can be improved.

[0083] In one embodiment of the present invention, the S14 includes:

[0084] For data assessed as having a high encryption level, the edge task allocation unit immediately uses a strong encryption algorithm to encrypt the data and directs the data to be stored in a specially configured high-security storage area; each data access is recorded in detail.

[0085] According to the specific usage scenarios of the data, a dynamic encryption strategy is implemented for data with a medium encryption level, that is, data is encrypted during data transmission and decrypted as needed during internal processing. The data is stored in a partition with a medium security level. For some sensitive fields that do not need to be fully encrypted, a data desensitization strategy is implemented;

[0086] Encrypt low-level encryption levels through basic encryption algorithms, and ensure data integrity through checksums or hash values, and store such data through distributed file systems or object storage services;

[0087] Implement data lifecycle management strategies to automatically archive data that is no longer frequently accessed to low-cost storage media, while setting data retention periods and automatic deletion rules;

[0088] Through machine learning algorithms, it intelligently identifies and classifies continuously inflowing data and dynamically adjusts its encryption level and processing strategy;

[0089] Regularly evaluate the effectiveness of existing processing strategies and automatically adjust encryption levels and storage strategies based on changes in data usage patterns, new trends in security threats, or more.

[0090] The working principle of the above technical solution is as follows: for data assessed as high-level encryption, the edge task allocation unit uses a strong encryption algorithm (such as AES-256) for encryption processing; the encrypted data is directed to be stored in a specially configured high-security storage area. This area uses physical isolation and multiple authentication mechanisms to ensure that only authorized personnel can access it; each data access will be recorded in detail, including access time, visitor identity and operation type, to provide a basis for auditing and tracing; according to the specific usage scenario of the data, a dynamic encryption strategy is implemented for data at the intermediate encryption level. Encrypt during data transmission and decrypt as needed during internal processing; Data is stored in a partition with a medium security level, which supports fine-grained access control, allowing specific business logic components to access data on demand after identity verification; For fields that are partially sensitive but do not need to be fully encrypted, implement data desensitization strategies, such as partial masking or hashing, to reduce the risk of data leakage; Encrypt low-level encryption levels through basic encryption algorithms (such as AES-128); Use checksums or hash values ​​to ensure data integrity; Store such data through distributed file systems or object storage services, implement data lifecycle management strategies, and automatically archive data that is no longer frequently accessed to low-cost storage media; Set data retention periods and automatic deletion rules to ensure data compliance and privacy protection; Intelligently identify and classify continuously inflowing data through machine learning algorithms, and dynamically adjust its encryption level and processing strategy; Automatically adjust encryption levels and storage strategies based on changes in data usage patterns, new trends in security threats, or to adapt to changing security needs; Regularly evaluate the effectiveness of existing processing strategies to ensure that they can adapt to changes in data usage patterns and new trends in security threats; Optimize encryption levels and storage strategies based on evaluation results to improve data security and availability.

[0091] The effects of the above technical solutions are as follows: strong encryption algorithms (such as AES-256) are used to encrypt data at high encryption levels to ensure the security of data during storage and transmission. Specially configured high-security storage areas, combined with physical isolation and multiple authentication mechanisms, effectively prevent unauthorized access and data leakage. Dynamic encryption strategies are implemented to encrypt data during transmission according to specific usage scenarios and decrypt it on demand during internal processing, which not only ensures data security but also improves the flexibility of data processing. The application of data desensitization strategies, such as partial masking or hashing, further reduces the risk of sensitive data leakage. Data at low encryption levels are encrypted using basic encryption algorithms (such as AES-128), and checksums or hash values ​​are used to ensure data integrity. The application of distributed file systems or object storage services improves data availability and scalability while maintaining basic security protection. Data lifecycle management strategies are implemented to automatically archive data that is no longer frequently accessed to low-cost storage media, effectively saving storage space and reducing storage costs. Data retention periods and automatic deletion rules are set to ensure data compliance and privacy protection while avoiding the accumulation of useless data. Through machine learning algorithms, the continuously inflowing data is intelligently identified and classified, which improves the automation and intelligence level of data processing. Dynamically adjust the encryption level and processing strategy of data to make data processing more flexible and efficient. The medium security level partition supports fine-grained access control, allowing specific business logic components to access data on demand after identity verification, which improves the efficiency and flexibility of data processing. According to changes in data usage patterns and new trends in security threats, the effectiveness of existing processing strategies is regularly evaluated, and the encryption level and storage strategy are automatically adjusted. The dynamic adjustment mechanism enables the system to quickly adapt to changing security needs, thereby improving the security and stability of the system. By regularly evaluating the effectiveness of existing processing strategies and optimizing them according to the evaluation results, the system can continue to maintain its optimal state. The evaluation and optimization mechanism enhances the maintainability of the system and reduces the cost and complexity of system maintenance.

[0092] In one embodiment of the present invention, the S15 includes:

[0093] S151. Dynamically adjust the preset segmentation strategy according to the historical data processing efficiency and system load;

[0094] S152, cutting the continuous data stream into independent segments or blocks according to a preset time window or data size, and classifying the data according to data type;

[0095] S154. Assign a globally unique identifier to each data group. For frequently updated data, introduce a version control mechanism to record the UUID and change time of each data update.

[0096] S155, generating metadata tags for each data group, annotating the data group according to the needs of subsequent data processing and analysis; and recording the association relationship between the data groups;

[0097] S156. Store the data group in a distributed file system or database based on the importance and access frequency of the data, and build an index for the data group and metadata tags.

[0098] The working principle of the above technical solution is as follows: according to the historical data processing efficiency and system load, the preset segmentation strategy is dynamically adjusted, including adjusting the time window or data size of data segmentation to adapt to the processing requirements under different time periods or different data flows; according to the preset time window or data size, the continuous data stream is cut into independent fragments or blocks. At the same time, the data is classified according to the data type (such as sensor readings, device logs, external data sources); a universal unique identifier (UUID) is assigned to each data group. For frequently updated data, a version control mechanism is introduced to record the UUID and change time of each data update; metadata tags are generated for each data group, including basic information such as data source, timestamp, encryption level, etc. At the same time, according to the needs of subsequent data processing and analysis, the data group is annotated, including processing priority, expected output format, whether preprocessing is required, etc. In addition, the association between data groups is recorded, such as the continuity of time series data, the correlation between different data sources, etc.; according to the importance and access frequency of the data, the data group is stored in a distributed file system or database. At the same time, indexes are built for data groups and metadata tags.

[0099] The effect of the above technical solution is: by dynamically adjusting the preset segmentation strategy according to the historical data processing efficiency and system load, it can ensure that the data maintains efficient flow during the processing process, avoiding processing bottlenecks or resource waste caused by over-large or under-sized segmentation. Cutting the continuous data stream into independent fragments or blocks and classifying them according to the data type helps to simplify the data processing process and improve processing efficiency. At the same time, the classified data is easier to manage and analyze. Assigning a globally unique UUID to each data group can ensure the uniqueness and traceability of the data. During the data processing and analysis process, the specific data group can be quickly located through the UUID, which is convenient for subsequent operation and management. For frequently updated data, introducing a version control mechanism and recording the UUID and change time of each data update can ensure the integrity and consistency of the data. At the same time, it also helps to track the change history of the data, which is convenient for data auditing and backtracking. Generating metadata tags containing basic information such as data source, timestamp, encryption level, etc. for each data group helps to quickly understand the basic attributes and sources of the data. At the same time, labeling the data group according to the needs of subsequent data processing and analysis can further improve the pertinence and accuracy of data processing. Recording the associations between data groups, such as the continuity of time series data and the correlation between different data sources, helps to explore the potential connections and values ​​between data and provides strong support for subsequent data analysis and application. Storing data groups in a distributed file system or database according to the importance and access frequency of the data, and building indexes for data groups and metadata tags can significantly improve the efficiency of data retrieval and query. At the same time, it also helps to optimize the data storage structure and improve the utilization of storage space. Marking the encryption level of data (high, medium, low) in the metadata tag helps to take corresponding security measures according to the sensitivity of the data and ensure the security of data during storage and transmission. The introduction of version control and UUID also helps to prevent data from being maliciously tampered with or misused, and improves the integrity and credibility of the data.

[0100] In one embodiment of the present invention, the S151 includes:

[0101] Collect the completion time, resource consumption (CPU, memory, disk I / O), and success rate of various data processing tasks (such as cleaning, conversion, aggregation, etc.) in the past period of time (for example, one week), and use machine learning algorithms (such as time series analysis and regression models) to predict the processing efficiency trend in the future under different segmentation strategies;

[0102] The integrated monitoring tool continuously tracks the usage of system resources, including key indicators such as CPU usage, memory usage, network bandwidth, and disk I / O. It uses deep learning models (such as LSTM and GRU) to predict changes in system load in future periods. The system load changes are predicted using the following formula:

[0103]

[0104] Among them, Y t is the system load at time t; X i,t is the i-th resource usage indicator at time t (i=1, 2, 3, 4 represent CPU usage, memory usage, network bandwidth usage and disk I / O usage respectively); β i , γ i , δ i , α i , is the corresponding coefficient; θ1, θ2, θ3 are the coefficients of the lag term; w j , p j is the weight of the lag term; n and m are the negative and positive time ranges of the lag term, respectively; λ and μ are the coefficients of the seasonal component, T is the seasonal period; ψ ij is the interaction coefficient between the i-th and j-th resource use indicators; ε t is the error term.

[0105] Based on the above historical efficiency analysis and system load prediction results, the granularity, frequency and data type classification strategy of data segmentation are dynamically adjusted through intelligent algorithms. For example, during the peak period of system load, a finer-grained segmentation strategy is adopted to reduce the occupation of system resources by a single processing task; for data types with low processing efficiency, the number of parallel processing units after segmentation is increased;

[0106] After executing the new segmentation strategy, continuously monitor the execution efficiency of data processing tasks and system load changes, and evaluate the effectiveness of the strategy through A / B testing or progressive deployment strategies; based on the evaluation results, continuously adjust and optimize algorithm parameters to form a closed-loop feedback mechanism.

[0107] The working principle of the above technical solution is as follows: First, the system collects the completion time, resource consumption (CPU, memory, disk I / O) and success rate of various data processing tasks (such as cleaning, conversion, aggregation, etc.) in the past period of time (such as a week). Then, these historical data are analyzed using machine learning algorithms (such as time series analysis, regression models, etc.) to predict the processing efficiency trend in the future under different segmentation strategies. The integrated monitoring tool continuously tracks the system resource usage, including key indicators such as CPU usage, memory usage, network bandwidth and disk I / O. These monitoring data are processed through deep learning models (such as LSTM, GRU, etc.) to predict the system load changes in the future period. Based on the above historical efficiency analysis and system load prediction results, the granularity of data segmentation (such as time window size, data block size), segmentation frequency and data type classification strategy are dynamically adjusted through intelligent algorithms. The new segmentation strategy is applied to the actual data processing process to observe its effect; after the new segmentation strategy is implemented, the execution efficiency of data processing tasks and system load changes are continuously monitored. The effectiveness of the strategy is evaluated through A / B testing or progressive deployment strategies to determine whether it has achieved the expected goals; based on the evaluation results, the optimization algorithm parameters are continuously adjusted to further improve data processing efficiency and system resource utilization. This process forms a closed-loop feedback mechanism that enables the system to continuously learn and improve.

[0108] The effect of the above technical solution is: by predicting the processing efficiency trend in the future period through machine learning algorithms, the system can perceive the performance under different segmentation strategies in advance, so as to select the optimal strategy for implementation; avoiding the waste of resources and time delay caused by blindly trying different strategies in traditional methods. According to the system load prediction results, the granularity, segmentation frequency and data type classification strategy of data segmentation are dynamically adjusted, so that the data processing tasks can better adapt to the current state of the system. During the peak period of system load, the use of a finer-grained segmentation strategy can reduce the occupation of system resources by a single processing task and avoid system overload; for data types with low processing efficiency, increasing the number of parallel processing units after segmentation can significantly improve the processing speed. By dynamically adjusting the data segmentation strategy through intelligent algorithms, the system can better achieve load balancing. It avoids the situation where other tasks cannot run normally due to a certain processing task occupying too many resources, and improves the overall stability and reliability of the system. Continuously monitoring the execution efficiency of data processing tasks and system load changes enables the system to timely discover and prevent potential failure points. By evaluating the effectiveness of the strategy through A / B testing or progressive deployment strategies, the system can be gradually optimized and improved without affecting normal operation, reducing the risk of business interruption caused by system failures. Dynamic adjustment of data segmentation strategies enables data processing tasks to be completed more quickly, thereby improving the response speed of the system. By improving data processing efficiency and resource utilization, the system can more effectively utilize limited computing resources. Forming a closed-loop feedback mechanism means that the system can continuously learn and optimize its own performance. It helps to promote technological innovation and sustainable development, allowing the system to continuously adapt to new data processing needs and challenges. With the continuous advancement of machine learning and deep learning technologies, the system can gradually improve its prediction and analysis capabilities. This will enable the system to better cope with complex data processing scenarios and changes in demand in the future.By introducing multiple types of variables and interaction terms, the above formula enables the model to more accurately capture the complex relationships in the data, thereby improving the accuracy of predictions; nonlinear transformations and polynomial terms allow the model to capture nonlinear dependencies between variables, which is crucial for many practical problems; lagged variables and seasonal factors enable the model to take into account the autocorrelation and periodicity in time series data, thereby more accurately predicting future trends and patterns; the model can more accurately predict system load and reduce prediction errors, which is crucial for resource planning and system maintenance; through accurate predictions, resources can be allocated more efficiently, reducing resource waste and improving resource utilization; prediction models can help identify risk points that may cause system overload, so that measures can be taken in advance to avoid system crashes; by optimizing resource usage, operating costs can be reduced, such as reducing unnecessary hardware investment and energy consumption; accurate load predictions can ensure stable system operation and improve user satisfaction and loyalty; provide data-driven decision support for management to help make long-term strategic plans and short-term operational decisions; complex models may be more likely to detect abnormal patterns in data, so that problems can be discovered and intervened in a timely manner.

[0109] In one embodiment of the present invention, the S2 includes:

[0110] S21. The edge task allocation unit monitors the key indicators of the edge task processor (ETP) in real time, wherein the key indicators include load, computing power, memory usage, and energy consumption;

[0111] S22. Based on the monitoring data, evaluate the real-time processing capability and remaining resources of the edge task processor;

[0112] S23, combining the task priority (urgency, importance), data size, transmission delay requirements and data encryption level, the edge task allocation unit formulates a dynamic allocation strategy based on load balancing;

[0113] S24, according to the allocation strategy, the edge task allocation unit allocates the data group to the most suitable edge task processor for processing;

[0114] S25, and generate an identification sequence including data group allocation information and edge task processor identifier, and use the identification sequence to recycle subsequent results and track errors.

[0115] The working principle of the above technical solution is as follows: the edge task allocation unit (or management unit) monitors the key indicators of each ETP, such as load, computing power, memory usage and energy consumption, in real time. Based on the monitoring data, the edge task allocation unit evaluates the real-time processing capability of the ETP. At the same time, the remaining resources of the ETP, including idle computing power, memory space, etc., are calculated to determine whether it can undertake new tasks. The edge task allocation unit formulates a dynamic allocation strategy based on factors such as task priority (urgency, importance), data size, transmission delay requirements and data encryption level. The goal of the strategy is to ensure that tasks can be assigned to the most suitable ETP to achieve load balancing and efficient processing. According to the formulated allocation strategy, the edge task allocation unit allocates data groups to the most suitable ETP for processing. The processing capacity, remaining resources and characteristics and requirements of the ETP are considered in the allocation process to ensure that the task can be completed efficiently and accurately. The edge task allocation unit generates an identification sequence containing data group allocation information and ETP identifier for each allocated data group. Through this identification sequence, the system can track the processing progress of the data, recycle the processing results, and track and locate errors when they occur.

[0116] The effect of the above technical solution is that the edge task allocation unit can monitor the key indicators of the edge task processor (ETP) in real time, such as load conditions, computing power, memory usage and energy consumption, and can timely discover and handle potential problems of the ETP to ensure the stable operation of the system. Based on the monitoring data, the real-time processing capacity and remaining resources of the ETP are evaluated, which provides an important basis for the subsequent dynamic allocation strategy and helps to achieve the optimal allocation of resources. Combined with the task priority (urgency, importance), data size, transmission delay requirements and data encryption level, the edge task allocation unit formulates a dynamic allocation strategy based on load balancing, which can flexibly adjust the task allocation according to the needs of different tasks and the real-time status of the ETP, and improve the overall performance of the system. According to the allocation strategy, the edge task allocation unit allocates the data group to the most suitable ETP for processing, which can ensure that the task is executed efficiently and accurately, while reducing the energy consumption and cost of the system. Generate an identification sequence containing data group allocation information and ETP identifiers, which provides an important basis for the subsequent result recovery and error tracking. Through the identification sequence, the processing results can be easily recovered and errors can be tracked, and problems can be discovered and handled in time to improve the reliability and stability of the system. Through real-time monitoring, resource evaluation and dynamic allocation strategies, this technical solution can make full use of ETP resources and improve the overall performance of the system. By optimizing task allocation and result recovery, this technical solution can reduce the energy consumption and cost of the system and improve resource utilization efficiency. Through the generation of identification sequences and error tracking mechanisms, this technical solution can improve the reliability and stability of the system and ensure that tasks are executed accurately and efficiently.

[0117] In one embodiment of the present invention, S3 includes:

[0118] S31, the edge task processor fuses the data in each data group based on a weighted average algorithm to generate a preliminary fusion result;

[0119] S32. For data groups that require in-depth analysis, they are integrated through machine learning algorithms;

[0120] S33. For data in a specific field, logical reasoning and judgment are performed based on a predefined set of rules;

[0121] S34. After completing the data fusion, the processing results of each data group are encapsulated into a standardized data packet, which contains the processing results, processing time, ETP identifier and the identifier of the original data group, and is sent back to the edge task allocation unit.

[0122] The working principle of the above technical solution is as follows: the edge task processor first assigns weights to the data items in each data group according to the reliability and importance of the data. Based on the assigned weights, ETP uses a weighted average algorithm to fuse the data in each data group, comprehensively considers the importance of each data item, and generates a preliminary fusion result. For data groups that require in-depth analysis, ETP uses a machine learning algorithm for fusion, which can mine complex patterns and relationships in the data and improve the accuracy and depth of fusion. During the fusion process, ETP comprehensively considers the spatiotemporal characteristics, correlation and complementarity of the data. For data in a specific field, ETP performs logical reasoning and judgment based on a predefined set of rules. The rule set reflects the expertise and experience in the field and helps to generate more refined and meaningful conclusions. In the process of rule fusion, ETP considers not only the direct meaning of the data, but also the implicit and causal relationships between the data. After completing the data fusion, ETP encapsulates the processing results of each data group into a standardized data packet. The data packet contains key information such as the processing result, processing time, ETP identifier, and the identifier of the original data group. ETP sends the encapsulated data packet back to the edge task allocation unit.

[0123] The effect of the above technical solution is: by assigning weights to each data item according to the reliability and importance of the data, and fusing based on the weighted average algorithm, the technical solution can flexibly respond to the characteristics of different data groups and ensure that key data is fully reflected in the fusion results. For data groups that require in-depth analysis, the use of machine learning algorithms for fusion can mine deep information in the data, improve the accuracy and efficiency of fusion, and enable the technical solution to adapt to complex and changeable data processing needs. For data in specific fields, logical reasoning and judgment based on pre-defined rule sets can generate more refined and meaningful conclusions, ensuring the effectiveness and accuracy of the technical solution in specific fields. In the process of machine learning algorithm fusion, comprehensive consideration of the spatiotemporal characteristics, correlation and complementarity of the data helps to improve the comprehensiveness and accuracy of the fusion results, reveal the internal connection between the data, and improve the quality of data fusion. Rule fusion not only considers the direct meaning of the data, but also considers the implicit relationship and causal relationship between the data, which helps to generate more in-depth and meaningful conclusions, so that the technical solution can reveal the deep information behind the data and improve the value of data fusion. After completing the data fusion, the processing results of each data group are encapsulated into a standardized data packet, which contains information such as the processing results, processing time, ETP identifier, and the identifier of the original data group. Standardized encapsulation helps the smooth progress of the subsequent processing process, and also facilitates data traceability and management. The data packet is sent back to the edge task allocation unit to achieve timely feedback on the processing results. This helps the edge task allocation unit understand the processing progress and results of each ETP, and provides an important basis for subsequent task allocation and scheduling. Through flexible data fusion strategies and methods, this technical solution can efficiently process large amounts of data, improve the speed and efficiency of data processing, reduce the energy consumption and cost of the system, and improve resource utilization efficiency. Different data fusion strategies and methods can be dynamically adjusted according to the real-time status and task requirements of the ETP to achieve load balancing. This helps to avoid overloading or idleness of certain ETPs and improve the stability and reliability of the entire system.

[0124] In one embodiment of the present invention, the S34 includes:

[0125] Before packaging, the data fusion results are verified. After passing the verification, the processing results are converted into a unified and structured data format according to the preset data format standard;

[0126] The timestamp of the whole process of receiving, processing and generating fusion results of each data group is recorded, and this time information is used as part of the data packet;

[0127] Assign a unique ETP identifier to each data group and append the ETP identifier to the data packet;

[0128] The identification information of the original data group is retained, and a mapping relationship between the original data group identification and the processing result is established in the data packet; according to the sensitivity and security requirements of the data, the data packet is encrypted and encapsulated using a secure encapsulation protocol;

[0129] Compress the data packets and dynamically adjust the parameters of the compression algorithm according to the network bandwidth and delay. Set the priority of the data packets according to the importance and urgency of the data. Use the routing algorithm in the edge computing network to plan the optimal transmission path for the data packets. The parameters of the compression algorithm are dynamically adjusted by the following formula:

[0130]

[0131] Among them, C p is the compression parameter; B t is the current total network bandwidth; B a is the used bandwidth; L is the network delay; S is the packet size; R is the target compression ratio; T is the packet transmission time; C m is the maximum compression parameter; C r is the resource utilization threshold; α and β are adjustment coefficients used to control the impact of bandwidth and delay on compression parameters; γ and ξ are additional adjustment coefficients used to control the impact of resource utilization and transmission time on compression parameters; n is the number of packets; S i and R i is the size of the ith data packet and the target compression ratio; K is the upper limit.

[0132] The encapsulated data packet is sent back to the edge task allocation unit through the edge computing network, and a confirmation message from the edge task allocation unit is received. If no confirmation message is received, retransmission is performed according to the retransmission strategy.

[0133] The working principle of the above technical solution is as follows: before encapsulation, the results of data fusion are first verified, including logic checking, range verification and outlier detection. According to the preset data format standard, the processing results are converted into a unified and structured data format. Including encoding the results into specific data types (such as integers, floating point numbers, strings, etc.), and organizing the data into nested data structures (such as JSON, XML, etc.). The timestamp of the entire process from receiving, processing to generating fusion results for each data group is recorded, including the start processing time, the end processing time and the time node of any key steps. A unique ETP identifier is assigned to each data group, which is used to uniquely identify the data group and its processing task in the edge computing environment. The identification information of the original data group, such as the UUID and data source identifier of the data group, is retained, and a mapping relationship between the original data group identifier and the processing result is established in the data packet. The data packet is encrypted according to the sensitivity and security requirements of the data. The data packet is encapsulated using a secure encapsulation protocol (such as HTTPS, TLS, etc.) to ensure the security and privacy of the data during transmission. The data packet is compressed to reduce the bandwidth occupancy and delay of data transmission. Dynamically adjust the parameters of the compression algorithm according to the network bandwidth and latency. Set priorities for data packets according to the importance and urgency of the data. Use the routing algorithm in the edge computing network to plan the optimal transmission path for the data packet. Send the encapsulated data packet back to the edge task allocation unit through the edge computing network. Receive the confirmation message from the edge task allocation unit to confirm that the data packet has been successfully received. If the confirmation message is not received, retransmit according to the retransmission strategy to ensure the reliability and integrity of the data.

[0134] The effects of the above technical solutions are: through verification methods such as logic checking, range verification and outlier detection, the accuracy and reliability of data fusion results are ensured, the impact of erroneous data and abnormal data on subsequent processing and analysis is reduced, and data quality is improved. Converting the processing results into a unified and structured data format is conducive to the standardization and normalization of subsequent processing and analysis, reducing the complexity of data processing and parsing, and improving data processing efficiency. Recording the timestamp of the entire process of each data group from receiving, processing to generating fusion results is conducive to data traceability, helping to quickly locate the problem when a problem occurs, and improving problem solving efficiency. Assigning a unique ETP identifier to each data group and retaining the identification information of the original data group is conducive to uniquely identifying the data group and its processing task in the edge computing environment, facilitating data tracking and verification, and improving data security. According to the sensitivity and security requirements of the data, the data packet is encrypted and encapsulated using a secure encapsulation protocol, which helps to protect the security and privacy of the data during transmission and prevent data leakage and tampering. Compressing the data packet and dynamically adjusting the parameters of the compression algorithm according to the network bandwidth and delay conditions helps to reduce the bandwidth occupancy and delay of data transmission and improve data transmission efficiency. Setting priorities for data packets based on their importance and urgency helps achieve efficient data scheduling and transmission in edge computing networks, ensuring that critical data can be transmitted and processed first. Using routing algorithms in edge computing networks to plan the optimal transmission path for data packets helps reduce the number of hops and delays in data transmission and improve the stability and reliability of data transmission. If no confirmation message is received from the edge task allocation unit, retransmission is performed according to the retransmission strategy, which helps ensure the integrity and reliability of the data and prevents the impact of data loss or transmission failure on subsequent processing and analysis.The above dynamic adjustment formula can optimize the compression degree of data packets according to the current network bandwidth usage (the ratio of used bandwidth to total bandwidth) by dynamically adjusting the compression parameters, thereby effectively improving bandwidth utilization and reducing network resource waste when the network bandwidth is tight; the formula takes into account the impact of network delay on compression parameters, so that in a high-delay environment, the system can adjust the compression algorithm accordingly to reduce the additional delay caused by compression, thereby improving the real-time performance of data transmission; by setting the resource utilization rate threshold and the maximum compression parameter, it can ensure that the system does not consume excessive computing resources when processing data, thereby maintaining the overall performance and stability of the system; based on the importance and urgency of the data packet, In terms of urgency, dynamic adjustment of compression parameters can give priority to the processing of high-priority data packets, ensuring that critical data can arrive in time even when the network environment is poor; the system can adaptively adjust the compression strategy according to changes in the network environment (such as bandwidth, latency, resource utilization, etc.), enhancing the flexibility and adaptability of the network and improving the reliability of overall data transmission; the formula takes into account various aspects of the network through the combination of multiple parameters, making the setting of compression parameters more scientific and reasonable, and able to achieve the best transmission effect under different network conditions; through dynamic adjustment, network congestion or data loss caused by static settings can be effectively avoided, thereby improving the success rate and stability of data transmission.

[0135] In one embodiment of the present invention, the S4 includes:

[0136] S41, the edge task allocation unit integrates the processing results of each received data group according to the identification sequence, and during the integration process, checks the consistency of the results and resolves conflicts;

[0137] S42, the integrated results are first uploaded to the cloud platform, and further analyzed, stored and backed up through the cloud platform;

[0138] S43. At the same time, the edge task allocation unit distributes key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism to make real-time decisions and responses.

[0139] The working principle of the above technical solution is as follows: the edge task allocation unit first integrates the processing results of each data group received according to the identification sequence (such as the ETP identifier). During the integration process, the edge task allocation unit will check the consistency of the results to ensure that the processing results between different data groups are coordinated and not contradictory. At the same time, for possible conflicts or inconsistencies, the edge task allocation unit will perform corresponding solutions, such as eliminating conflicts through algorithm optimization, data correction, etc. The integrated data results will be uploaded to the cloud platform. As the center of data processing and analysis, the cloud platform has powerful computing power and storage resources. On the cloud platform, the data can be further analyzed, stored and backed up. The analysis process involves advanced algorithms such as data mining and machine learning to extract valuable information from the data. Storage and backup ensure the persistence and security of the data, and the data can be recovered from the cloud platform even if the edge device fails or the data is lost. At the same time, the edge task allocation unit will also distribute key results or real-time updates to the corresponding edge devices through efficient messaging mechanisms (such as WebSocket, RESTfulAPI, etc.). These edge devices are usually deployed at the edge of the network, close to the data source and the user end, so they can respond to user requests faster and make real-time decisions. By distributing key results or real-time updates, edge devices can respond and make decisions quickly based on the latest data, thereby improving the response speed and decision-making efficiency of the entire system.

[0140] The effect of the above technical solution is that the edge task allocation unit can quickly integrate the processing results of each data group received according to the identification sequence, which not only improves the efficiency of data processing, but also ensures the integrity and consistency of the data. During the integration process, the edge task allocation unit will check the consistency of the results, promptly discover and resolve possible conflicts, ensure that the integrated data results are accurate and reliable, and provide strong support for subsequent analysis and decision-making. After the integrated data results are uploaded to the cloud platform, the powerful computing power of the cloud platform can be used for further analysis. The analysis involves advanced algorithms such as data mining and machine learning, which helps to extract valuable information from the data and provide a basis for business decisions. The cloud platform also provides safe and reliable storage and backup services. By storing data on the cloud platform, the persistence and security of the data can be ensured. Even if the edge device fails or data is lost, the data can be quickly restored from the cloud platform to ensure business continuity. The edge task allocation unit can distribute key results or real-time updates to the corresponding edge devices through efficient messaging mechanisms (such as WebSocket, RESTfulAPI, etc.). This distribution mechanism ensures the real-time and accuracy of the data, enabling edge devices to respond and make decisions quickly based on the latest data. Since edge devices are usually deployed at the edge of the network, close to data sources and users, they can respond to user requests faster and make real-time decisions. This technical solution supports flexible deployment and configuration of edge devices and cloud platforms. Edge devices can be dynamically adjusted according to business needs, while cloud platforms can be expanded and upgraded according to data processing and analysis needs. As business develops and data grows, this technical solution can easily cope with the increase in data volume and complexity. By adding edge devices and expanding cloud platform resources, it can ensure that the system always maintains good performance and scalability.

[0141] One embodiment of the present invention, as Figure 2 As shown, a multi-data fusion system based on edge computing, the system includes:

[0142] Data receiving unit: receives heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divides each task data into identified data groups according to the preset segmentation strategy according to the encryption level corresponding to each task in the heterogeneous resource data;

[0143] Allocation calculation unit: according to the running status of the edge task processor, the data group is allocated and calculated according to the preset allocation strategy; after the allocation is completed, the identification sequence of the allocation method is obtained;

[0144] Computation processing unit: performs computation on the assigned data group through the edge task processor. After the data computation is completed, the result in the data group is transmitted back to the edge task assignment unit;

[0145] Result return unit: The edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge devices.

[0146] The working principle of the above technical solution is as follows: the edge task allocation unit, as the core component of the system, is responsible for receiving heterogeneous resource data from different sensors, devices and sources. The data includes environmental parameters such as temperature, humidity, light, as well as device status, location information, etc. According to the encryption level corresponding to each task in the heterogeneous resource data, the edge task allocation unit will perform security processing on each task data. Then, according to the preset segmentation strategy, each task data is divided into a data group with a unique identifier. The edge task allocation unit formulates a data allocation strategy based on the current operating status of the edge task processor (such as computing power, load status, etc.), as well as factors such as task priority, data size and transmission delay. According to the allocation strategy, the preprocessed data group is allocated to the corresponding edge task processor for calculation. At the same time, the identification sequence of the allocation method is recorded. After receiving the allocated data group, the edge task processor will perform a series of calculations. The data is weighted according to the reliability and importance of the data. The data is integrated and analyzed using mathematical models and algorithms to obtain more comprehensive and accurate information. The data is logically inferred and judged according to the pre-set rules to achieve comprehensive analysis of the data and deduction of conclusions. After the calculation is completed, the result data group is transmitted back to the edge task allocation unit. The edge task allocation unit obtains the calculation results of each task according to the previously recorded identification sequence. These results are summarized and sorted to form the final calculation result set. The final calculation result set is transmitted to the cloud platform, and at the same time, the relevant calculation results are transmitted to the corresponding edge devices to support real-time decision-making and response.

[0147] The effect of the above technical solution is: through edge computing, data can be processed at a location closer to the data source, significantly reducing the time delay of data transmission to the cloud, enabling the system to respond to data changes faster, especially in application scenarios with high real-time requirements (such as autonomous driving, intelligent manufacturing, etc.), which can significantly improve the response speed and performance of the system. The edge task allocation unit can intelligently allocate data groups for calculation according to the operating status of the edge task processor. It improves the utilization rate of computing resources and avoids computing bottlenecks caused by insufficient resources. By receiving heterogeneous resource data from different sensors, devices and sources and effectively fusing them, the system can obtain more comprehensive and accurate information and provide a more reliable basis for decision-making. Through weighted fusion, the system can weight the data according to the reliability and importance of the data, thereby improving the accuracy of the overall fusion result. Model fusion uses mathematical models and algorithms to integrate data, further improving the comprehensiveness and accuracy of the data. According to the encryption level corresponding to each task in the heterogeneous resource data, the data is graded, which not only protects the security of sensitive data, but also avoids the risk of data leakage. Some sensitive data can be processed locally, reducing the risk of transmission on the network, further improving the security of the system and protecting the privacy of users. The edge task allocation unit can dynamically adjust the allocation strategy according to the operating status and data characteristics of the edge task processor, so that the system can flexibly respond to different application scenarios and data requirements. With the continuous increase and upgrade of edge devices, this method can easily expand the processing capacity of the system; it provides convenience for future expansion and upgrade, and reduces the maintenance cost of the system. In the field of intelligent manufacturing, the above technical solution can monitor the operating status of the production line in real time, and make intelligent decisions based on the data fusion results. This helps to optimize the production process, improve production efficiency and reduce production costs. In application scenarios such as smart homes and smart cities, this method can process and analyze user data in real time. This helps to provide a more personalized and intelligent service experience and improve user satisfaction and loyalty.

[0148] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-data fusion method based on edge computing, characterized in that: The method comprises: S1. Receive heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divide each task data into data groups with identification according to the encryption level corresponding to each task in the heterogeneous resource data according to the preset segmentation strategy; S2. According to the running status of the edge task processor, the data group is allocated and calculated according to the preset allocation strategy; after the allocation is completed, an identification sequence of the allocation method is obtained; S3, the edge task processor performs calculation processing on the assigned data group, and after the data calculation processing is completed, the result in the data group is transmitted back to the edge task allocation unit; S4. The edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge device.

2. According to the multi-data fusion method based on edge computing according to claim 1, it is characterized in that: Said S1 comprises: S11, the edge task allocation unit receives heterogeneous resource data from various sensors, smart devices and external data sources through high-speed network interfaces and Internet of Things protocols; S12. In addition, during the heterogeneous data reception process, the edge task allocation unit performs a preliminary check on the integrity of the data; S13, based on each received task data, the edge task allocation unit evaluates its encryption level according to the sensitivity and privacy requirements of the data; S14. Based on the evaluation results, different processing strategies and storage methods are adopted for different levels of data; among which, the encryption level is divided into three levels: high, medium and low; S15. According to a preset segmentation strategy, each task data is divided into multiple data groups with unique identifiers, and a metadata tag is generated for each data group.

3. According to the multi-data fusion method based on edge computing as claimed in claim 2, it is characterized in that: The S14 comprises: For data assessed as having a high encryption level, the edge task allocation unit immediately uses a strong encryption algorithm to encrypt the data and directs the data to be stored in a specially configured high-security storage area. Each data access is recorded in detail. According to the specific usage scenarios of the data, a dynamic encryption strategy is implemented for data with a medium encryption level, that is, data is encrypted during data transmission and decrypted as needed during internal processing. The data is stored in a partition with a medium security level. For some sensitive fields that do not need to be fully encrypted, a data desensitization strategy is implemented; Encrypt low-level encryption levels through basic encryption algorithms, and ensure data integrity through checksums or hash values, and store such data through distributed file systems or object storage services; Implement data lifecycle management strategies to automatically archive data that is no longer frequently accessed to low-cost storage media, while setting data retention periods and automatic deletion rules; Through machine learning algorithms, it intelligently identifies and classifies continuously inflowing data and dynamically adjusts its encryption level and processing strategy; Regularly evaluate the effectiveness of existing processing strategies and automatically adjust encryption levels and storage strategies based on changes in data usage patterns, new trends in security threats, or more.

4. According to the multi-data fusion method based on edge computing as claimed in claim 2, it is characterized in that: The S15 comprises: S151. Dynamically adjust the preset segmentation strategy according to the historical data processing efficiency and system load; S152, cutting the continuous data stream into independent segments or blocks according to a preset time window or data size, and classifying the data according to data type; S153. Assign a globally unique identifier to each data group. For frequently updated data, introduce a version control mechanism to record the UUID and change time of each data update. S154, generating metadata tags for each data group, annotating the data group according to the needs of subsequent data processing and analysis; and recording the association relationship between the data groups; S155. Store the data group in a distributed file system or database based on the importance and access frequency of the data, and build an index for the data group and metadata tags.

5. According to the multi-data fusion method based on edge computing according to claim 4, it is characterized in that: The S151 includes: Collect the completion time, resource consumption, and success rate of various data processing tasks in the past period of time, and use machine learning algorithms to predict the processing efficiency trend in the future under different segmentation strategies; Integrated monitoring tools continuously track system resource usage and use deep learning models to predict changes in system load in future periods. Based on the above historical efficiency analysis and system load prediction results, the granularity, frequency and data type classification strategy of data segmentation are dynamically adjusted through intelligent algorithms; After executing the new segmentation strategy, continuously monitor the execution efficiency of data processing tasks and system load changes, and evaluate the effectiveness of the strategy through A / B testing or progressive deployment strategies; based on the evaluation results, continuously adjust and optimize algorithm parameters to form a closed-loop feedback mechanism.

6. According to the multi-data fusion method based on edge computing in claim 1, it is characterized in that: The S2 comprises: S21. The edge task allocation unit monitors the key indicators of the edge task processor in real time, wherein the key indicators include load, computing power, memory usage, and energy consumption; S22. Based on the monitoring data, evaluate the real-time processing capability and remaining resources of the edge task processor; S23. Formulate a dynamic allocation strategy based on load balancing based on task priority, data size, transmission delay requirements and data encryption level; S24, according to the allocation strategy, the edge task allocation unit allocates the data group to the most suitable edge task processor for processing; S25, and generate an identification sequence including data group allocation information and edge task processor identifier, and use the identification sequence to recycle subsequent results and track errors.

7. The multi-data fusion method based on edge computing according to claim 1, characterized in that: The S3 includes: S31, the edge task processor fuses the data in each data group based on a weighted average algorithm to generate a preliminary fusion result; S32. For data groups that require in-depth analysis, they are integrated through machine learning algorithms; S33. For data in a specific field, logical reasoning and judgment are performed based on a predefined set of rules; S34. After completing the data fusion, the processing results of each data group are encapsulated into a standardized data packet, which contains the processing results, processing time, ETP identifier and the identifier of the original data group, and is sent back to the edge task allocation unit.

8. The multi-data fusion method based on edge computing according to claim 7 is characterized in that: The S34 comprises: Before packaging, the data fusion results are verified. After passing the verification, the processing results are converted into a unified and structured data format according to the preset data format standard; The timestamp of the whole process of receiving, processing and generating fusion results of each data group is recorded, and this time information is used as part of the data packet; Assign a unique ETP identifier to each data group and append the ETP identifier to the data packet; The identification information of the original data group is retained, and a mapping relationship between the original data group identification and the processing result is established in the data packet; according to the sensitivity and security requirements of the data, the data packet is encrypted and encapsulated using a secure encapsulation protocol; Compress data packets and dynamically adjust the parameters of the compression algorithm according to network bandwidth and latency. Set priorities for data packets based on the importance and urgency of the data. Use routing algorithms in edge computing networks to plan the optimal transmission path for data packets. The encapsulated data packet is sent back to the edge task allocation unit through the edge computing network, and a confirmation message from the edge task allocation unit is received. If no confirmation message is received, retransmission is performed according to the retransmission strategy.

9. The multi-data fusion method based on edge computing according to claim 1, characterized in that: The S4 comprises: S41, the edge task allocation unit integrates the processing results of each received data group according to the identification sequence, and during the integration process, performs consistency check on the results and resolves conflicts; S42, the integrated results are first uploaded to the cloud platform, and further analyzed, stored and backed up through the cloud platform; S43. At the same time, the edge task allocation unit distributes key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism to make real-time decisions and responses.

10. A multi-data fusion system based on edge computing, characterized in that: The system comprises: Data receiving unit: receives heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divides each task data into identified data groups according to the preset segmentation strategy according to the encryption level corresponding to each task in the heterogeneous resource data; Allocation calculation unit: according to the running status of the edge task processor, the data group is allocated and calculated according to the preset allocation strategy; after the allocation is completed, the identification sequence of the allocation method is obtained; Computation processing unit: performs computation on the assigned data group through the edge task processor. After the data computation is completed, the result in the data group is transmitted back to the edge task assignment unit; Result return unit: The edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge devices.

Images