A multi-data fusion method and system based on edge computing

Abstract

The application provides a multi-data fusion method and system based on edge computing. It belongs to the technical field of edge computing. The method comprises the following steps: receiving heterogeneous resource data from different sensors, devices and sources through an edge task allocation unit; dividing each task data into data groups with marks according to a preset division strategy according to the encryption level corresponding to each task in the heterogeneous resource data; distributing and calculating the data groups according to a preset distribution strategy according to the running state of an edge task processor; obtaining the mark sequence of the distribution mode after the distribution is completed; calculating and processing the distributed data groups through the edge task processor, and transmitting the results in the data groups back to the edge task allocation unit after the data calculation and processing are completed. Through the intelligent scheduling of the edge task allocation unit, the data from different sensors and devices can be received and processed in real time, so that the efficiency and response speed of data processing are improved.

Description

TECHNICAL FIELD

[0001] The application provides a multi-data fusion method and system based on edge computing, and belongs to the technical field of edge computing. BACKGROUND

[0002] With the rapid development of Internet of Things and big data technology, various intelligent devices have generated massive multi-modal data. Due to network delay and bandwidth limitation, the traditional cloud computing mode is difficult to process these data in real time. Edge computing reduces the delay and cost of data transmission by migrating computing tasks and data storage to the network edge. However, in the edge computing environment, how to realize effective fusion of multi-source heterogeneous data is still a problem to be solved. SUMMARY

[0003] The application provides a multi-data fusion method and system based on edge computing to solve the problems mentioned in the background.

[0004] The application provides a multi-data fusion method based on edge computing, which comprises the following steps:

[0005] S1, receiving heterogeneous resource data from different sensors, devices and sources through an edge task allocation unit; according to the encryption level corresponding to each task in the heterogeneous resource data, dividing each task data into a data group with an identifier according to a preset segmentation strategy;

[0006] S2, according to the running state of an edge task processor, performing distributed computing on the data group according to a preset distribution strategy; after the distribution is completed, obtaining an identification sequence of the distribution mode;

[0007] S3, performing computing and processing on the distributed data group through the edge task processor; after the data computing and processing is completed, transmitting the result in the data group back to the edge task allocation unit;

[0008] S4, obtaining the computing result of each task according to the identification sequence by the edge task allocation unit, and transmitting the computing result to a cloud platform and a corresponding edge device.

[0009] Further, the S1 comprises the following steps:

[0010] S11, the edge task allocation unit receives heterogeneous resource data from various sensors, intelligent devices and external data sources through a high-speed network interface and an Internet of Things protocol;

[0011] S12, and in the process of receiving heterogeneous data, the edge task allocation unit preliminarily checks the integrity of the data;

[0012] S13, based on each received task data, the edge task allocation unit evaluates the encryption level of the data according to the sensitivity and privacy requirements of the data;

[0013] S14, based on the evaluation result, different processing strategies and storage methods are adopted for data of different levels; wherein the encryption level is divided into three levels of high, medium and low;

[0014] S15, according to the preset segmentation strategy, each task data is divided into multiple data groups with unique identification, and metadata tags are generated for each data group.

[0015] Further, the S14 comprises:

[0016] For data evaluated as high encryption level, the edge task allocation unit immediately adopts strong encryption algorithm for encryption processing, and stores such data in a specially configured high security storage area; and detailed records are kept for each data access;

[0017] According to the specific use scenario of the data, dynamic encryption strategy is implemented for data of medium encryption level, that is, encryption during data transmission, and decryption according to needs during internal processing, and the data is stored in a partition with medium security level. For some sensitive but not completely encrypted fields, data desensitization strategy is implemented;

[0018] Low-level encryption is encrypted through basic encryption algorithm, and data integrity is guaranteed through checksum or hash value. Distributed file system or object storage service is used to store such data;

[0019] Implement data life cycle management strategy, automatically archive data that is no longer frequently accessed to low-cost storage medium, and set data retention period and automatic deletion rules;

[0020] Through machine learning algorithm, the continuously flowing data is intelligently identified and classified, and the encryption level and processing strategy are dynamically adjusted;

[0021] Periodically evaluate the effectiveness of the existing processing strategy, and automatically adjust the encryption level and storage strategy according to the changes of data use mode, new trends of security threats.

[0022] Further, the S15 comprises:

[0023] S151, according to the historical data processing efficiency and system load condition, the preset segmentation strategy is dynamically adjusted;

[0024] S152, according to the preset time window or data size, the continuous data stream is cut into independent fragments or blocks, and the data is classified according to the data type;

[0025] S153, assign a globally unique identifier to each data group, introduce a version control mechanism for frequently updated data, record the UUID and change time of each data update;

[0026] S154, generate metadata tags for each data group, label the data group according to the needs of subsequent data processing and analysis, and record the association between data groups;

[0027] S155, according to the importance and access frequency of data, store the data group in a distributed file system or database, and build an index for the data group and metadata tag.

[0028] Further, the S151 comprises:

[0029] Collect the completion time, resource consumption and success rate of various data processing tasks in the past period, and predict the processing efficiency trend in the future period under different splitting strategies through machine learning algorithms;

[0030] Integrate monitoring tools to continuously track system resource usage, and predict future system load changes through deep learning models;

[0031] Based on the above historical efficiency analysis and system load prediction results, dynamically adjust the granularity, splitting frequency and data type classification strategy of data splitting through intelligent algorithms;

[0032] After executing the new splitting strategy, continuously monitor the execution efficiency of data processing tasks and the change of system load, evaluate the strategy effect through A / B test or gradual deployment strategy; according to the evaluation result, continuously adjust and optimize the algorithm parameters, form a closed loop feedback mechanism;

[0033] Further, the S2 comprises:

[0034] S21, the edge task allocation unit monitors the key indicators of the edge task processor in real time, including load, computing power, memory occupation and energy consumption;

[0035] S22, based on the monitoring data, evaluate the real-time processing capacity and remaining resources of the edge task processor;

[0036] S23, combined with task priority, data size, transmission delay requirement and data encryption level, develop a dynamic allocation strategy based on load balancing;

[0037] S24, according to the allocation strategy, the edge task allocation unit allocates data groups to the most suitable edge task processor for processing;

[0038] S25, and generate an identification sequence containing data group allocation information and edge task processor identifier, through which the subsequent results are recycled and error tracked.

[0039] Further, the S3 comprises:

[0040] S31, the edge task processor fuses the data in each data group based on a weighted average algorithm to generate a preliminary fusion result;

[0041] S32, for the data group that needs in-depth analysis, the machine learning algorithm is used for fusion;

[0042] S33, for the data of specific field, logical reasoning and judgment are made based on the pre-defined rule set;

[0043] S34, after the data fusion, the processing result of each data group is packaged into a standardized data packet, the data packet contains the processing result, processing time, ETP identifier and the identification of the original data group, and is sent back to the edge task allocation unit.

[0044] Further, the S34 comprises:

[0045] Before packaging, the result of data fusion is verified, and after the verification, the processing result is converted into a unified and structured data format according to the pre-set data format standard;

[0046] The time stamp of the whole process from receiving, processing to generating the fusion result of each data group is recorded, and the time information is taken as a part of the data packet;

[0047] A unique ETP identifier is allocated for each data group, and the ETP identifier is attached to the data packet;

[0048] The identification information of the original data group is reserved, and the mapping relationship between the original data group identification and the processing result is established in the data packet; according to the sensitivity and security requirements of the data, the data packet is encrypted, and the data packet is packaged using a safe packaging protocol;

[0049] The data packet is compressed, the parameters of the compression algorithm are dynamically adjusted according to the network bandwidth and delay, the priority of the data packet is set according to the importance and urgency of the data, and the optimal transmission path of the data packet is planned by using the routing algorithm in the edge computing network;

[0050] The packaged data packet is sent back to the edge task allocation unit through the edge computing network, the confirmation message of the edge task allocation unit is received, and if the confirmation message is not received, the retransmission strategy is used for retransmission.

[0051] Further, the S4 comprises:

[0052] S41, the edge task allocation unit integrates the processing results of each data group received according to the identification sequence, and in the integration process, consistency checking is performed on the results and conflicts are resolved;

[0053] S42, the integrated results are first uploaded to the cloud platform for further analysis, storage and backup through the cloud platform;

[0054] S43, at the same time, the edge task allocation unit distributes the key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism for real-time decision-making and response.

[0055] The application provides a multi-data fusion system based on edge computing, which comprises:

[0056] The data receiving unit receives heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; according to the encryption level corresponding to each task in the heterogeneous resource data, each task data is divided into a data group with an identification according to a preset division strategy;

[0057] The allocation calculation unit performs allocation calculation on the data group according to the running state of the edge task processor according to a preset allocation strategy; after the allocation is completed, the identification sequence of the allocation mode is obtained;

[0058] The calculation processing unit performs calculation processing on the allocated data group through the edge task processor, and after the data calculation processing is completed, the results in the data group are transmitted back to the edge task allocation unit;

[0059] The result returning unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge device.

[0060] The application has the following advantages: through intelligent scheduling of the edge task allocation unit, data from different sensors and devices can be received and processed in real time, improving the efficiency and response speed of data processing; by adopting corresponding processing and storage strategies for data of different encryption levels, the security and privacy of the data can be effectively guaranteed; strong encryption algorithms and high-security storage are used for high-sensitivity data, while dynamic encryption and data desensitization strategies are implemented for low-sensitivity data, ensuring the security of data in different life cycles; the edge task allocation unit can monitor the key performance indicators of the task processor in real time, dynamically allocate resources according to system load and task priority, and improve the resource utilization of the system; through machine learning and logical reasoning algorithms, the data can be deeply analyzed and fused, effectively improving the accuracy and reliability of data processing; in specific data processing, the application of the rule set can make more accurate logical judgments; the data lifecycle management strategy can automatically archive and clean up data that is no longer frequently accessed, reducing storage costs and improving the management efficiency of the system; through dynamic adjustment of the data splitting strategy, the system can adapt to different data flow characteristics and system load conditions, thereby optimizing the efficiency of data processing; the processing results are distributed to the cloud platform and edge devices in a timely manner through an efficient message passing mechanism, ensuring real-time decision-making and response, and enhancing the real-time and agility of the system; a unique identifier and its metadata tag are generated for each data group, which can facilitate tracking of the entire data processing process, improving the transparency and traceability of data management; by processing and analyzing data in the edge computing environment, the time required for data transmission to the cloud can be reduced, the overall delay of the system can be reduced, and the user experience can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 A method step diagram according to the present application is shown in the following figure:

[0062] Figure 2 A system module diagram according to the present application is shown in the following figure: DETAILED DESCRIPTION

[0063] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0064] One embodiment of the present application is shown in the following figure: Figure 1 A multi-data fusion method based on edge computing, the method comprising:

[0065] S1, receiving heterogeneous resource data from different sensors, devices and sources through an edge task allocation unit; according to the encryption level corresponding to each task in the heterogeneous resource data, dividing each task data into data groups with identifiers according to a preset splitting strategy;

[0066] S2, according to the running status of the edge task processor, the data groups are allocated for calculation according to a preset allocation strategy; after the allocation is completed, an identification sequence of the allocation mode is obtained;

[0067] S3, the allocated data groups are calculated and processed by the edge task processor, and after the data calculation and processing are completed, the results in the data groups are transmitted back to the edge task allocation unit;

[0068] S4, the edge task allocation unit obtains the calculation results of each task according to the identification sequence, and transmits the calculation results to the cloud platform and the corresponding edge device.

[0069] The working principle of the above technical solution is that the edge task allocation unit is the core component of the system, responsible for receiving heterogeneous resource data from different sensors, devices and sources. The data includes environmental parameters such as temperature, humidity, illumination, and device status, location information, etc. According to the encryption level corresponding to each task in the heterogeneous resource data, the edge task allocation unit will perform security processing on each task data. Then, according to the preset splitting strategy, each task data is divided into data groups with unique identification. The edge task allocation unit formulates a data allocation strategy according to the current running status of the edge task processor (such as computing power, load condition, etc.), as well as the task priority, data size and transmission delay, etc. According to the allocation strategy, the preprocessed data groups are allocated to the corresponding edge task processor for calculation. At the same time, the identification sequence of the allocation mode is recorded. After receiving the allocated data groups, the edge task processor will perform a series of calculation and processing. The data is weighted according to its reliability and importance. Mathematical models and algorithms are used to integrate and analyze the data to obtain more comprehensive and accurate information. According to the pre-set rules, logical reasoning and judgment are performed on the data to realize comprehensive analysis and conclusion derivation of the data. After the calculation and processing are completed, the result data groups are transmitted back to the edge task allocation unit. The edge task allocation unit obtains the calculation results of each task according to the identification sequence recorded previously. These results are summarized and arranged to form the final calculation result set. The final calculation result set is transmitted to the cloud platform, and the relevant calculation results are transmitted to the corresponding edge device to support real-time decision-making and response.

[0070] The technical scheme has the following effects: through edge computing, data can be processed closer to the data source, significantly reducing the time delay of data transmission to the cloud, enabling the system to respond to data changes more quickly, especially in application scenarios with high real-time requirements (such as autonomous driving, intelligent manufacturing, etc.), significantly improving the response speed and performance of the system. The edge task allocation unit can intelligently allocate data groups for computing based on the running status of the edge task processors. This improves the utilization of computing resources and avoids computing bottlenecks caused by insufficient resources. By receiving heterogeneous resource data from different sensors, devices, and sources and effectively fusing them, the system can obtain more comprehensive and accurate information to provide a more reliable basis for decision-making. Through weighted fusion, the system can weight the data according to their reliability and importance, thereby improving the accuracy of the overall fusion result. Model fusion uses mathematical models and algorithms to integrate data, further improving the comprehensiveness and accuracy of the data. According to the encryption level of each task in the heterogeneous resource data, the data is processed in stages, not only protecting the security of sensitive data but also avoiding the risk of data leakage. Some sensitive data can be processed locally, reducing the risk of transmission over the network and further improving the security of the system and protecting the privacy of users. The edge task allocation unit can dynamically adjust the allocation strategy based on the running status of the edge task processors and the characteristics of the data, enabling the system to flexibly respond to different application scenarios and data requirements. As edge devices continue to increase and upgrade, this method can easily expand the processing capacity of the system; it provides convenience for future expansion and upgrades and reduces the maintenance cost of the system. In the field of intelligent manufacturing, the above technical scheme can monitor the running status of the production line in real time and make intelligent decisions based on the data fusion results. This helps to optimize the production process, improve production efficiency, and reduce production costs. In smart home and smart city application scenarios, this method can process and analyze user data in real time. This helps to provide more personalized and intelligent service experiences and improve user satisfaction and loyalty.

[0071] In one embodiment of the present application, the S1 comprises:

[0072] S11, the edge task allocation unit receives heterogeneous resource data from various sensors, intelligent devices, and external data sources through a high-speed network interface and an Internet of Things protocol;

[0073] S12, and during the reception of heterogeneous data, the edge task allocation unit performs preliminary verification of the integrity of the data;

[0074] S13, based on each received task data, the edge task allocation unit evaluates the encryption level of the data according to its sensitivity and privacy requirements;

[0075] S14, based on the evaluation results, different processing strategies and storage methods are adopted for data of different levels; wherein the encryption level is divided into three levels of high, medium and low;

[0076] S15, according to the preset segmentation strategy, each task data is divided into multiple data groups with unique identification, and a metadata tag containing source, timestamp, encryption level and data processing requirement is generated for each data group.

[0077] The working principle of the above technical solution is that the edge task allocation unit is the core component of the system, responsible for receiving heterogeneous resource data from various sensors, intelligent devices and external data sources (such as cloud services, remote databases); the edge task allocation unit communicates efficiently and reliably with various data sources through high-speed network interfaces (such as Ethernet, Wi-Fi, etc.) and Internet of Things protocols (such as MQTT, CoAP, etc.). In the data receiving process, the edge task allocation unit will perform preliminary verification on the integrity of the data; which includes checking the format, length, checksum, etc. of the data to ensure that the data has not been damaged or lost during transmission. The edge task allocation unit will evaluate the encryption level of each received task data according to its sensitivity and privacy requirements. The encryption level is divided into three levels of high, medium and low to reflect the sensitivity and privacy protection requirements of the data. According to the evaluation results of the encryption level, the edge task allocation unit will adopt different processing strategies and storage methods for data of different levels.

[0078] High encryption level data: needs to be encrypted with more complex encryption algorithm and stored in storage medium with higher security performance.

[0079] Medium encryption level data: uses medium-strength encryption algorithm and is stored in ordinary storage medium, and requires additional access control mechanism.

[0080] Low encryption level data: uses a simpler encryption algorithm or does not encrypt, and is stored in a storage medium that is easy to access.

[0081] According to the preset segmentation strategy (such as based on time window, data type, data size, etc.), the edge task allocation unit will divide each task data into multiple data groups with unique identification; a metadata tag containing source, timestamp, encryption level and data processing requirement is generated for each data group.

[0082] The effect of the above technical scheme is that the edge task allocation unit can receive heterogeneous resource data from various sensors, intelligent devices and external data sources (such as cloud services and remote databases) through a high-speed network interface and an Internet of Things protocol (such as MQTT and CoAP), can be compatible with multiple data sources and data formats, and improves the data receiving and processing efficiency. During the reception of heterogeneous data, the edge task allocation unit performs preliminary checking on the integrity of the data to ensure the accuracy and reliability of the data and reduce errors and losses caused by data damage or loss. The edge task allocation unit evaluates the encryption level of each received task data according to the sensitivity and privacy requirements of the data, which helps to determine the protection level of the data and provides guidance for subsequent data processing and storage. Based on the evaluation results of the encryption level, different processing strategies and storage methods are adopted for data of different levels; high sensitivity and high privacy requirement data are protected more strictly, and the processing cost of low sensitivity and low privacy requirement data is reduced. A metadata tag containing the source, timestamp, encryption level and data processing requirements is generated for each data group, and the tag helps to track and manage data during the data lifecycle to ensure the traceability and security of the data. According to the preset splitting strategy (such as based on time window, data type, data size, etc.), each task data is divided into multiple data groups with unique identifiers, which helps to divide data into smaller and more easily processed parts, improves the efficiency and flexibility of data processing. By dividing data into multiple data groups, data processing and fusion can be performed in parallel. The data processing time can be shortened, and the response speed and performance of the system can be improved. The information in the metadata tag (such as source, timestamp, encryption level and data processing requirements) is crucial for subsequent data fusion and analysis, which can determine the source, time and processing requirements of the data, so as to more accurately fuse and analyze the data. By generating unique identifiers and metadata tags for each data group, the coherence and consistency of the data processing process can be ensured, data confusion and errors can be avoided, and the accuracy and reliability of data processing can be improved.

[0083] In one embodiment of the present application, the S14 comprises:

[0084] For data evaluated as high encryption level, the edge task allocation unit immediately adopts strong encryption algorithm for encryption processing, and stores such data in a specially configured high security storage area; and detailed records are kept for each data access

[0085] According to the specific use scenario of the data, a dynamic encryption strategy is implemented for data of medium encryption level, that is, the data is encrypted during transmission, and is decrypted as needed during internal processing, and the data is stored in a partition with medium security level. For some sensitive but not completely encrypted fields, a data desensitization strategy is implemented.

[0086] The low-level encryption level is encrypted by a basic encryption algorithm, and the data integrity is guaranteed by a checksum or hash value pair. The data is stored by a distributed file system or object storage service.

[0087] Implement data lifecycle management policies to automatically archive data that is no longer frequently accessed to low-cost storage media, while setting data retention periods and automatic deletion rules.

[0088] Intelligently identify and classify continuously incoming data through machine learning algorithms, dynamically adjust their encryption levels and processing strategies.

[0089] Periodically evaluate the effectiveness of existing processing strategies, and automatically adjust encryption levels and storage strategies according to changes in data usage patterns, new trends in security threats.

[0090] The working principle of the above technical solution is as follows: for data evaluated as high-level encryption level, the edge task allocation unit uses strong encryption algorithm (such as AES-256) for encryption processing; the encrypted data is stored in a specially configured high-security storage area. This area uses physical isolation and multi-factor authentication mechanism to ensure that only authorized personnel can access it; each data access will be recorded in detail, including access time, access identity and operation type, to provide the basis for audit and traceability; according to the specific use scenario of the data, dynamic encryption strategy is implemented for the data of medium-level encryption level. Encrypt during data transmission, and decrypt as needed during internal processing; the data is stored in a partition with medium security level, which supports fine-grained access control and allows specific business logic components to access data on demand after identity verification; for some sensitive but not fully encrypted fields, implement data desensitization strategy such as partial masking or hashing to reduce the risk of data leakage; encrypt the low-level encryption level by a basic encryption algorithm (such as AES-128); use checksum or hash value to guarantee data integrity; store such data by a distributed file system or object storage service; implement data lifecycle management policies to automatically archive data that is no longer frequently accessed to low-cost storage media; set data retention periods and automatic deletion rules to ensure data compliance and privacy protection; intelligently identify and classify continuously incoming data through machine learning algorithms, dynamically adjust their encryption levels and processing strategies; adjust encryption levels and storage strategies automatically according to changes in data usage patterns, new trends in security threats, to adapt to changing security needs; periodically evaluate the effectiveness of existing processing strategies to ensure they can adapt to changes in data usage patterns and new trends in security threats; optimize encryption levels and storage strategies according to evaluation results to improve data security and availability.

[0091] The effect of the above technical scheme is that: the high-level encryption data is encrypted by using a strong encryption algorithm (such as AES-256), to ensure the security of the data during storage and transmission. The specially configured high-security storage area, combined with physical isolation and multiple identity verification mechanisms, effectively prevents unauthorized access and data leakage. The dynamic encryption strategy is implemented, which encrypts the data during transmission according to the specific use scenario, and decrypts the data as needed during internal processing, ensuring the security of the data and improving the flexibility of data processing. The application of data desensitization strategy, such as partial masking or hash processing, further reduces the risk of sensitive data leakage. The low-level encryption data is encrypted by using a basic encryption algorithm (such as AES-128), and the checksum or hash value is used to ensure the integrity of the data. The application of distributed file system or object storage service improves the availability and scalability of data while maintaining basic security protection. The data lifecycle management strategy is implemented, which automatically archives data that is no longer frequently accessed to low-cost storage media, effectively saving storage space and reducing storage costs. Set data retention period and automatic deletion rules to ensure data compliance and privacy protection, while avoiding the accumulation of useless data. Intelligent identification and classification of continuously flowing data through machine learning algorithms improve the automation and intelligence level of data processing. Dynamically adjust the encryption level and processing strategy of the data to make data processing more flexible and efficient. The medium security level partition supports fine-grained access control, allowing specific business logic components to access data as needed after verifying identity, improving data processing efficiency and flexibility. According to the changes in data usage patterns and new trends in security threats, periodically evaluate the effectiveness of existing processing strategies and automatically adjust encryption levels and storage strategies. Through the dynamic adjustment mechanism, the system can quickly adapt to changing security needs, improving the security and stability of the system. By periodically evaluating the effectiveness of existing processing strategies and optimizing them based on the evaluation results, the system can maintain optimal state, and the evaluation and optimization mechanism enhances the maintainability of the system, reducing the cost and complexity of system maintenance.

[0092] In one embodiment of the present application, the S15 comprises:

[0093] S151, dynamically adjust the preset splitting strategy according to historical data processing efficiency and system load;

[0094] S152, according to the preset time window or data size, cut the continuous data stream into independent fragments or blocks, and classify the data according to the data type;

[0095] S154, assign a globally unique identifier to each data group, and for frequently updated data, introduce a version control mechanism to record the UUID and change time of each data update;

[0096] S155, generate metadata tags for each data group, label the data group according to the needs of subsequent data processing and analysis, and record the association between data groups;

[0097] S156, store the data group in the distributed file system or database according to the importance and access frequency of the data, and build an index for the data group and the metadata tag.

[0098] The working principle of the above technical solution is: according to the historical data processing efficiency and system load, dynamically adjust the preset splitting strategy, including adjusting the time window or data size of data cutting, to adapt to the processing needs under different time periods or different data traffic; according to the preset time window or data size, cut the continuous data stream into independent fragments or blocks. At the same time, classify the data according to the data type (such as sensor readings, device logs, external data sources); assign a globally unique identifier (UUID) to each data group. For frequently updated data, introduce a version control mechanism to record the UUID and change time of each data update; generate metadata tags for each data group, including data source, timestamp, encryption level, and other basic information. At the same time, label the data group according to the needs of subsequent data processing and analysis, including processing priority, expected output format, whether it needs preprocessing, etc. In addition, the association between data groups is also recorded, such as the continuity of time series data, the correlation between different data sources, etc.; store the data group in the distributed file system or database according to the importance and access frequency of the data. At the same time, build an index for the data group and the metadata tag.

[0099] The effect of the above technical solution is: by dynamically adjusting the preset splitting strategy according to the historical data processing efficiency and system load, the efficient flow of data in the processing process can be ensured, and the processing bottleneck or resource waste caused by too large or too small splitting can be avoided. Cutting the continuous data stream into independent fragments or blocks and classifying them according to the data type can help simplify the data processing process and improve the processing efficiency. At the same time, the classified data is easier to manage and analyze. Assigning a globally unique UUID to each data group can ensure the uniqueness and traceability of the data. In the data processing and analysis process, the specific data group can be quickly located through the UUID, which is convenient for subsequent operation and management. For frequently updated data, introduce a version control mechanism and record the UUID and change time of each data update to ensure the integrity and consistency of the data. At the same time, it is also helpful to track the change history of the data, which is convenient for data auditing and backtracking. Generating metadata tags for each data group containing basic information such as data source, timestamp, encryption level, etc. can help quickly understand the basic attributes and source of the data. At the same time, according to the needs of subsequent data processing and analysis, the data group is labeled, which can further improve the pertinence and accuracy of data processing. Recording the association between data groups, such as the continuity of time series data and the correlation between different data sources, can help mine the potential relationship and value between data, and provide strong support for subsequent data analysis and application. According to the importance and access frequency of the data, the data group is stored in a distributed file system or database, and an index is constructed for the data group and the metadata tag, which can significantly improve the efficiency of data retrieval and query. At the same time, it is also helpful to optimize the data storage structure and improve the utilization rate of storage space. Labeling the encryption level (high, medium, low) of the data in the metadata tag can help take appropriate security measures according to the sensitivity of the data to ensure the security of the data in the storage and transmission process. The introduction of version control and UUID can also help prevent data from being tampered with or misused, and improve the integrity and credibility of the data.

[0100] In one embodiment of the present application, the S151 comprises:

[0101] Collect the completion time, resource consumption (CPU, memory, disk I / O) and success rate of various data processing tasks (such as cleaning, conversion, aggregation, etc.) in the past period (such as a week), and predict the processing efficiency trend in the future period under different splitting strategies through machine learning algorithms (such as time series analysis, regression model);

[0102] The integrated monitoring tool continuously tracks system resource usage, including CPU usage, memory occupation, network bandwidth, and disk I / O, and other key indicators, and predicts the system load change in the future period through a deep learning model (such as LSTM, GRU). The system load change is predicted by the following formula:

[0103]

[0104] where Y t is the system load at time t; X i,t is the i-th resource usage indicator at time t (i = 1, 2, 3, 4 representing CPU usage, memory occupation rate, network bandwidth usage rate, and disk I / O usage rate, respectively); β i , γ i , δ i , α i , are the corresponding coefficients; θ1, θ2, θ3 are the coefficients of the lag term; w j , p j are the weights of the lag term; n and m are the negative and positive time ranges of the lag term, respectively; λ and μ are the coefficients of the seasonal component, and T is the seasonal period; ψ ij is the interaction term coefficient between the i-th and j-th resource usage indicators; and ε t is the error term.

[0105] Based on the above historical efficiency analysis and system load prediction results, the granularity, frequency, and data type classification strategy of data partitioning are dynamically adjusted through intelligent algorithms; for example, during the system load peak period, a finer-grained partitioning strategy is used to reduce the system resource occupation of single processing task; for data types with lower processing efficiency, the number of parallel processing units after partitioning is increased;

[0106] After implementing the new partitioning strategy, the execution efficiency of data processing tasks and the system load change are continuously monitored, and the strategy effect is evaluated through A / B testing or gradual deployment strategy; according to the evaluation results, the algorithm parameters are continuously adjusted and optimized, forming a closed-loop feedback mechanism.

[0107] The working principle of the above technical solution is as follows: first, the system collects the completion time, resource consumption (CPU, memory, disk I / O) and success rate of various data processing tasks (such as cleaning, conversion, aggregation, etc.) in the past period (such as a week). Then, use machine learning algorithms (such as time series analysis, regression models, etc.) to analyze these historical data and predict the processing efficiency trend in the future period under different splitting strategies. The integrated monitoring tool continuously tracks system resource usage, including CPU usage, memory usage, network bandwidth, and disk I / O, among other key indicators. Through deep learning models (such as LSTM, GRU, etc.), these monitoring data are processed to predict future system load changes. Based on the above historical efficiency analysis and system load prediction results, the intelligent algorithm dynamically adjusts the granularity of data splitting (such as time window size, data block size), splitting frequency, and data type classification strategy. Apply the new splitting strategy to the actual data processing process to observe its effect; after executing the new splitting strategy, continuously monitor the execution efficiency of data processing tasks and the change of system load. Through A / B testing or gradual deployment strategy to evaluate the effect of the strategy to determine whether it has achieved the expected goal; according to the evaluation results, continuously adjust and optimize the algorithm parameters to further improve the data processing efficiency and system resource utilization. This process forms a closed-loop feedback mechanism, enabling the system to continuously learn and improve.

[0108] The effect of the above technical solution is: through the machine learning algorithm to predict the processing efficiency trend in the future period, the system can perceive the performance of different segmentation strategies in advance, so as to select the optimal strategy for implementation; Avoid the waste of resources and time delay caused by blind trial of different strategies in traditional methods. According to the system load prediction result, the granularity of data segmentation, segmentation frequency and data type classification strategy are dynamically adjusted, so that the data processing task can better adapt to the current state of the system. In the system load peak period, a finer granularity segmentation strategy can reduce the occupation of system resources by single processing task, and avoid system overload; For the data type with low processing efficiency, increasing the number of parallel processing units after segmentation can significantly improve the processing speed. Through intelligent algorithm to dynamically adjust the data segmentation strategy, the system can better realize load balancing. Avoid the situation that some processing tasks occupy too much resources and other tasks cannot run normally, improve the overall stability and reliability of the system. Continuously monitor the execution efficiency of data processing task and the change of system load, so that the system can find and prevent potential fault points in time. Through A / B test or gradual deployment strategy to evaluate the effect of the strategy, the system can gradually optimize and improve without affecting the normal operation, reduce the risk of business interruption caused by system failure. Dynamic adjustment of data segmentation strategy makes the data processing task complete more quickly, thereby improving the response speed of the system. By improving the data processing efficiency and resource utilization, the system can more effectively utilize the limited computing resources. Forming a closed loop feedback mechanism means that the system can continuously learn and optimize its performance. Help to promote technological innovation and continuous development, so that the system can continuously adapt to new data processing needs and challenges. With the continuous progress of machine learning and deep learning technology, the system can gradually improve its prediction and analysis ability. This will enable the system to better cope with complex data processing scenarios and changes in demand in the future.The above formula can more accurately capture the complex relationship in the data by introducing various types of variables and interaction terms, thereby improving the accuracy of the prediction; the nonlinear transformation and the polynomial term allow the model to capture the nonlinear dependence between variables, which is crucial for many practical problems; the lag variable and the seasonal factor enable the model to consider the autocorrelation and periodicity in time series data, thereby more accurately predicting future trends and patterns; the model can more accurately predict system load and reduce prediction error, which is crucial for resource planning and system maintenance; by accurately predicting, resources can be allocated more efficiently, reducing resource waste and improving resource utilization; the prediction model can help identify risk points that may cause system overload, so that measures can be taken in advance to avoid system collapse; by optimizing resource usage, operating costs such as unnecessary hardware investment and energy consumption can be reduced; accurate load prediction can ensure stable system operation, improve user satisfaction and loyalty; provide data-driven decision support for management, help make long-term strategic planning and short-term operational decisions; complex models may be more likely to detect abnormal patterns in data, thereby discovering problems and intervening in a timely manner.

[0109] In one embodiment of the present application, S2 comprises:

[0110] S21, the edge task allocation unit monitors the key indicators of the edge task processor (ETP) in real time, including load, computing power, memory usage and energy consumption;

[0111] S22, based on the monitoring data, the real-time processing capacity and remaining resources of the edge task processor are evaluated;

[0112] S23, combined with task priority (urgency, importance), data size, transmission delay requirement and data encryption level, the edge task allocation unit formulates a dynamic allocation strategy based on load balancing;

[0113] S24, according to the allocation strategy, the edge task allocation unit allocates data groups to the most suitable edge task processor for processing;

[0114] S25, and generate an identification sequence containing data group allocation information and edge task processor identifier, through the identification sequence to recycle and error tracking of subsequent results.

[0115] The working principle of the above technical solution is that the edge task allocation unit (or management unit) monitors the key indicators of each ETP in real time, such as load, computing power, memory usage, and energy consumption. Based on the monitoring data, the edge task allocation unit evaluates the real-time processing capacity of the ETP. At the same time, it calculates the remaining resources of the ETP, including idle computing power and memory space, to determine whether it can undertake new tasks. The edge task allocation unit combines factors such as task priority (urgency, importance), data size, transmission delay requirements, and data encryption level to develop a dynamic allocation strategy. The goal of the strategy is to ensure that tasks are allocated to the most suitable ETP to achieve load balancing and efficient processing. According to the developed allocation strategy, the edge task allocation unit allocates data groups to the most suitable ETP for processing. The allocation process takes into account the processing capacity, remaining resources of the ETP, and the characteristics and requirements of the task to ensure that the task is completed efficiently and accurately. The edge task allocation unit generates an identification sequence for each allocated data group, which contains data group allocation information and ETP identifiers. Through this identification sequence, the system can track the processing progress of the data, recover the processing results, and trace and locate errors when they occur.

[0116] The effect of the above technical solution is that the edge task allocation unit can monitor the key indicators of the edge task processor (ETP) in real time, such as load, computing power, memory usage, and energy consumption, and can discover and handle potential problems in the ETP in a timely manner to ensure stable operation of the system. Based on the monitoring data, the real-time processing capacity and remaining resources of the ETP are evaluated, providing an important basis for subsequent dynamic allocation strategies, which helps to optimize resource allocation. By combining task priority (urgency, importance), data size, transmission delay requirements, and data encryption level, the edge task allocation unit develops a dynamic allocation strategy based on load balancing, which can flexibly adjust task allocation according to the needs of different tasks and the real-time state of the ETP, improving the overall performance of the system. According to the allocation strategy, the edge task allocation unit allocates data groups to the most suitable ETP for processing, ensuring efficient and accurate execution of tasks while reducing system energy consumption and cost. An identification sequence containing data group allocation information and ETP identifiers is generated, providing an important basis for subsequent result recovery and error tracking. Through the identification sequence, processing results can be easily recovered and errors can be traced, problems can be discovered and handled in a timely manner, improving the reliability and stability of the system. Through real-time monitoring, resource evaluation, and dynamic allocation strategies, this technical solution can fully utilize the resources of the ETP and improve the overall performance of the system. By optimizing task allocation and result recovery, this technical solution can reduce system energy consumption and cost, and improve resource utilization efficiency. Through the generation of the identification sequence and the error tracking mechanism, this technical solution can improve the reliability and stability of the system, ensuring accurate and efficient execution of tasks.

[0117] In one embodiment of the present application, the S3 comprises:

[0118] S31, the edge task processor fuses the data in each data group based on a weighted average algorithm to generate a preliminary fusion result;

[0119] S32, for data groups that need in-depth analysis, the fusion is performed through a machine learning algorithm;

[0120] S33, for data in a specific field, logical reasoning and judgment are performed based on a predefined rule set;

[0121] S34, after completing the data fusion, the processing result of each data group is packaged into a standardized data packet, the data packet contains the processing result, processing time, ETP identifier and the identifier of the original data group, and is sent back to the edge task allocation unit.

[0122] The working principle of the above technical solution is as follows: the edge task processor first assigns weights to the data items in each data group according to the reliability and importance of the data. Based on the assigned weights, the ETP uses a weighted average algorithm to fuse the data in each data group, taking into account the importance of each data item, to generate a preliminary fusion result. For data groups that need in-depth analysis, the ETP uses a machine learning algorithm for fusion, which can mine complex patterns and relationships in the data, improving the accuracy and depth of fusion. In the fusion process, the ETP considers the spatio-temporal characteristics, correlation and complementarity of the data. For data in a specific field, the ETP performs logical reasoning and judgment based on a predefined rule set, which reflects the professional knowledge and experience in this field and helps to generate more detailed and meaningful conclusions. In the rule fusion process, the ETP not only considers the direct meaning of the data, but also considers the implicit relationships and causal relationships between the data. After completing the data fusion, the ETP packages the processing result of each data group into a standardized data packet. The data packet contains key information such as the processing result, processing time, ETP identifier and the identifier of the original data group. The ETP sends the packaged data packet back to the edge task allocation unit.

[0123] The effect of the above technical solution is that by assigning weights to each data item according to the reliability and importance of the data, and fusing based on the weighted average algorithm, the technical solution can flexibly cope with the characteristics of different data groups and ensure that key data is fully reflected in the fusion result. For data groups that need in-depth analysis, machine learning algorithms are used for fusion, which can mine deep information in the data and improve the accuracy and efficiency of fusion, so that the technical solution can adapt to complex and variable data processing needs. For data in specific fields, logical reasoning and judgment are made based on a predefined rule set, which can generate more detailed and meaningful conclusions and ensure the effectiveness and accuracy of the technical solution in specific fields. In the machine learning algorithm fusion process, the temporal and spatial characteristics, correlation and complementarity of the data are comprehensively considered, which helps to improve the comprehensiveness and accuracy of the fusion result, reveals the internal relationship between the data, and improves the quality of data fusion. Rule fusion not only considers the direct meaning of the data, but also considers the implicit relationship and causal relationship between the data, which helps to generate more in-depth and meaningful conclusions and enables the technical solution to reveal the deep information behind the data and improve the value of data fusion. After completing data fusion, the processing result of each data group is packaged into a standardized data packet, which contains processing result, processing time, ETP identifier and identification of the original data group. Standardized packaging helps the subsequent processing flow to proceed smoothly, and also facilitates data tracing and management. The data packet is sent back to the edge task allocation unit to realize timely feedback of the processing result. This helps the edge task allocation unit to understand the processing progress and result of each ETP, providing an important basis for subsequent task allocation and scheduling. Through flexible data fusion strategies and methods, the technical solution can efficiently process a large amount of data, improve the speed and efficiency of data processing, reduce the energy consumption and cost of the system, and improve the resource utilization efficiency. Different data fusion strategies and methods can be dynamically adjusted according to the real-time state of the ETP and the task demand, realizing load balancing. This helps to avoid overloading or idling of some ETPs, improving the stability and reliability of the entire system.

[0124] In an embodiment of the present application, the S34 comprises:

[0125] Before packaging, the result of data fusion is verified, and after verification, the processing result is converted into a unified and structured data format according to a preset data format standard;

[0126] The time stamp of the whole process from receiving, processing to generating the fusion result of each data group is recorded, and these time information is taken as part of the data packet;

[0127] An unique ETP identifier is assigned to each data group, and the ETP identifier is attached to the data packet;

[0128] The mapping relationship between the original data set identifier and the processing result is established in the data packet; the data packet is encrypted according to the sensitivity and security requirements of the data, and the data packet is encapsulated using a secure encapsulation protocol;

[0129] The data packet is compressed, the parameters of the compression algorithm are dynamically adjusted according to the network bandwidth and delay, the priority of the data packet is set according to the importance and urgency of the data, and the optimal transmission path of the data packet is planned using a routing algorithm in the edge computing network; wherein the parameters of the compression algorithm are dynamically adjusted by the following formula:

[0130]

[0131] Wherein, C p is the compression parameter; B t is the current total network bandwidth; B a is the used bandwidth; L is the network delay; S is the data packet size; R is the target compression ratio; T is the data packet transmission time; C m is the maximum compression parameter; C r is the resource usage threshold; alpha and beta are adjustment coefficients for controlling the influence of bandwidth and delay on the compression parameter; gamma and xi are additional adjustment coefficients for controlling the influence of resource usage and transmission time on the compression parameter; n is the number of data packets; S i and R i are the size and target compression ratio of the ith data packet; wherein K is the upper limit value.

[0132] The encapsulated data packet is sent back to the edge task allocation unit through the edge computing network, an acknowledgement message from the edge task allocation unit is received, and if no acknowledgement message is received, retransmission is performed according to a retransmission strategy.

[0133] The working principle of the above technical solution is as follows: Before packaging, first check the result of data fusion, including logical check, range verification and outlier detection. According to the preset data format standard, convert the processing result into a unified and structured data format. Including encoding the result into a specific data type (such as integer, floating point, string, etc.), and organizing the data into nested data structure (such as JSON, XML, etc.). Record the time stamp of the whole process of each data group from receiving, processing to generating fusion result, including starting processing time, ending processing time and any key step time node. Assign a unique ETP identifier to each data group, which is used to uniquely identify the data group and its processing task in the edge computing environment. Keep the identification information of the original data group, such as the UUID of the data group, the data source identifier, etc., and establish the mapping relationship between the original data group identification and the processing result in the data packet. According to the sensitivity and security requirements of data, encrypt the data packet. Use a secure packaging protocol (such as HTTPS, TLS, etc.) to package the data packet to ensure the security and privacy of the data in the transmission process. Compress the data packet to reduce the bandwidth occupation and delay of data transmission. According to the network bandwidth and delay, dynamically adjust the parameters of the compression algorithm. According to the importance and urgency of the data, set the priority of the data packet. Use the routing algorithm in the edge computing network to plan the optimal transmission path of the data packet. Send the packaged data packet back to the edge task allocation unit through the edge computing network. Receive the confirmation message of the edge task allocation unit to confirm that the data packet has been successfully received. If no confirmation message is received, retransmit according to the retransmission strategy to ensure the reliability and integrity of the data.

[0134] The effect of the above technical solution is: through logical check, range verification and outlier detection and other verification means, the accuracy and reliability of the data fusion result are ensured, the influence of error data and abnormal data on subsequent processing and analysis is reduced, and the data quality is improved. Converting the processing result into a unified and structured data format helps to standardize and normalize subsequent processing and analysis, reduces the complexity of data processing and analysis, and improves the data processing efficiency. Recording the full process timestamp of each data group from receiving, processing to generating fusion result helps the traceability of data, helps to quickly locate the problem when problems occur, and improves the problem solving efficiency. Assign a unique ETP identifier to each data group and keep the original data group identification information, which helps to uniquely identify the data group and its processing task in the edge computing environment, helps to track and verify the data, and improves the security of the data. According to the sensitivity and security requirements of the data, the data packet is encrypted and encapsulated using a secure encapsulation protocol, which helps to protect the security and privacy of the data in the transmission process and prevent data leakage and tampering. The data packet is compressed and the parameters of the compression algorithm are dynamically adjusted according to the network bandwidth and delay, which helps to reduce the bandwidth occupation and delay of data transmission and improve the data transmission efficiency. According to the importance and urgency of the data, the priority of the data packet is set, which helps to realize the efficient scheduling and transmission of data in the edge computing network and ensures that critical data can be transmitted and processed in priority. The optimal transmission path of the data packet is planned by using the routing algorithm in the edge computing network, which helps to reduce the hop count and delay of data transmission and improve the stability and reliability of data transmission. If no confirmation message is received from the edge task allocation unit, retransmission is performed according to the retransmission strategy, which helps to ensure the integrity and reliability of the data and prevent the influence of data loss or transmission failure on subsequent processing and analysis.The dynamic adjustment formula can optimize the compression degree of data packets according to the use condition (the ratio of the used bandwidth to the total bandwidth) of the current network bandwidth by dynamically adjusting the compression parameter, thereby effectively improving the bandwidth utilization and reducing the waste of network resources when the network bandwidth is tight. The formula considers the influence of network delay on the compression parameter, so that in a high-delay environment, the system can adjust the compression algorithm accordingly to reduce the additional delay caused by compression, thereby improving the real-time performance of data transmission. By setting the resource usage threshold and the maximum compression parameter, the system can ensure that the calculation resources are not excessively consumed when processing data, thereby maintaining the overall performance and stability of the system. In terms of the importance and urgency of data packets, the dynamic adjustment of the compression parameter can prioritize the processing of high-priority data packets to ensure that critical data can be delivered in time even in poor network environment. The system can adaptively adjust the compression strategy according to the changes of the network environment (such as bandwidth, delay, resource usage, etc.), thereby enhancing the flexibility and adaptability of the network and improving the reliability of the overall data transmission. The formula considers various aspects of the network by combining multiple parameters, making the setting of the compression parameter more scientific and reasonable, and achieving the best transmission effect under different network conditions. Through dynamic adjustment, network congestion or data loss caused by static setting can be effectively avoided, thereby improving the success rate and stability of data transmission.

[0135] In one embodiment of the present application, the S4 comprises:

[0136] S41, the edge task allocation unit integrates the processing results of each data group received according to the identification sequence, and performs consistency check on the results and solves the conflicts during the integration process;

[0137] S42, the integrated results are first uploaded to the cloud platform for further analysis, storage and backup through the cloud platform;

[0138] S43, at the same time, the edge task allocation unit distributes the key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism for real-time decision-making and response.

[0139] The working principle of the above technical solution is as follows: the edge task allocation unit first integrates the processing results of each data group received according to the identification sequence (such as ETP identifier). During the integration process, the edge task allocation unit will perform consistency checking on the results to ensure that the processing results between different data groups are coordinated and not contradictory. At the same time, for possible conflicts or inconsistencies, the edge task allocation unit will perform corresponding solution processing, such as eliminating conflicts through algorithm optimization, data correction, etc. The integrated data results will be uploaded to the cloud platform. The cloud platform, as the center of data processing and analysis, has powerful computing power and storage resources. On the cloud platform, data can be further analyzed, stored and backed up. The analysis process involves advanced algorithms such as data mining and machine learning to extract valuable information from the data. Storage and backup ensure the persistence and security of data, so that even in the event of edge device failure or data loss, data can be recovered from the cloud platform. At the same time, the edge task allocation unit will also distribute key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism (such as WebSocket, RESTful API, etc.). These edge devices are usually deployed at the edge of the network, close to the data source and user end, so they can respond to user requests and make real-time decisions more quickly. By distributing key results or real-time updates, edge devices can respond and make decisions quickly based on the latest data, thereby improving the response speed and decision efficiency of the entire system.

[0140] The technical scheme has the effects that the edge task allocation unit can quickly integrate the processing results of each data group received according to the identification sequence, not only improves the efficiency of data processing, but also ensures the integrity and consistency of data. In the integration process, the edge task allocation unit will perform consistency checking on the results, timely discover and solve possible conflicts, and ensure that the integrated data results are accurate and reliable, providing strong support for subsequent analysis and decision-making. After the integrated data results are uploaded to the cloud platform, further analysis can be performed using the powerful computing power of the cloud platform. Analysis involves advanced algorithms such as data mining and machine learning, which helps to extract valuable information from data and provides a basis for business decisions. The cloud platform also provides secure and reliable storage and backup services. By storing data on the cloud platform, the persistence and security of data can be ensured. Even if the edge device fails or data is lost, data can be quickly recovered from the cloud platform to ensure business continuity. The edge task allocation unit can distribute key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism (such as WebSocket, RESTful API, etc.). This distribution mechanism ensures the real-time and accuracy of data, allowing edge devices to respond quickly and make decisions based on the latest data. Since edge devices are usually deployed at the edge of the network, close to data sources and user terminals, they can respond to user requests and make real-time decisions more quickly. This technical scheme supports flexible deployment and configuration of edge devices and cloud platforms. Edge devices can be dynamically adjusted according to business needs, while cloud platforms can be expanded and upgraded according to data processing and analysis needs. As the business grows and data grows, this technical scheme can easily handle increasing data volume and complexity. By adding edge devices and expanding cloud platform resources, the system can always maintain good performance and scalability.

[0141] One embodiment of the present application, as shown in Figure 2 A multi-data fusion system based on edge computing, the system comprising:

[0142] Data receiving unit: receiving heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; according to the encryption level corresponding to each task in the heterogeneous resource data, dividing each task data into data groups with identification according to the preset division strategy;

[0143] Allocation calculation unit: according to the running status of the edge task processor, performing allocation calculation on the data groups according to the preset allocation strategy; after allocation is completed, obtaining the identification sequence of the allocation mode;

[0144] Calculation processing unit: performing calculation processing on the allocated data groups through the edge task processor; after data calculation processing is completed, transmitting the results in the data groups back to the edge task allocation unit;

[0145] Result Return Unit: The edge task allocation unit obtains the calculation results of each task according to the identifier sequence and transmits the calculation results to the cloud platform and the corresponding edge devices.

[0146] The working principle of the above technical solution is as follows: The edge task allocation unit, as the core component of the system, is responsible for receiving heterogeneous resource data from different sensors, devices, and sources. This data includes environmental parameters such as temperature, humidity, and light intensity, as well as device status and location information. Based on the encryption level corresponding to each task in the heterogeneous resource data, the edge task allocation unit performs secure processing on each task's data. Then, according to a preset segmentation strategy, each task's data is divided into data groups with unique identifiers. The edge task allocation unit formulates a data allocation strategy based on the current operating status of the edge task processors (such as computing power and load), as well as factors such as task priority, data size, and transmission latency. According to the allocation strategy, the pre-processed data groups are allocated to the corresponding edge task processors for computation. Simultaneously, the identifier sequence of the allocation method is recorded. After receiving the allocated data groups, the edge task processors perform a series of computational processes. The data is weighted according to its reliability and importance. Mathematical models and algorithms are used to integrate and analyze the data to obtain more comprehensive and accurate information. Logical reasoning and judgment are performed on the data according to pre-set rules to achieve comprehensive data analysis and conclusion derivation. After the computation is completed, the result data set is transmitted back to the edge task allocation unit. The edge task allocation unit retrieves the computation results for each task based on the previously recorded identifier sequence. These results are then aggregated and organized to form the final computation result set. This final computation result set is transmitted to the cloud platform, and simultaneously, relevant computation results are transmitted to the corresponding edge devices to support real-time decision-making and response.

[0147] The effect of the above technical solution is that through edge computing, data can be processed closer to the data source, significantly reducing the time delay of data transmission to the cloud, enabling the system to respond to data changes more quickly, especially in application scenarios with high real-time requirements (such as autonomous driving, intelligent manufacturing, etc.), significantly improving the response speed and performance of the system. The edge task allocation unit can intelligently allocate data groups for calculation according to the running status of the edge task processor, improving the utilization of computing resources and avoiding computing bottlenecks caused by insufficient resources. By receiving heterogeneous resource data from different sensors, devices, and sources and effectively fusing them, the system can obtain more comprehensive and accurate information to provide a more reliable basis for decision-making. Through weighted fusion, the system can weight the data according to their reliability and importance, improving the accuracy of the overall fusion result. Model fusion uses mathematical models and algorithms to integrate data, further improving the comprehensiveness and accuracy of the data. According to the encryption level of each task in the heterogeneous resource data, the data is processed in stages, not only protecting the security of sensitive data but also avoiding the risk of data leakage. Some sensitive data can be processed locally, reducing the risk of transmission over the network, further improving the security of the system, and protecting the privacy of users. The edge task allocation unit can dynamically adjust the allocation strategy according to the running status of the edge task processor and the characteristics of the data, enabling the system to flexibly respond to different application scenarios and data requirements. As edge devices continue to increase and upgrade, this method can easily expand the processing capacity of the system; it provides convenience for future expansion and upgrade, and reduces the maintenance cost of the system. In the field of intelligent manufacturing, the above technical solution can monitor the running status of the production line in real time and make intelligent decisions based on the data fusion result. This helps to optimize the production process, improve production efficiency, and reduce production costs. In smart home and smart city application scenarios, this method can process and analyze user data in real time. This helps to provide more personalized and intelligent service experiences and improve user satisfaction and loyalty.

[0148] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A multi-data fusion method based on edge computing, characterized in that, The method includes: S1. Receive heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; divide each task data into data groups with labels according to the encryption level corresponding to each task in the heterogeneous resource data and the preset segmentation strategy. S2. Based on the operating status of the edge task processor, perform allocation calculations on the data group according to the preset allocation strategy; after the allocation is completed, obtain the identification sequence of the allocation method; S3. The edge task processor performs calculations on the allocated data group. After the data calculations are completed, the results in the data group are sent back to the edge task allocation unit. S4. The edge task allocation unit obtains the calculation results of each task according to the identifier sequence and transmits the calculation results to the cloud platform and the corresponding edge device. The S3 includes: S31. The edge task processor uses a weighted average algorithm to fuse the data within each data group and generate a preliminary fusion result. S32. For data sets that require in-depth analysis, they are fused using machine learning algorithms; S33. For data in a specific domain, logical reasoning and judgment are performed based on a predefined set of rules. S34. After completing the data fusion, the processing results of each data group are encapsulated into standardized data packets and sent back to the edge task allocation unit. S34 includes: Before encapsulation, the results of data fusion are verified, and the processing results are converted into a unified, structured data format according to the preset data format standard. Record the complete timestamp for each data group; Assign a unique ETP identifier to each data group and append the ETP identifier to the data packet; The identification information of the original data group is preserved, and a mapping relationship between the original data group identification and the processing result is established in the data packet; the data packet is encrypted and encapsulated. The data packets are compressed, and the parameters of the compression algorithm are dynamically adjusted based on network bandwidth and latency. Data packets are prioritized according to their importance and urgency. Routing algorithms from the edge computing network are used to plan the optimal transmission path for the data packets. The parameters of the compression algorithm are dynamically adjusted using the following formula: in, These are compression parameters; This represents the current total network bandwidth. Bandwidth already used; L Due to network latency; S This refers to the data packet size. R The target compression ratio is T; T is the data packet transmission time. This is the maximum compression parameter; For resource utilization thresholds; α and β These are adjustment coefficients used to control the impact of bandwidth and latency on compression parameters; and This is an additional adjustment factor used to control the impact of resource utilization and transmission time on compression parameters; n is the number of data packets; and It is the first i The size of each data packet and the target compression ratio; where K is the upper limit; The encapsulated data packet is sent back to the edge task allocation unit through the edge computing network. The unit receives an acknowledgment message from the edge task allocation unit. If no acknowledgment message is received, the packet is retransmitted according to the retransmission strategy.

2. The multi-data fusion method based on edge computing according to claim 1, characterized in that, S1 includes: S11. The edge task allocation unit receives heterogeneous resource data from various sensors, smart devices and external data sources through a high-speed network interface and IoT protocol. S12. Furthermore, during the heterogeneous data reception process, the edge task allocation unit performs a preliminary verification of the data integrity. S13. Based on each received task data, the edge task allocation unit evaluates its encryption level according to the data's sensitivity and privacy requirements; S14. Based on the evaluation results, different processing strategies and storage methods are adopted for data of different levels; among them, the encryption level is divided into three levels: high, medium and low. S15. According to the preset segmentation strategy, divide each task data into multiple data groups with unique identifiers, and generate metadata tags for each data group.

3. The multi-data fusion method based on edge computing according to claim 2, characterized in that, S14 includes: For data assessed as having a high level of encryption, the edge task allocation unit immediately encrypts it using a strong encryption algorithm and stores it in a high-security storage area; and records each data access in detail. Based on the specific use case of the data, a dynamic encryption strategy is implemented for data with a medium encryption level, that is, encryption is carried out during data transmission and decryption is carried out as needed during internal processing, and the data is stored in a partition with a medium security level. For some sensitive fields that do not need to be fully encrypted, a data desensitization strategy is implemented. Encrypting low-level encryption using basic encryption algorithms and ensuring data integrity through checksums or hash values, then storing this type of data through a distributed file system or object storage service. Implement a data lifecycle management strategy to automatically archive data that is no longer frequently accessed to low-cost storage media, while setting data retention periods and automatic deletion rules; Through machine learning algorithms, the continuously flowing data is intelligently identified and classified, and its encryption level and processing strategy are dynamically adjusted. Regularly assess the effectiveness of existing processing strategies and adjust encryption levels and storage strategies automatically based on changes in data usage patterns, new trends in security threats, or other factors.

4. The multi-data fusion method based on edge computing according to claim 2, characterized in that, S15 includes: S151. Dynamically adjust the preset splitting strategy based on historical data processing efficiency and system load. S152. Cut the continuous data stream into independent segments or blocks according to the preset time window or data size, and classify the data according to the data type. S153. Assign a globally unique identifier to each data group. For frequently updated data, introduce a version control mechanism to record the UUID and change time of each data update. S154. Generate metadata tags for each data group, label the data groups according to the needs of subsequent data processing and analysis, and record the relationships between data groups; S155. Based on the importance and access frequency of the data, store the data group in a distributed file system or database, and build an index for the data group and metadata tags.

5. The multi-data fusion method based on edge computing according to claim 4, characterized in that, S151 includes: The completion time, resource consumption, and success rate of various data processing tasks over a period of time are collected, and machine learning algorithms are used to predict the processing efficiency trend in the future under different segmentation strategies. Integrated monitoring tools continuously track system resource usage and use deep learning models to predict system load changes in future time periods; Based on the above historical efficiency analysis and system load prediction results, the granularity of data segmentation, segmentation frequency, and data type classification strategy are dynamically adjusted through intelligent algorithms. After implementing the new segmentation strategy, continuously monitor the execution efficiency of data processing tasks and changes in system load, and evaluate the effectiveness of the strategy through A / B testing or incremental deployment strategies; based on the evaluation results, continuously adjust and optimize algorithm parameters to form a closed-loop feedback mechanism.

6. The multi-data fusion method based on edge computing according to claim 1, characterized in that, S2 includes: S21. The edge task allocation unit monitors the key indicators of the edge task processor in real time, including load status, computing power, memory usage and energy consumption. S22. Based on monitoring data, assess the real-time processing capability and remaining resources of the edge task processor; S23. Based on task priority, data size, transmission delay requirements and data encryption level, formulate a dynamic allocation strategy based on load balancing. S24. According to the allocation strategy, the edge task allocation unit allocates the data group to the most suitable edge task processor for processing. S25, and generate an identifier sequence containing data group allocation information and edge task processor identifiers, and use the identifier sequence to reclaim subsequent results and track errors.

7. The multi-data fusion method based on edge computing according to claim 1, characterized in that, The S4 includes: S41. The edge task allocation unit integrates the processing results of each received data group according to the identification sequence. During the integration process, the consistency of the results is checked and conflicts are resolved. S42. The integrated results are first uploaded to the cloud platform for further analysis, storage and backup. S43. At the same time, the edge task allocation unit distributes key results or real-time updates to the corresponding edge devices through an efficient message passing mechanism for real-time decision-making and response.

8. A system for implementing the edge computing-based multi-data fusion method as described in claim 1, characterized in that, The system includes: Data receiving unit: Receives heterogeneous resource data from different sensors, devices and sources through the edge task allocation unit; according to the encryption level of each task in the heterogeneous resource data, divides each task data into data groups with labels according to a preset segmentation strategy; Allocation and calculation unit: Based on the operating status of the edge task processor, it performs allocation and calculation on the data group according to the preset allocation strategy; after the allocation is completed, it obtains the identification sequence of the allocation method; Computation and processing unit: Performs computation and processing on the allocated data group through the edge task processor, and after the data computation and processing is completed, transmits the results in the data group back to the edge task allocation unit; Result Return Unit: The edge task allocation unit obtains the calculation results of each task according to the identifier sequence and transmits the calculation results to the cloud platform and the corresponding edge devices.

Images