Iot master service platform data analysis and cooperation method

Abstract

The application discloses an Internet of Things main service platform data analysis cooperation method, relates to the technical field of data analysis, and has the technical scheme as follows: obtaining terminal uploaded data groups to be analyzed, detecting the real-time transmission time delay and data encryption level of each data segment in the terminal uploaded data groups; extracting the fault tolerance coefficient between the real-time transmission time delay and the analysis fault tolerance rate based on historical data analysis cooperation logs, obtaining the analysis fault tolerance deviation value caused by the transmission time delay change according to the fault tolerance coefficient and the real-time transmission time delay; if the real-time transmission time delay of the current Internet of Things main service platform does not interfere with the data encryption analysis algorithm, then the data encryption level and the analysis calculation power consumption are processed to obtain the calculation loss coefficient, the initial analysis calculation power allocation amount is compensated according to the calculation loss coefficient, the data encryption level and the analysis fault tolerance deviation value, and the first actual analysis calculation power is obtained; and the effect is to improve the cooperation efficiency and stability of the whole platform data analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, more particularly, it relates to a data analysis collaboration method for a main service platform of the Internet of Things. BACKGROUND

[0002] With the rapid development of the Internet of Things technology, terminal devices continuously upload various types of data to the main service platform, resulting in an explosive growth in data size and increasing diversity of data types. However, the existing main service platform of the Internet of Things may experience unstable data transmission latency during data analysis. The network environment of different terminal devices varies greatly, resulting in significant fluctuations in the time it takes for data to be transmitted from the terminal to the platform. Traditional data analysis methods ignore the impact of transmission latency on analysis fault tolerance, making it difficult to ensure the accuracy of the analysis results. Due to varying levels of data encryption, the computational power required for analyzing different levels of encrypted data varies significantly. However, existing data analysis collaboration methods for the main service platform of the Internet of Things lack an adjustment mechanism for computational power allocation, typically using a rough allocation method that can lead to waste of computational power resources, i.e., allocating excessive unnecessary computational power to low encryption level data. At the same time, it can cause high encryption level data to be analyzed slowly due to insufficient computational power, thereby affecting the efficiency and collaboration of data analysis for the main service platform of the Internet of Things, making it difficult to meet the demand for real-time, efficient, and accurate data analysis in the Internet of Things scenario. SUMMARY

[0003] In view of the deficiencies in the prior art, the purpose of the present application is to provide a data analysis collaboration method for the main service platform of the Internet of Things.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0005] The data analysis collaboration method for the main service platform of the Internet of Things comprises the following steps:

[0006] Obtain a terminal uploaded data group to be analyzed, and detect the real-time transmission latency and data encryption level of each data segment in the terminal uploaded data group;

[0007] Extract the fault tolerance coefficient between real-time transmission latency and analysis fault tolerance based on historical data analysis collaboration logs, and obtain the analysis fault tolerance deviation value caused by changes in transmission latency based on the fault tolerance coefficient and real-time transmission latency;

[0008] If the real-time transmission latency of the current main service platform of the Internet of Things does not interfere with the data encryption analysis algorithm, then process the data encryption level and analysis power consumption to obtain the algorithm loss coefficient, perform power compensation on the initial analysis power allocation based on the algorithm loss coefficient, data encryption level, and analysis fault tolerance deviation value, and obtain the first actual analysis power;

[0009] If the real-time transmission delay of the current Internet of Things main service platform interferes with the data encryption and analysis algorithm, the dynamic analysis priority obtained according to the node congestion interference value and the congestion node type of the data segment to which the data encryption level belongs is used to perform power compensation on the initial analysis power allocation amount according to the dynamic analysis priority, the data encryption level, the algorithm loss coefficient and the analysis fault tolerance deviation value, and a second actual analysis power is obtained.

[0010] A data analysis cooperative scheduling framework is constructed based on the first actual analysis power or the second actual analysis power, and a data analysis cooperative optimization scheme is output.

[0011] Preferably, it further comprises:

[0012] The analysis power requirement of each data segment in the terminal uploaded data group is detected to obtain an initial analysis power allocation amount.

[0013] Preferably, the real-time transmission delay and the data encryption level of each data segment in the terminal uploaded data group are detected, and the detection specifically comprises the following steps:

[0014] The real-time transmission state of the terminal uploaded data group is monitored to obtain a transmission delay value, and the real-time transmission delay greater than a preset fault tolerance threshold value in the transmission delay value is extracted;

[0015] The encryption attribute of each data segment in the terminal uploaded data group is detected to obtain a data encryption level.

[0016] Preferably, the fault tolerance loss coefficient between the real-time transmission delay and the analysis fault tolerance rate is extracted based on the historical data analysis cooperative log, and the analysis fault tolerance deviation value caused by the change of the transmission delay is obtained according to the fault tolerance loss coefficient and the real-time transmission delay, and the detection specifically comprises the following steps:

[0017] The historical abnormal delay value of the historical analysis data whose transmission delay is greater than the fault tolerance threshold value is extracted from the historical data analysis cooperative log, and the delay change amplitude is obtained by subtracting the historical abnormal delay value from the fault tolerance threshold value;

[0018] The historical fault tolerance loss value of the historical analysis data affected by the delay change amplitude is extracted from the historical data analysis cooperative log;

[0019] The ratio of the historical fault tolerance loss value and the delay change amplitude is obtained to obtain the fault tolerance loss coefficient;

[0020] The current delay difference value is obtained by subtracting the real-time transmission delay from the fault tolerance threshold value;

[0021] The analysis fault tolerance deviation value caused by the change of the transmission delay is obtained according to the fault tolerance loss coefficient and the current delay difference value.

[0022] Preferably, the algorithm loss coefficient is obtained by processing the data encryption level and the analysis power consumption, and the processing specifically further comprises the following steps:

[0023] extracting, from the historical data analysis collaboration log, a historical encryption level of each data segment in the terminal upload data group and a corresponding historical computing power loss value;

[0024] calculating a computing power loss coefficient by ratio calculation of the historical computing power loss value and the analysis complexity weight of the historical encryption level.

[0025] Preferably, the initial analysis computing power allocation amount is compensated according to the computing power loss coefficient, the data encryption level, and the analysis fault tolerance deviation value to obtain a first actual analysis computing power, specifically including the following steps:

[0026] multiplying the computing power loss coefficient and the analysis complexity weight of the data encryption level to obtain a first computing power loss value;

[0027] Compensating the initial analysis computing power allocation amount according to the first computing power loss value and the analysis fault tolerance deviation value to obtain the first actual analysis computing power.

[0028] Preferably, the dynamic analysis priority is obtained according to the node congestion interference value and the congestion node type of the data segment to which the data encryption level belongs, specifically including the following steps:

[0029] detecting the node congestion interference value and the congestion node type of the data segment to which the data encryption level belongs;

[0030] analyzing the node congestion interference value and the congestion node type to obtain a to-be-tested effective interference value;

[0031] extracting, from the historical data analysis collaboration log, a historical node effective interference value and a corresponding historical priority offset value;

[0032] calculating a priority interference sensitivity coefficient by ratio calculation of the historical priority offset value and the historical node effective interference value;

[0033] obtaining the dynamic analysis priority according to the product of the priority interference sensitivity coefficient and the to-be-tested effective interference value.

[0034] Preferably, the node congestion interference value and the congestion node type are analyzed to obtain a to-be-tested effective interference value, specifically including the following steps:

[0035] determining a congestion priority deviation value according to the congestion node type and the data segment analysis basic priority;

[0036] splitting the node congestion interference value according to the congestion priority deviation value to obtain the to-be-tested effective interference value.

[0037] Preferably, the initial analysis computing power allocation amount is compensated according to the dynamic analysis priority, the data encryption level, the computing power loss coefficient, and the analysis fault tolerance deviation value to obtain a second actual analysis computing power, specifically including the following steps:

[0038] The preprocessing parsing weight is obtained by combining the parsing complexity weight of the data encryption level with the weight increment of the dynamic parsing priority.

[0039] The second computing power loss value is obtained by multiplying the computing loss coefficient with the preprocessing analysis weight;

[0040] The initial parsing power allocation is compensated based on the second computing power loss value and the parsing fault tolerance deviation value to obtain the second actual parsing power.

[0041] Preferably, after constructing a data parsing collaborative scheduling framework based on the first or second actual parsing computing power, a data parsing collaborative optimization scheme is output, specifically including the following steps:

[0042] A data parsing collaborative scheduling framework is obtained by mapping collaborative scheduling rules based on the first or second actual parsing computing power.

[0043] The marking results are obtained based on the data parsing and collaborative scheduling framework; wherein, the marking results include the data segment with the highest computing power consumption and the data segment with the largest fault tolerance deviation;

[0044] Generate a collaborative optimization scheme for data parsing based on the labeling results.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] This invention acquires data uploaded from terminals, detects the real-time transmission latency and data encryption level of each data segment, and calculates the parsing fault tolerance deviation value caused by changes in transmission latency based on historical logs and a tolerance coefficient. This quantifies the impact of transmission latency on parsing fault tolerance, providing an accurate basis for subsequent computing power compensation and reducing parsing errors caused by latency. Differentiated computing power compensation allows computing power allocation to better match the actual parsing needs of data segments. After constructing a data parsing collaborative scheduling framework based on the first or second actual parsing computing power, an optimization scheme is output. This enables the platform to plan data parsing from a global perspective, and to specifically optimize the parsing process, algorithm, or resource allocation based on the data segments with the highest computing power consumption and the largest fault tolerance deviation, improving the collaborative efficiency and stability of the entire platform's data parsing and better adapting to the complex and ever-changing data transmission and parsing scenarios in the IoT environment. Attached Figure Description

[0047] Fig. 1 This is a schematic diagram illustrating the steps of the IoT master service platform data parsing and collaboration method proposed in this invention;

[0048] Fig. 2 This diagram illustrates the steps involved in obtaining dynamic parsing priority in the IoT master service platform data parsing collaboration method proposed in this invention. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0052] Reference Figs. 1-2 As shown.

[0053] The embodiments further illustrate the data parsing and collaboration method for the IoT master service platform proposed in this invention.

[0054] The IoT master service platform data parsing and collaboration method includes the following steps:

[0055] Obtain the terminal upload data group to be parsed, and detect the real-time transmission latency and data encryption level of each data segment in the terminal upload data group;

[0056] Based on historical data analysis of collaborative logs, the tolerance coefficient between real-time transmission latency and parsing fault tolerance rate is extracted. Based on the tolerance coefficient and real-time transmission latency, the parsing fault tolerance deviation value caused by changes in transmission latency is obtained.

[0057] If the real-time transmission latency of the current IoT main service platform does not interfere with the data encryption and parsing algorithm, the data encryption level and parsing computing power consumption are processed to obtain the computing loss coefficient. The initial parsing computing power allocation is compensated based on the computing loss coefficient, the data encryption level and the parsing fault tolerance deviation value to obtain the first actual parsing computing power.

[0058] If the real-time transmission latency of the current IoT main service platform interferes with the data encryption and parsing algorithm, then the dynamic parsing priority is obtained based on the node congestion interference value and congestion node type of the data segment to which the data encryption level belongs. The initial parsing computing power allocation is then compensated based on the dynamic parsing priority, data encryption level, computing loss coefficient, and parsing fault tolerance deviation value to obtain the second actual parsing computing power.

[0059] Transmission latency values ​​obtained by real-time monitoring of data groups uploaded from terminals, especially those exceeding a preset fault tolerance threshold, are considered. This threshold is a pre-defined upper limit for security latency based on the type of data encryption and parsing algorithm and the timing dependence of the parsing process. If the real-time transmission latency does not exceed this threshold, it means that the time difference between the data segments arriving at the main service platform is within the algorithm-compatible range. For example, during segment-by-segment decryption, after the previous segment of data is parsed, the next segment arrives just in time, without waiting or caching. In this case, the latency will not disrupt the continuity of the parsing process, nor will it cause data to expire due to timeout and trigger parsing errors. Therefore, it is determined that the latency does not interfere with the encryption and parsing algorithm. Conversely, if the latency exceeds this threshold, it is determined that the latency interferes with the encryption and parsing algorithm.

[0060] After constructing a data parsing collaborative scheduling framework based on the first or second actual parsing computing power, a data parsing collaborative optimization scheme is output.

[0061] Also includes:

[0062] The initial parsing computing power allocation is obtained by analyzing the parsing computing power requirements of each data segment in the data group uploaded by the detection terminal.

[0063] IoT terminals continuously upload various data sets to the main service platform. These data sets consist of numerous different data fragments. Due to their data characteristics, encryption methods, and the business requirements they carry, different data fragments have significantly different computing power requirements when parsing them.

[0064] Assume an IoT environment includes two types of terminal devices: temperature sensors and high-definition image sensors. The data segments uploaded by temperature sensors are relatively small in size and have a simple data structure, consisting only of simple numerical values ​​and a small amount of identification information. Parsing these values ​​requires basic format conversion and validity verification, thus the computational demand for parsing is relatively low. However, the data segments uploaded by high-definition image sensors contain a large amount of pixel information, resulting in a massive data volume. Furthermore, they may have undergone complex encryption processing. Parsing them requires not only decryption but also complex image processing operations such as noise reduction and feature extraction, thus demanding substantial computational resources.

[0065] The main service platform detects the individual parsing computing power requirements for each data segment of the uploaded data group. These requirements include the computational power needed to complete data decryption, format conversion, content extraction, and logical verification operations, such as CPU processing power and memory usage. After detecting and evaluating the requirements for each data segment, the main service platform allocates a corresponding initial parsing computing power to each segment, thus forming the initial parsing computing power allocation.

[0066] The real-time transmission latency and data encryption level of each data segment in the data group uploaded by the detection terminal are determined by the following steps:

[0067] Real-time transmission status monitoring of the data group uploaded by the terminal is performed to obtain the transmission delay value, and the real-time transmission delay values ​​that are greater than the preset fault tolerance threshold are extracted.

[0068] The encryption level of the data is obtained by detecting the encryption attributes of each data segment in the data group uploaded by the detection terminal.

[0069] The main service platform continuously monitors the real-time transmission status of data groups uploaded by the terminals to obtain the transmission latency value of each data segment from the terminal to the platform. Transmission latency refers to the time taken for data to be completely received by the main service platform from the terminal. To determine whether the transmission latency is within a normal and acceptable range, the platform presets a fault tolerance threshold. When the monitored transmission latency value is greater than the preset fault tolerance threshold, it indicates that there is an abnormal delay in the transmission of that data segment.

[0070] The main service platform detects the encryption attributes of each data segment in the data group uploaded by the terminal to determine the encryption level of each data segment. Different encryption algorithms and encryption strengths correspond to different encryption levels, which directly affect the computing power and processing complexity required for subsequent data parsing.

[0071] Suppose an IoT system has two types of terminal devices: one is a general environmental sensor, whose uploaded data segments use a simple symmetric encryption algorithm with a low encryption level, resulting in relatively stable transmission with latency generally within a preset fault tolerance threshold; the other is industrial control equipment, whose uploaded data segments involving core control commands use a complex asymmetric encryption algorithm with a high encryption level. Due to the high importance of this data, network congestion during transmission can cause latency exceeding the preset fault tolerance threshold. The main service platform can promptly detect abnormal transmission latency of data segments from the industrial control equipment through real-time monitoring, and simultaneously detect its high encryption level. This provides a basis for subsequent data parsing, computing power allocation, and collaborative scheduling.

[0072] Based on historical data analysis of collaborative logs, a tolerance coefficient is extracted between real-time transmission latency and parsing fault tolerance rate. Then, based on this tolerance coefficient and real-time transmission latency, the parsing fault tolerance deviation caused by changes in transmission latency is obtained. This process includes the following steps:

[0073] Extract historical abnormal latency values ​​from the historical data parsing collaborative logs, where the transmission latency of the historical parsing data is greater than the fault tolerance threshold. Calculate the difference between the historical abnormal latency values ​​and the fault tolerance threshold to obtain the latency change amplitude.

[0074] Extract the historical fault tolerance and loss tolerance values ​​that affect the historical data parsing from the collaborative logs of historical data parsing;

[0075] The tolerance coefficient is obtained by comparing the historical fault tolerance value with the delay change amplitude.

[0076] The current latency difference is obtained by calculating the difference between the real-time transmission latency and the fault tolerance threshold.

[0077] The analytical fault tolerance deviation value caused by the change in transmission delay is obtained based on the tolerance coefficient and the current delay difference.

[0078] From the historical data parsing collaboration logs, extract historical abnormal latency values ​​where the transmission latency exceeds the fault tolerance threshold. Subtract the historical abnormal latency value from the fault tolerance threshold to obtain the latency change amplitude. Assuming the fault tolerance threshold is 50 milliseconds and the transmission latency of a certain historical parsed data is 70 milliseconds, the latency change amplitude is 70 - 50 = 20 milliseconds. Extract the historical fault tolerance loss value from the historical data parsing collaboration logs, which represents the loss of fault tolerance capability due to latency changes. Calculate the ratio of the historical fault tolerance loss value to the latency change amplitude to obtain the tolerance coefficient, which reflects the degree of fault tolerance loss per unit latency change amplitude.

[0079] The current real-time transmission latency is calculated by subtracting it from the fault tolerance threshold. Based on the current latency difference and the tolerance coefficient, the parsing fault tolerance deviation caused by the transmission latency change is calculated. For example, if the tolerance coefficient is 0.5 and the current latency difference is 30 milliseconds, the parsing fault tolerance deviation is 0.5 × 30 = 15. This means that the change in transmission latency has reduced the parsing fault tolerance capability by 15 units, providing a basis for subsequent operations such as computing power compensation based on this deviation value.

[0080] The computational loss coefficient is obtained by processing the data encryption level and the computing power consumption for parsing. This process also includes the following steps:

[0081] Extract the historical encryption level and corresponding historical computing power loss value of each data segment in the terminal uploaded data group from the historical data parsing collaborative logs;

[0082] The computational loss coefficient is obtained by calculating the ratio of historical computing power loss values ​​to the parsing complexity weights of historical encryption levels.

[0083] The initial parsing computing power allocation is compensated based on the computational loss coefficient, data encryption level, and parsing fault tolerance deviation value to obtain the first actual parsing computing power. The specific steps include:

[0084] The first computing power loss value is obtained by multiplying the computing loss coefficient with the parsing complexity weight of the data encryption level.

[0085] The initial parsing ...

[0086] The historical encryption level and corresponding historical computing power loss value of each data segment in the terminal uploaded data group are extracted from the historical data parsing collaborative logs. Assuming the encryption level of a historical data segment is 3, the corresponding historical computing power loss value is 15. Then, the computing power loss value is calculated by ratioing the historical computing power loss value to the parsing complexity weight of that historical encryption level. If the parsing complexity weight of encryption level 3 is 5, then the computing power loss coefficient is 15 ÷ 5 = 3.

[0087] The first computational power loss value is obtained by multiplying the computational loss coefficient and the parsing complexity weight of the current data encryption level. Assuming the current data encryption level is 3, the parsing complexity weight is 5, and the computational loss coefficient is 3, then the first computational power loss value is 3 × 5 = 15. Next, the initial parsing computational power allocation is compensated based on the first computational power loss value and the parsing fault tolerance deviation value. For example, if the initial parsing computational power allocation is 100 and the parsing fault tolerance deviation value is 5, then the initial parsing computational power allocation minus the first computational power loss value, plus the parsing fault tolerance deviation value, yields the first actual parsing computational power of 100 - 15 + 5 = 90. This is used to adapt to changes in computational power requirements caused by factors such as encryption level and transmission latency, ensuring efficient and accurate data parsing.

[0088] The dynamic parsing priority, derived from the node congestion interference value and congested node type of the data segment corresponding to the data encryption level, specifically includes the following steps:

[0089] Detect the node congestion interference value and congested node type of the data segment to which the data encryption level belongs;

[0090] The effective interference value to be measured is obtained by analyzing the node congestion interference value and the type of congested node;

[0091] Extract the effective interference values ​​of historical nodes and their corresponding historical priority offsets from the historical data parsing and collaborative logs.

[0092] The priority interference sensitivity coefficient is obtained by calculating the ratio of the historical priority offset to the effective interference value of the historical node.

[0093] The dynamic analytical priority is obtained by multiplying the priority interference sensitivity coefficient by the effective interference value to be measured.

[0094] First, the node congestion interference value and congested node type of the data segment belonging to the data encryption level are detected. The node congestion interference value reflects the degree of interference of node congestion on the transmission and subsequent parsing of the data segment, and the impact logic of different congested node types on data parsing also varies. Assuming that a node is congested during the transmission of a data segment, the node congestion interference value = (traffic anomaly score × traffic weight) + (data packet transmission anomaly score × data packet weight) + (node ​​performance utilization score × performance weight). Assuming that the congestion interference value of this node for this data segment is 8, and this congested node is a data forwarding type node.

[0095] The effective interference value to be measured is obtained by combining the node congestion interference value with the type of congested node. This is because the functional attributes of different types of congested nodes determine the interference priority deviation for data parsing. Data forwarding nodes act as relays in the data transmission link; congestion at these nodes affects the transmission timing and integrity of data segments, thus impacting the parsing priority. Therefore, the node congestion interference value is broken down into parsing requirement interference based on the congestion priority deviation value corresponding to the data forwarding node. Assuming the congestion priority deviation value of the data forwarding node is 0.5, it means that the interference level of this type of node congestion on the parsing requirement is multiplied by a coefficient of 0.5, resulting in an effective interference value of 8 × 0.5 = 4.

[0096] Extract the effective interference values ​​of historical nodes and their corresponding historical priority offsets from the historical data parsing collaboration logs. The historical data records the changes in data parsing priority under different effective interference conditions at different nodes. For example, if a node had an effective interference value of 5 in the past, the data parsing priority would have shifted by 10 compared to the normal situation under this interference; that is, the historical priority offset is 10. Calculate the ratio of the historical priority offset to the effective interference value of the historical node to obtain the priority interference sensitivity coefficient. This coefficient reflects the degree of priority shift caused by a unit of effective interference value at a node. Therefore, the priority interference sensitivity coefficient is 10 ÷ 5 = 2.

[0097] The dynamic parsing priority is obtained by multiplying the priority interference sensitivity coefficient by the effective interference value to be measured. For example, if the priority interference sensitivity coefficient is 2 and the effective interference value to be measured is 4, then the dynamic parsing priority is 2 × 4 = 8. The dynamic parsing priority reflects the priority that the current data segment should be assigned during parsing due to node congestion.

[0098] The effective interference value to be measured is obtained by analyzing the node congestion interference value and the type of congested node. The specific steps include:

[0099] The congestion priority deviation value is determined based on the congestion node type and the basic priority of data segment parsing.

[0100] The node congestion interference value is analyzed and the demand interference is decomposed based on the congestion priority deviation value to obtain the effective interference value to be measured.

[0101] First, the congestion priority deviation value is determined based on the type of congested node and the basic priority of data segment parsing. Different types of congested nodes and the basic priority of the data segment itself affect the priority deviation caused by congestion. Assume there are two types of congested nodes: core data forwarding nodes and ordinary data relay nodes. Also, assume a data segment has a high basic priority, for example, this data segment carries critical device control commands, and its basic priority is set to 9. When the high-priority data segment passes through a core data forwarding node, due to the core node's critical position in the overall network architecture, its congestion will have a more significant impact on the high-priority data. The priority deviation value = (node ​​weight + basic priority of data segment parsing) ÷ impact coefficient, for example, a congestion priority deviation value of 5. However, if it passes through an ordinary data relay node, the congestion priority deviation value is 2.

[0102] The node congestion interference value is analyzed and broken down into required interference values ​​based on the congestion priority deviation value to obtain the effective interference value to be measured. The node congestion interference value reflects the overall degree of interference caused by node congestion to data transmission, but for data parsing needs, it needs to be further broken down in conjunction with the congestion priority deviation. Assuming the node congestion interference value is 10 and the congestion priority deviation value is 5, the effective interference value to be measured = node congestion interference value × (congestion priority deviation value / baseline deviation value). If the baseline deviation value is 10, then the effective interference value to be measured is 10 × (5 / 10) = 5; if the congestion priority deviation value is 2 and the baseline deviation value is also 10, then the effective interference value to be measured is 10 × (2 / 10) = 2. The effective interference value to be measured more closely reflects the actual interference requirements caused by node congestion during data segment parsing, providing a basis for subsequent operations such as determining dynamic parsing priorities.

[0103] The initial parsing computing power allocation is compensated based on dynamic parsing priority, data encryption level, computational loss coefficient, and parsing fault tolerance deviation value to obtain the second actual parsing computing power. The specific steps include:

[0104] The preprocessing parsing weight is obtained by combining the parsing complexity weight of the data encryption level with the weight increment of the dynamic parsing priority.

[0105] The second computing power loss value is obtained by multiplying the computing loss coefficient with the preprocessing analysis weight;

[0106] The initial parsing power allocation is compensated based on the second computing power loss value and the parsing fault tolerance deviation value to obtain the second actual parsing power.

[0107] First, the preprocessing parsing weight is obtained by combining the parsing complexity weight of the data encryption level with the weight increment of the dynamic parsing priority. For example, suppose a data segment has a high encryption level and its parsing complexity weight is 6, while the weight increment of the dynamic parsing priority due to node congestion is 3. By adding them together, the preprocessing parsing weight is 6 + 3 = 9.

[0108] The second computing power loss value is obtained by multiplying the computational loss coefficient by the preprocessing analysis weight. If the computational loss coefficient is 0.5, then the second computing power loss value is 0.5 × 9 = 4.5.

[0109] The second actual parsing computing power is obtained by compensating the initial parsing computing power allocation based on the second computing power loss value and the parsing fault tolerance deviation value. For example, if the initial parsing computing power allocation is 100 and the parsing fault tolerance deviation value is 5, then the second actual parsing computing power is obtained by subtracting the second computing power loss value from the initial parsing computing power allocation and adding the parsing fault tolerance deviation value: 100 - 4.5 + 5 = 100.5. This is to adapt to changes in computing power requirements caused by transmission delay interference and other factors, ensuring that data parsing can be performed efficiently and accurately.

[0110] After constructing a data parsing collaborative scheduling framework based on the first or second actual parsing computing power, a data parsing collaborative optimization scheme is output, which specifically includes the following steps:

[0111] A data parsing collaborative scheduling framework is obtained by mapping collaborative scheduling rules based on the first or second actual parsing computing power.

[0112] The marking results are obtained based on the data parsing and collaborative scheduling framework; the marking results include the data segments with the highest computing power consumption and the data segments with the largest fault tolerance deviation.

[0113] Generate a collaborative optimization scheme for data parsing based on the labeling results.

[0114] First, a collaborative scheduling framework for data parsing is obtained by mapping collaborative scheduling rules based on the first or second actual parsing computing power. Collaborative scheduling rule mapping refers to combining the actual computing power with a pre-set scheduling strategy to determine how computing power is allocated and in what order different data segments are parsed during the parsing process. If the second actual parsing computing power of a certain data segment is higher, it indicates that it is difficult to parse and has a high demand for computing power. In the collaborative scheduling framework, more computing resources will be allocated to it, and it will be arranged for priority parsing.

[0115] The marking results are obtained based on the data parsing collaborative scheduling framework. The marking results include the data segment with the highest computing power consumption and the data segment with the largest fault tolerance deviation. Assuming that multiple terminals upload data groups, a data segment carrying high-definition images and with a high level of encryption requires a lot of computing power during parsing and becomes the data segment with the highest computing power consumption; another data segment has the largest parsing fault tolerance deviation due to large fluctuations in transmission latency.

[0116] A collaborative optimization scheme for data parsing is generated based on the labeling results. For the data segment with the highest computing power consumption, the optimization scheme determines whether there is excessive computing power allocation, such as improving the parsing algorithm to reduce computing power requirements; for the data segment with the largest fault tolerance deviation, the transmission strategy or parsing algorithm parameters are adjusted after investigating the impact of transmission latency factors to reduce fault tolerance deviation, thereby improving the overall efficiency and reliability of data parsing of the IoT main service platform.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data parsing and collaboration method for an IoT main service platform, characterized in that, The method includes the following steps: Obtain the terminal upload data group to be parsed, and detect the real-time transmission latency and data encryption level of each data segment in the terminal upload data group; Extracting the tolerance coefficient between real-time transmission latency and parsing fault tolerance rate based on historical data analysis of collaborative logs includes the following steps: Extract historical abnormal latency values ​​from the historical data parsing collaborative logs, where the transmission latency of the historical parsing data is greater than the fault tolerance threshold. Calculate the difference between the historical abnormal latency values ​​and the fault tolerance threshold to obtain the latency change amplitude. Extract the historical fault tolerance and loss tolerance values ​​that affect the historical data parsing from the collaborative logs of historical data parsing; The tolerance coefficient is obtained by comparing the historical fault tolerance value with the delay change amplitude. The analytical fault tolerance deviation value caused by the change in transmission delay is obtained based on the fault tolerance coefficient and the real-time transmission delay. If the real-time transmission latency of the current IoT main service platform does not interfere with the data encryption and parsing algorithm, then the data encryption level and the parsing computing power consumption are processed to obtain the loss coefficient. Specifically, the following steps are also included: Extract the historical encryption level and corresponding historical computing power loss value of each data segment in the terminal uploaded data group from the historical data parsing collaborative logs; The computational loss coefficient is obtained by calculating the ratio of historical computing power loss values ​​to the parsing complexity weights of historical encryption levels. The initial parsing computing power allocation is compensated based on the computing loss coefficient, data encryption level, and parsing fault tolerance deviation value to obtain the first actual parsing computing power. If the real-time transmission latency of the current IoT main service platform interferes with the data encryption and parsing algorithm, then the dynamic parsing priority is obtained based on the node congestion interference value and congestion node type of the data segment to which the data encryption level belongs. The initial parsing computing power allocation is then compensated based on the dynamic parsing priority, data encryption level, computing loss coefficient, and parsing fault tolerance deviation value to obtain the second actual parsing computing power. After constructing a data parsing collaborative scheduling framework based on the first or second actual parsing computing power, a data parsing collaborative optimization scheme is output.

2. The IoT master service platform data parsing and collaboration method according to claim 1, characterized in that, Also includes: The initial parsing computing power allocation is obtained by analyzing the parsing computing power requirements of each data segment in the data group uploaded by the detection terminal.

3. The IoT master service platform data parsing and collaboration method according to claim 1, characterized in that, The real-time transmission latency and data encryption level of each data segment in the data group uploaded by the detection terminal are determined by the following steps: Real-time transmission status monitoring of the data group uploaded by the terminal is performed to obtain the transmission delay value, and the real-time transmission delay values ​​that are greater than the preset fault tolerance threshold are extracted. The encryption level of the data is obtained by detecting the encryption attributes of each data segment in the data group uploaded by the detection terminal.

4. The IoT master service platform data parsing and collaboration method according to claim 1, characterized in that, The analytical fault tolerance deviation value caused by changes in transmission delay is obtained based on the fault tolerance coefficient and real-time transmission delay. The specific steps include: The current latency difference is obtained by calculating the difference between the real-time transmission latency and the fault tolerance threshold. The analytical fault tolerance deviation value caused by the change in transmission delay is obtained based on the tolerance coefficient and the current delay difference.

5. The IoT master service platform data parsing and collaboration method according to claim 1, characterized in that, The initial parsing computing power allocation is compensated based on the computational loss coefficient, data encryption level, and parsing fault tolerance deviation value to obtain the first actual parsing computing power. The specific steps include: The first computing power loss value is obtained by multiplying the computing loss coefficient with the parsing complexity weight of the data encryption level. The initial parsing ...

6. The IoT master service platform data parsing and collaboration method according to claim 5, characterized in that, The dynamic parsing priority, derived from the node congestion interference value and congested node type of the data segment corresponding to the data encryption level, specifically includes the following steps: Detect the node congestion interference value and congested node type of the data segment to which the data encryption level belongs; The effective interference value to be measured is obtained by analyzing the node congestion interference value and the type of congested node; Extract the effective interference values ​​of historical nodes and their corresponding historical priority offsets from the historical data parsing and collaborative logs. The priority interference sensitivity coefficient is obtained by calculating the ratio of the historical priority offset to the effective interference value of the historical node. The dynamic analytical priority is obtained by multiplying the priority interference sensitivity coefficient by the effective interference value to be measured.

7. The IoT master service platform data parsing and collaboration method according to claim 6, characterized in that, The effective interference value to be measured is obtained by analyzing the node congestion interference value and the type of congested node. The specific steps include: The congestion priority deviation value is determined based on the congestion node type and the basic priority of data segment parsing. The node congestion interference value is analyzed and the demand interference is decomposed based on the congestion priority deviation value to obtain the effective interference value to be measured.

8. The IoT master service platform data parsing and collaboration method according to claim 7, characterized in that, The initial parsing computing power allocation is compensated based on dynamic parsing priority, data encryption level, computational loss coefficient, and parsing fault tolerance deviation value to obtain the second actual parsing computing power. The specific steps include: The preprocessing parsing weight is obtained by combining the parsing complexity weight of the data encryption level with the weight increment of the dynamic parsing priority. The second computing power loss value is obtained by multiplying the computing loss coefficient with the preprocessing analysis weight; The initial parsing power allocation is compensated based on the second computing power loss value and the parsing fault tolerance deviation value to obtain the second actual parsing power.

9. The IoT master service platform data parsing and collaboration method according to claim 1, characterized in that, After constructing a data parsing collaborative scheduling framework based on the first or second actual parsing computing power, a data parsing collaborative optimization scheme is output, which specifically includes the following steps: A data parsing collaborative scheduling framework is obtained by mapping collaborative scheduling rules based on the first or second actual parsing computing power. The marking results are obtained based on the data parsing and collaborative scheduling framework; wherein, the marking results include the data segment with the highest computing power consumption and the data segment with the largest fault tolerance deviation; Generate a collaborative optimization scheme for data parsing based on the labeling results.

Images