Organic data reporting method and platform

Abstract

The invention relates to the technical field of dynamic data processing, and discloses an organic data reporting method, which comprises the following steps of: constructing a dynamic characteristic weight model, comprehensively quantifying the importance of data through time characteristics, spatial characteristics and context relevance, introducing differential privacy noise protection to high-weight data by utilizing a hierarchical privacy protection strategy, and reporting the data to the dynamic characteristic weight model; the low-weight data is subjected to lightweight encryption processing; a multi-level cache queue and a dynamic bandwidth allocation mechanism are combined, and low-delay reporting of high-priority data is preferentially guaranteed; and recalculating and comparing the dynamic characteristics of the data after privacy protection through a collaborative verification mechanism to ensure the context consistency of the data. According to the invention, the balance of privacy protection, security and real-time performance is realized through cooperation of multiple modules, the contradiction between privacy protection and data integrity is broken through, the transmission reliability and efficiency of high-priority data are remarkably improved in a high-traffic scene, and full-link optimization from data acquisition to data reporting is formed.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic data processing, and in particular to an organic data reporting method and platform. Background Art

[0002] Dynamic data processing technology has a wide range of applications in the fields of data collection, privacy protection and real-time transmission, especially in scenarios such as the Internet of Things, multi-source data transmission and intelligent analysis. The working principle of such systems usually includes the collection of multi-source heterogeneous data, privacy protection processing, transmission scheduling and final storage or reporting. In these scenarios, the timeliness and privacy protection of data are particularly important. For example, in intelligent transportation systems, vehicle location data needs to be uploaded in real time; in smart medical care, patient health monitoring data needs to ensure both real-time transmission and privacy. However, the performance of existing technologies in the above scenarios has significant limitations. For example, in order to achieve data privacy protection and transmission scheduling in the industry, the following solutions are usually adopted:

[0003] 1. Traditional full encryption method: Existing technologies usually perform unified full encryption on data to protect data privacy. However, full encryption consumes a lot of computing resources and bandwidth, especially in high-traffic scenarios, which can easily cause computing bottlenecks and reporting delays, and cannot meet real-time requirements. In addition, since the fully encrypted data cannot be directly analyzed, some contextual information will be lost, resulting in reduced data integrity and effectiveness.

[0004] 2. Data desensitization or weak encryption: In order to reduce the computing burden, some technologies use data desensitization or weak encryption to process data, but this method often fails to achieve a sufficient level of privacy protection, especially when processing highly sensitive data, there is a greater risk of leakage.

[0005] 3. Insufficient priority scheduling schemes: In the prior art, data transmission priority scheduling usually adopts static rules, such as based on preset fixed priorities. However, this static scheduling method lacks real-time perception of data importance and dynamic network conditions, resulting in critical data not being given priority and insufficient transmission reliability, especially when data traffic surges.

[0006] In addition, in order to further optimize the contradiction between privacy protection and real-time performance, the industry has also tried to introduce distributed architecture and edge computing technologies. Although these technologies have alleviated the computing bottleneck problem to a certain extent, their deployment and maintenance costs are high, and they are not good enough in terms of dynamic changes in multi-source data and maintaining contextual characteristics. Especially in high-frequency data collection and processing scenarios, existing technologies lack accurate analysis of the dynamic characteristics of data and cannot flexibly adapt to changing actual scenarios, which ultimately makes it difficult to achieve a balance between privacy protection, real-time performance, and system resource utilization.

[0007] Therefore, how to find the best balance between privacy protection and computing efficiency, data integrity and transmission scheduling efficiency to form a dynamic optimization solution for the entire link has become a technical problem to be solved by the present invention. Summary of the invention

[0008] The technical problem solved by the present invention is to provide an organic data reporting method and platform in response to the defects existing in the above-mentioned prior art, so as to solve the problems of the contradiction between privacy protection and computing resource consumption, loss of data integrity and insufficient transmission scheduling efficiency raised in the above-mentioned background technology.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0010] A method for reporting organic data, comprising the following steps:

[0011] Step 1: Establishment of dynamic characteristic weight model:

[0012] Analyze the spatiotemporal dynamic characteristics of organic data and build a dynamic characteristic weight model to extract and quantify the temporal characteristics, spatial characteristics and contextual relevance of data, including:

[0013] Based on time series analysis technology, the time distribution law and change trend of organic data are extracted to generate time characteristic scores;

[0014] Based on spatial association analysis technology, the correlation strength and geographical distribution characteristics between sources of organic data are extracted to generate spatial characteristic scores;

[0015] Based on the contextual correlation analysis technology, the relationship between the data and its previous and subsequent related data is extracted to generate a contextual correlation score;

[0016] The dynamic characteristic weight of each piece of data is calculated based on the time characteristic score, space characteristic score and context relevance score. The calculation formula of the dynamic characteristic weight is:

[0017] W i =α·T i +β·S i +γ·C i

[0018] Among them, W i is the dynamic characteristic weight value of the data, T i is the time characteristic score, S i is the spatial characteristic score, C i is the context relevance score, α, β, γ are the parameters for assigning weights, and α+β+γ=1;

[0019] Step 2: Hierarchical privacy protection processing:

[0020] Based on the weight values ​​of the dynamic characteristic weight model, the organic data is subjected to hierarchical privacy protection, including:

[0021] According to the dynamic characteristic weight value, the organic data is divided into high-weight data and low-weight data, wherein the data with a weight value higher than a preset weight threshold is the high-weight data, and the data with a weight value lower than the preset weight threshold is the low-weight data;

[0022] Differential privacy protection is used for high-weight data. The data is disturbed by introducing noise that satisfies the Laplace distribution. The calculation formula for the processed data value is:

[0023] D′ k =D k +Noise(∈)

[0024] Among them, D k is high-weight data, Noise(∈) is the noise that satisfies the Laplace distribution, and ∈ is the privacy protection parameter;

[0025] For low-weight data, a lightweight encryption algorithm is used to encrypt the data to reduce the usage of computing resources;

[0026] Step 3, priority reporting mechanism:

[0027] The reporting priority of organic data is dynamically allocated based on the dynamic characteristic weight model, and hierarchical reporting is achieved through multi-level cache queues, including:

[0028] According to the dynamic characteristic weight value, the data is divided into high priority data and low priority data, wherein the weight value of the high priority data is higher than the preset weight threshold, and the weight value of the low priority data is lower than the preset weight threshold;

[0029] Allocate cache queues for high-priority data and transmit them through real-time reporting channels;

[0030] Allocate a low-speed cache queue for low-priority data and use batch transmission for delayed reporting;

[0031] In the case of a surge in data traffic, the network bandwidth usage status is monitored in real time, the bandwidth allocation ratio of the high-speed cache queue and the low-speed cache queue is dynamically adjusted, and bandwidth resources are allocated to the high-speed cache queue first to ensure low-latency transmission of high-priority data;

[0032] Step 4: Collaborative verification of privacy and spatiotemporal characteristics:

[0033] Before data is reported, a collaborative verification mechanism is used to verify whether the organic data after privacy protection processing retains the dynamic characteristics of the original data, including:

[0034] Recalculate the contextual relevance score of the organic data after privacy protection based on the dynamic characteristic weight model and compare it with the contextual relevance score of the original data;

[0035] If the comparison result meets the set contextual consistency threshold, the data is submitted to the reporting process;

[0036] If the comparison result does not meet the contextual consistency threshold, the parameters of the privacy protection processing are readjusted, including adjusting the noise amplitude or encryption strength, and the data is reprocessed and verified again;

[0037] Step 5, Adaptive Optimization Module:

[0038] The weight parameters and weight thresholds of the dynamic characteristic weight model are dynamically adjusted through the adaptive optimization module, including:

[0039] Real-time monitoring of the reporting delay, privacy protection effect and bandwidth utilization of high-priority data during data reporting;

[0040] According to the monitoring results, the values ​​of weight parameters α, β, and γ are dynamically adjusted to change the proportion of temporal characteristics, spatial characteristics, and contextual relevance in weight calculation, so as to adapt to the data characteristics of different scenarios;

[0041] Adjust the weight threshold according to the privacy protection effect of real-time monitoring to reduce or increase the sensitivity of data classification, so as to adapt to different privacy protection intensity requirements.

[0042] As a further solution of the present invention, the method for extracting time characteristic scores by time series analysis comprises the following steps:

[0043] Analyze the collection frequency, time distribution and trend changes of organic data based on time series models;

[0044] Calculate the time score T for each piece of data i , the T i The calculation formula is:

[0045]

[0046] Among them, t c is the current time, t s is the data acquisition time, and λ is the adjustment parameter.

[0047] As a further solution of the present invention, the method for extracting spatial characteristic scores by spatial association analysis comprises the following steps:

[0048] Calculate the spatial distance between data source nodes based on the geographic coordinate information of the organic data and the physical location of the source nodes;

[0049] By calculating the inverse function value of the spatial distance, the spatial score S of each data is generated i , the S i The calculation formula is:

[0050]

[0051] Among them, d ij Represents the spatial distance between data source node i and node j.

[0052] As a further solution of the present invention, the method for extracting context relevance scores by context relevance analysis comprises the following steps:

[0053] Construct a context-dependent graph based on the logical dependency between the current data and the previous and next data;

[0054] Use graph neural network to calculate the context-related graph and generate the context-related score C for each piece of data. i .

[0055] As a further solution of the present invention, the initial values ​​of the weight parameters α, β, and γ in the dynamic characteristic weight model are preset according to the application scenario and dynamically adjusted by the adaptive optimization module in the following manner:

[0056] Calculate parameter adjustment direction in real time based on data reporting delay, privacy protection effect, and system bandwidth utilization;

[0057] The adjusted weight parameter value satisfies the formula: α+β+γ=1

[0058] And the adjustment range is between [0,1].

[0059] As a further solution of the present invention, the privacy noise Noise(∈) in the differential privacy protection process satisfies the following Laplace distribution:

[0060]

[0061] Among them, u is a random number in the interval [0,1], and ∈ is a privacy protection parameter.

[0062] As a further solution of the present invention, the bandwidth allocation ratio of the multi-level cache queue is calculated by the following formula:

[0063]

[0064] R l =1-R h

[0065] Among them, R h and R l Respectively represent the bandwidth allocation ratio of the high priority queue and the low priority queue, W h and W l Respectively represent the total weight values ​​of high priority data and low priority data.

[0066] As a further solution of the present invention, the collaborative verification mechanism recalculates the context relevance score C′ of the privacy-protected data when verifying the context consistency. i , and the original score C i Compare, if the following formula is met, then enter the data reporting process:

[0067] |C′ i -C i |≤δ

[0068] Where δ is the context consistency threshold.

[0069] An organic data reporting platform based on spatiotemporal dynamic characteristics, including:

[0070] A data collection module for collecting temporal, spatial and contextual information of organic data;

[0071] The dynamic characteristic weight model module is used to calculate the dynamic characteristic weight value based on the collected data, where the weight value is calculated comprehensively based on the time characteristic, spatial characteristic and context relevance score, and the calculation formula is:

[0072] W i =α·T i +β·S i +γ·C i

[0073] A hierarchical privacy protection module is used to perform hierarchical privacy protection on data according to dynamic characteristic weight values;

[0074] The data cache and reporting module is used to allocate data to multi-level cache queues according to their priority, and dynamically allocate bandwidth resources by monitoring bandwidth occupancy status in real time;

[0075] The adaptive optimization module is used to dynamically adjust the weight parameters α, β, γ and weight thresholds to optimize the data reporting process.

[0076] As a further solution of the present invention, the hierarchical privacy protection module includes:

[0077] A high-weight data processing unit, used to perform differential privacy protection processing on data whose dynamic characteristic weight value is higher than a weight threshold, wherein the differential privacy protection is achieved by introducing noise that satisfies the Laplace distribution;

[0078] A low-weight data processing unit, used for performing lightweight encryption processing on data whose dynamic characteristic weight value is lower than a weight threshold;

[0079] The privacy verification unit is used to verify whether the data processed by privacy protection meets the context consistency requirements.

[0080] Compared with the prior art, the present invention has the following beneficial effects:

[0081] 1. By organically combining the dynamic feature weight model, hierarchical privacy protection, multi-level cache queue and collaborative verification mechanism, a full-link technical solution covering data collection, privacy protection, transmission scheduling and consistency verification is formed, which solves the problem of balancing privacy protection, data integrity and real-time performance in existing technologies.

[0082] 2. The dynamic feature weight model comprehensively quantifies the importance of each piece of data based on three dimensions: time feature, spatial feature, and context relevance. The model formula W i =α·T i +β·S i +γ·C i It not only quantifies the characteristics of multi-dimensional data, but also adaptively optimizes according to the actual scenario by dynamically adjusting the weight parameters α, β, and γ, thereby accurately identifying the importance of data. Compared with existing simple rules or static indicators, this three-dimensional characteristic modeling greatly improves the recognition accuracy of key data in multi-source heterogeneous data streams, allowing key data to receive more reasonable resource allocation in subsequent privacy protection and reporting processes.

[0083] 3. Dynamic feature weight values ​​achieve hierarchical privacy protection, with differential privacy used for high-weight data and lightweight encryption used for low-weight data. This hierarchical processing method not only ensures the privacy security of high-weight data, but also significantly reduces the computing resource consumption of low-weight data. This dynamic grading strategy not only solves the defects of full encryption in real-time and computing efficiency, but also recalculates and compares the contextual relevance score C of the data after privacy processing through a collaborative verification mechanism. i , ensuring that the privacy-protected data can still retain key contextual information, and the multi-level cache queue architecture combines dynamic feature weight values ​​to achieve data priority scheduling, and dynamically allocates bandwidth resources by real-time monitoring of bandwidth occupancy status. The bandwidth allocation ratio is dynamically adjusted through a formula, so that high-priority data (critical data) can still be reported with low latency in traffic surge scenarios. Compared with the traditional priority queue method, it realizes efficient transmission of multi-source heterogeneous data streams, significantly improving the transmission reliability and real-time performance of high-priority data.

[0084] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0086] Figure 1 The figure is a flow chart of the organic data reporting method of the present invention. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] See also Figure 1 In an embodiment of the present invention, an organic data reporting method includes the following steps:

[0089] Step 1: Establishment of dynamic characteristic weight model:

[0090] Analyze the spatiotemporal dynamic characteristics of organic data and build a dynamic characteristic weight model to extract and quantify the temporal characteristics, spatial characteristics and contextual relevance of data, including:

[0091] Based on time series analysis technology, the time distribution law and change trend of organic data are extracted to generate time characteristic scores;

[0092] Based on spatial association analysis technology, the correlation strength and geographical distribution characteristics between sources of organic data are extracted to generate spatial characteristic scores;

[0093] Based on the contextual correlation analysis technology, the relationship between the data and its previous and subsequent related data is extracted to generate a contextual correlation score;

[0094] The dynamic characteristic weight of each piece of data is calculated based on the time characteristic score, space characteristic score and context relevance score. The calculation formula of the dynamic characteristic weight is:

[0095] W i =α·T i +β·S i +γ·Ci

[0096] Among them, W i is the dynamic characteristic weight value of the data, T i is the time characteristic score, S i is the spatial characteristic score, C i is the context relevance score, α, β, γ are the parameters for assigning weights, and α+β+γ=1;

[0097] Step 2: Hierarchical privacy protection processing:

[0098] Based on the weight values ​​of the dynamic characteristic weight model, the organic data is subjected to hierarchical privacy protection, including:

[0099] According to the dynamic characteristic weight value, the organic data is divided into high-weight data and low-weight data, wherein the data with a weight value higher than a preset weight threshold is the high-weight data, and the data with a weight value lower than the preset weight threshold is the low-weight data;

[0100] Differential privacy protection is used for high-weight data. The data is disturbed by introducing noise that satisfies the Laplace distribution. The calculation formula for the processed data value is:

[0101] D′ k =D k +Noise(∈)

[0102] Among them, D k is high-weight data, Noise(∈) is the noise that satisfies the Laplace distribution, and ∈ is the privacy protection parameter;

[0103] For low-weight data, a lightweight encryption algorithm is used to encrypt the data to reduce the usage of computing resources;

[0104] Step 3, priority reporting mechanism:

[0105] The reporting priority of organic data is dynamically allocated based on the dynamic characteristic weight model, and hierarchical reporting is achieved through multi-level cache queues, including:

[0106] According to the dynamic characteristic weight value, the data is divided into high priority data and low priority data, wherein the weight value of the high priority data is higher than the preset weight threshold, and the weight value of the low priority data is lower than the preset weight threshold;

[0107] Allocate cache queues for high-priority data and transmit them through real-time reporting channels;

[0108] Allocate a low-speed cache queue for low-priority data and use batch transmission for delayed reporting;

[0109] In the case of a surge in data traffic, the network bandwidth usage status is monitored in real time, the bandwidth allocation ratio of the high-speed cache queue and the low-speed cache queue is dynamically adjusted, and bandwidth resources are allocated to the high-speed cache queue first to ensure low-latency transmission of high-priority data;

[0110] Step 4: Collaborative verification of privacy and spatiotemporal characteristics:

[0111] Before data is reported, a collaborative verification mechanism is used to verify whether the organic data after privacy protection processing retains the dynamic characteristics of the original data, including:

[0112] Recalculate the contextual relevance score of the organic data after privacy protection based on the dynamic characteristic weight model and compare it with the contextual relevance score of the original data;

[0113] If the comparison result meets the set contextual consistency threshold, the data is submitted to the reporting process;

[0114] If the comparison result does not meet the contextual consistency threshold, the parameters of the privacy protection processing are readjusted, including adjusting the noise amplitude or encryption strength, and the data is reprocessed and verified again;

[0115] Step 5, Adaptive Optimization Module:

[0116] The weight parameters and weight thresholds of the dynamic characteristic weight model are dynamically adjusted through the adaptive optimization module, including:

[0117] Real-time monitoring of the reporting delay, privacy protection effect and bandwidth utilization of high-priority data during data reporting;

[0118] According to the monitoring results, the values ​​of weight parameters α, β, and γ are dynamically adjusted to change the proportion of temporal characteristics, spatial characteristics, and contextual relevance in weight calculation, so as to adapt to the data characteristics of different scenarios;

[0119] Adjust the weight threshold according to the privacy protection effect of real-time monitoring to reduce or increase the sensitivity of data classification, so as to adapt to different privacy protection intensity requirements.

[0120] As a further solution of the present invention, the method for extracting time characteristic scores by time series analysis comprises the following steps:

[0121] Analyze the collection frequency, time distribution and trend changes of organic data based on time series models;

[0122] Calculate the time score T for each piece of data i , the T i The calculation formula is:

[0123]

[0124] Among them, t c is the current time, t s is the data acquisition time, and λ is the adjustment parameter.

[0125] As a further solution of the present invention, the method for extracting spatial characteristic scores by spatial association analysis comprises the following steps:

[0126] Calculate the spatial distance between data source nodes based on the geographic coordinate information of the organic data and the physical location of the source nodes;

[0127] By calculating the inverse function value of the spatial distance, the spatial score S of each data is generated i , the S i The calculation formula is:

[0128]

[0129] Among them, d ij Represents the spatial distance between data source node i and node j.

[0130] As a further solution of the present invention, the method for extracting context relevance scores by context relevance analysis comprises the following steps:

[0131] Construct a context-dependent graph based on the logical dependency between the current data and the previous and next data;

[0132] Use graph neural network to calculate the context-related graph and generate the context-related score C for each piece of data. i .

[0133] As a further solution of the present invention, the initial values ​​of the weight parameters α, β, and γ in the dynamic characteristic weight model are preset according to the application scenario and dynamically adjusted by the adaptive optimization module in the following manner:

[0134] Calculate parameter adjustment direction in real time based on data reporting delay, privacy protection effect, and system bandwidth utilization;

[0135] The adjusted weight parameter value satisfies the formula: α+β+γ=1

[0136] And the adjustment range is between [0,1].

[0137] As a further solution of the present invention, the privacy noise Noise(∈) in the differential privacy protection process satisfies the following Laplace distribution:

[0138]

[0139] Among them, u is a random number in the interval [0,1], and ∈ is a privacy protection parameter.

[0140] As a further solution of the present invention, the bandwidth allocation ratio of the multi-level cache queue is calculated by the following formula:

[0141]

[0142] R l =1-R h

[0143] Among them, R h and R l Respectively represent the bandwidth allocation ratio of the high priority queue and the low priority queue, W h and W l Respectively represent the total weight values ​​of high priority data and low priority data.

[0144] As a further solution of the present invention, the collaborative verification mechanism recalculates the context relevance score C′ of the privacy-protected data when verifying the context consistency. i , and the original score C i Compare, if the following formula is met, then enter the data reporting process:

[0145] |C′ i -C i |≤δ

[0146] Where δ is the context consistency threshold.

[0147] An organic data reporting platform based on spatiotemporal dynamic characteristics, including:

[0148] A data collection module for collecting temporal, spatial and contextual information of organic data;

[0149] The dynamic characteristic weight model module is used to calculate the dynamic characteristic weight value based on the collected data, where the weight value is calculated comprehensively based on the time characteristic, spatial characteristic and context relevance score, and the calculation formula is:

[0150] W i =α·T i +β·S i +γ·C i

[0151] A hierarchical privacy protection module is used to perform hierarchical privacy protection on data according to dynamic characteristic weight values;

[0152] The data cache and reporting module is used to allocate data to multi-level cache queues according to their priority, and dynamically allocate bandwidth resources by monitoring bandwidth occupancy status in real time;

[0153] The adaptive optimization module is used to dynamically adjust the weight parameters α, β, γ and weight thresholds to optimize the data reporting process.

[0154] As a further solution of the present invention, the hierarchical privacy protection module includes:

[0155] A high-weight data processing unit, used to perform differential privacy protection processing on data whose dynamic characteristic weight value is higher than a weight threshold, wherein the differential privacy protection is achieved by introducing noise that satisfies the Laplace distribution;

[0156] A low-weight data processing unit, used for performing lightweight encryption processing on data whose dynamic characteristic weight value is lower than a weight threshold;

[0157] The privacy verification unit is used to verify whether the data processed by privacy protection meets the context consistency requirements.

[0158] Embodiment 1:

[0159] In the intelligent transportation system, real-time vehicle location information upload is an important technical requirement to ensure traffic flow optimization, accident handling and public safety. However, vehicle location information often involves user privacy. At the same time, the large amount of data upload during peak traffic hours places extremely high demands on the real-time performance of the transmission system. In order to cope with the contradiction between privacy protection and real-time performance and insufficient allocation of transmission resources, the organic data reporting method of the present invention has been specifically implemented in this scenario.

[0160] In this smart transportation application scenario, the system collects vehicle location information, speed information, and surrounding environment information through vehicle terminals and roadside units (RSUs) deployed in the urban road network. The dynamic characteristics of data determine its importance in traffic scheduling. For example, vehicle information at traffic accident sites and dynamic data on congested sections are highly time-sensitive, while the location information of vehicles on normal sections is relatively less important.

[0161] When a vehicle enters the coverage area of ​​the urban traffic management system, the vehicle terminal will collect dynamic data such as its location information, speed, lane occupancy, etc. in real time. These data are first transmitted to the roadside unit, where the importance of the data is quantified in real time through the dynamic characteristic weight model of the present invention. The dynamic characteristic weight model integrates information from three dimensions:

[0162] Time characteristics: The system analyzes the relationship between data collection time and current system requirements. For example, the closer the vehicle information is to the current time, the greater its impact on traffic prediction and scheduling, and thus the time characteristics have a higher weight.

[0163] Spatial characteristics: The spatial characteristics of the data are determined by analyzing the distance between the location of the vehicle to which the data belongs and the accident point, congested road section or key intersection. The data of vehicles closer to the key area have higher priority.

[0164] Contextual relevance: By analyzing vehicle behavior and traffic flow characteristics in the previous and next time periods, the relevance of the data in the overall traffic flow forecast is determined. Contextual information, such as whether it is in a coherent congestion chain or close to a key point of traffic transfer, will significantly increase the importance of the data.

[0165] The dynamic characteristic weight model generates a dynamic characteristic weight value for each piece of data based on these analysis results. This weight value directly determines the subsequent data processing priority.

[0166] The collected dynamic data is divided into high-weight data and low-weight data according to the weight value.

[0167] High-weight data (such as vehicle information near the accident site) is processed by differential privacy technology. The system introduces noise that satisfies the Laplace distribution on these data to achieve high-intensity privacy protection. Differential privacy protection ensures that even if the data is illegally obtained, it is difficult for attackers to infer the specific information of a single vehicle.

[0168] Low-weight data (such as vehicle location information on ordinary roads) is processed using lightweight encryption technology. While protecting data privacy, lightweight encryption significantly reduces computing resources and bandwidth consumption, allowing the system to handle larger data flows.

[0169] Through this hierarchical privacy protection method, the present invention not only ensures the privacy protection of highly sensitive data, but also takes into account the processing efficiency of low-sensitivity data.

[0170] The processed data is allocated to multi-level cache queues according to the weight values. High-weight data enters the high-speed cache queue, and low-weight data enters the low-speed cache queue. During peak traffic hours, when data traffic surges, the system dynamically adjusts the bandwidth allocation ratio of the high-speed cache queue and the low-speed cache queue by monitoring bandwidth occupancy in real time. High-weight data is uploaded first to ensure that information on accident points and congested sections can reach the traffic control center in real time, thereby providing timely support for measures such as signal light control and emergency lane opening. Compared with traditional fixed priority scheduling, the dynamic bandwidth allocation mechanism of the present invention effectively improves the transmission reliability and real-time performance of high-priority data.

[0171] Before uploading the data, the system verifies the consistency of all data processed after privacy protection. The system recalculates the contextual characteristic values ​​of high-weight data and compares them with the original characteristic values. If the privacy protection process causes the contextual characteristics of the data to deviate significantly, for example, the dynamic information of vehicles near the accident point cannot correctly reflect the continuity of the congestion chain, the system will automatically adjust the privacy protection parameters (such as reducing the noise amplitude) and reprocess the data to ensure the validity of the uploaded data. This collaborative verification mechanism not only ensures that the data after privacy protection can still maintain the consistency of dynamic characteristics, but also avoids the negative impact of privacy protection on traffic prediction and control effects.

[0172] The present invention uses hierarchical privacy protection to provide high-intensity protection for high-weight data, while lightweight processing of low-weight data significantly reduces computing resource consumption. Compared with traditional full encryption methods, it is less costly, and multi-level cache queues and dynamic bandwidth allocation mechanisms ensure low-latency transmission of high-priority data. Even during peak traffic hours, the delay in uploading accident point data is reduced; through the collaborative verification mechanism, privacy-protected data can still correctly reflect the dynamic characteristics of traffic flow, solving the problem of context information loss in traditional privacy protection schemes, and through dynamic characteristic weight models and adaptive parameter optimization, the system can dynamically adjust weight allocation and privacy processing strategies according to traffic flow, accident frequency, etc., to adapt to the changing needs of traffic scenarios, so that the present invention not only solves the contradiction between privacy protection and real-time performance in the prior art, but also significantly improves the efficiency and safety of traffic scheduling in high-traffic scenarios.

[0173] Embodiment 2:

[0174] In order to verify the superiority of the present invention in terms of privacy protection and real-time performance, data integrity, and resource utilization, this embodiment designs a set of comparative experiments, such as selecting a smart medical scenario as the test environment to simulate the multi-source heterogeneous data upload process, including patient health data (such as heart rate, blood pressure, blood oxygen) and dynamic monitoring data (such as geographic location, emergency call status). In the experimental environment, the following two data upload scenarios are set:

[0175] Scenario 1: In emergency situations, real-time upload of patient data is crucial, and high-priority data such as heart rate and blood oxygen need to be quickly transmitted to the medical platform.

[0176] Scenario 2: In routine monitoring status, patient data with lower real-time requirements (such as geographic location) accounts for the majority of the uploaded data, but privacy and security still need to be guaranteed.

[0177] The experiment uses the existing full encryption upload technology (comparison technology 1), desensitizing encryption combined with fixed priority upload technology (comparison technology 2) and the technical solution of the present invention to upload data. The test indicators include: real-time: average upload delay of high-priority data; privacy protection effect: evaluate the privacy protection strength through differential attack simulation; data integrity: the degree of preservation of contextual relevance of data after privacy processing; resource consumption: system CPU and bandwidth occupancy.

[0178] In the emergency state (scenario 1), the comparison technology 1 leads to a computing bottleneck due to full encryption, and the average upload delay of high-priority data is 320ms; although the comparison technology 2 reduces the encryption pressure, the fixed priority scheduling lacks real-time adaptability and the delay is 200ms; the present invention gives higher weights to high-priority data through a dynamic characteristic weight model, and combines multi-level cache queues with dynamic bandwidth allocation to achieve an average upload delay of only 110ms for high-priority data, and the delay reduction is 65.6%, which is significantly better than the existing technology. In the privacy protection strength test, the privacy leakage rate of high-priority data is analyzed by simulating differential attacks. Due to the use of weak encryption, the privacy leakage rate of comparison technology 2 is as high as 8.2%; the leakage rate of comparison technology 1 is lower, at 1.1%, but its full encryption method sacrifices a lot of computing resources. The present invention uses differential privacy processing for high-priority data through dynamic characteristic weight values, and the privacy leakage rate is as low as 0.8%, which not only enhances the privacy protection strength, but also significantly reduces resource consumption. The full encryption of the comparative technology 1 leads to the complete loss of data context characteristics, and the context relevance retention rate is only 45%; the data desensitization processing of the comparative technology 2 is also difficult to retain the dynamic characteristics of the data, and its relevance retention rate is 60%. The present invention verifies the consistency of the dynamic characteristics of the data after privacy processing through a collaborative verification mechanism, and dynamically adjusts the privacy processing parameters according to the verification results. The data context relevance retention rate reaches 92%, which is close to the integrity level of unencrypted data. Under the normal monitoring state (scenario 2), the full encryption method of the comparative technology 1 leads to a system CPU occupancy rate of up to 78%. Although the desensitization encryption combined with fixed priority scheduling of the comparative technology 2 is optimized, it still takes up 58% of the system CPU. The present invention uses a hierarchical privacy protection strategy to use lightweight encryption processing for low-weight data, so that the CPU occupancy rate is only 37%, which is 52.6% lower than that of the comparative technology 1, and significantly improves the resource utilization of the system in high-traffic scenarios.

[0179] It can be seen from the experimental results that the present invention not only achieves significant progress in privacy protection, real-time performance and data integrity compared with the prior art, but the dynamic characteristic weight model not only improves the recognition ability of high-priority data, but also achieves a dynamic balance between privacy protection strength and computing resource consumption through the joint action of hierarchical privacy protection and multi-level cache scheduling; the collaborative verification mechanism ensures the preservation of data context characteristics during the privacy protection process. This dynamic consistency verification has not been achieved in the prior art and greatly improves the availability of encrypted data; dynamic bandwidth allocation further optimizes system resource allocation, especially in high-traffic scenarios. Through real-time adjustment, it solves the bandwidth waste problem of the existing fixed allocation method and optimizes the overall performance of the system.

[0180] Embodiment 3:

[0181] This embodiment takes the smart industrial scenario as an example to illustrate the specific implementation steps of the dynamic characteristic weight model, the specific adjustment method of the optimization parameters, and the specific operation process of context characteristic preservation and dynamic verification during the privacy protection processing.

[0182] For example, in smart industrial scenarios, sensor networks are deployed on multiple production equipment to collect dynamic data such as temperature, vibration, and energy consumption. These data are used to predict equipment failures, optimize production efficiency, and schedule resources. The system needs to upload these data to the central server in real time, while ensuring the privacy and security of key data and giving priority to transmitting important data during peak network traffic. The existing full encryption and static priority scheduling methods have serious computing bottlenecks and lack of real-time performance in this scenario.

[0183] In this embodiment, the dynamic characteristic weight model is used to quantify the importance of each piece of collected data, and the weight value W i It is calculated by the time characteristic score, space characteristic score and context relevance score. The specific implementation steps are as follows:

[0184] Time characteristic score calculation: Sensor data is assigned different time characteristic scores according to the real-time requirements of the equipment's operating status. For example, equipment vibration data has high real-time requirements in predicting equipment failures, so it has a higher priority score. The time characteristic score is quantified by setting a time threshold (e.g., vibration data within 5 minutes is a high priority).

[0185] Spatial characteristic score calculation: Analyze the geographical distribution information between devices. For example, when a device is close to a high failure rate area or a key resource consumption point, the importance of its collected data increases. By dynamically updating the association matrix between devices, the score is adjusted to a higher priority based on the regional weight.

[0186] Calculation of context relevance score: By constructing a context relevance graph, the dynamic relevance of data between devices is extracted. For example, if the data of multiple devices show fault chain characteristics (such as vibration and temperature data are abnormal at the same time), these data will be assigned a higher context relevance score. The context relevance graph is dynamically updated through a sliding window mechanism to ensure the real-time relevance score.

[0187] In this embodiment, the calculation result of the dynamic characteristic weight value is directly used for the dynamic adjustment of privacy protection and data transmission priority.

[0188] To ensure a dynamic balance between privacy protection strength and computing efficiency, this embodiment provides a specific parameter adjustment mechanism:

[0189] High-weight data processing: The amplitude of differential privacy noise ∈ is dynamically adjusted according to the distribution of weight values. For example, when the proportion of high-weight data is too high (more than 50%), the system will automatically increase the value of ∈\epsilon∈ to reduce the noise intensity, ensuring that the privacy protection strength is moderate and does not affect the consistency of data context.

[0190] Low-weight data processing: Low-weight data uses lightweight encryption algorithms (such as AES-128) to reduce the amount of calculation through group encryption. For scheduled upload of sensor data, the system will batch transmit low-weight data during non-peak hours to further reduce bandwidth pressure.

[0191] At the same time, in order to ensure that the data after privacy protection processing can still accurately reflect the operating status of the device, this embodiment adopts the following context consistency verification steps:

[0192] Recalculate context feature scores: Build a new context-related graph for the privacy-protected data and compare it with the original context-related graph to verify whether the data features remain consistent.

[0193] Consistency deviation judgment: When the deviation of the context feature score after privacy processing exceeds the threshold (such as 5%), the system will dynamically adjust the differential privacy noise amplitude or re-optimize the encryption parameters to ensure the integrity of the context information.

[0194] Verification mechanism optimization: The system uses incremental learning technology to optimize the context consistency verification process. For example, when device data is frequently updated, the system only verifies the context characteristics of high-weight data, thereby reducing verification time and computing resource consumption.

[0195] Furthermore, in order to adapt to peak data traffic, this embodiment dynamically adjusts the bandwidth allocation ratio of the multi-level cache queue by real-time monitoring of the traffic proportion and bandwidth utilization of the device uploading data. The specific operations are as follows: high-weight data is uploaded in real time through the cache queue. The system obtains the bandwidth occupancy status of the current queue through a real-time bandwidth monitoring tool (such as an SNMP-based network traffic analyzer), and dynamically allocates more bandwidth resources to the high-weight data queue; low-weight data is uploaded in delayed batches through a low-speed cache queue to save network bandwidth during peak periods. The interval time of the delayed batch is dynamically adjusted by the data traffic, for example, a 10-minute interval is used during peak periods and a 30-minute interval is used during non-peak periods.

[0196] Through the technical solution of this embodiment, the sensor data collected in the smart industrial scene is efficiently protected and transmitted in real time. Experimental data show that the leakage rate of high-weight data after differential privacy protection is reduced to 0.6%, which is significantly improved compared with the full encryption method (1.0%) and the weak encryption method (6.8%); the average upload delay of high-weight data is 120ms, which is 61% lower than the full encryption scheme (310ms) and the fixed priority scheme (200ms); through the dynamic bandwidth allocation mechanism, the bandwidth utilization of the system during peak hours is increased to 85%, which is 20% higher than the static allocation method; the context consistency retention rate after privacy protection reaches 95%, which is significantly higher than the traditional differential privacy scheme (70%), enhancing the ability to balance privacy protection and real-time performance in smart industrial scenarios.

[0197] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

Claims

1. A method for reporting organic data, characterized in that: The following steps are involved: Step 1: Establishment of dynamic characteristic weight model: Analyze the spatiotemporal dynamic characteristics of organic data and build a dynamic characteristic weight model to extract and quantify the temporal characteristics, spatial characteristics and contextual relevance of data, including: Based on time series analysis technology, the time distribution law and change trend of organic data are extracted to generate time characteristic scores; Based on spatial association analysis technology, the correlation strength and geographical distribution characteristics between sources of organic data are extracted to generate spatial characteristic scores; Based on contextual correlation analysis technology, the relationship between data and its previous and subsequent related data is extracted to generate contextual correlation scores; The dynamic characteristic weight of each piece of data is calculated based on the time characteristic score, space characteristic score and context relevance score. The calculation formula of the dynamic characteristic weight is: W i =α·T i +β·S i +γ·C i Among them, W i is the dynamic characteristic weight value of the data, T i is the time characteristic score, S i is the spatial characteristic score, C i is the context relevance score, α, β, γ are the parameters for assigning weights, and α+β+γ=1; Step 2: Hierarchical privacy protection processing: Based on the weight values ​​of the dynamic characteristic weight model, the organic data is subjected to hierarchical privacy protection, including: According to the dynamic characteristic weight value, the organic data is divided into high-weight data and low-weight data, wherein the data with a weight value higher than a preset weight threshold is the high-weight data, and the data with a weight value lower than the preset weight threshold is the low-weight data; Differential privacy protection is used for high-weight data. The data is disturbed by introducing noise that satisfies the Laplace distribution. The calculation formula for the processed data value is: D′ k =D k +Noise(∈) Among them, D k is high-weight data, Noise(∈) is the noise that satisfies the Laplace distribution, and ∈ is the privacy protection parameter; For low-weight data, a lightweight encryption algorithm is used to encrypt the data to reduce the usage of computing resources; Step 3, priority reporting mechanism: The reporting priority of organic data is dynamically allocated based on the dynamic characteristic weight model, and hierarchical reporting is achieved through multi-level cache queues, including: According to the dynamic characteristic weight value, the data is divided into high priority data and low priority data, wherein the weight value of the high priority data is higher than the preset weight threshold, and the weight value of the low priority data is lower than the preset weight threshold; Allocate cache queues for high-priority data and transmit them through real-time reporting channels; Allocate a low-speed cache queue for low-priority data and use batch transmission for delayed reporting; In the case of a surge in data traffic, the network bandwidth usage status is monitored in real time, the bandwidth allocation ratio of the high-speed cache queue and the low-speed cache queue is dynamically adjusted, and bandwidth resources are allocated to the high-speed cache queue first to ensure low-latency transmission of high-priority data; Step 4: Collaborative verification of privacy and spatiotemporal characteristics: Before data is reported, a collaborative verification mechanism is used to verify whether the organic data after privacy protection processing retains the dynamic characteristics of the original data, including: Recalculate the contextual relevance score of the organic data after privacy protection based on the dynamic characteristic weight model and compare it with the contextual relevance score of the original data; If the comparison result meets the set contextual consistency threshold, the data is submitted to the reporting process; If the comparison result does not meet the contextual consistency threshold, the parameters of the privacy protection processing are readjusted, including adjusting the noise amplitude or encryption strength, and the data is reprocessed and verified again; Step 5, Adaptive Optimization Module: The weight parameters and weight thresholds of the dynamic characteristic weight model are dynamically adjusted through the adaptive optimization module, including: Real-time monitoring of the reporting delay, privacy protection effect and bandwidth utilization of high-priority data during data reporting; According to the monitoring results, the values ​​of weight parameters α, β, and γ are dynamically adjusted to change the proportion of temporal characteristics, spatial characteristics, and contextual relevance in weight calculation, so as to adapt to the data characteristics of different scenarios; Adjust the weight threshold according to the privacy protection effect of real-time monitoring to reduce or increase the sensitivity of data classification, so as to adapt to different privacy protection intensity requirements.

2. The method for reporting organic data according to claim 1, characterized in that: The method for extracting time characteristic scores through time series analysis comprises the following steps: Analyze the collection frequency, time distribution and trend changes of organic data based on time series models; Calculate the time score T for each piece of data i , the T i The calculation formula is: Among them, t c is the current time, t s is the data acquisition time, and λ is the adjustment parameter.

3. The method for reporting organic data according to claim 1, characterized in that: The method for extracting spatial characteristic scores through spatial association analysis comprises the following steps: Calculate the spatial distance between data source nodes based on the geographic coordinate information of the organic data and the physical location of the source nodes; By calculating the inverse function value of the spatial distance, the spatial score S of each data is generated i , the S i The calculation formula is: Among them, d ij Represents the spatial distance between data source node i and node j.

4. The method for reporting organic data according to claim 1, characterized in that: The method for extracting context relevance scores through context relevance analysis comprises the following steps: Construct a context-dependent graph based on the logical dependency between the current data and the previous and next data; Use graph neural network to calculate the context-related graph and generate the context-related score C for each piece of data. i .

5. The method for reporting organic data according to claim 1, characterized in that: The initial values ​​of the weight parameters α, β, and γ in the dynamic characteristic weight model are preset according to the application scenario and dynamically adjusted by the adaptive optimization module in the following manner: Calculate parameter adjustment direction in real time based on data reporting delay, privacy protection effect, and system bandwidth utilization; The adjusted weight parameter value satisfies the formula: α+β+γ=1 And the adjustment range is between [0,1].

6. The method for reporting organic data according to claim 1, characterized in that: The privacy noise Noise(∈) in the differential privacy protection process satisfies the following Laplace distribution: Among them, u is a random number in the interval [0,1], and ∈ is a privacy protection parameter.

7. The method for reporting organic data according to claim 1, characterized in that: The bandwidth allocation ratio of the multi-level cache queue is calculated by the following formula: R l =1-R h Among them, R h and R l Respectively represent the bandwidth allocation ratio of the high priority queue and the low priority queue, W h and W l Respectively represent the total weight values ​​of high priority data and low priority data.

8. The method for reporting organic data according to claim 1, characterized in that: The collaborative verification mechanism recalculates the context relevance score C′ of the privacy-protected data when verifying the context consistency. i , and the original score C i Compare, if the following formula is met, then enter the data reporting process: |C′ i -C i |≤δ Where δ is the context consistency threshold.

9. An organic data reporting platform based on spatiotemporal dynamic characteristics, characterized in that: include: A data collection module for collecting temporal, spatial and contextual information of organic data; The dynamic characteristic weight model module is used to calculate the dynamic characteristic weight value based on the collected data, where the weight value is calculated comprehensively based on the time characteristic, spatial characteristic and context relevance score, and the calculation formula is: W i =α·T i +β·S i +γ·C i A hierarchical privacy protection module is used to perform hierarchical privacy protection on data according to dynamic characteristic weight values; The data cache and reporting module is used to allocate data to multi-level cache queues according to their priority, and dynamically allocate bandwidth resources by monitoring bandwidth occupancy status in real time; The adaptive optimization module is used to dynamically adjust the weight parameters α, β, γ and weight thresholds to optimize the data reporting process.

10. The organic data reporting platform based on spatiotemporal dynamic characteristics according to claim 9, characterized in that: It is characterized in that The hierarchical privacy protection module includes: A high-weight data processing unit, used to perform differential privacy protection processing on data whose dynamic characteristic weight value is higher than a weight threshold, wherein the differential privacy protection is achieved by introducing noise that satisfies the Laplace distribution; A low-weight data processing unit, used for performing lightweight encryption processing on data whose dynamic characteristic weight value is lower than a weight threshold; The privacy verification unit is used to verify whether the data processed by privacy protection meets the context consistency requirements.

Images