IoT data processing system supporting heterogeneous data fusion and AI services
By constructing a semantic graph of temporal causal semantic edges and prediction edges, the semantic differences and storage resource management problems of multi-source heterogeneous data in IoT data processing systems are solved, achieving efficient data fusion and dynamic storage scheduling, and improving AI service and security protection capabilities.
Patent Information
- Application Number
- CN202611372050.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-05
- Publication Date
- 2026-11-17
AI Technical Summary
Existing IoT data processing systems struggle to effectively handle semantic differences in multi-source heterogeneous data, resulting in low data fusion accuracy. Storage resource management lacks the ability to predict causal evolution trends, leading to delayed expansion or resource waste. Security analysis is limited to single access logs, making it difficult to identify indirect attacks.
By extracting semantic atoms from the edge side and anchoring and matching with the domain digital twin semantic template library, a semantic graph of temporal causal semantic edges and prediction edges is constructed. Combined with the graph attention credibility propagation network and the causal constraint rule set, the semantic unified mapping, causal encoding and dynamic storage scheduling of data are realized, and a data behavior profile is constructed for risk assessment.
It achieves unified semantic fusion of multi-source heterogeneous data, dynamically optimizes storage resource allocation, improves the efficiency and security of AI services, can predict storage needs in advance, identify hidden threats, and improve the system's response speed and security.
Smart Images

Figure CN123114375A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and more specifically, to an Internet of Things (IoT) data processing system that supports heterogeneous data fusion and AI services. Background Technology
[0002] With the rapid development of IoT technology, massive amounts of multi-source heterogeneous sensing data are experiencing explosive growth. In typical IoT application scenarios such as smart manufacturing, intelligent transportation, and environmental monitoring, the data often comes from a variety of sensing devices, which have significant differences in data format, communication protocols, semantic expression, and temporal characteristics, resulting in severe data heterogeneity. How to effectively integrate, govern, and service these multi-source heterogeneous data has become a core challenge in the field of IoT data processing.
[0003] In existing technologies, the processing of multi-source heterogeneous data typically employs traditional ETL (Extract-Transform-Load) tools or rule-based data cleaning methods, achieving initial data integration through format conversion and field mapping. However, these methods primarily address the heterogeneity problem at the syntactic level, struggling to handle semantic differences between different data sources. This results in the same real-world object being repeatedly described as multiple heterogeneous data entities across different data sources, leading to low data fusion accuracy. Furthermore, in terms of storage resource management, existing IoT data storage systems often employ static management strategies based on physical capacity thresholds. Expansion is triggered when storage space reaches a preset limit, or data migration is performed based on a simple hot / cold tier strategy. These methods treat data merely as an undifferentiated byte stream, failing to consider differences in semantic value, causal importance, and AI service dependence. This leads to a disconnect between storage resource scheduling and the actual data demands of upper-layer AI services. Particularly when facing the explosive growth of IoT data over time, existing solutions lack the ability to predict storage demand based on the causal evolution trend of data, often falling into a dilemma of passive expansion leading to resource waste or delayed expansion leading to service interruption.
[0004] In view of this, the present invention proposes an IoT data processing system that supports heterogeneous data fusion and AI services to solve the above problems. Summary of the Invention
[0005] To overcome the aforementioned deficiencies of the prior art and achieve the above objectives, the present invention provides the following technical solution: An IoT data processing system that supports heterogeneous data fusion and AI services, including: The data processing module is used to acquire multi-source heterogeneous sensing data, map the multi-source heterogeneous sensing data into a unified semantic graph, and perform data governance on the semantic graph to obtain reliable data. The data storage module includes multiple expandable storage bins for acquiring reliable data sources and storing the reliable data in the corresponding storage bins in the cloud according to the data sources. The storage management module is used to obtain the reliable data storage volume of each storage warehouse, calculate the resource stress of each storage warehouse based on the reliable data storage volume, and dynamically adjust the storage space of each storage warehouse based on the resource stress. The AI service module is used to extract reliable data from each storage bin and send the extracted reliable data to each preset service intelligence body. The service intelligence body then performs AI model training and inference calculations based on the reliable data to generate corresponding AI service outputs. The security management module obtains the data flow records of reliable data corresponding to each storage warehouse and constructs a data behavior profile based on the data flow records; calculates the risk score of each storage warehouse based on the data behavior profile; receives the current request and classifies the risk level of the current request based on the risk score.
[0006] Furthermore, the method for mapping multi-source heterogeneous sensing data into a unified semantic graph is as follows: Perform edge-side semantic atomization extraction on multi-source heterogeneous sensing data to construct a semantic atom set; based on the pre-built domain digital twin semantic template library, anchor the semantic atom set with each digital twin semantic template in the domain digital twin semantic template library to obtain an anchored semantic entity set; Each semantic entity in the anchored semantic entity set is mapped to a unified semantic vector. The semantic similarity between the unified semantic vectors is calculated. Based on the semantic similarity, cross-source entity disambiguation and semantic association weaving are performed on the semantic entities from different data sources in the anchored semantic entity set to construct a preliminary semantic graph. Temporal association features are extracted based on the temporal attributes of each semantic entity in the preliminary semantic graph. State transition causal patterns are mined based on the temporal association features, temporal causal semantic edges are constructed, and prediction edges are generated based on the state transition causal patterns. Each semantic entity is mapped to a state node by combining its corresponding temporal attributes, and the initial semantic graph is expanded into a semantic graph containing state nodes, temporal causal semantic edges, and prediction edges.
[0007] Furthermore, the method for obtaining reliable data is as follows: Multidimensional semantic quality features are extracted from each state node, temporal causal semantic edge, and prediction edge in the semantic graph, and multidimensional quality feature vectors are constructed for each state node. The semantic graph and its multi-dimensional quality feature vectors are input into a pre-trained graph attention credibility propagation network for credibility propagation inference to obtain the initial credibility score of each state node; a causal constraint rule set is constructed to verify the causal compliance of the semantic graph to obtain the verification result of each state node; based on the initial credibility score and verification result of each state node, a data governance strategy is matched for each state node. After implementing the data governance strategy, the semantic graph is re-performed with credibility propagation reasoning and causal consistency verification, iterating until convergence. The data of each state node in the converged semantic graph is then used as reliable data.
[0008] Furthermore, methods for storing reliable data in corresponding storage repositories in the cloud include: Multiple data features are extracted from the data source corresponding to the reliable data. Based on the pre-configured storage warehouse matching rule table for each storage warehouse, each data feature is matched to obtain the number of data feature matches for each storage warehouse. The storage warehouse with the largest number of data feature matches is selected as the target storage warehouse for storing the corresponding reliable data.
[0009] Furthermore, methods for calculating the resource stress of each storage warehouse include: Based on the state nodes corresponding to each reliable data in the storage bin, the number of temporal causal semantic edges associated with each state node is counted to obtain the temporal causal semantic edge density, and the number of predicted edges pointing to each state node is counted to obtain the predicted edge pointing frequency. Based on the temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score of each state node in the storage bin, a causal profile of the data value of each storage bin is constructed. Based on the predicted edges in the semantic graph, causal inference is performed on each storage bin to obtain the causal derivative storage demand of each storage bin in a preset future time period. The AI service module obtains the historical data extraction frequency of each storage bin, the training load index and inference load of the current service agent, calculates the data dependency based on the historical data extraction frequency, calculates the computing power pressure based on the training load index and inference load, and weights and fuses the data dependency and computing power pressure to obtain the coupling demand factor of each storage bin. The resource stress of each storage warehouse is obtained by weighting and summing the reliable data storage volume, the causal profile of data value, the storage demand derived from the causal relationship, and the coupling demand factors.
[0010] Furthermore, methods for dynamically adjusting the storage space of each storage compartment include: Set corresponding expansion thresholds and recycling thresholds for each storage warehouse. When the resource stress is greater than the expansion threshold, the storage warehouse is expanded and the corresponding storage warehouse is recorded as the expansion warehouse. When the resource stress is less than the recycling threshold, the storage warehouse is recycled and the corresponding storage warehouse is recorded as the recycling warehouse.
[0011] Furthermore, methods for dynamically adjusting the storage space of each storage compartment also include: Calculate the expansion capacity of each expansion compartment, sort them in descending order of expansion capacity, and obtain the expansion sorting table; Calculate the recycling volume of each recycling bin, sort them in descending order of recycling volume, and obtain a recycling sorting table; Based on the priority order determined by the expansion sorting table and the recycling sorting table, a spatial resource matching relationship is established between the recycling bins and the expansion bins. The amount of recycling to be recycled in the recycling bins is then redistributed to the corresponding expansion bins until the expansion capacity requirements of each expansion bin are met or the recycling amount of each recycling bin is exhausted.
[0012] Furthermore, the methods for generating the corresponding AI service output include: Acquire reliable data from each storage bin, perform service preprocessing on the reliable data, and obtain training sample data and inference input data; The training sample data is sent to the corresponding service agent, which then performs AI model training based on the training sample data to obtain the trained target model. The trained target model is encapsulated into a standardized service interface, and the corresponding calling parameters are configured to generate an AI service instance. The inference input data is then input into the AI service instance to perform inference calculations and obtain the corresponding AI service output.
[0013] Furthermore, the method for constructing data behavior profiles is as follows: Multidimensional behavioral feature parameters are extracted from data flow records. Based on these parameters, time-series behavioral feature vectors are constructed for each storage warehouse according to a preset time window. Based on the time-series behavioral feature vectors of each storage warehouse, corresponding data behavior profiles are generated through preset profile construction rules. These profiles include access behavior profiles, flow behavior profiles, and storage operation behavior profiles.
[0014] Furthermore, the method for classifying the risk level of the current request is as follows: Based on the data behavior profile, the historical behavior baseline of the corresponding storage warehouse is extracted, and the request feature information is extracted from the current request. Based on the request feature information and the historical behavior baseline, the behavior deviation is calculated. Based on the behavioral deviation, risk anchoring state nodes are determined in the semantic graph. Based on the temporal causal semantic edges and prediction edges associated with the risk anchoring state nodes, risk causal propagation is deduced to obtain the cumulative causal risk value. Identify the request type of the current request, and match the corresponding risk coupling factor based on the request type; The risk score for the current request is obtained by weighting and fusing behavioral deviation, cumulative causal risk value, and risk coupling factors. Multiple risk level threshold ranges are preset, and the corresponding risk level is determined based on the risk level threshold range in which the risk score falls.
[0015] The technical effects and advantages of this invention's IoT data processing system that supports heterogeneous data fusion and AI services are as follows: This invention achieves unified semantic mapping of multi-source heterogeneous sensing data by extracting semantic atoms at the edge and anchoring and matching with a domain digital twin semantic template library. This effectively eliminates the repetitive heterogeneous representation of the same real-world object by different data sources. At the same time, by constructing temporal causal semantic edges and prediction edges, the dynamic temporal characteristics and state transition causal laws of the data are explicitly encoded in the graph structure. This overcomes the shortcomings of existing static semantic association graphs that cannot express the causal evolution law of IoT data, and provides a structured causal knowledge foundation for subsequent storage scheduling, AI services and security management.
[0016] This invention constructs a causal profile of data value by introducing temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score, thereby upgrading storage resource evaluation from being driven by physical capacity to being driven by data value and causal importance. By predicting storage demand based on causal derivation of predicted edges, storage scheduling is endowed with causal inference capabilities, enabling the pre-allocation of resources for future data growth and avoiding the lag of traditional threshold-triggered expansion. By coupling demand factors to incorporate the real-time training and inference loads of AI services into storage scheduling decisions, the coordinated optimization of storage and computing resources is achieved, effectively preventing the problem of storage expansion being disconnected from AI computing power requirements.
[0017] This invention constructs a data behavior profile by integrating multi-dimensional data flow records such as access requests, data extraction, and storage adjustments, achieving a three-dimensional depiction of the entire lifecycle flow behavior of IoT data. This overcomes the limitations of traditional security auditing, which only focuses on a single access log. By mapping behavioral deviations to semantic graphs for risk causal propagation deduction, security analysis is upgraded from isolated request detection to networked risk transmission assessment, enabling the identification of hidden threats that use semantic associations for indirect attacks. Finally, through the mapping of dynamic risk scores to risk level threshold ranges, an automatic closed loop from risk quantification to control strategies is formed, significantly improving the system's security protection capabilities and response speed in complex environments. Attached Figure Description
[0018] Figure 1 This is a system module block diagram of the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] In one embodiment, an IoT data processing system supporting heterogeneous data fusion and AI services is disclosed, such as... Figure 1 As shown, the system includes: The data processing module is used to acquire multi-source heterogeneous sensing data, map the multi-source heterogeneous sensing data into a unified semantic graph, and perform data governance on the semantic graph to obtain reliable data.
[0021] The method for mapping multi-source heterogeneous sensing data into a unified semantic graph is as follows: perform edge-side semantic atomization extraction on the multi-source heterogeneous sensing data to construct a set of semantic atoms; based on a pre-built domain digital twin semantic template library, anchor the set of semantic atoms with each digital twin semantic template in the domain digital twin semantic template library to obtain an anchored set of semantic entities. Each semantic entity in the anchored semantic entity set is mapped to a unified semantic vector. The semantic similarity between the unified semantic vectors is calculated. Based on the semantic similarity, cross-source entity disambiguation and semantic association weaving are performed on the semantic entities from different data sources in the anchored semantic entity set to construct a preliminary semantic graph. Temporal association features are extracted based on the temporal attributes of each semantic entity in the preliminary semantic graph. State transition causal patterns are mined based on the temporal association features, temporal causal semantic edges are constructed, and prediction edges are generated based on the state transition causal patterns. Each semantic entity is mapped to a state node by combining its corresponding temporal attributes, and the initial semantic graph is expanded into a semantic graph containing state nodes, temporal causal semantic edges, and prediction edges. The method for obtaining reliable data is as follows: extract multi-dimensional semantic quality features from each state node, temporal causal semantic edge, and prediction edge in the semantic graph, and construct a multi-dimensional quality feature vector for each state node; The semantic graph and its multi-dimensional quality feature vectors are input into a pre-trained graph attention credibility propagation network for credibility propagation inference to obtain the initial credibility score of each state node; a causal constraint rule set is constructed to verify the causal compliance of the semantic graph to obtain the verification result of each state node; based on the initial credibility score and verification result of each state node, a data governance strategy is matched for each state node. After implementing the data governance strategy, the semantic graph is re-performed with credibility propagation reasoning and causal consistency verification, iterating until convergence. The data of each state node in the converged semantic graph is then used as reliable data.
[0022] Specifically, multi-source heterogeneous sensing data originates from various sensing devices deployed at the edge of the Internet of Things (IoT), including but not limited to sensor nodes, smart terminals, industrial PLCs, vehicle terminals, and video surveillance equipment. The data processing module collects the corresponding raw sensing data from these sensing devices via edge gateways or direct connections. Because these sensing data exhibit significant differences in data format, communication protocols, sampling frequency, and semantic expression, they possess multi-source heterogeneity, creating semantic barriers. Therefore, to eliminate these semantic barriers, semantic mapping is necessary, specifically including: The data processing module performs edge-side semantic atomization extraction on the acquired multi-source heterogeneous sensing data. At the edge computing node, it performs lightweight parsing on the multi-source heterogeneous sensing data to identify basic semantic units in the multi-source heterogeneous sensing data, such as entity names, attribute values, timestamps, spatial coordinates, and event types, and constructs a set of semantic atoms. A semantic atom is the smallest indivisible unit that constitutes a semantic entity. Each semantic atom carries the attribute features of the original sensing data. Attribute features include, but are not limited to, attribute identifiers (such as attribute names), attribute value types (such as numeric, character, enumeration, and time-series types), attribute value ranges (such as upper and lower bounds of values and enumeration value sets), spatiotemporal marker features (such as sampling timestamps and spatial coordinate precision), and data source markers (such as sensing device type identifiers and communication protocol identifiers).
[0023] The system pre-builds a domain-specific digital twin semantic template library. This library is constructed using existing knowledge engineering methods based on industry knowledge ontology, device semantic specifications, and expert experience within the IoT application domain; the specific construction process is not described in detail here. The domain-specific digital twin semantic template library stores predefined digital twin semantic templates for specific IoT application domains (such as intelligent manufacturing, smart transportation, and environmental monitoring). Each digital twin semantic template represents the semantic structure of a standard physical or logical entity within that domain, and each template has a pre-defined attribute feature framework. The data processing module anchors and matches the obtained set of semantic atoms with each digital twin semantic template in the domain-specific digital twin semantic template library to obtain an anchored set of semantic entities. The anchoring and matching process calculates the similarity between the attribute features of each semantic atom and the attribute feature framework of each digital twin semantic template. For example, the matching is performed by comparing the attribute features of each semantic atom with the template attribute words in the template attribute framework. Each segment is matched one by one, and the string similarity of attribute identifiers is calculated (e.g., edit distance or Jaccard similarity coefficient), the compatibility similarity of attribute value types (e.g., consistent types are recorded as 1, inconsistent types as 0), the overlap similarity of attribute value domains (e.g., the actual value of the semantic atom falls within the template value domain as 1, partial overlap is scored proportionally, no overlap is recorded as 0), and the data source similarity (consistent data sources are recorded as 1, inconsistent data sources as 0). The above similarities are weighted and summed according to preset weights (based on experience) to obtain the field-level similarity between the semantic atom and the corresponding template attribute field. Then, the field-level similarity between the semantic atom and all template attribute fields in the template is aggregated (e.g., the average value is taken) to obtain the overall similarity between the semantic atom and the corresponding digital twin semantic template. When the overall similarity exceeds the preset anchoring threshold (based on experience), it is determined that the semantic atom and the digital twin semantic template are successfully anchored and matched, and the mapping relationship between the semantic atom and the corresponding template attribute field is recorded. Traverse all semantic atoms in the semantic atom set and repeat the above anchoring matching process to obtain a mapping relationship set between each semantic atom and its corresponding digital twin semantic template and template attribute fields. Based on the mapping relationship set, each semantic atom that matches the same digital twin semantic template is classified and combined according to the template attribute framework, and the template attribute fields of the digital twin semantic template are filled to generate structured anchored semantic entities. Then, each anchored semantic entity is composed of an anchored semantic entity set. In this way, each semantic entity in the anchored semantic entity set is bound to the corresponding digital twin semantic template, thereby giving heterogeneous data a unified domain semantic skeleton.
[0024] The process maps each semantic entity in the anchored semantic entity set to a unified semantic vector. This mapping can be performed using a pre-trained semantic encoding model, which is an existing technology, such as pre-training based on a large-scale general or domain-specific text corpus through self-supervised masked language modeling or contrastive learning mechanisms (details omitted here). The semantic encoding model encodes the semantic entities into unified semantic vectors of fixed dimensions. Subsequently, the semantic similarity between these unified semantic vectors is calculated, using cosine similarity. Based on the calculated semantic similarity, cross-source entity disambiguation is performed on semantic entities from different data sources in the anchored semantic entity set: when the semantic similarity of two or more semantic entities from different data sources exceeds a preset disambiguation threshold (set empirically), they are determined to be the same data. Heterogeneous representations from different data sources are merged into a single semantic entity. Simultaneously, semantic relationships are established between semantic entities based on semantic similarity. For example, after disambiguation, the semantic similarity between any two semantic entities is calculated using their corresponding unified semantic vectors. When the semantic similarity exceeds a preset association threshold (based on experience), a semantic relationship is determined between the two semantic entities, and an association edge is constructed. Furthermore, association edges are constructed based on the type of attribute intersection between semantic entities (such as spatial proximity, device subordination, temporal co-occurrence, or business logic association), with semantic similarity used as the edge weight. Finally, semantic association weaving is performed, using each semantic entity as a node and the established association edges as edges to organize the associations and form a preliminary semantic graph.
[0025] Each semantic entity node in the preliminary semantic graph carries timestamp information of the perceived data, forming the temporal attributes of the semantic entity. Based on the preliminary semantic graph, the data processing module maps each semantic entity node to a state node by combining its temporal attributes. Mapping means that for the same semantic entity, it is divided into one or more time slice instances according to the timestamp information it carries. Each time slice instance, combined with the attribute value and running state at that moment, generates an independent state node. Thus, a semantic entity corresponds to one semantic entity node in the preliminary semantic graph, and can be expanded into one or more state nodes in the unified semantic graph. The data processing module extracts temporal association features based on the temporal attributes of each semantic entity in the preliminary semantic graph. The temporal association features include, but are not limited to, the change sequence of attribute values of semantic entity nodes within adjacent time windows (for example, if the temperature attribute value of a semantic entity node is 25.3℃ in the first time window and the temperature attribute value is 26.1℃ in the next adjacent time window, then the change sequence of the attribute value of the semantic entity node is [25.3, 26.1℃).1) State transition frequency (the ratio of the number of times a semantic entity node transitions from one state category to another within a preset statistical time window to the duration of that statistical time window; for example, if a device semantic entity transitions from running state to standby state 3 times and from standby state to running state 3 times within 1 hour, then the state transition frequency within that statistical window is 6 times / hour) and state dwell time (the length of time a semantic entity node remains in a specific state category; for example, if a device semantic entity enters the running state at 10:00:00 and remains in the running state until 10: If the system transitions to standby mode at 45:00, the state dwell time in the running state is 45 minutes. Within a preset statistical time window, the average dwell time can be further calculated as the state dwell time. Based on the extracted temporal correlation features, the data processing module mines state transition causal patterns. These patterns characterize the causal driving relationship of semantic entities transitioning from one state to another. The mining employs existing temporal causal inference methods, such as Granger causality tests and dynamic Bayesian networks, which will not be described in detail here. Based on the state transition causal patterns, a system is constructed... Temporal causal semantic edges connect state nodes with causal relationships. Based on the discovered state transition causal patterns, prediction edges are generated using existing temporal prediction models, such as Long Short-Term Memory networks or temporal graph neural networks. The future evolution trend of state nodes is inferred based on the state transition causal patterns, yielding inference results and corresponding inference confidence levels. The inference results include the inferred future state nodes. When the inference confidence level exceeds a preset threshold (pre-set based on experience), a prediction edge is established between the current state node and the inferred future state node. This is existing technology and will not be described in detail here. Predictive edges represent predictive associations from the current state node to the future state node. Thus, the initial semantic graph is expanded into a unified semantic graph containing state nodes, temporal causal semantic edges, and prediction edges. State nodes in the semantic graph are instantiations of semantic entities in specific time slices, while temporal causal semantic edges and prediction edges explicitly encode the temporal dynamics and causal regularities of the data, providing a structured causal knowledge foundation for subsequent storage scheduling, AI services, and security management. .
[0026] For each state node in the semantic graph, the data processing module extracts the ontology semantic quality features of each state node, as well as the edge attribute quality features of the temporal causal semantic edges and predicted edges associated with each state node. The ontology semantic quality features characterize the data quality of the state node itself, including semantic integrity features (such as attribute missing rate and field fill rate) and cross-source confidence features (such as the number of data sources fused to the state node and the consistency between data sources). The edge attribute quality features characterize the semantic relationship quality between the state node and its neighboring state nodes, including temporal consistency features (such as whether the logical order of the timestamps of the state nodes at both ends of the temporal causal semantic edge conforms to causal temporal constraints) and causal compliance features (such as whether the state transition conforms to the mined state transition causal pattern). Based on the ontology semantic quality features and the edge attribute quality features, a multi-dimensional quality feature vector corresponding to each state node is constructed by fusing them. The fusion method can be feature concatenation, so that each state node corresponds to a multi-dimensional quality feature vector, which comprehensively reflects the quality status of the corresponding state node itself and its associated relationships.
[0027] The data processing module inputs the semantic graph and the multi-dimensional quality feature vectors corresponding to each state node into a pre-trained graph attention credibility propagation network. The pre-trained graph attention credibility propagation network is an existing technology, and its pre-training process includes: acquiring historical IoT semantic graphs and credibility labels for each state node obtained through manual annotation or rule verification as training samples; using multi-dimensional quality feature vectors as initial node features; learning the attention weights for credibility propagation between nodes using a graph attention mechanism; optimizing to minimize the credibility score output by the network; and obtaining network parameters through end-to-end supervised learning. Specific training algorithms and hyperparameter tuning are detailed below. The process is not part of the improvement of this invention and will not be described in detail here. The graph attention credibility propagation network is based on the graph attention mechanism and performs multi-hop credibility propagation reasoning on the semantic associations, temporal causal semantic edges and prediction edges between each state node. For example, the network uses multi-dimensional quality feature vectors as the initial features of the nodes, aggregates the credibility information of neighboring nodes by weighting through the built-in attention coefficients, and performs credibility propagation and correction in the causal direction along the temporal causal semantic edges and prediction edges, and finally obtains the initial credibility score of each state node. The initial credibility score is a numerical quality confidence index. The higher the score, the more reliable the data quality of the state node.
[0028] Simultaneously, the data processing module constructs a causal constraint rule set based on the temporal causal semantic edges and prediction edges in the semantic graph. This causal constraint rule set can be pre-constructed using domain expert knowledge combined with historical state transition data mining to identify frequent causal patterns. The data processing module then uses this causal constraint rule set to perform causal analysis on each state transition path in the semantic graph (a state transition path is a continuous sequence of state transitions that starts from a state node, follows a directed connection formed by temporal causal semantic edges and prediction edges, passes through at least one intermediate state node, or directly reaches another state node; for example, if state node A points to state node B through a temporal causal semantic edge, and state node B then points to state node C through a prediction edge, then A→B→C constitutes a state transition path). The compliance check identifies anomalous edges and nodes that violate the causal constraint rules within the causal constraint rule set. Specifically, each hop in each state transition path (i.e., each temporal causal semantic edge or prediction edge) is compared item by item with the corresponding preset rules in the causal constraint rule set. For example, it checks whether the transition direction belongs to the set of state transitions allowed by the rules, whether the transition delay is within the reasonable range specified by the rules, and whether the confidence of each edge meets the threshold requirements set by the rules. If any of these are violated, the edge is marked as an anomalous edge, and the state node associated with the edge is marked as an anomalous node. The verification result is output in the form of the causal compliance verification result of each state node, for example, using a binary label (normal / abnormal) to represent the compliance of each state node at the causal logic level.
[0029] The data processing module adaptively matches data governance strategies for each state node based on its initial credibility score and causal compliance verification results. These strategies include semantic completion (completing missing attributes for state nodes with insufficient semantic integrity based on domain knowledge or neighbor node information), temporal interpolation (performing time-series interpolation based on temporal causal semantic edges for state nodes with missing temporal attributes), and causal correction (correcting state values or relationships for causal anomaly nodes based on a set of causal constraint rules). Each state node matches and executes the optimal data governance strategy based on the combined characteristics of its initial credibility score and verification results. After executing the data governance strategy, the data processing module re-performs credibility propagation inference and causal compliance verification of the semantic graph. Since data governance operations change the attribute values and relationships of state nodes, leading to changes in credibility scores and causal verification results, iterative execution is required until the change in credibility scores and causal verification results between adjacent iterations is less than a preset convergence threshold (set empirically), or the maximum number of iterations is reached, at which point convergence is considered achieved. The data of each state node in the converged semantic graph is then output as reliable data for subsequent modules.
[0030] By using edge-side semantic atomization extraction and anchoring matching with a domain-specific digital twin semantic template library, unified mapping and precise fusion of multi-source heterogeneous perceptual data at the semantic level are achieved, effectively eliminating repetitive heterogeneous expressions of the same real-world object from different data sources. Simultaneously, by constructing temporal causal semantic edges and prediction edges, the temporal dynamics and causal regularities of the data are explicitly encoded into the graph structure, providing a structured causal knowledge foundation for subsequent storage scheduling, AI services, and security management. Furthermore, driven governance based on a graph attention credibility propagation network and a causal constraint rule set enables deep data quality governance oriented towards semantic association and causal logic, ensuring that the output reliable data simultaneously meets credibility and causal compliance requirements, thus providing high-quality data support for subsequent AI services.
[0031] The data storage module includes multiple scalable storage bins for acquiring reliable data sources and storing the reliable data in the corresponding storage bins in the cloud according to the data sources.
[0032] The method for storing reliable data in the corresponding storage warehouses in the cloud includes: extracting multiple data features from the data source corresponding to the reliable data, matching each data feature based on the storage warehouse matching rule table pre-configured for each storage warehouse, obtaining the number of data feature matches for each storage warehouse, and selecting the storage warehouse with the largest number of matches as the target storage warehouse for storing the corresponding reliable data.
[0033] Specifically, after receiving reliable data output from the data processing module, the data storage module first obtains the reliable data and its corresponding data source. The data source is metadata attached by the edge gateway or sensing device during the data access phase, used to identify the source attributes of the data. The data storage module parses the data source, extracting data features such as data source type (e.g., temperature and humidity sensor, industrial camera, vehicle GPS, etc.), device identifier (e.g., device unique code MAC address or device serial number), and data protocol attributes (e.g., communication protocol types such as MQTT, CoAP, HTTP, or Modbus). The data storage module has a pre-configured storage warehouse mapping rule table, which defines the matching rules for the data source types, device identifier ranges, and data protocol attributes supported by each storage warehouse. The data storage module matches the parsed data features with each storage warehouse in the storage warehouse mapping rule table, counts the number of data features matched by each storage warehouse, and obtains the number of data feature matches for each storage warehouse. The storage warehouse with the largest number of data feature matches is selected as the target storage warehouse for the reliable data. After determining the target storage warehouse, the reliable data is formatted and stored according to the preset data format requirements of the target storage warehouse.
[0034] Through the above technical solution, by resolving data source identifiers and matching multi-dimensional data features, the automated and accurate classification and storage of heterogeneous reliable data in the cloud storage warehouse is achieved, avoiding retrieval difficulties caused by mixed data storage.
[0035] The storage management module is used to obtain the reliable data storage volume of each storage warehouse, calculate the resource stress of each storage warehouse based on the reliable data storage volume, and dynamically adjust the storage space of each storage warehouse based on the resource stress.
[0036] The method for calculating the resource stress of each storage bin includes: based on the state nodes corresponding to each reliable data in the storage bin, counting the number of temporal causal semantic edges associated with each state node to obtain the temporal causal semantic edge density, and counting the number of prediction edges pointing to each state node to obtain the prediction edge pointing frequency. Based on the temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score of each state node in the storage bin, a causal profile of the data value of each storage bin is constructed. Based on the predicted edges in the semantic graph, causal inference is performed on each storage bin to obtain the causal derivative storage demand of each storage bin in a preset future time period. The AI service module obtains the historical data extraction frequency of each storage bin, the training load index and inference load of the current service agent, calculates the data dependency based on the historical data extraction frequency, calculates the computing power pressure based on the training load index and inference load, and weights and fuses the data dependency and computing power pressure to obtain the coupling demand factor of each storage bin. The resource stress of each storage warehouse is obtained by weighting and summing the reliable data storage volume, the causal profile of data value, the storage demand derived from the causal relationship, and the coupling demand factors.
[0037] The method for dynamically adjusting the storage space of each storage bin includes: setting corresponding expansion thresholds and recovery thresholds for each storage bin; when the resource stress is greater than the expansion threshold, the storage bin is expanded and the corresponding storage bin is recorded as the expansion bin; when the resource stress is less than the recovery threshold, the storage bin is recovered and the corresponding storage bin is recorded as the recovery bin. Calculate the expansion capacity of each expansion bin, sort them in descending order according to the expansion capacity to obtain an expansion sorting table; calculate the recovery amount of each recovery bin, sort them in descending order according to the recovery amount to obtain a recovery sorting table. Based on the priority order determined by the expansion sorting table and the recycling sorting table, a spatial resource matching relationship is established between the recycling bins and the expansion bins. The amount of recycling to be recycled in the recycling bins is then redistributed to the corresponding expansion bins until the expansion capacity requirements of each expansion bin are met or the recycling amount of each recycling bin is exhausted.
[0038] Specifically, the reliable data storage volume of each storage bin is obtained, and for the state nodes corresponding to the reliable data in each storage bin, the number of temporal causal semantic edges associated with each state node is counted to obtain the temporal causal semantic edge density of that state node. The temporal causal semantic edge density reflects the association complexity and causal activity of that state node in the causal network. At the same time, the number of prediction edges pointing to each state node is counted to obtain the prediction edge pointing frequency. The prediction edge pointing frequency reflects the degree to which the prediction model focuses on that state node as a potential future state, and the higher the frequency, the greater the importance of that state node in future causal evolution.
[0039] Based on the temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score of each state node in each storage bin, a data value causal profile is constructed for each storage bin. The data value causal profile is a quantitative representation of the comprehensive value of the data in the storage bin. It can be constructed using a weighted fusion method, such as normalizing the temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score to fall within the range of 0-1, and then setting their respective weights according to the experience of those in the field, and performing weighted accumulation to obtain the data value causal profile of each storage bin. The data value causal profile comprehensively reflects the causal complexity, future predictive value, and current quality credibility of the data in the storage bin.
[0040] Based on the predicted edges in the semantic graph, causal inference is performed on each storage bin. Specifically, the causal derivative states (i.e., potential state nodes that the current state node may transition to within the future preset time period) that may be triggered by the state nodes in each storage bin are tracked along the predicted edges. The data capacity occupied at the storage level of the newly generated state node and its associated temporal causal semantic edges, attribute values, timestamps and other structured data after the causal derivative state is triggered and actually generated is estimated. All the occupied data capacity is summarized to obtain the causal derivative storage demand of each storage bin in the preset future time period. This demand not only considers the current storage capacity, but also proactively incorporates the storage growth pressure brought about by causal evolution.
[0041] The system obtains the historical data extraction frequency of the AI service module for each storage warehouse, as well as the current training load metrics (such as GPU utilization and training batch processing time) and inference load (such as inference request QPS and average inference latency) for each service agent. Based on this data, the storage management module constructs the coupling demand factors for each storage warehouse. Specifically, it counts the number of data extraction requests made by the AI service module to each storage warehouse within a preset historical time window (such as the last 24 hours or 7 days) to obtain the historical data extraction frequency. The ratio of the historical data extraction frequency to the maximum reference frequency set by the system is normalized to obtain the data dependency. The closer the data dependency is to 1, the stronger the AI service's data dependency on that storage warehouse. The system obtains the training load metrics and inference load metrics of the service agent corresponding to the currently executing data extraction request. The training load metrics include GPU utilization and training batch processing time, while the inference load includes inference request QPS and average inference latency. Each metric is normalized: GPU utilization is directly expressed as a percentage, training... Batch processing time is inversely normalized to the preset maximum allowable time (shorter time means less pressure), inference request QPS is directly normalized to the system's maximum capacity QPS, and average inference latency is directly normalized to the preset service level target. The normalized GPU utilization, normalized training batch processing time, normalized inference request QPS, and normalized average inference latency are weighted and summed to obtain the computing power pressure. Finally, the weights of preset data dependency and computing power pressure are weighted and accumulated to obtain the coupling demand factor for each storage warehouse. The specific values of the corresponding weights can be dynamically adjusted according to the system operation strategy. For example, the weight of computing power pressure is increased in high-load AI service scenarios, and the weight of data dependency is increased in data explosion scenarios. The coupling demand factor characterizes the AI service's dependence on storage warehouse data and its real-time requirements. The higher the historical extraction frequency and the greater the current training / inference load, the more critical the data in that storage warehouse is to support the AI service, and its storage resources should be prioritized.
[0042] Finally, the reliable data storage volume, data value causal profile, causal derivative storage demand, and coupling demand factors are weighted and accumulated to calculate the comprehensive resource stress of each storage warehouse. During the weighting and accumulation process, the respective weight coefficients can be dynamically adjusted according to the system operation strategy. For example, the weight of coupling demand factors is increased in high AI service load scenarios, and the weight of causal derivative storage demand is increased in data explosion growth scenarios.
[0043] After obtaining the overall resource stress of each storage warehouse, corresponding expansion and recovery thresholds are preset for each warehouse. These thresholds are comprehensively set based on the physical capacity limit of each warehouse, historical resource stress statistics, and preset service level agreement requirements. When the overall resource stress of a storage warehouse exceeds the expansion threshold, it is determined that the warehouse needs expansion, is designated as an expansion warehouse, and its expansion capacity is calculated. The expansion capacity is obtained by subtracting the expansion threshold from the overall resource stress and multiplying by the expansion coefficient. When the overall resource stress of a storage warehouse is less than the recovery threshold, it is determined that the warehouse has redundant space and is designated as a recovery warehouse. The system calculates the recovery volume of each storage bin. The recovery volume is calculated based on the difference between the recovery threshold and the overall resource stress level, as well as a preset recovery coefficient. The recovery volume is obtained by subtracting the overall resource stress level from the recovery threshold and then multiplying it by the recovery coefficient. The preset expansion coefficient and recovery coefficient are set by the management personnel based on the total physical capacity of each storage bin, the fluctuation range of resource stress level in historical periods, and the preset business tolerance latency requirements. These parameters will not be described in detail here. The storage management module sorts each expansion bin in descending order by expansion capacity to obtain an expansion sorting table. The larger the expansion capacity, the higher the ranking, indicating a more urgent expansion need. At the same time, the system sorts each recovery bin in descending order by recovery volume to obtain a recovery sorting table. The larger the recovery volume, the higher the ranking, indicating more recoverable space.
[0044] Based on the priority order represented by the expansion and recycling priority tables, the total amount of recyclable space in the recycling warehouse and the total expansion demand in the expansion warehouse are calculated in a coordinated manner to establish a space resource matching relationship between the recycling and expansion warehouses. The storage management module allocates the storage space released from the recycling warehouse to the expansion warehouse across warehouses. If the total amount of recyclable space is greater than or equal to the total expansion demand, it indicates that the recycling space is sufficient to meet the expansion demand, and the storage management module includes the remaining recycling space in the redundant resource pool for subsequent dynamic allocation. If the total amount of recyclable space is less than the total expansion demand, it indicates that the recycling space is insufficient to meet all expansion demands, and the storage management module marks the unmet expansion capacity as a queue to be expanded and triggers external storage resource applications (such as cloud storage elastic expansion interfaces) to make up the difference.
[0045] By introducing temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score to construct a causal profile of data value, the above-mentioned solution enables storage resource assessment to go beyond the physical capacity dimension and deeply integrate the semantic value and causal importance of data, thus achieving data value-driven storage management. Through causal-derived storage demand prediction based on predicted edges, storage resource scheduling gains causal inference capabilities, allowing for pre-allocation of resources for future data growth and avoiding the lag of traditional threshold-triggered expansion. Simultaneously, by coupling demand factors to incorporate the real-time load of AI services into storage scheduling decisions, the solution achieves coordinated optimization of storage and computing resources, preventing a disconnect between storage expansion and AI computing power demands. Finally, through a collaborative mechanism of cross-warehouse resource allocation and external resource requests, the system achieves internal storage space rebalancing and external elastic expansion, balancing resource utilization and system scalability.
[0046] The AI service module is used to extract reliable data from each storage bin and send the extracted reliable data to each preset service intelligence body. The service intelligence body then performs AI model training and inference calculations based on the reliable data to generate corresponding AI service outputs.
[0047] The methods for generating corresponding AI service outputs include: acquiring reliable data from each storage warehouse, performing service preprocessing on the reliable data, and obtaining training sample data and inference input data; The training sample data is sent to the corresponding service agent, which then performs AI model training based on the training sample data to obtain the trained target model. The trained target model is encapsulated into a standardized service interface, and the corresponding calling parameters are configured to generate an AI service instance. The inference input data is then input into the AI service instance to perform inference calculations and obtain the corresponding AI service output.
[0048] Specifically, the AI service module initiates a data extraction request to the data storage module to obtain reliable data from each storage bin. The obtained reliable data first undergoes service preprocessing, which includes data cleaning (removing residual noise after processing), data augmentation (expanding training samples), and format standardization (converting to model input tensor format). After service preprocessing, the reliable data is divided into training sample data and inference input data. The training sample data is used in the model training stage, and the inference input data is used in the model inference stage.
[0049] The AI service module delivers training sample data to the corresponding service agents. Each service agent corresponds to a specific AI service task (such as equipment failure prediction, environmental anomaly detection, traffic flow prediction, etc.) and has a pre-set corresponding model architecture (such as temporal neural network, graph neural network, Transformer architecture, etc.). The service agent performs AI model training based on the received training sample data to obtain the trained AI model. During the training process, the service agent can use temporal causal semantic edges and prediction edges in the semantic graph as auxiliary supervision signals to enhance the model's ability to capture causal relationships.
[0050] After obtaining the trained AI model, the AI service module encapsulates it into a standardized service interface. This standardized service interface can take the form of a RESTful API, gRPC, or message queue, defining a unified input / output format, authentication method, and rate limiting strategy. Simultaneously, the AI service module configures corresponding call parameters for the standardized service interface, including model version identifier, inference batch size, timeout threshold, and resource quota. After encapsulation and configuration, a corresponding AI service instance is generated. This AI service instance is an independently deployable and horizontally scalable service entity that provides continuous AI inference capabilities. Finally, the inference input data is input into the generated AI service instance, which calls the internally loaded trained AI model to perform forward computation on the inference input data, obtaining the inference result data. This inference result data is the AI service output, which the AI service module returns to the requesting party (such as upper-layer business applications or other system modules).
[0051] The security management module acquires data flow records of reliable data from each storage warehouse and constructs data behavior profiles based on these records. It then calculates the risk score for each storage warehouse based on the data behavior profiles, receives current requests, and classifies the risk level of each request based on the risk score.
[0052] The method for constructing data behavior profiles is as follows: extract multi-dimensional behavior feature parameters from data flow records; based on the multi-dimensional behavior feature parameters, construct time-series behavior feature vectors for each storage warehouse according to a preset time window; based on the time-series behavior feature vectors of each storage warehouse, generate corresponding data behavior profiles through preset profile construction rules. The data behavior profiles include access behavior profiles, flow behavior profiles, and storage operation behavior profiles. The method for classifying the risk level of the current request is as follows: extract the historical behavior baseline of the corresponding storage warehouse based on the data behavior profile, extract the request feature information from the current request, and calculate the behavior deviation based on the request feature information and the historical behavior baseline; Based on the behavioral deviation, risk anchoring state nodes are determined in the semantic graph. Based on the temporal causal semantic edges and prediction edges associated with the risk anchoring state nodes, risk causal propagation is deduced to obtain the cumulative causal risk value. Identify the request type of the current request, and match the corresponding risk coupling factor based on the request type; The risk score for the current request is obtained by weighting and fusing behavioral deviation, cumulative causal risk value, and risk coupling factors. Multiple risk level threshold ranges are preset, and the corresponding risk level is determined based on the risk level threshold range in which the risk score falls.
[0053] Specifically, the security management module acquires data flow records from each storage warehouse. These records are collections of operation logs automatically collected by the system during operation, including: access request records for reliable data (recording external entities' read and query operations on reliable data within the storage warehouse), data extraction request records from the AI service module (recording operations by which the AI service module extracts data from the storage warehouse for training or inference), and storage space adjustment operation records performed by the storage management module on the storage warehouse (recording storage management operations such as expansion and reclamation). Furthermore, it extracts multi-dimensional behavioral feature parameters from these records. These parameters include: access subject identifiers (user ID, service agent ID, or device identifier, etc.), access timing features (timestamp of the request, time interval distribution, periodic patterns, etc.), data flow characteristics (data source storage warehouse, target module, transmission path, etc.), and operation type characteristics (read, write, extract, expansion, reclamation, etc.). Based on these extracted multi-dimensional behavioral feature parameters, the module aggregates behavioral features for each storage warehouse according to preset time windows (such as 5-minute windows, 1-hour windows, or daily windows) to construct a temporal behavioral feature vector for each storage warehouse within that time window.
[0054] Based on the time-series behavioral feature vectors of each storage warehouse, corresponding data behavior profiles are generated using preset profile construction rules. These preset profile construction rules are obtained using existing clustering analysis methods, such as calculating feature similarity and dividing behavioral pattern clusters from historical time-series behavioral feature vectors, and using the centroid feature vectors of each cluster as the representation rules for the corresponding data behavior profiles. The specific process is not described in detail here. The data behavior profiles include access behavior profiles, flow behavior profiles, and storage operation behavior profiles. For example, in generating access behavior profiles, the access subject identifier and access time-series features are selected as effective feature subsets from the time-series behavioral feature vectors. The access frequency distribution, access time concentration, and access permissions of each access subject within a preset statistical period (based on empirical presets) are statistically analyzed. The scope is defined, and the statistical results are encoded into access behavior profile vectors. For the generation of flow behavior profiles, data flow direction features are selected as an effective feature subset from the time-series behavior feature vectors. The frequency of data flow transmission paths between modules and the proportion of cross-warehouse flow during flow are statistically analyzed, and the statistical results are encoded into flow behavior profile vectors. For the generation of storage operation behavior profiles, operation type features are selected as an effective feature subset from the time-series behavior feature vectors. The expansion frequency, recycling frequency, and cross-warehouse migration frequency of each storage warehouse within a preset statistical period are statistically analyzed, and the statistical results are encoded into storage operation behavior profile vectors. Thus, access behavior profiles represent the access of external entities to storage warehouses, flow behavior profiles represent the flow of data between modules, and storage operation behavior profiles represent the operation of storage warehouse space adjustment.
[0055] The security management module acquires currently received requests and identifies their request types. Request types include reliable data access requests (data access initiated by external entities) and data extraction requests (data extraction initiated by the AI service module). Simultaneously, the security management module extracts the historical behavior baseline of the corresponding storage warehouse based on data behavior profiles. This historical behavior baseline is a representation of normal behavior patterns obtained through statistical analysis of historical time-series behavior feature vectors. The security management module extracts request feature information (such as request subject, request time, target data range, and operation type) from the currently received requests, compares this information with the historical behavior baseline, and calculates the behavior deviation. The behavior deviation can be calculated using Mahalanobis distance; a larger deviation indicates a greater difference between the current request and the normal behavior pattern. Based on the calculated behavior deviation, the security management module determines risk anchoring state nodes in the semantic graph. Specifically, it then... To map deviation to the set of state nodes affected by the current request in the semantic graph, the state node with the largest deviation is selected as the risk anchor state node. Based on the temporal causal semantic edges and prediction edges associated with the risk anchor state node, the security management module performs risk causal propagation deduction: tracing the risk transmission path along the temporal causal semantic edges and prediction edges to determine the risk transmission impact range (i.e., starting from the risk anchor state node, traversing all connected state nodes sequentially through the temporal causal semantic edges and prediction edges to obtain the risk transmission impact range). Within the determined risk transmission impact range, the security management module calculates the risk contribution of each state node within the risk transmission impact range, and then weights and accumulates the risk contributions to obtain the cumulative causal risk value. The weight of each state node's risk contribution is set autonomously based on experience; the closer the node is to the risk anchor state node, the greater its weight. The calculation method for the risk contribution of each state node is as follows: For each state node within the risk transmission impact range, the corresponding node importance (a normalized representation of the sum of the number of temporal causal semantic edges and prediction edges associated with the state node; the larger the value, the more important the node), data sensitivity (a normalized representation of the security level of the reliable data corresponding to the state node; the larger the value, the higher the data sensitivity; the security level of each reliable data is preset by the system according to the data source, sensitive attribute category, etc.), and correlation tightness (the inverse normalized representation of the Euclidean distance between the state node and the risk anchoring state node; the larger the value, the higher the correlation tightness). The node importance, data sensitivity, and correlation tightness are weighted and summed to obtain the risk contribution of each state node. Here, the respective weight coefficients are dynamically set according to the actual situation. For example, when focusing on data sensitivity, the weight ratio of data sensitivity can be increased to ensure that the sum of all weights is 1.
[0056] The security management module matches the corresponding risk coupling factor based on the request type of the current request. When the request type is a data extraction request, it obtains the current load indicators (such as CPU / GPU utilization and memory usage) and data extraction mode (such as data extraction volume, extraction frequency, and extraction time distribution) of the service agent executing the current request. It then compares the data extraction volume, extraction frequency, and extraction time distribution in the data extraction mode with the corresponding preset statistical benchmark values in the historical behavior baseline, calculates the mode deviation rate for each dimension, and weights and fuses them to obtain the data extraction deviation degree. Simultaneously, it normalizes and weights and fuses the CPU / GPU utilization and memory usage in the current load indicators to obtain the load stress degree. Finally, it fuses the data extraction deviation degree and the load stress degree to obtain the coupling risk factor. The coupling risk factor reflects the AI service's performance in... Risks associated with abnormal data extraction under abnormal loads; When the request type is a reliable data access request, the security management module obtains the access subject identifier and the security level of the reliable data (e.g., public level 1, internal level 2, confidential level 3, top secret level 4). It calculates a coupling risk factor based on the access subject identifier's permission level and the reliable data's security level. For example, it maps the permission level and security level numerically according to a preset level mapping table, obtaining permission level values and security level values respectively, and calculates the level difference between the security level value and the permission level value. When the level difference is greater than zero, the coupling risk factor equals the product of the level difference and a preset risk coefficient; when the level difference is less than or equal to zero, the coupling risk factor equals a preset baseline risk value. The preset risk coefficient and preset baseline risk value are pre-set according to the system security policy.
[0057] Finally, the security management module performs a weighted fusion of behavioral deviation, causal risk accumulation value, and matched risk coupling factors to generate a risk score for the currently received request. The corresponding weight coefficients are set based on the system's preset security protection level strategy and the backtracking analysis results of historical risk events. In highly sensitive data scenarios, the weight coefficient of the causal risk accumulation value is increased; in high service load scenarios, the weight coefficient of the risk coupling factor is increased. The sum of all weight coefficients is 1 and can be dynamically adjusted by the system administrator according to actual business needs. After obtaining the risk score, the security management module presets multiple risk level threshold ranges (e.g., low risk range [0, 0.3], medium risk range [0.3, 0.7], and high risk range [0.7, 1.0]). Based on the risk level threshold range in which the dynamic risk score is located, the corresponding risk level (e.g., low risk, medium risk, high risk) is determined. The security management module executes differentiated control strategies on the current request based on the determined risk level. The higher the risk level, the stricter the control. Control strategies include access permission, traffic restriction, request blocking, and service degradation.
[0058] The above scheme constructs a data behavior profile by integrating multi-dimensional data flow records such as access requests, data extraction, and storage adjustments, achieving a three-dimensional depiction of the entire lifecycle flow behavior of IoT data. Simultaneously, risk causal propagation is deduced based on semantic graphs, upgrading security analysis from isolated request detection to networked risk transmission assessment, enabling the identification of hidden threats that exploit semantic associations for indirect attacks. By matching differentiated risk coupling factors to data extraction requests and reliable data access requests, accurate risk assessment adaptively based on request types is achieved. Finally, through the mapping of dynamic risk scores to risk level threshold ranges, an automatic closed loop from risk quantification to control strategies is formed, significantly improving the system's security protection capabilities and response speed in complex environments.
[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
[0060] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.
[0061] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention.
Claims
1. An IoT data processing system supporting heterogeneous data fusion and AI services, characterized in that: The system includes: The data processing module is used to acquire multi-source heterogeneous sensing data, map the multi-source heterogeneous sensing data into a unified semantic graph, and perform data governance on the semantic graph to obtain reliable data. The data storage module includes multiple expandable storage bins for acquiring reliable data sources and storing the reliable data in the corresponding storage bins in the cloud according to the data sources. The storage management module is used to obtain the reliable data storage volume of each storage warehouse, calculate the resource stress of each storage warehouse based on the reliable data storage volume, and dynamically adjust the storage space of each storage warehouse based on the resource stress. The AI service module is used to extract reliable data from each storage bin and send the extracted reliable data to each preset service intelligence body. The service intelligence body then performs AI model training and inference calculations based on the reliable data to generate corresponding AI service outputs. The security management module obtains the data flow records of reliable data corresponding to each storage warehouse, and builds a data behavior profile based on the data flow records; Based on the data behavior profile, calculate the risk score for each storage warehouse; Receive the current request and classify its risk level based on the risk score.
2. The system according to claim 1, characterized in that, The method for mapping multi-source heterogeneous sensing data into a unified semantic graph is as follows: Perform edge-side semantic atomization extraction on multi-source heterogeneous sensing data to construct a semantic atom set; based on the pre-built domain digital twin semantic template library, anchor the semantic atom set with each digital twin semantic template in the domain digital twin semantic template library to obtain an anchored semantic entity set; Each semantic entity in the anchored semantic entity set is mapped to a unified semantic vector. The semantic similarity between the unified semantic vectors is calculated. Based on the semantic similarity, cross-source entity disambiguation and semantic association weaving are performed on the semantic entities from different data sources in the anchored semantic entity set to construct a preliminary semantic graph. Temporal association features are extracted based on the temporal attributes of each semantic entity in the preliminary semantic graph. State transition causal patterns are mined based on the temporal association features, temporal causal semantic edges are constructed, and prediction edges are generated based on the state transition causal patterns. Each semantic entity is mapped to a state node by combining its corresponding temporal attributes, and the initial semantic graph is expanded into a semantic graph containing state nodes, temporal causal semantic edges, and prediction edges.
3. The system according to claim 2, characterized in that, The method for obtaining reliable data is as follows: Multidimensional semantic quality features are extracted from each state node, temporal causal semantic edge, and prediction edge in the semantic graph, and multidimensional quality feature vectors are constructed for each state node. The semantic graph and its multi-dimensional quality feature vectors are input into a pre-trained graph attention credibility propagation network for credibility propagation inference to obtain the initial credibility score of each state node; a causal constraint rule set is constructed to verify the causal compliance of the semantic graph to obtain the verification result of each state node; based on the initial credibility score and verification result of each state node, a data governance strategy is matched for each state node. After implementing the data governance strategy, the semantic graph is re-performed with credibility propagation reasoning and causal consistency verification, iterating until convergence. The data of each state node in the converged semantic graph is then used as reliable data.
4. The system according to claim 3, characterized in that, Methods for storing reliable data in corresponding storage repositories in the cloud include: Multiple data features are extracted from the data source corresponding to the reliable data. Based on the pre-configured storage warehouse matching rule table for each storage warehouse, each data feature is matched to obtain the number of data feature matches for each storage warehouse. The storage warehouse with the largest number of data feature matches is selected as the target storage warehouse for storing the corresponding reliable data.
5. The system according to claim 4, characterized in that, Methods for calculating the resource stress of each storage facility include: Based on the state nodes corresponding to each reliable data in the storage bin, the number of temporal causal semantic edges associated with each state node is counted to obtain the temporal causal semantic edge density, and the number of predicted edges pointing to each state node is counted to obtain the predicted edge pointing frequency. Based on the temporal causal semantic edge density, predicted edge pointing frequency, and initial credibility score of each state node in the storage bin, a causal profile of the data value of each storage bin is constructed. Based on the predicted edges in the semantic graph, causal inference is performed on each storage bin to obtain the causal derivative storage demand of each storage bin in a preset future time period. The AI service module obtains the historical data extraction frequency of each storage bin, the training load index and inference load of the current service agent, calculates the data dependency based on the historical data extraction frequency, calculates the computing power pressure based on the training load index and inference load, and weights and fuses the data dependency and computing power pressure to obtain the coupling demand factor of each storage bin. The resource stress of each storage warehouse is obtained by weighting and summing the reliable data storage volume, the causal profile of data value, the storage demand derived from the causal relationship, and the coupling demand factors.
6. The system according to claim 5, characterized in that, Methods for dynamically adjusting the storage space of each storage compartment include: Set corresponding expansion thresholds and recycling thresholds for each storage warehouse. When the resource stress is greater than the expansion threshold, the storage warehouse is expanded and the corresponding storage warehouse is recorded as the expansion warehouse. When the resource stress is less than the recycling threshold, the storage warehouse is recycled and the corresponding storage warehouse is recorded as the recycling warehouse.
7. The system according to claim 6, characterized in that, Methods for dynamically adjusting the storage space of each storage compartment also include: Calculate the expansion capacity of each expansion compartment, sort them in descending order of expansion capacity, and obtain the expansion sorting table; Calculate the recycling volume of each recycling bin, sort them in descending order of recycling volume, and obtain a recycling sorting table; Based on the priority order determined by the expansion sorting table and the recycling sorting table, a spatial resource matching relationship is established between the recycling bins and the expansion bins. The amount of recycling to be recycled in the recycling bins is then redistributed to the corresponding expansion bins until the expansion capacity requirements of each expansion bin are met or the recycling amount of each recycling bin is exhausted.
8. The system according to claim 7, characterized in that, The methods for generating corresponding AI service outputs include: Acquire reliable data from each storage bin, perform service preprocessing on the reliable data, and obtain training sample data and inference input data; The training sample data is sent to the corresponding service agent, which then performs AI model training based on the training sample data to obtain the trained target model. The trained target model is encapsulated into a standardized service interface, and the corresponding calling parameters are configured to generate an AI service instance. The inference input data is then input into the AI service instance to perform inference calculations and obtain the corresponding AI service output.
9. The system according to claim 8, characterized in that, The method for constructing data behavior profiles is as follows: Multidimensional behavioral feature parameters are extracted from data flow records. Based on these parameters, time-series behavioral feature vectors are constructed for each storage warehouse according to a preset time window. Based on the time-series behavioral feature vectors of each storage warehouse, corresponding data behavior profiles are generated through preset profile construction rules. These profiles include access behavior profiles, flow behavior profiles, and storage operation behavior profiles.
10. The system according to claim 9, characterized in that, The method for classifying the risk level of the current request is as follows: Based on the data behavior profile, the historical behavior baseline of the corresponding storage warehouse is extracted, and the request feature information is extracted from the current request. Based on the request feature information and the historical behavior baseline, the behavior deviation is calculated. Based on the behavioral deviation, risk anchoring state nodes are determined in the semantic graph. Based on the temporal causal semantic edges and prediction edges associated with the risk anchoring state nodes, risk causal propagation is deduced to obtain the cumulative causal risk value. Identify the request type of the current request, and match the corresponding risk coupling factor based on the request type; The risk score for the current request is obtained by weighting and fusing behavioral deviation, cumulative causal risk value, and risk coupling factors. Multiple risk level threshold ranges are preset, and the corresponding risk level is determined based on the risk level threshold range in which the risk score falls.
