Smart industry park equipment big data storage method and system based on Internet of Things

Abstract

The invention relates to the technical field of smart industry park data management, and discloses a smart industry park equipment big data storage method and system based on the Internet of Things, and the method comprises the steps: collecting park equipment original data flow through an Internet of Things interface; constructing a multi-level topological structure by adopting a graph database; extracting a real-time state sub-graph to determine a high-frequency access path; fusing service task data to dynamically adjust the storage level; executing concurrent query simulation on the layered task integration data, screening hotspot data blocks and activating a cache mechanism; a scheduling response improvement index is obtained through difference analysis, and data management configuration is optimized. According to the method, rapid access and low-delay response of the equipment data can be realized, and the efficiency and reliability of big data storage and management of the intelligent industrial park equipment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart industrial park data management, and particularly relates to a smart industrial park equipment big data storage method and system based on Internet of Things. BACKGROUND

[0002] In the prior art, the data management of smart industrial park equipment mostly adopts a decentralized or single storage mode, and the device basic configuration, real-time state and business scheduling data are independently processed, and the heterogeneous data is parsed through simple hierarchical division or hard coding. However, in actual application, the types of park equipment are various, and the data formats are quite different. The single storage mode cannot adapt to the demand of multi-source heterogeneous data, resulting in scattered data and inconsistent semantics, which not only increases the complexity of system maintenance, but also affects the timeliness and accuracy of information acquisition. At the same time, the existing scheme does not optimize the device node association relationship, and it is difficult to quickly respond to real-time query or scheduling demand in a high-concurrency scenario, such as delay of park vehicle scheduling instruction, which limits the dynamic response capability of the park.

[0003] In summary, the prior art cannot efficiently integrate multi-source heterogeneous data and optimize device node association. In the actual application scenario of gradually popularizing big data storage, due to low data access efficiency and high task response delay, it is difficult to meet the efficient data processing demand of intelligent management of smart industrial park. SUMMARY

[0004] The present application provides a smart industrial park equipment big data storage and system based on Internet of Things to solve the problem of low data access efficiency and high task response delay in the prior art, which makes it difficult to meet the efficient data processing demand of intelligent management of smart industrial park.

[0005] In a first aspect, to solve the above technical problems, the present application provides a smart industrial park equipment big data storage method based on Internet of Things, comprising: Collecting the original data stream of park equipment through an Internet of Things interface, and performing standardization processing on the original data stream to obtain standardized equipment data; Processing the standardized equipment data using a graph database structure, constructing a multi-level topology structure, associating device basic information and real-time state through node attributes, and generating an interconnected data graph; Calculating the connection strength between nodes in the interconnected data graph, if the connection strength does not reach a preset strength threshold, marking the corresponding node as a weak connection node, and optimizing the weak connection node through clustering grouping to obtain an enhanced data graph; Extracting a real-time state subgraph from the enhanced data graph, processing the real-time state subgraph through a depth-first search algorithm to obtain a priority access path, and generating an optimized access sequence according to the priority access path; Obtain business task data from a park business management platform, fuse the optimized access sequence and the business task data to obtain a fusion feature data set, and if a matching degree reflected by the fusion feature data set reaches a preset correlation threshold, dynamically adjust a data storage level to obtain layered task integration data; Perform concurrent query simulation on the layered task integration data, filter hot data blocks based on simulation results and record response delay indicators, if the response delay indicators reach a preset delay threshold, load the hot data blocks to a memory mapping space and establish a fast retrieval index to obtain a response query result; Compare the response query result with an integrity check code of the original data stream, obtain a scheduling response improvement indicator through difference analysis, optimize data management configuration based on the scheduling response improvement indicator, and complete data management optimization.

[0006] In a second aspect, the present application provides a smart industrial park equipment big data storage system based on the Internet of Things, comprising: A data standardization module is configured to collect an original data stream of park equipment through an Internet of Things interface, perform standardization processing on the original data stream, and obtain standardized equipment data. A graph construction module is configured to process the standardized equipment data using a graph database structure, construct a multi-level topology structure, associate equipment basic information with real-time states through node attributes, and generate an interconnected data graph. A graph optimization module is configured to calculate the connection strength between nodes in the interconnected data graph, mark corresponding nodes as weak connection nodes if the connection strength does not reach a preset strength threshold, perform clustering and grouping optimization on the weak connection nodes, and obtain an enhanced data graph. An access sequence generation module is configured to extract a real-time state subgraph in the enhanced data graph, process the real-time state subgraph through a depth-first search algorithm to obtain a priority access path, and generate an optimized access sequence according to the priority access path. A storage level adjustment module is configured to obtain business task data from a park business management platform, fuse the optimized access sequence and the business task data to obtain a fusion feature data set, and if a matching degree reflected by the fusion feature data set reaches a preset correlation threshold, dynamically adjust a data storage level to obtain layered task integration data. A low-delay query module is configured to perform concurrent query simulation on the layered task integration data, filter hot data blocks based on simulation results and record response delay indicators, if the response delay indicators reach a preset delay threshold, load the hot data blocks to a memory mapping space and establish a fast retrieval index to obtain a response query result. The management optimization module is used to compare the response query result with the integrity check code of the original data stream, obtain scheduling response improvement indicators through difference analysis, optimize data management configuration based on the scheduling response improvement indicators, and complete data management optimization.

[0007] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention collects multi-source heterogeneous raw data streams through IoT interfaces, achieves unified data format through standardized mapping processing, and constructs a multi-level interconnected data graph with graph database to associate basic information and real-time status of devices. This solves the problems of scattered data storage, semantic inconsistency and loose association in existing technologies, and provides structured data support for centralized management of equipment data in smart industrial parks.

[0008] (2) This invention optimizes weak connection nodes through clustering algorithm to form an enhanced data graph, generates an optimized access sequence, dynamically adjusts storage level and activates caching mechanism, solves the problems of inefficient access path and high response latency in high concurrency scenarios of traditional solutions, and realizes fast retrieval and low-latency access of device data.

[0009] (3) This invention compares the low-latency query results with the integrity check code of the original data stream, obtains scheduling response improvement indicators through difference analysis and optimizes the resource allocation matrix, forming a closed-loop optimization mechanism for data management. This solves the problem that existing technologies are difficult to meet the high-efficiency data processing needs of intelligent management of smart industrial parks due to low data access efficiency and high task response latency. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the process of the IoT-based smart industrial park equipment big data storage method provided in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a smart industrial park equipment big data storage system based on the Internet of Things provided in the second embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] Reference Figure 1 The first embodiment of the present invention provides a method for storing big data of equipment in a smart industrial park based on the Internet of Things, including the following steps: S101: Collect raw data streams from park equipment via IoT interface, and perform standardized processing on the raw data streams to obtain standardized equipment data; S102, The standardized equipment data is processed using a graph database structure to construct a multi-level topology structure. The basic information of the equipment and its real-time status are associated through node attributes to generate an interconnected data graph. S103, calculate the connection strength between nodes in the interconnected data graph. If the connection strength does not reach the preset strength threshold, mark the corresponding node as a weak connection node, and perform clustering and grouping optimization on the weak connection nodes to obtain an enhanced data graph. S104, extract the real-time state subgraph from the enhanced data graph, process the real-time state subgraph using a depth-first search algorithm to obtain the priority access path, and generate an optimized access sequence based on the priority access path; S105, obtain business task data from the park business management platform, merge the optimized access sequence and the business task data to obtain a fused feature dataset. If the matching degree reflected by the fused feature dataset reaches a preset association threshold, dynamically adjust the data storage level to obtain hierarchical task integration data. S106, perform concurrent query simulation on the integrated data of the hierarchical task, filter hot data blocks based on the simulation results and record the response latency index. If the response latency index reaches the preset latency threshold, load the hot data block into the memory mapping space and establish a fast retrieval index to obtain the response query result. S107, compare the response query result with the integrity check code of the original data stream, obtain the scheduling response improvement index through difference analysis, optimize the data management configuration based on the scheduling response improvement index, and complete the data management optimization.

[0013] In step S101, raw data streams from park devices are collected via an IoT interface, and the raw data streams are standardized to obtain standardized device data, including: The device identifier is determined by capturing the raw data stream in transmission through the Internet of Things interface and using the header feature code of the raw data stream. Based on the device identifier, a pre-stored message structure template is matched, and the original data stream is parsed to extract a set of heterogeneous data fields. The heterogeneous data field set is cleaned and normalized to obtain the data items to be processed; Align the data items to be processed with a preset data format to construct a mapping table; Based on the mapping table, the format of the data items to be processed is reconstructed to obtain standardized device data.

[0014] It should be noted that the header signature of the raw data stream is 4 bytes long, and the correspondence between it and the device identifier is stored in a pre-stored signature library. The device is quickly identified by comparing fields. The device identifier is named according to the "device type-serial number" rule to ensure uniqueness.

[0015] It is worth noting that the pre-stored message structure template includes information such as field names, byte offsets, and data types. Based on the device identifier, the template is called to parse the original data stream segment by segment, stripping away heterogeneous data field sets. The cleaning operation includes removing invalid values, such as 65535, empty strings, and other common invalid flags; performing unit conversions according to the protocol; and anomaly detection using the normal value range of ±15% of the device's rated parameters. Normalization uses the minmax method, statistically analyzing the historical operating data of the field to determine the historical minimum and maximum values, mapping the cleaned field values ​​to the [0, 1] interval to ensure comparability of field values ​​of different magnitudes.

[0016] In step S102, standardized device data is processed using a graph database structure to construct a multi-level topology. By associating basic device information with real-time status through node attributes, an interconnected data graph is generated, including: The device identifier is extracted from the standardized device data to generate a graph node entity, and a unique graph database index key is assigned to the graph node entity using a hash algorithm; Based on the graph database index key, the graph node entities are quickly retrieved, directional edges are defined, and a multi-level topology is constructed based on the directional edges. Based on the node attributes of the multi-level topology, associate the device's basic information and real-time status; Extract the static attribute fields from the device's basic information and write them into the node storage area corresponding to the multi-level topology; The dynamically changing values ​​in the real-time state are overwritten into the dynamic attribute slots of the node storage area to generate an interconnected data graph.

[0017] It should be noted that the device identifier is directly extracted from the "deviceId" field in the standardized device data. Each device identifier corresponds to a unique graph node entity, and the node entity name is completely identical to the device identifier, ensuring a direct association between devices and nodes. The SHA-1 hash algorithm is used. The specific process for assigning the graph database index key is as follows: first, the device identifier is converted into a UTF-8 encoded byte array; then, the byte array is hashed using the SHA-1 algorithm to obtain a 20-byte hash value; finally, the hash value is hexadecimal encoded, and the first 12 characters are used as the graph database index key. The graph database index key is stored in the index table as the node's primary key. During retrieval, the node's storage address is directly located using the index key, eliminating the need to traverse all nodes.

[0018] It's worth noting that after retrieving graph node entities based on the graph database index key, the definition of directional edges is performed according to the spatial affiliation and communication link relationships of the devices. Spatial affiliation is determined using the "location" and "gatewayId" fields in the standardized device data. If the device's "location" field contains a floor identifier and the "gatewayId" field corresponds to a gateway node identifier, then the directional edge between the device node and the gateway node / floor node is defined as an affiliation relationship, labeled "belongs_to," with the direction from the device node to the gateway node and from the gateway node to the floor node. Communication link relationships are determined using interaction records in the device communication logs. If two nodes have data exchange records (such as sending or responding to commands) within the last 24 hours, then the directional edge between them is defined as a connection relationship, labeled "cto," with the direction from the data initiating node to the data receiving node. The attributes of the directional edges also include an interaction frequency statistic, which is the number of interactions between nodes within the last 24 hours, used for subsequent connection strength calculations.

[0019] It should be noted that the construction of the multi-level topology follows a hierarchical order of "Campus - Building - Floor - Gateway - Device". First, a campus master node is created as the root node of the topology. Then, building identifiers are extracted from the "building" field in the standardized device data, and building nodes are created, connected to the campus master node via directional edges based on attribution relationships. Next, floor identifiers are extracted from the "floor" field, and floor nodes are created, connected to the corresponding building nodes via directional edges based on attribution relationships. Then, gateway node identifiers are extracted from the "gatewayId" field, and gateway nodes are created, connected to the corresponding floor nodes via directional edges based on attribution relationships. Finally, all device nodes are connected to their respective gateway nodes via directional edges based on attribution relationships, and nodes with communication links are interconnected via directional edges based on connection relationships, forming a complete multi-level topology. The topology construction process uses a batch processing mode, processing 1000 device nodes per batch. Nodes are retrieved in batches using index keys, and directional edges are created in batches to improve construction efficiency.

[0020] It's worth noting that the association between basic equipment information and real-time status is achieved through node attribute mapping. Basic equipment information is extracted from the "deviceInfo" field in standardized equipment data, including static attribute fields such as equipment model, rated power, installation date, and manufacturer name. Real-time status is extracted from the "realTimeData" field in standardized equipment data, including dynamic attribute fields such as operating voltage, operating current, equipment status code, and data acquisition timestamp. Node attributes are divided into static attribute groups and dynamic attribute groups. Static attribute groups are mapped one-to-one with basic equipment information fields, and dynamic attribute groups are mapped one-to-one with real-time status fields. The mapping relationship is determined by a pre-defined attribute mapping table, which clearly defines the corresponding names and data type conversion rules for basic equipment information fields, real-time status fields, and node attribute fields.

[0021] It should be noted that the writing process for static attribute fields involves first locating the node storage area using the graph database index key, and then writing each static attribute field from the device's basic information to its corresponding storage location according to the field order of the node's static attribute group. After the static attribute fields are written, updates are only triggered when the device's basic information changes (through a change command issued by the park's device management platform). Updates use an overwrite method and simultaneously record an update log, including the update time, updated fields, old values, and new values.

[0022] It is worth noting that the process for overwriting dynamically changing values ​​is as follows: Real-time monitoring of standardized device data pushes is performed. When new real-time status data is received, the device identifier is extracted, and an index key is obtained through a hash algorithm to locate the dynamic attribute slot in the node's storage area. The new dynamically changing values ​​are then overwritten to the corresponding slots one by one according to the field order of the dynamic attribute group. During overwriting, old values ​​are not deleted; the new values ​​directly overwrite the binary data in the storage address. A cyclic overwrite mechanism is used for dynamic attribute slots, and historical dynamic data is not retained. Historical data is archived to a separate time-series database by timestamp, with an archiving frequency of once per hour. During archiving, the device identifier and index key are associated for easy traceability and querying. The response time for dynamic attribute overwriting is controlled within 50 milliseconds to ensure timely updates of real-time status data. Both the archiving frequency and the response time are derived from experimental data statistics.

[0023] In step S103, the connection strength between nodes in the interconnected data graph is calculated. If the connection strength does not reach a preset strength threshold, the corresponding node is marked as a weak connection node. Clustering and grouping optimization is performed on the weak connection nodes to obtain an enhanced data graph, including: Traverse the interconnected data graph, extract the interaction frequency and data flow characteristics between nodes, and calculate the connection strength value based on the interaction frequency and data flow characteristics; If the connection strength value does not reach the preset strength threshold, the connection edge between the corresponding nodes is marked as a weak connection edge, all associated nodes of the weak connection edge are marked as weak connection nodes, and multiple discrete connection nodes are extracted to form a discrete node set to be optimized. Calculate the topological distance and business relevance of the discrete node set to be optimized, and construct a similarity matrix; Traverse the similarity matrix and divide the nodes whose topological distance meets the preset short distance requirement into logical groups; For the logical group, the gateway nodes within the logical group are associated with the directed edges to generate virtual aggregation nodes, and directed edges are established between the virtual aggregation nodes and each node within the logical group to form an enhanced data graph.

[0024] It should be noted that the traversal of the interconnected data graph adopts a depth-first traversal approach, starting from the overall node of the park and traversing each level of nodes and the connecting edges between nodes, recording the node pairs and interaction data corresponding to each connecting edge. The interaction frequency is extracted as the average daily number of data interactions between node pairs in the past 7 days, and the data flow characteristic is extracted as the average daily total data transmission between node pairs in the past 7 days. Before performing weighted summation, both need to be normalized.

[0025] To determine the maximum interaction frequency and maximum data flow required for normalization, two temporary variables, `max_freq` and `max_flow`, are first initialized to 0. While traversing the interconnected data graph and recording the interaction data of each connection edge, the interaction frequency of the current edge is synchronously compared to `max_freq`. If it is, `max_freq` is updated to its current value. Similarly, the data flow of the current edge is synchronously compared to `max_flow`. If it is, `max_flow` is updated to its current value. After the traversal is complete, `max_freq` and `max_flow` represent the extreme values ​​of the entire graph in the current period, used for normalization calculations. Specifically, the normalized interaction frequency equals the interaction frequency of a connection edge divided by the maximum interaction frequency in that period, and the normalized data flow equals the data flow of a connection edge divided by the maximum data flow in that period. The connection strength value is calculated by a weighted sum of the normalized interaction frequency and the normalized data flow.

[0026] The weighting coefficients were determined as follows: First, 1000 labeled node pairs covering different types were selected within the park, and their actual interaction data for three months was collected. Using the fit between the connection strength calculation results and the expert annotation results as the optimization objective, multiple weight combinations were tested, including normalized interaction frequency weight α∈[0.5,0.7] and normalized data flow weight β=1-α. The ability of each weight combination to distinguish between strongly and weakly correlated node pairs labeled by the expert was calculated based on the connection strength value obtained from that weight combination. The area under the receiver operating characteristic (ROC) curve was used as a comprehensive evaluation index; a higher area value indicates better performance in distinguishing between strongly and weakly correlated node pairs. Finally, the weight combination with the highest area value was selected, for example, α=0.6 and β=0.4.

[0027] The preset strength threshold was determined by statistically analyzing the connection strength data of 5000 node pairs within the park, using stratified sampling to ensure sample coverage of different business scenarios such as energy consumption monitoring, equipment control, and alarm response. Outliers were removed during data collection using a 95% confidence interval. The threshold was determined based on business requirements; for example, if optimization of the bottom 20% of connections is needed, the value at the 20th percentile after sorting the valid data in ascending order was used as the threshold. Through multiple rounds of sample testing, this threshold has been verified to consistently identify a expected proportion of weak connections.

[0028] It's worth noting that when marking weak links, the `weak_link` label is directly added to the corresponding link in the interconnected data graph, and the label is stored in the link's attribute field. When marking weak nodes, all links with the `weak_link` label are traversed, the nodes at both ends of the link are extracted, the `weak_node` label is added to these nodes, and the number of weak links associated with each node is recorded.

[0029] The extraction rules for the discrete node set to be optimized are as follows: nodes labeled "weak_node" with at least two associated weak edges are selected. Simultaneously, the average connection strength between this node and all nodes directly connected via CTO edges is calculated. If this average value is lower than a preset strength threshold, the node is included in the discrete node set to be optimized. During the extraction process, nodes are batch-filtered using the graph database's label query function, generating a list of discrete node sets to be optimized. This list includes information such as node identifier, number of associated weak edges, and average connection strength.

[0030] It should be noted that the topology distance calculation uses a hierarchical path length statistical method between nodes, that is, the number of edges on the shortest path between two nodes in the interconnected data graph. Business relevance calculation is based on the degree of matching between device type and functional attributes. Device types are divided into 10 categories, such as meters, sensors, gateways, and controllers; functional attributes are divided into 8 categories, such as energy consumption monitoring, environmental sensing, equipment control, and data forwarding. The business relevance assignment rules are as follows: Same device type and same functional attributes, business relevance is 0.9; different device types but same functional attributes, business relevance is 0.7; same device type but different functional attributes, business relevance is 0.5; different device types and different functional attributes, business relevance is 0.2.

[0031] It should be noted that the construction of the node feature matrix aims to generate a feature vector for each node that reflects its topological and business attributes. For each node in the discrete node set to be optimized, its feature vector is constructed as follows: First, the topological feature component of the node directly uses its own hierarchical depth, calculated starting from the total node of the park, with a depth of 1. The depth is increased by 1 for each belonging relationship edge, and this depth value is normalized to the [0,1] interval. Second, the business feature component of the node is represented by a one-hot encoded vector. The vector dimension is determined by the product of the number of device types and the number of functional attributes. The corresponding position of the device type and functional attribute combination of the node in the vector is set to 1, and the other positions are set to 0. The feature vector of each node is formed by concatenating the normalized hierarchical depth and the one-hot encoded business vector. The feature vectors of all nodes together constitute the node feature matrix. This method generates a fixed-dimensional feature representation for each node that directly reflects its key attributes, and is suitable for subsequent clustering algorithms.

[0032] It is worth noting that the parameters of the DBSCAN density clustering model use optimized preset values. Since the node features have been normalized to the [0,1] interval, the neighborhood radius ε is searched within the (0,1) range, and the minimum number of points MinPts is set according to the expected cluster size. This set of parameters is obtained by training on historical data using a grid search method. Specifically, the training process involves collecting weakly connected node sample data covering multiple types of devices in the park over the past 6 months, constructing a feature matrix, and labeling it with true grouping labels based on business function consistency and topological distance requirements. Using cluster purity as the evaluation index, the parameter combinations are iterated, and finally, the highest cluster purity is determined when the neighborhood radius ε=0.3 and the minimum number of points MinPts=3. Under this parameter combination, the model can effectively identify density regions formed by nodes that are sufficiently close in both hierarchical depth and business attribute feature dimensions.

[0033] It should be noted that after inputting the node feature matrix into the DBSCAN model, the model automatically performs density clustering based on feature similarity and outputs the cluster results, with each cluster being a logical group. Noise points that are not assigned to any cluster are temporarily retained in the set of discrete nodes to be optimized and can participate in clustering again in subsequent optimization cycles.

[0034] For logical groups, virtual aggregation nodes are generated. These virtual aggregation nodes are named according to the core functional attributes of the logical group, using the naming format: Function Type - Group Number. The function type is determined based on the mainstream functional attributes of the nodes within the group, and the group number increments from 01 according to the generation order of the logical groups. Using the edge creation function of the graph database, directed edges are created in batches from the virtual aggregation nodes to each node within the logical group. The edge label is defined as "contains," and the attribute field records the relationships within the group. Simultaneously, the original gateway nodes to which each node within the logical group belong are counted. The gateway node containing the most nodes is determined as the upper-level gateway node of that logical group, and directed edges are created from the virtual aggregation nodes to this upper-level gateway node. The edge label is also "contains," and the attribute field records the upper-level gateway relationship. By introducing virtual aggregation nodes and creating these directed edges, the logical connections between the original weakly connected nodes are strengthened, forming an enhanced data graph.

[0035] In step S104, the real-time state subgraph is extracted from the enhanced data graph, processed by a depth-first search algorithm to obtain the preferred access path, and an optimized access sequence is generated based on the preferred access path, including: Traverse the enhanced data graph, filter active nodes and associated edges, and construct a real-time state subgraph; The interaction frequency weights of the real-time state subgraph are analyzed to generate an adjacency matrix. The adjacency matrix is ​​then input into a depth-first search algorithm to output connected traversal paths. The weights of the traversed paths are summed to obtain the path popularity value. The traversed paths whose path popularity value reaches a preset popularity threshold are determined as priority access paths. Obtain the topology depth and path popularity value of the preferred access path, generate a sorting index, and reorganize the preferred access path according to the sorting index to obtain an optimized access sequence.

[0036] It should be noted that the enhanced data graph traversal adopts a breadth-first traversal approach, starting from the gateway node and traversing its associated terminal device nodes, virtual aggregation nodes, and corresponding connection edges layer by layer, recording the latest activity timestamps and data interaction records of the nodes. The selection criteria for active nodes are: the node has any type of data interaction record, such as data sending, receiving, or status update, within the most recent hour.

[0037] The selection criteria for associated edges are as follows: edges connecting two active nodes, or edges connecting an active node and a gateway node, ensure that the constructed real-time state subgraph only contains currently active nodes and their relationships. When constructing the real-time state subgraph, the node attributes of the subgraph retain the static and dynamic attributes from the original enhanced data graph, and the associated edges retain the interaction frequency and connection strength attributes. The subgraph adopts a temporary in-memory storage mode to avoid consuming persistent storage resources.

[0038] It's worth noting that the interaction frequency weight is calculated based on the average daily interaction count between node pairs over the past 7 days. The average daily interaction count of a node pair is divided by the historical maximum average daily interaction count of all node pairs in the enhanced data graph over the past 7 days. The weight value ranges from [0,1]. When parsing the interaction frequency weight of the real-time state subgraph, all connecting edges in the subgraph are traversed, the interaction frequency corresponding to each edge is extracted, and the weight value is calculated in the above manner. If the average daily interaction count of a node pair is 0, the weight value is set to 0.01. This setting aims to avoid division by zero errors and prevent zero weight from filtering out valid paths.

[0039] It should be noted that the adjacency matrix is ​​a square matrix of active nodes in the real-time state subgraph. The matrix dimension is the same as the number of active nodes, and the matrix element values ​​are the interaction frequency weights of the edges connecting corresponding two nodes. The rows and columns of the matrix are arranged in lexicographical order according to the node identifiers to ensure a fixed correspondence between nodes and their matrix positions. For example, if the real-time state subgraph contains nodes A, B, and C, arranged lexicographically as rows A, B, C and columns A, B, C, and the interaction frequency weight between nodes A and B is 0.8, between nodes A and C it is 0.3, and between nodes B and C it is 0.5, then the adjacency matrix would be [[0,0.8,0.3],[0.8,0,0.5],[0.3,0.5,0]]. After the matrix is ​​constructed, it is stored as a two-dimensional array.

[0040] It's worth noting that during the execution of the depth-first search algorithm, the sequence of nodes at each jump is recorded to form a connected traversal path. Each path starts at a gateway node and ends at a terminal device node or virtual aggregation node. The path length (i.e., the number of nodes contained in the path) does not exceed 8. This limitation is determined based on the actual hierarchical distribution characteristics of the devices in the campus, avoiding decreased access efficiency due to excessively long paths. If multiple gateway nodes exist in the real-time state subgraph, a depth-first search is performed starting from each gateway node, generating multiple independent traversal paths.

[0041] It should be noted that the path popularity value is the sum of the interaction frequency weights of all connecting edges in the path. The preset popularity threshold is determined by statistically analyzing historical traversal path data within the park over the past three months, removing outliers using a 95% confidence interval, and then sorting the valid data in ascending order. The threshold is determined in conjunction with specific business objectives. For example, if resources need to be allocated primarily to the top 25% of the most active paths, the 75th percentile value is used as the preset popularity threshold. Through multiple rounds of sample testing, this threshold has been verified to reliably filter out the expected proportion of high-popularity paths.

[0042] It's worth noting that topology depth is the difference between the level of the endpoint node and the level of the starting node. The starting gateway node has a fixed four levels, and the node levels are divided as follows: "Campus General Node (Level 1) - Building Node (Level 2) - Floor Node (Level 3) - Gateway Node / Virtual Aggregator Node (Level 4) - Terminal Device Node (Level 5)". For paths containing intermediate nodes, the topology depth is always calculated based on the starting and ending node levels, without considering the intermediate nodes. For example, if the starting point of the path is a gateway node (Level 4) and the ending point is a terminal device node (Level 5), then the topology depth is 1; if the starting point of the path is a gateway node (Level 4) and the ending point is a virtual aggregation node (Level 4), then the topology depth is 0. The topology depth feature value used in the sorted index calculation needs to be normalized to the [0,1] interval.

[0043] It is worth noting that the ranking index is calculated by weighting path popularity value and normalized topology depth, where the path popularity value weight λ is 0.7 and the normalized topology depth weight μ is 0.3. This weight allocation is based on experiments, using 500 historical access paths covering different business scenarios such as energy consumption monitoring, equipment control, and alarm response within the park over the past 6 months as the sample set. These paths are sorted in descending order of their historical access frequency as the benchmark ranking. Multiple weight combinations with λ∈[0.6,0.8] and μ=1-λ are tested, and the Spearman rank correlation coefficient between the generated path ranking and the benchmark ranking is calculated for each combination. Finally, the weight combination with the highest Spearman rank correlation coefficient and passing the significance test (p-value < 0.05) is selected as the final parameter. For example, when λ=0.7, the correlation coefficient reaches 0.85 and the p-value is less than 0.01, indicating that this weight combination is statistically significant.

[0044] The p-value is a statistical concept, short for probability value. It is used to quantify the significance level of a statistical test result.

[0045] It should be noted that the depth-first search algorithm requires parameter settings. Access markers are stored using a Boolean array with a length matching the number of active nodes. Path storage uses a stack structure with a maximum stack depth of 8. The algorithm terminates when the stack is empty and all reachable nodes have been visited, or when the number of generated traversal paths reaches a preset limit. This preset limit needs to be set based on specific business requirements. Its initial value is determined by analyzing the historical access patterns of the park, such as calculating the average number of device nodes involved in a single query over the past three months. A multiplier is determined based on the percentile target for business coverage (e.g., covering 95% of query scenarios), typically set to 2.5 times as the initial value for the preset limit, to ensure coverage of major query needs while avoiding the generation of too many low-value paths.

[0046] In step S105, business task data is obtained from the park's business management platform. The access sequence and business task data are then merged and optimized to obtain a fused feature dataset. If the matching degree reflected by the fused feature dataset reaches a preset association threshold, the data storage hierarchy is dynamically adjusted to obtain hierarchical task integration data, including: The optimized access sequence is parsed to obtain the sequence topology, and the business task data is extracted to generate a task feature vector. The sequence topology is fused with the task feature vector to output a fused feature dataset, and the matching degree of the fused feature dataset is calculated. If the matching degree reaches the preset association threshold, the target storage level label is determined by combining the access frequency. Data migration instructions are generated based on the target storage tier label, and the data migration instructions are executed to obtain tiered task integration data.

[0047] It's worth noting that when parsing and optimizing the access sequence to obtain the sequence topology, a path-by-path traversal method is used to extract the core topological features of each path, including topological depth (the difference in levels between the start and end points of the path), number of branches (the total number of branch nodes in the path), and path length (the number of nodes contained in the path). For example, if a path in the optimized access sequence is "gateway node → sensor node → terminal device node," then the topological depth is 2, the number of branches is 1, and the path length is 3. During the parsing process, a topological feature file is created for each path, containing information such as path identifier, topological depth, number of branches, and path length.

[0048] It should be noted that when extracting business task data to generate task feature vectors, four core feature dimensions are defined: task urgency, data volume, execution cycle, and number of associated devices. Task urgency is assigned a value based on task type: 0.9 for urgent alarm tasks, 0.5 for periodic data reporting tasks, and 0.1 for device configuration update tasks. This assignment rule is determined by prioritizing the park's business processes. Data volume is divided according to the storage space occupied by task data: 0.8 for data larger than 100KB, 0.5 for 10KB-100KB, and 0.2 for less than 10KB. This division is based on the data volume statistics of common business tasks in the park. Execution cycle is assigned a value based on the interval between repeated executions: 0.9 for real-time execution (no interval), 0.6 for hourly execution, and 0.3 for execution on a daily or longer basis. The number of associated devices is assigned a value based on the number of device nodes involved in the task: 0.8 for more than 10, 0.5 for 3-10, and 0.2 for less than 3. The task feature vector is a four-dimensional vector, which is combined in the order of "task urgency - data size - execution cycle - number of associated devices". For example, an emergency alarm task with a data size of 50KB, real-time execution, and associated with 2 devices has a feature vector of (0.9, 0.5, 0.9, 0.2).

[0049] It's worth noting that the fusion method between the sequence topology and the task feature vector is feature concatenation. The three core features of the sequence topology—normalized topology depth, normalized branch count, and normalized path length—are concatenated with the four dimensions of the task feature vector to form a seven-dimensional fused feature dataset. Normalization maps the topology feature values ​​to the [0, 1] interval. For example, if the maximum topology depth is 5 and a path's topology depth is 2, the normalized value is 0.4. The structure of the fused feature dataset is (normalized topology depth, normalized branch count, normalized path length, task urgency, data volume, execution cycle, number of associated devices). Each optimized access path is combined with data from each business task to generate one fused feature dataset.

[0050] It should be noted that the matching degree is calculated based on feature fit judgment, focusing on the adaptation relationship between topological structure features and task features. Shallow topological depth (normalized value ≤ 0.3) is suitable for high urgency (≥ 0.8) tasks, few branches (normalized value ≤ 0.4) are suitable for tasks with few associated devices (≤ 0.3), and short path length (normalized value ≤ 0.3) is suitable for real-time execution (≥ 0.8) tasks. Each matching relationship is scored as 1 point when it is satisfied and 0 points when it is not satisfied. The total score is divided by the total number of matching relationships (3) to obtain the matching degree, with a value range of [0, 1].

[0051] The preset association threshold was determined as follows: 1200 sets of business task and access sequence matching samples from the past 6 months were collected from the park. Stratified sampling was used to ensure that the samples covered four business scenarios: emergency alarms, periodic reporting, device configuration updates, and statistical analysis. The samples involved device nodes in 5 buildings and 15 floors. The sample data included sequence topology features, task feature vectors, and actual business adaptation result labels. During data statistics, outliers (such as samples with extreme matching degrees of 0 or 1) were removed using a 95% confidence interval, retaining 1120 valid samples. The matching degrees of the valid samples were sorted in ascending order, and the effective matching rate and false matching rate corresponding to different percentile values ​​were calculated. When the 70th percentile value was 0.7, the effective matching rate reached 93%, and the false matching rate was only 3.2%, fully meeting the target optimization function requirements. If the 65th percentile value was 0.65, the false matching rate rose to 6.8%, exceeding the threshold. If the 75th percentile value was 0.75, the effective matching rate dropped to 88%, failing to meet the target. The final preset association threshold was determined to be 0.7.

[0052] It should be noted that the access frequency statistics are the number of path queries corresponding to the optimized access sequence in the most recent hour. The statistical method is to extract query requests through log records, group them by path identifier, and count them to obtain the number of queries per hour for each path.

[0053] It's worth noting that data migration instructions include information such as instruction identifier, source storage address, target storage address, migration priority, and data identifier. The migration priority is consistent with the business task priority: urgent tasks have a migration priority of 1 (highest), regular tasks have a priority of 2, and low-priority tasks have a priority of 3. When executing data migration instructions, priority 1 tasks use a real-time migration method, immediately migrating data from the source storage medium to the target storage medium; priority 2 and 3 tasks use an off-peak migration method, performing migration during off-peak business hours in the park (2:00 AM to 4:00 AM) to avoid consuming business bandwidth during the migration process. After migration is complete, the hierarchical mapping table is updated, recording information such as data identifier, target storage address, migration completion time, and current storage level label. The hierarchical mapping table uses key-value pairs for storage, with the data identifier as the key, ensuring fast lookup of data storage location.

[0054] In step S106, concurrent query simulation is performed on the data integrated by the hierarchical task. Based on the simulation results, hot data blocks are selected and response latency indicators are recorded. If the response latency indicator reaches a preset latency threshold, the hot data blocks are loaded into the memory-mapped space and a fast retrieval index is established to obtain the response query results, including: A virtual request queue is constructed based on the hierarchical task integration data. A simulated retrieval operation is performed according to the virtual request queue. The identifier of the associated data block, the start timestamp, the end timestamp, and the number of accesses in the hierarchical task integration data corresponding to each retrieval are recorded to obtain the response timestamp sequence and access frequency statistics. The average latency, peak latency, and percentile latency are extracted as latency statistics based on the response timestamp sequence. Filter related data blocks whose access frequency exceeds a preset frequency threshold and whose response latency exceeds a preset sub-threshold, and mark them as hot data blocks; If the latency statistics exceed the preset latency threshold, the hot data block is loaded into the memory-mapped space, an index for fast hash retrieval is established, and a query task is executed through the index to obtain the response query result.

[0055] It should be noted that when constructing the virtual request queue, the storage hierarchy priority is sorted in the order of hot storage, warm storage, and cold storage. Within the same hierarchy, the priority is arranged according to the business priority of emergency alarms, periodic reporting, and configuration updates. The queue length is set to 500 entries, determined based on the statistics of the highest daily concurrent query volume in the park over the past 3 months, covering actual access pressure. Each request includes a data block identifier and query type. Among the query types, data reading accounts for 70%, status query accounts for 20%, and statistical analysis accounts for 10%, which closely matches the actual business query distribution in the park.

[0056] It is worth noting that the simulated retrieval calls the simulated interface corresponding to the actual storage medium. Hot storage corresponds to the high-speed NVMe SSD simulated interface, warm storage corresponds to the SATA SSD simulated interface, and cold storage corresponds to the mechanical hard drive simulated interface. The start and end timestamps are recorded with millisecond precision, and the access count is grouped and counted according to the data block identifier. The count is incremented by 1 for each retrieval. The statistical results correspond one-to-one with the data block identifier, forming an access frequency statistics table.

[0057] It's worth noting that, considering the requirement that the latency of the park's core business should not exceed 50 milliseconds, an average latency threshold of 30 milliseconds, a peak latency threshold of 100 milliseconds, and a percentile latency threshold of 60 milliseconds were set. Exceeding any of these thresholds triggers subsequent operations. The preset frequency threshold is 10 times / hour, and the preset sub-threshold is 20 milliseconds, based on statistical analysis of the park's data access patterns over six months. When filtering hot data blocks, the access frequency statistics table and response timestamp sequence are traversed to extract data blocks with ≥10 accesses and a latency ≥20 milliseconds. These are marked as hot data blocks, and their identifier, data size, storage address, and other information are recorded to form a list of hot data blocks.

[0058] It should be noted that the memory-mapped space is allocated at 40% of the server's physical memory; for example, 64GB of physical memory will be allocated 25.6GB of cache space. When loading frequently accessed data blocks, they are sorted in descending order of access frequency, prioritizing blocks with high access frequency and small data sizes. If the total size exceeds the cache space, the top N high-priority data blocks are retained. The loading process uses direct memory access, bypassing the operating system cache and reducing data copying overhead.

[0059] It's worth noting that the fast retrieval index uses a hash table structure. The key fields are the data block identifier, device identifier, and timestamp. The MurmurHash3 algorithm is used as the hash function, converting the key fields into a 32-bit hash value as the index key. During index construction, the index iterates through the hot data blocks already loaded into the memory-mapped space, extracts the key fields to generate the index key, and establishes a mapping between the index key and the data block's memory storage address. The hash table's load factor is set to 0.7, and it automatically expands to twice its original size if the load exceeds the threshold.

[0060] For example, the hierarchical task integration data includes 20 emergency alarm data blocks in the hot storage tier, 100 periodically reported data blocks in the warm storage tier, and 380 configuration update data blocks in the cold storage tier. A virtual request queue is constructed according to priority: the first 20 are emergency alarm requests, the middle 100 are periodically reported requests, and the last 380 are configuration update requests. After performing a simulated retrieval, the latency for emergency alarm requests is 5-15 milliseconds, for periodically reported requests it is 20-80 milliseconds, and for configuration update requests it is 50-200 milliseconds. Access frequency statistics show that 15 periodically reported data blocks are accessed ≥10 times and have a latency ≥20 milliseconds, and are marked as hot data blocks. Latency statistics are calculated, and the average latency is 45 milliseconds, the peak latency is 200 milliseconds, and the 95th percentile latency is 85 milliseconds, all exceeding the preset thresholds. Fifteen hot data blocks (total size 8GB) were loaded into a 25.6GB memory-mapped space. A hash index was built based on the data block identifier, device identifier, and timestamp. When a query was executed through the index, the original 40-millisecond periodic reporting request latency was reduced to less than 3 milliseconds, and the query result was obtained.

[0061] In step S107, the integrity check code of the response query result is compared with that of the original data stream. A scheduling response improvement index is obtained through difference analysis. Based on the scheduling response improvement index, the data management configuration is optimized to complete the data management optimization, including: Obtain the integrity check code of the response query result and the original data stream; if the response query result does not match the integrity check code, generate a difference feature vector. The scheduling log is parsed based on the difference feature vector, the response time deviation set is extracted, and the scheduling response improvement index is calculated based on the response time deviation set. Obtain the current resource allocation matrix, analyze the scheduling response improvement index, determine the adjustment range of the resource allocation matrix, and generate optimized configuration parameters; The optimized configuration parameters are applied to the data flow control layer to complete data management optimization.

[0062] It should be noted that the integrity check code is calculated using the SHA-256 algorithm. The calculation process is as follows: the response query result and the original data stream are both serialized into binary format. The serialized byte array is then subjected to a SHA-256 hash operation to obtain a 64-bit hexadecimal string as the integrity check code. During the verification process, the character sequences of the two check codes are compared byte by byte. If any character is inconsistent, the check code is considered mismatched, triggering the generation of a difference feature vector. The serialization process uses a uniform field order and encoding format to ensure that identical data generates consistent byte arrays. The field order is arranged as "device identifier - static attribute - dynamic attribute - timestamp," and the encoding format is UTF-8.

[0063] It's worth noting that the difference feature vector comprises five core dimensions: the difference field name, the field value in the original data stream, the field value in the response query result, the difference type, and the difference impact range. The difference type is categorized into four types: numerical deviation, timestamp misalignment, missing field, and data tampering, based on the actual difference situation. The difference impact range is divided according to the associated business modules, including energy consumption monitoring, equipment control, alarm response, and statistical analysis. When generating the difference feature vector, all fields in the response query result and the original data stream are traversed, and the field values ​​are compared one by one. The information of the fields with differences is recorded, and the vector is formed by combining them in the order of "difference field name - original value - query value - difference type - impact range." The vector format is a string array for easy subsequent parsing and processing.

[0064] It should be noted that the scheduling log contains core fields such as task identifier, scheduling initiation time, data transmission time, node processing time, response feedback time, associated device identifier, and storage level information. The log is stored in timestamp order, retaining records from the most recent 30 days. When parsing the scheduling log, the corresponding scheduling records are filtered based on the device identifier and timestamp in the difference feature vector. The total response time (response feedback time - scheduling initiation time) and the expected response time (determined based on the average response time of similar historical tasks) of each record are extracted. The difference between the two is calculated as the response time deviation, and all deviation values ​​form the response time deviation set. The expected response time is determined by statistically analyzing the response times of similar tasks within the past month and taking the 50th percentile value to ensure that it meets the normal business response level.

[0065] It is worth noting that the scheduling response improvement index is calculated using the deviation improvement rate, and the formula is: ,

[0066] Where R represents the scheduling response improvement metric. This represents the average response time before optimization. This represents the average response time after optimization. The value range is [0, 1], with a larger value indicating a more significant improvement. During calculation, if... If R=0, it indicates that optimization has no improvement; if The actual improvement percentage is then calculated using the formula. For example, if the average response time before optimization was 50 milliseconds and after optimization it is 30 milliseconds, then R=0.4, meaning the improvement is 0.4.

[0067] It is worth noting that the average response time before optimization was calculated by collecting response time data for similar business tasks within the park over the past three months, totaling 1000 valid samples. The arithmetic mean of these data was taken as the average response time before optimization. The average response time after optimization was calculated by using benchmarking tools to collect response time data for the same type of business tasks under the same simulated load environment, resulting in 800 valid samples. Outliers were removed using the same 95% confidence interval as before optimization, retaining 750 valid data points, and their arithmetic mean was calculated as the average response time after optimization.

[0068] It should be noted that the resource allocation matrix is ​​a two-dimensional matrix. The rows represent server nodes within the park, the columns represent business task types, and the matrix element values ​​represent the proportion of computing resources allocated to the corresponding business task for that server node.

[0069] When analyzing scheduling response improvement indicators, adjustment levels are categorized based on these indicators: R < 0.2 indicates a low improvement level with an adjustment of 10%; 0.2 ≤ R < 0.5 indicates a medium improvement level with an adjustment of 30%; and R ≥ 0.5 indicates a high improvement level with an adjustment of 50%. The adjustment increases the resource allocation for high-priority tasks and decreases the resource allocation for low-priority tasks. For example, if a server node allocates 20% of its resources to emergency alarm tasks, it would be adjusted to 20% × (1 + 30%) = 26% under the medium improvement level. Simultaneously, a corresponding percentage is deducted from the resource allocation for configuration update tasks.

[0070] It's worth noting that the optimized configuration parameters include the computing resource allocation ratio, storage tier threshold adjustment value, cache eviction policy parameters, and data flow routing priority. The computing resource allocation ratio directly uses the result after adjusting the resource allocation matrix; the storage tier threshold adjustment value is determined based on the improvement index: at a high improvement level, the access frequency threshold for the hot storage tier is reduced by 20%, at a medium improvement level by 10%, and at a low improvement level, it remains unchanged; the cache eviction policy parameter adopts the LRU strategy, with the eviction threshold set at 80% of the cache space. When the cache occupancy reaches the threshold, the hottest data block with the fewest accesses in the last 30 minutes is evicted; the data flow routing priority is sorted by business task type: emergency alarms > periodic reporting > statistical analysis > configuration updates, ensuring that high-priority tasks have priority access to high-speed transmission channels.

[0071] It's important to note that the data flow control layer is the core layer of the park's data management system, responsible for data flow routing, resource scheduling, and priority control. When applications optimize configuration parameters, the control layer's configuration files, including the resource allocation table, routing rule table, and threshold configuration table, are updated through the system interface. The update process uses an incremental update method, replacing only the changed parameter items to avoid business interruptions caused by system restarts. After the update is complete, the control layer processes subsequent data flows according to the new configuration: allocating CPU and memory resources according to the new resource allocation ratio, determining the data storage level according to the adjusted storage level threshold, managing memory-mapped space according to the cache eviction policy, and distributing data flows according to routing priority.

[0072] For example, in the response query results, the active power of a sensor "Sensor-040" is 156.8 kWh, while in the original data stream, the value of this field is 158.2 kWh. The SHA-256 checksums of the two are "a7f3d2e8b1c94560a3b7d8e2c1f9a0b3c5d7e9f0a2b4c6d8e0f1a3b5c7d9e0f1" and "b7f3d2e8b1c94560a3b7d8e2c1f9a0b3c5d7e9f0a2b4c6d8e0f1a3b5c7d9e0f2" respectively, indicating a checksum mismatch. A difference feature vector is generated ("activeEnergy", 158.2, 156.8, "numerical deviation", "energy consumption monitoring"). Analyzing the scheduling logs and filtering the scheduling records corresponding to this sensor, the response time before optimization was 45 milliseconds, the expected response time was 35 milliseconds, and the response time deviation was 10 milliseconds; after optimization, the response time was 28 milliseconds, the expected response time was 35 milliseconds, and the response time deviation was -7 milliseconds, with the response time deviation set being [10, -7]. The scheduling response improvement index R = (45-28) / 45 ≈ 0.378 was calculated, which belongs to the medium improvement level, with an adjustment range of 30%. In the current resource allocation matrix, the resource ratio of server "Server-03" for emergency alarm tasks is 25%, which will be adjusted to 25% × 1.3 = 32.5%. 7.5% will be deducted from the resource ratio of configuration update tasks (originally 15%), resulting in an adjustment to 7.5%. Optimized configuration parameters are generated: computing resource allocation ratio (Server-03: emergency alarms 32.5%, periodic reporting 30%, statistical analysis 20%, configuration update 7.5%), hot storage level access frequency threshold reduced by 10%, cache eviction threshold set to 80% of cache space, and routing priority (emergency alarms > periodic reporting > statistical analysis > configuration update). These parameters are applied to the data flow control layer. After updating the configuration file, subsequent data flows from this sensor are distributed according to the new configuration, emergency alarm tasks receive more computing resources, and response speed is further improved.

[0073] In summary, this invention provides a method for storing big data of smart industrial park equipment based on the Internet of Things. It can solve the problem that existing technologies, in practical application scenarios where big data storage is becoming increasingly widespread, suffer from low data access efficiency and high task response latency, making it difficult to meet the high-efficiency data processing needs of intelligent management in smart industrial parks. This invention provides reliable technical support for intelligent big data storage.

[0074] refer to Figure 2 The second embodiment of the present invention provides a smart industrial park equipment big data storage system based on the Internet of Things, comprising: The data standardization module is used to collect raw data streams from park equipment through an IoT interface, and to standardize the raw data streams to obtain standardized equipment data. The graph construction module is used to process the standardized equipment data using a graph database structure, construct a multi-level topology, and generate an interconnected data graph by associating basic equipment information and real-time status through node attributes. The graph optimization module is used to calculate the connection strength between nodes in the interconnected data graph. If the connection strength does not reach a preset strength threshold, the corresponding node is marked as a weak connection node. The weak connection nodes are then clustered and optimized to obtain an enhanced data graph. The access sequence generation module is used to extract the real-time state subgraph from the enhanced data graph, process the real-time state subgraph through a depth-first search algorithm to obtain the priority access path, and generate an optimized access sequence based on the priority access path. The storage tier adjustment module is used to obtain business task data from the park business management platform, merge the optimized access sequence and the business task data to obtain a fused feature dataset. If the matching degree reflected by the fused feature dataset reaches a preset association threshold, the data storage tier is dynamically adjusted to obtain hierarchical task integration data. The low-latency query module is used to perform concurrent query simulation on the integrated data of the hierarchical task, filter hot data blocks based on the simulation results and record the response latency index. If the response latency index reaches the preset latency threshold, the hot data block is loaded into the memory-mapped space and a fast retrieval index is established to obtain the response query results. The management optimization module is used to compare the response query result with the integrity check code of the original data stream, obtain scheduling response improvement indicators through difference analysis, optimize data management configuration based on the scheduling response improvement indicators, and complete data management optimization.

[0075] It should be noted that the IoT-based smart industrial park equipment big data storage system provided in this embodiment of the invention is used to execute all the process steps of the IoT-based smart industrial park equipment big data storage method described in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0076] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for storing big data on equipment in a smart industrial park based on the Internet of Things, characterized in that, include: The raw data streams from park equipment are collected through an IoT interface, and the raw data streams are then standardized to obtain standardized equipment data. The standardized equipment data is processed using a graph database structure to construct a multi-level topology. The basic equipment information and real-time status are associated through node attributes to generate an interconnected data graph. Calculate the connection strength between nodes in the interconnected data graph. If the connection strength does not reach a preset strength threshold, mark the corresponding node as a weak connection node. Perform clustering and grouping optimization on the weak connection nodes to obtain an enhanced data graph. Extract the real-time state subgraph from the enhanced data graph, process the real-time state subgraph using a depth-first search algorithm to obtain the priority access path, and generate an optimized access sequence based on the priority access path; Business task data is obtained from the park's business management platform. The optimized access sequence and the business task data are then integrated to obtain a fused feature dataset. If the matching degree reflected by the fused feature dataset reaches a preset association threshold, the data storage level is dynamically adjusted to obtain hierarchical task integration data. The data of the hierarchical task is integrated and concurrent query simulation is performed. Based on the simulation results, hot data blocks are filtered and response latency indicators are recorded. If the response latency indicators reach a preset latency threshold, the hot data blocks are loaded into the memory-mapped space and a fast retrieval index is established to obtain the response query results. By comparing the response query result with the integrity check code of the original data stream, a scheduling response improvement index is obtained through difference analysis. Based on the scheduling response improvement index, the data management configuration is optimized to complete the data management optimization.

2. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 1, characterized in that, The process of collecting raw data streams from park equipment via an IoT interface and standardizing these raw data streams to obtain standardized equipment data includes: The device identifier is determined by capturing the raw data stream in transmission through the Internet of Things interface and using the header feature code of the raw data stream. Based on the device identifier, a pre-stored message structure template is matched, and the original data stream is parsed to extract a set of heterogeneous data fields. The heterogeneous data field set is cleaned and normalized to obtain the data items to be processed; Align the data items to be processed with a preset data format to construct a mapping table; Based on the mapping table, the format of the data items to be processed is reconstructed to obtain standardized device data.

3. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 2, characterized in that, The process employs a graph database structure to process the standardized device data, constructing a multi-level topology. It generates an interconnected data graph by associating basic device information with real-time status through node attributes, including: The device identifier is extracted from the standardized device data to generate a graph node entity, and a unique graph database index key is assigned to the graph node entity using a hash algorithm; Based on the graph database index key, the graph node entities are quickly retrieved, directional edges are defined, and a multi-level topology is constructed based on the directional edges. Based on the node attributes of the multi-level topology, associate the device's basic information and real-time status; Extract the static attribute fields from the device's basic information and write them into the node storage area corresponding to the multi-level topology; The dynamically changing values ​​in the real-time state are overwritten into the dynamic attribute slots of the node storage area to generate an interconnected data graph.

4. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 3, characterized in that, The process involves calculating the connection strength between nodes in the interconnected data graph. If the connection strength does not reach a preset strength threshold, the corresponding node is marked as a weak connection node. Clustering and grouping optimization is then performed on the weak connection nodes to obtain an enhanced data graph, including: Traverse the interconnected data graph, extract the interaction frequency and data flow characteristics between nodes, and calculate the connection strength value based on the interaction frequency and data flow characteristics; If the connection strength value does not reach the preset strength threshold, the connection edge between the corresponding nodes is marked as a weak connection edge, all associated nodes of the weak connection edge are marked as weak connection nodes, and multiple discrete connection nodes are extracted to form a discrete node set to be optimized. Calculate the topological distance and business relevance of the discrete node set to be optimized, and construct a similarity matrix; Traverse the similarity matrix and divide the nodes whose topological distance meets the preset short distance requirement into logical groups; For the logical group, the gateway nodes within the logical group are associated with the directed edges to generate virtual aggregation nodes, and directed edges are established between the virtual aggregation nodes and each node within the logical group to form an enhanced data graph.

5. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 1, characterized in that, The process of extracting the real-time state subgraph from the enhanced data graph, processing the real-time state subgraph using a depth-first search algorithm to obtain a priority access path, and generating an optimized access sequence based on the priority access path includes: Traverse the enhanced data graph, filter active nodes and associated edges, and construct a real-time state subgraph; The interaction frequency weights of the real-time state subgraph are analyzed to generate an adjacency matrix. The adjacency matrix is ​​then input into a depth-first search algorithm to output connected traversal paths. The weights of the traversed paths are summed to obtain the path popularity value. The traversed paths whose path popularity value reaches a preset popularity threshold are determined as priority access paths. Obtain the topology depth and path popularity value of the preferred access path, generate a sorting index, and reorganize the preferred access path according to the sorting index to obtain an optimized access sequence.

6. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 1, characterized in that, The process involves obtaining business task data from the park's business management platform, integrating the optimized access sequence with the business task data to obtain a fused feature dataset. If the matching degree reflected by the fused feature dataset reaches a preset association threshold, the data storage hierarchy is dynamically adjusted to obtain hierarchical task integration data, including: The optimized access sequence is parsed to obtain the sequence topology, and the business task data is extracted to generate a task feature vector. The sequence topology is fused with the task feature vector to output a fused feature dataset, and the matching degree of the fused feature dataset is calculated. If the matching degree reaches the preset association threshold, the target storage level label is determined by combining the access frequency. Data migration instructions are generated based on the target storage tier label, and the data migration instructions are executed to obtain tiered task integration data.

7. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 1, characterized in that, The process involves performing concurrent query simulations on the integrated data of the hierarchical tasks, filtering hot data blocks based on the simulation results, and recording response latency metrics. If the response latency metrics reach a preset latency threshold, the hot data blocks are loaded into the memory-mapped space and a fast retrieval index is established to obtain the response query results, including: A virtual request queue is constructed based on the hierarchical task integration data. A simulated retrieval operation is performed according to the virtual request queue. The identifier of the associated data block, the start timestamp, the end timestamp, and the number of accesses in the hierarchical task integration data corresponding to each retrieval are recorded to obtain the response timestamp sequence and access frequency statistics. The average latency, peak latency, and percentile latency are extracted as latency statistics based on the response timestamp sequence. Filter related data blocks whose access frequency exceeds a preset frequency threshold and whose response latency exceeds a preset sub-threshold, and mark them as hot data blocks; If the latency statistics exceed the preset latency threshold, the hot data block is loaded into the memory-mapped space, an index for fast hash retrieval is established, and a query task is executed through the index to obtain the response query result.

8. The method for storing big data of smart industrial park equipment based on the Internet of Things according to claim 1, characterized in that, The process of comparing the response query result with the integrity check code of the original data stream, obtaining scheduling response improvement indicators through difference analysis, and optimizing data management configuration based on the scheduling response improvement indicators to complete data management optimization includes: Obtain the integrity check code of the response query result and the original data stream; if the response query result does not match the integrity check code, generate a difference feature vector. The scheduling log is parsed based on the difference feature vector, the response time deviation set is extracted, and the scheduling response improvement index is calculated based on the response time deviation set. Obtain the current resource allocation matrix, analyze the scheduling response improvement index, determine the adjustment range of the resource allocation matrix, and generate optimized configuration parameters; The optimized configuration parameters are applied to the data flow control layer to complete data management optimization.

9. A smart industrial park equipment big data storage system based on the Internet of Things, characterized in that, include: The data standardization module is used to collect raw data streams from park equipment through an IoT interface, and to standardize the raw data streams to obtain standardized equipment data. The graph construction module is used to process the standardized equipment data using a graph database structure, construct a multi-level topology, and generate an interconnected data graph by associating basic equipment information and real-time status through node attributes. The graph optimization module is used to calculate the connection strength between nodes in the interconnected data graph. If the connection strength does not reach a preset strength threshold, the corresponding node is marked as a weak connection node. The weak connection nodes are then clustered and optimized to obtain an enhanced data graph. The access sequence generation module is used to extract the real-time state subgraph from the enhanced data graph, process the real-time state subgraph through a depth-first search algorithm to obtain the priority access path, and generate an optimized access sequence based on the priority access path. The storage tier adjustment module is used to obtain business task data from the park business management platform, merge the optimized access sequence and the business task data to obtain a fused feature dataset. If the matching degree reflected by the fused feature dataset reaches a preset association threshold, the data storage tier is dynamically adjusted to obtain hierarchical task integration data. The low-latency query module is used to perform concurrent query simulation on the integrated data of the hierarchical task, filter hot data blocks based on the simulation results and record the response latency index. If the response latency index reaches the preset latency threshold, the hot data block is loaded into the memory-mapped space and a fast retrieval index is established to obtain the response query results. The management optimization module is used to compare the response query result with the integrity check code of the original data stream, obtain scheduling response improvement indicators through difference analysis, optimize data management configuration based on the scheduling response improvement indicators, and complete data management optimization.

Images