An ontology-based ground data observation whole-process management platform

Abstract

The application discloses a kind of ground data observation whole process management platform based on ontology, and the application relates to data acquisition and management technical field.The application includes local storage module, local preprocessing module, ontology rule base, networking synchronization module, backup recovery module, realizes offline full data storage, completes offline data standardization and abnormal screening with local preprocessing process, avoids the data loss and redundant upload problem of traditional system, guarantees the integrity of data.Threshold value judgment accurate screening abnormal data is relied on ontology rule base, local alarm is triggered when abnormal proportion exceeds standard, helps to investigate equipment problem in time, and data standardization also reduces cloud repeated cleaning work, improves data processing efficiency.In addition, the independent offline capability of observation terminal forms "local autonomy+networking synchronization" dual mode, combined with the dynamic capacity allocation of hierarchical storage, enhances the autonomy and reliability of system in complex field environment.

Description

Technical Field

[0001] This invention relates to the field of data acquisition and management technology, specifically to an ontology-based full-process management platform for ground data observation. Background Technology

[0002] Field observation is a core means of obtaining field data in fields such as geography, ecology, and meteorology. However, the complexity of the field environment often poses challenges to data management, among which network signal interruption is a prominent pain point of current observation systems.

[0003] Existing field observation systems generally adopt a "terminal acquisition - cloud-centralized management" architecture, which is highly dependent on the cloud and lacks a robust offline data processing mechanism designed for scenarios with unstable network connections in the field. When the network is normal, this architecture can achieve real-time data upload and unified analysis. However, when network interruptions occur in the field, existing observation terminals are only equipped with temporary cache modules with limited capacity, which can only store a small amount of real-time acquired data and cannot complete core operations such as data preprocessing and filtering locally.

[0004] This design flaw leads to a series of problems: First, during network outages, observation data exceeding the terminal's cache capacity is easily lost, compromising the integrity of the observation data. Second, after network recovery, the terminal needs to upload the cached data to the cloud in batches, but due to the lack of local preprocessing, the uploaded data contains a large amount of raw data with inconsistent formats and invalid outliers, increasing the redundant load on network transmission and the data cleaning pressure on the cloud, thus reducing data processing efficiency. Third, the existing system cannot identify data anomalies in real time offline. Even if a large number of anomalies occur in the observation data, they can only be detected after the data is uploaded to the cloud, making it difficult to promptly alert observers to troubleshoot equipment malfunctions. This may result in the long-term collection of invalid data, affecting the reliability of the observation mission.

[0005] In summary, current field observation systems suffer from insufficient data storage capacity, lack of local processing capabilities, and low data management efficiency in offline scenarios, making it difficult to meet the field observation requirements for data integrity, real-time performance, and reliability. There is an urgent need for an observation data management solution that can adapt to field network fluctuation scenarios and has local storage and intelligent processing capabilities. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] To address the shortcomings of existing technologies, this invention provides an ontology-based full-process management platform for ground data observation, which solves the problems mentioned in the background section.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the present invention provides the following technical solution: a ground data observation end-to-end management platform based on ontology, comprising:

[0010] The local storage module has a built-in hierarchical local storage unit in the observation terminal for storing all data in the event of a network interruption.

[0011] The local preprocessing module is used to automatically start the local preprocessing process on the observation terminal when the network is interrupted, including data format standardization and outlier filtering.

[0012] The network synchronization module is used for incremental synchronization between the observation terminal and the cloud after the network is restored.

[0013] As a further aspect of the present invention, when the network is interrupted, the full data is stored in the following manner:

[0014] Step A1, Data Classification and Labeling:

[0015] The raw data collected by the observation terminal is divided into observation metadata and observation value data, and a unique identifier is assigned to each type of data.

[0016] Step A2, Tiered Storage Allocation:

[0017] The tiered local storage unit is divided into a temporary cache area and a persistent storage area. When the amount of data in the temporary cache area reaches 90% of the temporary cache area capacity, the data preprocessing process is triggered, and the preprocessed valid data is transferred to the persistent storage area.

[0018] As a further aspect of the present invention, the unique identifier allocation method is as follows:

[0019] Let n be the total number of observation tasks, i = 1, 2, ..., n; and let M be the metadata set corresponding to the i-th observation task. i ={M i，1 M i，2 ,……M i，k}, where k is the number of metadata fields, M i，j M represents the j-th metadata field of the i-th observation task, where j = 1, 2, ..., k; where each metadata field M i，j The unique identifier is "observation task number i + metadata field number j + acquisition timestamp";

[0020] Let V be the set of observation data corresponding to the i-th observation task. i ={V i，1 V i，2 ,……V i，g}, where g is the number of observation fields, V i，p V represents the p-th observation for the i-th observation task, where p = 1, 2, ..., g; where each observation Vi，p The unique identifier is "task number i + observation field number p + collection timestamp";

[0021] Add an offline status flag S to each data item i When the network is disconnected, S i =1; when the network recovers, S i =0.

[0022] As a further aspect of the present invention, the capacity allocation formula for the temporary cache area and the persistent storage area is as follows:

[0023]

[0024] Where: C(Z) is the total storage capacity of the hierarchical local storage unit; α is the preset temporary cache capacity ratio; C(S) is the temporary cache capacity, used to store real-time acquired unprocessed data; C(J) is the persistent storage capacity, used to store preprocessed valid data.

[0025] As a further aspect of this invention, the data format standardization method is as follows:

[0026] pass: This converts different types of observations into a unified format;

[0027] Among them: V1 i，p V(min) represents the standardized result of the p-th observation for the i-th observation task. i,p V(max) is the preset minimum value for this observation term. i,p This is the preset maximum value for this observation item;

[0028] As a further aspect of the present invention, the outlier filtering method is as follows:

[0029] Anomaly detection is performed on the standardized data based on the threshold values ​​of observation items in a pre-established ontology rule base:

[0030] First, the normal interval of the p-th observation item in the i-th observation task is denoted as [T(low)]. i，p T (high) i，p ];

[0031] If V1 i，p <T (low) i，p Or V1 i，p >T (high) i，p If so, the data item is marked as an anomaly and denoted as E. i，p =1; otherwise, it is recorded as E. i，p =0;

[0032] Subsequently, through: Calculate the percentage of outlier data R for the i-th observation task. i :

[0033] If R i If the value is greater than R(Y), a local alarm will be triggered on the observation terminal, prompting the observers to check the equipment.

[0034] If R i If the value is less than or equal to R(Y), then normal data is retained and abnormal data is removed.

[0035] Where R(Y) is the preset abnormal percentage threshold.

[0036] As a further aspect of the present invention, the incremental synchronization method is as follows:

[0037] The set of identifiers for the currently stored data sent from the observation terminal to the cloud is denoted as: ID(H) = {ID} i，1 ID i，2 ,……ID i，u}; The set of identifiers already stored in the cloud feedback is denoted as: ID(G) = {ID} j，1 ID j，2 ,……ID j，v}; where u is the number of data items stored locally on the observation terminal, and v is the number of data items stored in the cloud;

[0038] Subsequently passed: Calculate the set of identifiers ID(N) for unsynchronized data; where the symbol "-" represents the difference operation of the sets, that is, only the identifiers in ID(H) that do not appear in ID(G) are retained;

[0039] Step C2, Incremental Data Upload:

[0040] The observation terminal only uploads the dataset corresponding to ID (N). After the upload is complete, the cloud sends a confirmation signal to the observation terminal. Upon receiving the confirmation signal, the observation terminal updates the offline status flag of the corresponding data in its local storage to S. i =0, and release the corresponding persistent storage area capacity;

[0041] If the network is interrupted again during the upload process, the terminal records the uploaded subset of identifiers ID(T). After the network is restored, it will be accessed via: Recalculate the unsynchronized identifier set ID (N0); then only upload the dataset corresponding to ID (N0).

[0042] As a further aspect of the present invention, the specific method of set difference operation is as follows:

[0043] Iterate through each element of the set ID(H), and extract each identifier ID from ID(H) one by one. i，kDetermine if it exists in ID(G);

[0044] Where k = 1, 2, ..., u;

[0045] The judgment rule is: [Identify ID] i，k Each identifier ID in ID(G) i，m , to perform a complete character match;

[0046] Where m = 1, 2, ..., v;

[0047] If ID i，k If it does not appear in ID(G), then ID will be... i，k Add the difference set ID(N);

[0048] If ID i，k If the element already appears in ID (G), skip it.

[0049] Ultimately, the elements in ID(N) are the identifiers of data items that exist only on the observation terminal and are not synchronized to the cloud.

[0050] As a further aspect of the present invention, the ontology rule base is deployed on the observation terminal to provide logical judgment basis for the data preprocessing process in the offline state of the network;

[0051] Among them, the ontology rule base includes threshold determination rules that correspond one-to-one with each observation item, as well as data association rules for logical verification between multiple observation items;

[0052] The data association rules include:

[0053] Time correlation rules: Verification of consecutive observation time intervals at the same observation point;

[0054] Location association rules: Grid consistency check of the region to which the observation point belongs;

[0055] Device association rules: Collaborative verification of data from multiple sensors under the same terminal;

[0056] In the offline data preprocessing stage, the observation terminal automatically calls the ontology rule base to perform correlation verification operations on the collected observation data;

[0057] As a further aspect of the present invention, the correlation verification operation process is as follows: sequentially retrieve all rules stored in the rule base, and for the i-th observation task, retrieve the standardized observation value V1 of all observation items. i,p And related attribute information, match and verify the preset trigger conditions of the current call rule one by one;

[0058] Among them, the associated attribute information is extracted from the observation metadata, which includes the time-related attributes of the observation task, namely the continuous observation time interval of the same observation point; the location-related attributes, namely the grid number of the area to which the observation point belongs; and the equipment-related attributes, namely the equipment group number of multiple sensors under the same terminal.

[0059] If the comparison determines that the dataset for the observation task meets the triggering conditions of the rule, the processing instruction corresponding to the rule is executed immediately, that is, the dataset is marked with an associated abnormal state identifier.

[0060] As a further aspect of the present invention, the backup and recovery module is used to write all data in the persistent storage area to a pre-established backup storage partition when the data volume in the persistent storage area reaches 80% of C(J).

[0061] When persistent storage data is corrupted or lost, backup data is read from the standby storage partition for recovery. The recovered data retains the original identifier and status information.

[0062] As a further aspect of the present invention, the observation terminal refers to an embedded hardware device deployed at a field observation site, capable of data acquisition, local storage, offline preprocessing, and network synchronization, and is a key node connecting the field observation scene with the cloud.

[0063] As a further aspect of the present invention, observation metadata is used to uniquely identify observation tasks, trace data sources, and associate data attributes; observation data is the direct object of data standardization processing, anomaly screening, and ontology rule verification.

[0064] As a further aspect of the present invention, the observation item refers to a specific quantifiable physical indicator determined for a ground observation task and required to be collected by an observation terminal, and its collected value corresponds to the observation data set V in the i-th observation task. i The p-th element V i,p ;

[0065] In addition, each observation item has a unique preset attribute parameter, including the lower limit and upper limit of the value of the observation item. The lower limit is the preset minimum value V (min). i,p The upper limit of the value is the preset maximum value V (max). i,p .

[0066] As a further aspect of the present invention, a data item refers to the specific collected value and corresponding attribute information of any observation item under a single observation task.

[0067] (III) Beneficial Effects

[0068] This invention provides an ontology-based end-to-end management platform for ground data observation. Compared with existing technologies, it has the following advantages:

[0069] (1) This ontology-based ground data observation full-process management platform solves the pain point of network interruption in field observation scenarios: by using hierarchical local storage units, it realizes full data storage when the network is disconnected, avoiding the problem of batch loss caused by traditional systems that can only cache a small amount of data. At the same time, the local preprocessing process can complete data standardization and anomaly screening in offline state, which not only reduces the uploading of redundant data after reconnection, but also ensures the integrity and validity of the data, so that the management of observation data is no longer limited by network status.

[0070] (2) This ontology-based ground data observation end-to-end management platform improves the quality and processing efficiency of field observation data: by using threshold judgments based on the ontology rule base, it can accurately filter abnormal data and trigger local alarms through abnormality ratio statistics, helping observers to troubleshoot equipment problems in a timely manner and reduce interference from invalid data. Standardized processing unifies the format of different types of observation values. After subsequent network synchronization, the cloud does not need to repeatedly clean the data, which greatly shortens the data cycle from collection to usability and improves the overall efficiency of observation tasks.

[0071] (3) This ontology-based full-process management platform for ground data observation enhances the autonomy and reliability of the field observation system: the observation terminal has independent offline storage and preprocessing capabilities, no longer overly relying on cloud management, forming a dual-mode management system of "local autonomous processing + network synchronization". The dynamic allocation of the layered storage capacity not only ensures the temporary caching of real-time data, but also provides persistent storage space for effective data. Combined with the anomaly early warning mechanism, the system can operate stably in complex field environments, improving the continuity and reliability of ground observation tasks. Attached Figure Description

[0072] Figure 1 This is a system block diagram of an ontology-based ground data observation end-to-end management platform according to the present invention. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] Please see Figure 1 As shown, the embodiments of the present invention provide the following technical solutions:

[0075] As an embodiment of the present invention:

[0076] This invention is an ontology-based end-to-end management platform for ground data observation, comprising:

[0077] The local storage module has a built-in hierarchical local storage unit in the observation terminal for storing all data in the event of a network interruption.

[0078] An observation terminal is an embedded hardware device deployed at a field observation site that has the capabilities of data acquisition, local storage, offline preprocessing, and network synchronization. It is a key node connecting the field observation scene with the cloud.

[0079] The specific steps are as follows:

[0080] Step A1, Data Classification and Labeling:

[0081] The raw data collected by the observation terminal is divided into observation metadata and observation value data, and a unique identifier is assigned to each type of data.

[0082] Observation metadata does not directly characterize the physical features of the observed object, but is used to uniquely identify the observation task, trace the data source, and associate data attributes. It is the core auxiliary information to ensure the traceability and manageability of observation data. Its specific content is determined by the type of observation task, and typical fields include:

[0083] Task identification category: Observation task number, task name;

[0084] Time attribute types: observation start time, observation end time, data collection timestamp;

[0085] Equipment attribute category: Observation terminal equipment number, sensor model;

[0086] Location attribute type: latitude and longitude of the observation point, observation area number;

[0087] Observational data is the core target data of ground observation missions. It is the direct object of data standardization, anomaly screening, and ontology rule verification. Its values ​​are specific physical quantities; its specific content corresponds one-to-one with the observation items, and typical fields include:

[0088] Soil environmental monitoring data: soil temperature, soil moisture, and soil pH.

[0089] Meteorological observation data: atmospheric temperature, precipitation, and wind speed.

[0090] Hydrological observation data: water level, water flow velocity, and water turbidity.

[0091] In this embodiment, observation metadata refers to observation time and device number, and observation data refers to temperature and humidity;

[0092] The specific method is as follows:

[0093] Let the total number of observation tasks be denoted as n, i = 1, 2, ..., n;

[0094] Let M be the metadata set corresponding to the i-th observation task. i ={M i，1 M i，2 ,……M i，k}, where k is the number of metadata fields, M i，j M represents the j-th metadata field of the i-th observation task, where j = 1, 2, ..., k; where each metadata field M i，j The unique identifier is "observation task number i + metadata field number j + acquisition timestamp";

[0095] In this embodiment, M i，1 M refers to the observation time. i，2 Refers to the equipment number;

[0096] Let V be the set of observation data corresponding to the i-th observation task. i ={V i，1 V i，2 ,……V i，g}, where g is the number of observation fields, V i，p V represents the p-th observation for the i-th observation task, where p = 1, 2, ..., g; where each observation V i，p The unique identifier is "task number i + observation field number p + collection timestamp";

[0097] In this embodiment, V i，1 V represents the temperature value. i，2 Represents humidity value;

[0098] Add an offline status flag S to each data item i :

[0099] When the network is disconnected, S i =1; when the network recovers, S i =0;

[0100] Step A2, Tiered Storage Allocation:

[0101] The tiered local storage unit is divided into a temporary cache area and a persistent storage area;

[0102] The capacity allocation formulas for the temporary cache and persistent storage are as follows:

[0103]

[0104] Where: C(Z) is the total storage capacity of the hierarchical local storage unit; α is the preset temporary cache capacity ratio; C(S) is the temporary cache capacity, used to store real-time acquired unprocessed data; C(J) is the persistent storage capacity, used to store preprocessed valid data;

[0105] In this embodiment, its value range is 0.2≤α≤0.3, which is preset by the terminal hardware configuration;

[0106] When the amount of data in the temporary cache reaches 90% of C(S), the data preprocessing process is triggered, and the preprocessed valid data is transferred to the persistent storage area.

[0107] The local preprocessing module is used to automatically start the local preprocessing process on the observation terminal when the network is interrupted, including data format standardization and outlier filtering.

[0108] Step B1, Data Format Standardization:

[0109] pass: This converts different types of observations into a unified format;

[0110] Among them: V1 i，p V(min) represents the standardized result of the p-th observation for the i-th observation task. i,p V(max) is the preset minimum value for this observation term. i,p This is the preset maximum value for this observation item;

[0111] In this embodiment, the standardized data range is uniformly [0, 100], which facilitates subsequent anomaly screening;

[0112] An observation term refers to a specific quantifiable physical indicator that needs to be collected by an observation terminal for a ground-based observation task. Its collected value corresponds to the set of observation data V in the i-th observation task. i The p-th element V i,p ;

[0113] In addition, each observation item has a unique preset attribute parameter, including the lower limit and upper limit of the value of the observation item. The lower limit is the preset minimum value V (min). i,p The upper limit of the value is the preset maximum value V (max). i,p ;

[0114] Step B2, Outlier Filtering:

[0115] Anomaly detection is performed on the standardized data based on the threshold values ​​of observation items in the ontology rule base:

[0116] First, the normal interval of the p-th observation item in the i-th observation task is denoted as [T(low)]. i，pT (high) i，p ];

[0117] If V1 i，p <T (low) i，p Or V1 i，p >T (high) i，p If so, the data item is marked as an anomaly and denoted as E. i，p =1; otherwise, it is recorded as E. i，p =0;

[0118] A data item refers to the specific collected value and corresponding attribute information of any observation item under a single observation task;

[0119] Subsequently, through: Calculate the percentage of outlier data R for the i-th observation task. i :

[0120] If R i If the value is greater than R(Y), a local alarm will be triggered on the observation terminal, prompting the observers to check the equipment.

[0121] Wherein, R(Y) is a preset abnormality percentage threshold, and in this embodiment, its value range is 0.1≤R(Y)≤0.3;

[0122] If R i If the value is less than or equal to R(Y), then normal data is retained and abnormal data is removed.

[0123] This embodiment effectively solves the challenges of data storage and quality control in field network outage scenarios through the collaborative design of hierarchical local storage units and offline preprocessing modules. Hierarchical storage rationally allocates cache and persistent storage capacity, ensuring that real-time data is not lost and that storage is efficient; data classification and offline status marking lay the foundation for subsequent synchronization. Format standardization unifies data into the [0,100] range, and combined with threshold-based anomaly filtering and anomaly percentage alarm mechanisms, it can accurately remove invalid data and promptly alert to equipment failures, significantly improving the integrity and reliability of observation data in offline states.

[0124] As a second embodiment of the present invention:

[0125] In its specific implementation, compared to Embodiment 1, the technical solution of this embodiment differs only in that it further includes: a network synchronization module, used for incremental synchronization between the observation terminal and the cloud after network recovery, to avoid data loss or redundant uploading. The steps are as follows:

[0126] Step C1, Data Consistency Verification:

[0127] The set of identifiers for the currently stored data sent from the observation terminal to the cloud is denoted as:

[0128] ID(H) = {ID i，1 ID i，2 ,……ID i，u};

[0129] The set of identifiers already stored in the cloud feedback is denoted as:

[0130] ID(G) = {ID j，1 ID j，2 ,……ID j，v};

[0131] Where u is the number of data items stored locally at the observation terminal, and v is the number of data items stored in the cloud;

[0132] Subsequently passed: Calculate the ID(N) set of identifiers for unsynchronized data;

[0133] In the formula, the symbol "-" represents the difference operation of sets, that is, only the identifiers in ID(H) that do not appear in ID(G) are retained; the specific method is as follows:

[0134] Iterate through each element of the set ID(H), and extract each identifier ID from ID(H) one by one. i，k Determine if it exists in ID(G);

[0135] Where k = 1, 2, ..., u;

[0136] The judgment rule is: [Identify ID] i，k Each identifier ID in ID(G) i，m , to perform a complete character match;

[0137] Where m = 1, 2, ..., v;

[0138] In this embodiment, since the identifier is globally unique, the matching result has only two possibilities: "exists" or "does not exist".

[0139] If ID i，k If it does not appear in ID(G), then ID will be... i，k Add the difference set ID(N);

[0140] If ID i，k If the element already appears in ID (G), skip it.

[0141] Ultimately, the elements in ID(N) are the identifiers of data items that exist only on the observation terminal and are not synchronized to the cloud;

[0142] Step C2, Incremental Data Upload:

[0143] The observation terminal only uploads the dataset corresponding to ID (N). After the upload is completed, the cloud sends a confirmation signal to the observation terminal, which is marked as ACK=1.

[0144] After receiving ACK=1, the observation terminal updates the offline status flag of the corresponding data in its local storage to S. i =0, and release the corresponding persistent storage area capacity;

[0145] If the network is interrupted again during the upload process, the terminal records the uploaded subset of identifiers ID(T). After the network is restored, it will be accessed via: Recalculate the set of unsynchronized identifiers;

[0146] Upload only the dataset corresponding to ID (N0) to ensure that the data is not duplicated or lost.

[0147] Example 2 focuses on the incremental synchronization requirements after network connectivity. Through data consistency verification and incremental upload logic, it avoids the redundant transmission and data conflict problems caused by traditional full synchronization. The terminal and cloud use a difference operation on the identifier set to lock out unsynchronized data, uploading only the incremental portion, significantly reducing network bandwidth consumption and synchronization time. Simultaneously, in the event of a network interruption during synchronization, the synchronization range is recalculated by recording the already uploaded subset of identifiers, ensuring that data is not duplicated or lost. This achieves high efficiency and stability in data synchronization after network outages, guaranteeing data consistency between the terminal and the cloud.

[0148] As an embodiment of the present invention:

[0149] In its specific implementation, compared to Embodiment 1 and Embodiment 2, the technical solution of this embodiment combines the solutions of Embodiment 1 and Embodiment 2. The only difference between this embodiment and Embodiment 1 and Embodiment 2 is that in this embodiment, a lightweight ontology rule base is deployed on the observation terminal for offline preprocessing logic support. Specific implementation:

[0150] Step D1, Local storage of ontology rules:

[0151] The ontology rule base is stored in the observation terminal in the form of a structured file. The ontology rule base includes threshold determination rules that correspond one-to-one with each observation item, as well as data association rules for logical verification between multiple observation items.

[0152] The data association rules include:

[0153] Time correlation rules: Verification of consecutive observation time intervals at the same observation point;

[0154] Location association rules: Grid consistency check of the region to which the observation point belongs;

[0155] Device association rules: Collaborative verification of data from multiple sensors under the same terminal;

[0156] In this embodiment, the "correlation rule between soil moisture and temperature" is stored as follows:

[0157] Rule identifier R (GZ) t Where t=1,2,……w, and w is the total number of rules;

[0158] Rule conditions: V1 i，土壤湿度 >T (low) i，土壤湿度 And V1 i，土壤温度 >T (low) i，土壤温度 ;

[0159] Rule result: Marked as associated anomaly E1 i，L ;

[0160] Step D2, rule invocation in offline mode:

[0161] During offline data preprocessing, the ontology rule base of the observation terminal is invoked to perform correlation verification on the collected observation data;

[0162] The correlation verification operation process is as follows: Retrieve all rules stored in the rule base sequentially, and calculate the standardized observation value V1 for all observation items corresponding to the i-th observation task. i,p And related attribute information, match and verify the preset trigger conditions of the current call rule one by one;

[0163] For the dataset of the i-th observation task, determine whether it satisfies rule R(GZ). t Triggering conditions;

[0164] If the comparison determines that the dataset of the observation task meets the triggering conditions of the rule, the processing instruction corresponding to the rule is executed immediately, that is, the dataset is marked with an associated abnormal state identifier.

[0165] Among them, the associated attribute information is extracted from the observation metadata, which includes the time-related attributes of the observation task, namely the continuous observation time interval of the same observation point; the location-related attributes, namely the grid number of the area to which the observation point belongs; and the equipment-related attributes, namely the equipment group number of multiple sensors under the same terminal.

[0166] In this embodiment, rule call log L is recorded simultaneously. t ={R(GZ)} t V i The execution result is used to synchronize to the cloud for rule optimization after connecting to the network;

[0167] This embodiment, building upon embodiments one and two, introduces a lightweight ontology rule base and supports offline rule invocation, further enhancing the intelligence level of offline data preprocessing. The observation thresholds and data association rules stored in the ontology rule base not only provide accurate criteria for outlier screening but also enable multi-observation association verification, identifying association anomalies that are difficult to detect with single threshold screening. Rule invocation logs provide data support for cloud-based rule optimization after network connection, forming a closed loop of "offline verification - cloud optimization." Furthermore, the lightweight design ensures efficient operation of the ontology rule base on embedded terminals without consuming excessive hardware resources, balancing intelligence with terminal adaptability.

[0168] As an embodiment of the present invention:

[0169] In specific implementation, compared with Embodiments 1, 2, and 3, the difference between this embodiment and Embodiments 1, 2, and 3 lies only in that this embodiment also includes:

[0170] The backup and recovery module is used to prevent data loss due to local storage failures on the terminal. It sets up a local backup mechanism: when the amount of data in the persistent storage area reaches 80% of C(J), the local backup process is triggered. The local backup process is as follows:

[0171] First, check if the backup storage partition exists and is readable and writable:

[0172] If the backup storage partition is unavailable, i.e., the backup storage partition does not exist, or the backup storage partition exists but is not readable or writable, an alarm will be triggered on the monitoring terminal and the alarm log will be recorded that the backup storage partition is unavailable.

[0173] If the spare storage partition is available (i.e., it exists and is readable and writable), then calculate the remaining capacity of the spare storage partition and compare it with the total capacity of all valid data stored in the persistent storage area:

[0174] If C(b) ≥ 1.2 × C(y), then proceed to the data reading stage;

[0175] If C(b) < 1.2 × C(y), an alarm will be triggered on the observation terminal, prompting the observer to clear the spare storage partition space;

[0176] Where C(b) is the remaining capacity of the spare storage partition, and C(y) is the total effective data capacity stored in the persistent storage area;

[0177] In this embodiment, multiplying the comparison by 1.2 represents reserving 20% ​​redundant space to avoid insufficient capacity during backup;

[0178] Read the complete information of all data items in the persistent storage area one by one in ascending order of observation task number, and then write them to the spare storage partition.

[0179] The complete information includes: metadata set M i Standardized observations V1 i Offline status indicator S i Anomaly identifier E i,p .

[0180] When persistent storage data is corrupted or lost, backup data is read from the standby storage partition for recovery. The recovered data retains the original identifier and status bit information.

[0181] This fourth embodiment adds a local backup and recovery module, specifically addressing the risk of data loss due to terminal storage failures. Backup is automatically triggered when the persistent storage data reaches a preset threshold. Before backup, the availability and capacity of the backup storage partition are rigorously verified, and 20% redundancy space is reserved to ensure a stable and reliable backup process. Backup data includes complete information such as metadata, standardized observations, and status identifiers, ensuring data traceability and recoverability. An alarm mechanism for when the backup storage partition is unavailable or insufficient in capacity promptly alerts observers, further strengthening data security and providing dual storage protection for ground observation data, thus improving the fault tolerance and data security of the entire management platform.

[0182] As a fifth embodiment of the present invention:

[0183] In specific implementation, compared with Embodiment 1, Embodiment 2, Embodiment 3 and Embodiment 4, the technical solution of this embodiment is to combine the solutions of Embodiment 1, Embodiment 2, Embodiment 3 and Embodiment 4.

[0184] This embodiment integrates the core technologies of the previous four embodiments, constructing a closed-loop management system encompassing offline storage, intelligent preprocessing, incremental synchronization, and local backup. Layered storage and backup modules provide dual protection for data storage security; ontology-driven preprocessing enables precise data quality control; and the incremental synchronization mechanism optimizes data transmission efficiency. The collaborative efforts of these modules comprehensively cover all scenarios requiring ground observation data from acquisition to synchronization. Whether it's offline processing during network interruptions, backup and recovery from storage failures, or efficient synchronization after network reconnection, it achieves secure, complete, and high-quality data management, significantly improving the stability, intelligence, and fault tolerance of the entire ground data observation process. It is suitable for practical applications in complex field observation scenarios.

[0185] It should be stated that all user data collected in this application was collected with the user's consent and authorization, and the use of user data is legal and compliant, and the use and processing of user data comply with the relevant laws, regulations and standards of the relevant regions.

[0186] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0187] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0188] The above formulas are all dimensionless calculations. Dimensionless calculation involves introducing a reference benchmark, such as the maximum, minimum, standard deviation, or theoretical extreme value of a physical quantity, to transform the original physical quantity into a dimensionless relative value. This value is usually mapped to a specific interval, such as [0,1] or [-1,1], which eliminates the influence of units while preserving the relative size relationship of the physical quantities. The formula is derived from software simulation based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0189] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0190] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A ground data observation end-to-end management platform based on ontology, characterized in that, include: The local storage module has a built-in hierarchical local storage unit in the observation terminal, which is divided into a temporary cache area and a persistent storage area. This system is used for storing all data during network outages. The temporary buffer is used to store unprocessed data collected in real time, while the persistent storage area is used to store preprocessed valid data. The method is as follows: the raw data collected by the observation terminal is divided into observation metadata and observation value data, and a unique identifier is assigned to each type of data. When the amount of data in the temporary buffer reaches 90% of the temporary buffer capacity, the data preprocessing process is triggered, and the preprocessed valid data is transferred to the persistent storage area. The local preprocessing module is associated with a pre-established ontology rule base. When the network is interrupted, the observation terminal will automatically start the local preprocessing process, which includes data format standardization and outlier filtering. Data format standardization is to convert different types of observations into a unified format. Outlier filtering is to judge the anomalies of the converted data based on the threshold of the observation items in the pre-established ontology rule base. The network synchronization module is used to perform incremental synchronization between the observation terminal and the cloud after the network is restored, based on the set difference operation of the unique data identifier.

2. The ontology-based ground data observation end-to-end management platform according to claim 1, characterized in that: The unique identifier allocation method is as follows: Let n be the total number of observation tasks, and M be the metadata set corresponding to the i-th observation task. i ={M i，1 M i，2 ,……M i，k }, where i=1,2,……n, k is the number of metadata fields, M i，j M represents the j-th metadata field of the i-th observation task, where j = 1, 2, ..., k; where each metadata field M i，j The unique identifier is "observation task number i + metadata field number j + acquisition timestamp"; Let V be the set of observation data corresponding to the i-th observation task. i ={V i，1 V i，2 ,……V i，g }, where g is the number of observation fields, V i，p V represents the p-th observation for the i-th observation task, where p = 1, 2, ..., g; where each observation V i，p The unique identifier is "task number i + observation field number p + collection timestamp"; Add an offline status flag S to each data item i When the network is disconnected, S i =1; when the network recovers, S i =0.

3. The ontology-based ground data observation end-to-end management platform according to claim 1, characterized in that: in, The capacity allocation formulas for the temporary cache and persistent storage are as follows: The temporary cache capacity is equal to the total storage capacity of the tiered local storage units multiplied by the preset temporary cache capacity ratio; the persistent storage capacity is equal to the total storage capacity of the tiered local storage units minus the temporary cache capacity.

4. The ontology-based ground data observation end-to-end management platform according to claim 1, characterized in that: The data format standardization method is as follows: First, subtract the preset minimum value of the corresponding observation item from the p-th observation value, then divide by the difference between the preset maximum value and the preset minimum value of the observation item, and finally multiply the result by 100 to obtain the standardized result of the p-th observation value in the i-th observation task, which is to convert different types of observation values ​​into a unified format.

5. The ontology-based ground data observation end-to-end management platform according to claim 1, characterized in that: The outlier filtering method is as follows: Anomaly detection is performed on the standardized data based on the threshold values ​​of observation items in a pre-established ontology rule base: First, extract the normal interval of the p-th observation item of the i-th observation task, that is, the range from the lower threshold to the upper threshold of the observation item. If the standardized result of the p-th observation of the i-th observation task is less than the lower threshold of the observation item or greater than the upper threshold of the observation item, then the data item is marked as an anomaly, and the anomaly identifier is recorded as 1. Otherwise, the exception flag is recorded as 0; Subsequently, the percentage of anomalous data in the i-th observation task is obtained by dividing the number of data items marked as anomalous in that observation task by the total number of data items in that observation task. If the proportion of abnormal data is greater than the preset abnormal proportion threshold, a local alarm is triggered on the observation terminal, prompting the observer to check the equipment; if the proportion of abnormal data is less than or equal to the preset abnormal proportion threshold, normal data is retained and abnormal data is removed.

6. The ontology-based ground data observation end-to-end management platform according to claim 1, characterized in that: Incremental synchronization methods are as follows: First, the set of identifiers for the currently stored data sent from the observation terminal to the cloud is denoted as: ID(H) = {ID} i，1 ID i，2 ,……ID i，u }; The set of identifiers already stored in the cloud feedback is denoted as: ID(G) = {ID} j，1 ID j，2 ,……ID j，v }; where u is the number of data items stored locally on the observation terminal, and v is the number of data items stored in the cloud; Subsequently, by traversing each element of the set ID(H), each identifier ID in ID(H) is retrieved one by one. i，k Determine if it exists in ID(G), and identify the set of identifiers for unsynchronized data, that is, retain only the identifiers in ID(H) that do not appear in ID(G), and denot them as ID(N); where k=1,2,……u; Afterwards, the observation terminal only uploads the dataset corresponding to ID(N). Once the upload is complete, the cloud sends a confirmation signal to the observation terminal. Upon receiving the confirmation signal, the observation terminal updates the offline status flag of the corresponding data in its local storage to S. i =0, and release the corresponding persistent storage area capacity.

7. The ontology-based ground data observation end-to-end management platform according to claim 6, characterized in that: If the network is interrupted again during the upload process, the terminal records the uploaded subset of identifiers ID(T). After the network is restored, it iterates through each element of the set ID(N), takes out each identifier in ID(N) one by one, determines whether it exists in ID(T), redetermines the set of identifiers of the unsynchronized data, and records it as ID(N0). Then, it only uploads the dataset corresponding to ID(N0).

8. The ontology-based ground data observation end-to-end management platform according to claim 6, characterized in that: Each identifier ID in ID(H) i，k The method for determining whether an element exists in ID(G) is as follows: ID i，k Each identifier ID in ID(G) i，m , to perform a complete character match; where m = 1, 2, ..., v; If ID i，k If it does not appear in ID(G), then ID will be... i，k Add the difference set ID(N); If ID i，k If the element already appears in ID (G), skip it. The elements in ID(N) are the identifiers of data items that exist only on the observation terminal and have not been synchronized to the cloud.

9. The ontology-based ground data observation end-to-end management platform according to claim 5, characterized in that: The ontology rule base includes preset threshold judgment rules that correspond one-to-one with each observation item, as well as preset data association rules corresponding to logical verification between multiple observation items. It is used to provide logical judgment basis for the data preprocessing process in the offline state of the network. The data association rules include time association rules: verification of continuous observation time intervals at the same observation point; location association rules: verification of grid consistency in the area to which the observation point belongs; and device association rules: collaborative verification of multi-sensor data under the same terminal.

10. A ground data observation end-to-end management platform based on ontology according to claim 3, characterized in that: It also includes a backup and recovery module, which is used to write all data in the persistent storage area to a pre-established backup storage partition when the data volume in the persistent storage area reaches 80% of the persistent storage area capacity; and to read backup data from the backup storage partition for recovery when the data in the persistent storage area is damaged or lost.

Images