An internet of things sensor data collection integrity verification exception compensation method

Abstract

The application discloses a kind of Internet of Things sensor data acquisition integrity check exception compensation method, it is related to Internet of Things data acquisition and data integrity check technical field, for solving the problem that sensor acquisition data is missing, out of order, tampering or abnormal value is difficult to identify and reliable compensation;Write sensor identification, serial number, time, calibration value, environmental reference, network state and historical reference when collecting, generate integrity check value and continuous abstract chain;Again, abnormal identification is carried out to field structure, check value, timing, network and numerical value change, according to exception type, select compensation, retain, reject or rearrange, and recheck compensation record, improve the credibility and continuity of collection.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) data acquisition and data integrity verification technology, and more specifically, to a method for compensating for anomalies in IoT sensor data acquisition integrity verification. Background Technology

[0002] IoT sensors are widely used in industrial monitoring, environmental sensing, and equipment control, with platforms relying on continuously acquired time-series data to determine the on-site status. Existing acquisition systems mostly use field validation, threshold judgment, communication retransmission, or simple interpolation to handle missing, out-of-bounds, and delayed data, which can meet the requirements for reception and anomaly alerts. In actual operation, the same acquisition record may be affected by factors such as sensor failure, time synchronization deviation, network disturbance, data tampering, lack of environmental reference, and real-world changes in the measured object. Existing methods struggle to accurately distinguish the source of anomalies. Furthermore, directly referencing compensated data or using fixed compensation rules within continuous anomaly intervals can easily cause offsets in the transmission of compensation values, reducing the reliability of subsequent control and analysis.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for compensating for anomalies in the integrity verification of IoT sensor data acquisition, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: In a preferred embodiment, it includes: Retrieve the current data records to be compensated; Read the sensor identifier, acquisition sequence number, and time field from the current data record to be compensated, and determine the continuous abnormal interval based on the sensor identifier, the acquisition sequence number, and the time field; Generate an interval position status based on the position of the currently collected data to be compensated within the continuous abnormal interval; Read the candidate compensation basis data according to the interval position status, and generate the compensation admission result according to the candidate compensation basis data; The data on the basis for access compensation shall be retained based on the aforementioned compensation access results; The compensation path is determined based on the interval position status and the admission compensation basis data, and candidate compensation values ​​are generated according to the compensation path. Perform back-substitution verification and transitivity suppression processing on the candidate compensation values ​​to obtain back-substitution verification results and transitivity suppression results; The compensation value is determined based on the back-substitution verification result and the transmission suppression result, and the compensation value is written into the compensation acquisition data record.

[0006] In a preferred embodiment, the step of determining a continuous abnormal interval based on the sensor identifier, the acquisition sequence number, and the time field, and generating an interval position status based on the position of the currently collected data to be compensated within the continuous abnormal interval, includes: Read adjacent data collection records of the same IoT sensor based on the sensor identifier; Based on the acquisition sequence number or the time field, adjacent acquisition data records that are continuous with the current acquisition data record to be compensated and have abnormal results are merged into the continuous abnormal interval. When an adjacent data record that has not been written with an abnormal type result and whose field structure is normal, integrity verification passes, and continuous summary verification passes is read, the expansion of the continuous abnormal interval is stopped. Based on the start position, middle position, end position, or single point position of the currently collected data to be compensated within the continuous abnormal interval, the corresponding interval position status is generated.

[0007] In a preferred embodiment, the step of reading candidate compensation basis data based on the interval position status and generating compensation admission results based on the candidate compensation basis data includes: Based on the interval position status, read the original valid data collection records, the compensated data collection records, the relevant valid data collection records of IoT sensors, and the environmental reference data, and use them as the candidate compensation basis data; Source attributes are generated based on the formation source of the candidate compensation basis data; The number of compensation transmission layers is generated based on the range of the compensation basis data of the candidate compensation basis data, and the original measurement support status is generated based on whether the candidate compensation basis data can be traced back to the original valid data acquisition record. Based on the time field and installation location identifier of the candidate compensation data, generate time proximity status and location proximity status; Based on the network status data, environmental reference data acquisition status, and original valid data record distribution within the continuous abnormal interval, the transmission disturbance status, environmental coverage status, and historical continuous status are generated. Based on the source attribute, the number of compensation transmission layers, the original measurement support status, the time proximity status, the location proximity status, the transmission disturbance status, the environmental coverage status, and the historical continuity status, the candidate compensation basis data is marked as admitted, restricted admitted, or not admitted, and the compensation admission result is generated.

[0008] In a preferred embodiment, determining the compensation path based on the interval position status and the admission compensation basis data, and generating candidate compensation values ​​according to the compensation path, includes: Based on the compensation access results, the candidate compensation basis data marked as access is determined as the access compensation basis data, and the candidate compensation basis data marked as restricted access is determined as the restricted access compensation basis data. Based on the interval location status, the transmission disturbance status, the environmental coverage status, and the historical continuity status, the compensation path is determined from the compensation path based on the original effective data acquisition records, the compensation path for correcting the original change trend with environmental reference data, and the compensation path for constraining the one-sided historical change with the effective data acquisition records of relevant IoT sensors. When the transmission disturbance state is a transmission disturbance untraceable state, the position of the current data to be compensated in the same IoT sensor acquisition sequence is determined according to the acquisition sequence number and continuous summary chain value, and the compensation path is corrected according to the position. According to the compensation path, extract valid calibration collection values ​​or reference values ​​from the access compensation basis data; The restricted access compensation basis data is used to determine the direction of numerical change within the continuous abnormal interval, and the restricted access compensation basis data is prohibited from directly providing numerical values ​​for generating candidate compensation values. The candidate compensation value is generated based on the effective calibration acquisition value or the reference value and the direction of the value change.

[0009] In a preferred embodiment, performing back-substitution verification and transitivity suppression processing on the candidate compensation values ​​to obtain back-substitution verification results and transitivity suppression results, and determining the compensation value based on the back-substitution verification results and the transitivity suppression results, includes: The candidate compensation value is temporarily written into the position corresponding to the current data record to be compensated; Based on the allowable range rules, numerical change rules, position matching rules, and the correspondence of environmental reference data, the temporarily written candidate compensation values ​​are verified to generate time-backward state, spatial-backward state, and environmental-backward state, and the time-backward state, spatial-backward state, and environmental-backward state are used as the backward verification results. Based on whether the compensation basis data range corresponding to the candidate compensation value contains compensation collection data records, a state of no transmission inhibition, a state of weak transmission inhibition, or a state of strong transmission inhibition is generated, and the state of no transmission inhibition, the state of weak transmission inhibition, or the state of strong transmission inhibition is taken as the transmission inhibition result. When the transmission suppression result is a strong transmission suppression state, the corresponding candidate compensation value is removed. Based on the back-substitution verification results and the propagation suppression results corresponding to the candidate compensation values ​​that have not been eliminated, a compensation selection state is generated; Based on the selected compensation status, the compensation value is determined from the candidate compensation values ​​that have never been eliminated.

[0010] In a preferred embodiment, before acquiring the current data record to be compensated, the method further includes: A temporary record of the data collected this time is created, and the sensor identifier corresponding to the IoT sensor that performed the data collection is written into the temporary record of the data collected this time. Read the output of the sensing element of the IoT sensor to generate the original collected value, or generate a collection missing indicator when the output of the sensing element is not read; Based on the sensor identifier, the deployment configuration record, deployment location record, and calibration record are read to generate the measurement type, measurement unit, installation location identifier, and calibration parameter version identifier, and calibration acquisition values ​​are generated based on the original acquisition values; A time field is generated based on the local timestamp and the reference time provided by the time synchronization source, and a collection sequence number is generated based on the sensor identifier; Based on the time field, the installation location identifier, and the measurement type, environmental reference data is matched, and network status data and historical data reference information are read; The sensor identifier, acquisition sequence number, time field, calibration acquisition value or acquisition missing identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, environmental reference data, network status data, and historical data reference information are concatenated according to fixed field rules to obtain the acquisition text to be authenticated. The integrity check value is obtained by calculating the integrity check value based on the integrity key of the collected text to be authenticated; The current continuous digest chain value is generated based on the previous continuous digest chain value, the digest value of the text to be authenticated, and the integrity verification value. The integrity check value and the current continuous summary chain value are written into the temporary record of the current data collection to form the current data collection record.

[0011] In a preferred embodiment, after forming the data record for this collection, the method further includes: Read the data record collected in this instance and determine it as the current data record collected. Perform field structure validation on the currently collected data record according to the fixed field rules, and generate field structure validation results; The text to be verified is regenerated based on the currently collected data record, and an integrity verification result is generated based on the text to be verified and the integrity verification value. A continuous summary verification result is generated based on the previous continuous summary chain value, the summary value of the collected text to be verified, and the integrity verification value. The collection sequence number is verified according to the collection sequence rules, and a collection sequence number verification result is generated. The time field is validated according to the sampling time rules, and a time field validation result is generated. A numerical verification result is generated based on the calibration acquisition value or the acquisition missing identifier, allowable range rules, and numerical change rules; Based on the environmental reference data, the network status data, and the historical data reference information, a data status verification result is generated; An initial anomaly marker is formed based on the field structure verification result, the integrity verification result, the continuous summary verification result, the collection sequence number verification result, the time field verification result, the numerical value verification result, and the data status verification result.

[0012] In a preferred embodiment, after forming the initial anomaly marker, the method further includes: Read the currently collected data record that has been written with the initial anomaly mark, and determine the currently collected data record with the initial anomaly mark as the current collected data record to be identified; An anomaly type is generated based on the field structure verification result, the integrity verification result, the continuous summary verification result, the collection sequence number verification result, the time field verification result, the numerical value verification result, and the data status verification result. When the anomaly type involves a change in calibration acquisition value, read the historical valid acquisition data record, the relevant IoT sensor valid acquisition data record, and the environmental reference data, and correct the anomaly type based on the historical valid acquisition data record, the relevant IoT sensor valid acquisition data record, and the environmental reference data; An anomaly type result is generated based on the anomaly type, the data source involved in the identification, and the data status. Based on the anomaly type results, the current data records to be identified that require numerical supplementation, numerical substitution, sequence correction, or record removal are determined as the current data records to be compensated. Based on the sensor identifier, acquisition sequence number or time field, measurement type and installation location identifier in the current data record to be compensated, determine whether the current data record to be compensated has the basic fields for compensation positioning.

[0013] In a preferred embodiment, after writing the compensation value into the compensation acquisition data record, the method further includes: Write the record identifier of the currently collected data record to be compensated into the compensation collected data record; According to fixed field rules, the fields in the compensation data collection records that require integrity protection are concatenated to obtain the compensation collection text to be authenticated; The compensation integrity verification value is calculated based on the integrity key and written into the compensation collection data record. A compensation continuous digest chain value is generated based on the previous continuous digest chain value, the digest value of the compensation text to be authenticated and collected, and the compensation integrity verification value, and the compensation continuous digest chain value is written into the compensation collection data record; Perform compensation and re-verification on the compensation collection data records that contain the compensation integrity verification value and the compensation continuous digest chain value to obtain the compensation and re-verification result. Based on the results of the compensation and re-verification, the compensation data collection records are written into the compensation data or the non-compensable records.

[0014] The technical effects and advantages of the present invention, a method for compensating for anomalies in IoT sensor data acquisition integrity verification, are as follows: This invention ensures that the collected records have verifiable relationships from generation to output through field normalization, integrity verification values, and continuous summary chains, reducing the risk of misjudgment caused by missing, out-of-order, or tampered data. By hierarchically identifying field structure, network transmission, time sequence number, range abrupt changes, and data support status, it can more accurately distinguish between sensor acquisition anomalies and actual anomalies of the measured object. Through continuous anomaly intervals, compensation transmission layers, access criteria, and back-substitution verification, it limits the repeated propagation of compensation values, improving the stability, traceability, and engineering usability of compensation results, making it suitable for application in continuous monitoring scenarios. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating the generation and integrity encapsulation of data acquisition records in the IoT sensor data acquisition integrity verification anomaly compensation method of the present invention.

[0016] Figure 2 This is a schematic diagram illustrating the continuous abnormal interval compensation and transmission suppression of an IoT sensor data acquisition integrity verification anomaly compensation method according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] An embodiment of the present invention discloses a method for compensating for anomalies in the integrity verification of IoT sensor data acquisition, comprising: In step one, when performing an IoT sensor data acquisition, a temporary record of the acquired data is created, using this acquisition as the processing object, and the sensor identifier corresponding to the IoT sensor performing this acquisition is read. The sensor identifier is obtained from the sensor registration record and written into the temporary record of the acquired data, thus establishing a correspondence between the acquired data and the actual IoT sensor performing the acquisition.

[0019] After the sensor identifier is written, the raw sensing signal generated by the sensing element of the IoT sensor during this data acquisition is read and converted into raw acquired values. The raw acquired values ​​are the values ​​formed by the sensing element directly measuring the object being measured, and are written to the temporary record of this data acquisition along with the sensor identifier. If no sensing element output is read during this acquisition, a acquisition missing flag is written to the temporary record of this data acquisition; no raw acquired values ​​are generated, and no predicted values, compensation values, or manually filled values ​​are used to replace the raw acquired values.

[0020] After the original acquired values ​​or missing acquisition flags are written, the deployment configuration record and deployment location record of the IoT sensor are read based on the sensor identifier. The measurement type and measurement unit are obtained from the deployment configuration record, the installation location identifier is obtained from the deployment location record, and the calibration parameter version identifier is obtained from the most recent valid calibration record. The measurement type, measurement unit, installation location identifier, and calibration parameter version identifier are written to the temporary record of this acquisition data, and together with the already written sensor identifier, original acquired values, or missing acquisition flags, constitute the source information of this acquisition.

[0021] After the source information for this data acquisition is generated, the corresponding calibration parameters and calibration conversion relationships are retrieved based on the measurement type and calibration parameter version identifier. Calibration parameters are obtained from calibration records containing calibration time, calibration source, and calibration parameter version identifiers; the calibration conversion relationships are recorded in these records. If original acquired values ​​exist, they are converted according to the calibration conversion relationships recorded in the calibration record to obtain calibrated acquired values, which are then written to the temporary record of this data acquisition. If the calibration record indicates linear calibration, piecewise calibration, lookup table calibration, or nonlinear calibration, the conversion is performed according to the corresponding record. If no original acquired values ​​exist, no calibrated acquired values ​​are generated, and a data acquisition missing identifier is retained.

[0022] After calibrating the acquired values ​​or writing the missing acquisition flag, read the local timestamp generated during this acquisition and read the reference time provided by the time synchronization source. The time synchronization source includes network time synchronization, precise time synchronization, positioning time synchronization, or trusted time synchronization records. If the local timestamp can be corrected according to the time deviation rules in the time synchronization configuration record, the corrected time is written to the temporary record of this acquisition data; if a time synchronization reference cannot be obtained or time correction cannot be completed, the local timestamp is written to the temporary record of this acquisition data, and a time synchronization status flag is also written.

[0023] After the time field is written, a collection sequence number corresponding to this collection is generated. The collection sequence number is generated by incrementing the collection order of the same IoT sensor; if no collection sequence number is generated for the same IoT sensor, the collection sequence number is generated based on the receiving order of the same IoT sensor and the previously saved collection sequence number. The collection sequence number is written to the temporary record of this collection data, and together with the already written time field, it indicates the position of this collection in the collection sequence of the same IoT sensor.

[0024] After the sensor identifier, installation location identifier, measurement type, and time fields are all written, environmental reference data matching the current data acquisition is read. Environmental reference data is obtained from independent reference sensors, standard measuring instruments, or externally trusted measurement records, and is matched according to the time field, installation location identifier, and measurement type. If environmental reference data is matched, the environmental reference data or corresponding reference information is written to the temporary record of the current data acquisition; if no environmental reference data is matched, an environmental reference data missing flag is written.

[0025] After the data transmission for this acquisition is completed or a transmission status record is generated, the corresponding network status data is read. Network status data is obtained from the communication detection results during this data transmission process. These results include link connectivity status, transmission confirmation status, retransmission status, and signal quality status. If network status data is obtained, it is written to the temporary record of this acquisition; if network status data is not obtained, a missing network status data flag is written.

[0026] After the acquisition sequence number and time fields are written, previously saved data from the same IoT sensor is read based on the sensor identifier, and related IoT sensor data is read based on the measurement type and installation location identifier. The read historical data is written to the temporary record of this acquisition data in the form of a historical record identifier, time range, or summary value. If no referable historical data is read, a historical verification data missing identifier is written.

[0027] After writing the temporary data record for this collection into the aforementioned fields, the sensor identifier, collection sequence number, time field, calibration collection value or missing collection identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, time synchronization status identifier, environmental reference data or missing environmental reference data identifier, network status data or missing network status data identifier, and historical data reference information or missing historical verification data identifier are standardized and concatenated according to fixed field names, fixed field order, fixed encoding format, and fixed numerical representation rules to obtain the collection text to be certified. Historical data uses reference information for concatenation, and historical values ​​of uncertain origin are not directly written into the collection text to be certified.

[0028] After the text to be authenticated is generated, the integrity key bound to the IoT sensor is read based on the sensor identifier. A key-based message authentication algorithm is then used to calculate the integrity check value from the text to be authenticated. The integrity key is obtained from the sensor registration record or deployment key configuration record. The integrity check value is written to the temporary record of this data collection. When using a digital signature, the same text to be authenticated is used as the signature object, and the signature result is written to the temporary record of this data collection as the integrity check value.

[0029] After the integrity verification value is written, the continuous digest chain value is read from an already encapsulated data record on the same IoT sensor. This continuous digest chain value, the digest value of the data to be authenticated, and the current integrity verification value are then concatenated in a fixed order and used to calculate the current continuous digest chain value. If this data is the first data record for this IoT sensor, the initial digest chain value generated during sensor registration is used in the digest calculation. The current continuous digest chain value is then written to the temporary record of this data.

[0030] After the continuous digest chain value is written, the temporary record of the collected data is designated as the current collected data record. This current collected data record includes sensor identifier, acquisition sequence number, time field, source reference of the original acquired value, calibration acquired value or acquisition missing identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, time synchronization status identifier, environmental reference data or environmental reference data missing identifier, network status data or network status data missing identifier, historical data reference information or historical verification data missing identifier, integrity check value, and continuous digest chain value, such as... Figure 1 As shown.

[0031] In step two, the data record for this acquisition generated in step one is read and designated as the current data record. The current data record is the object of processing in this step, and its fields include sensor identifier, acquisition sequence number, time field, source reference of the original acquisition value, calibration acquisition value or acquisition missing identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, time synchronization status identifier, environmental reference data or environmental reference data missing identifier, network status data or network status data missing identifier, historical data reference information or historical verification data missing identifier, integrity check value, and continuous summary chain value.

[0032] After the currently collected data record is read, the fields in the record are checked for existence and format according to the fixed field names, fixed field order, fixed encoding format, and fixed numerical representation rules adopted in step one. Field existence check refers to checking whether the fields that should be written into the currently collected data record exist; field format check refers to checking whether the encoding format, field order, and numerical representation of existing fields conform to the rules adopted in step one. If there are missing fields, incorrect field order, incorrect field encoding format, or incorrect numerical representation, a field structure anomaly marker is written to the currently collected data record; if none of the above situations exist, a field structure normal marker is written to the currently collected data record.

[0033] After the field structure is correctly marked and written, following the field range and concatenation rules used in step one when forming the text to be authenticated, the sensor identifier, acquisition sequence number, time field, calibration acquisition value or acquisition missing identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, time synchronization status identifier, environmental reference data or environmental reference data missing identifier, network status data or network status data missing identifier, and historical data reference information or historical verification data missing identifier are re-extracted from the current acquisition data record. The extracted fields are then re-normalized and concatenated to obtain the text to be verified. The text to be verified is a regenerated text of the fields already protected for integrity in the current acquisition data record, used to verify its correspondence with the integrity verification values ​​already written in the current acquisition data record.

[0034] After the collected text to be verified is formed, the integrity key bound to the IoT sensor is read based on the sensor identifier in the current collected data record. The same key-based message authentication algorithm as in step one is then used to calculate the integrity verification value to be compared. This integrity verification value is a recalculated value obtained in this step based on the current collected data record. The integrity verification value to be compared is compared with the integrity verification value already written in the current collected data record. If they match, an integrity verification pass flag is written to the current collected data record; otherwise, an integrity verification failure flag is written. If the integrity verification value in the current collected data record is formed using a digital signature, the corresponding signature verification public key is read based on the sensor identifier. This signature verification public key is the public key used to verify the signature result of the IoT sensor. Signature verification is then performed using the collected text to be verified and the integrity verification value in the current collected data record. If the signature verification passes, an integrity verification pass flag is written; if the signature verification fails, an integrity verification failure flag is written.

[0035] After the integrity verification pass flag is written, the previous data record from the same IoT sensor that has already completed integrity verification and been saved is read, and the previous continuous summary chain value is obtained from that data record. The previous continuous summary chain value is the continuous summary chain value written in the previous saved data record from the same IoT sensor. The previous continuous summary chain value, the summary value of the data to be verified, and the integrity verification value in the current data record are concatenated in the fixed order used in step one, and the summary of the concatenation result is calculated to obtain the continuous summary chain value to be compared. The continuous summary chain value to be compared is the continuous summary chain value recalculated in this step. The continuous summary chain value to be compared is compared with the continuous summary chain value already written in the current data record; if they match, a continuous summary verification pass flag is written to the current data record; if they do not match, a continuous summary verification failure flag is written to the current data record. If the current data record is the first data record of the IoT sensor, the initial summary chain value formed during sensor registration is read, and this initial summary chain value is used in the calculation of the continuous summary chain value to be compared.

[0036] After the continuous digest verification pass flag or continuous digest verification failure flag is written, the system reads the previous data acquisition record of the same IoT sensor that has been saved and has a normal field structure, based on the sensor identifier in the current data acquisition record. It then retrieves the previous acquisition sequence number and the previous time field from that data acquisition record. The previous acquisition sequence number is the acquisition sequence number in the previous data acquisition record of the same IoT sensor that has been saved and has a normal field structure, and the previous time field is the time field in the previous data acquisition record of the same IoT sensor that has been saved and has a normal field structure.

[0037] After obtaining the previous acquisition sequence number and the previous time field, the sequence number in the current acquisition data record is checked for continuity according to the acquisition sequence rules in the sensor deployment configuration record. The acquisition sequence rules are the generation rules for the same IoT sensor acquisition sequence number recorded in the sensor deployment configuration record. If the current acquisition sequence number continues from the previous acquisition sequence number according to the acquisition sequence rules, a consecutive acquisition sequence number marker is written into the current acquisition data record; if the current acquisition sequence number is greater than the acquisition sequence number that should appear according to the acquisition sequence rules, a jump acquisition sequence number marker is written into the current acquisition data record; if the current acquisition sequence number is equal to the previous acquisition sequence number, a duplicate acquisition sequence number marker is written into the current acquisition data record; if the current acquisition sequence number is less than the previous acquisition sequence number, a fallback acquisition sequence number marker is written into the current acquisition data record.

[0038] After the sequence number verification result is written, the time field in the current data acquisition record is checked for time consistency according to the sampling time rules in the sensor deployment configuration record. The sampling time rules are the sampling time interval, allowable time deviation, and time increment method recorded in the sensor deployment configuration record. If the current time field meets the sampling time rules, a time consistency flag is written to the current data acquisition record; if the current time field is earlier than the previous time field and does not fall within the allowable time deviation range, a time rollback flag is written to the current data acquisition record; if the interval between the current time field and the previous time field does not conform to the sampling time rules, a sampling time interval exception flag is written to the current data acquisition record; if the current time field is inconsistent with the acquisition sequence corresponding to the acquisition number, an acquisition timing inconsistency flag is written to the current data acquisition record.

[0039] In this embodiment, the acquisition sequence rules, sampling time rules, allowable range rules, numerical change rules, position matching rules, and environmental reference data correspondence can all be pre-written into the sensor deployment configuration record or calibration record, and read when forming field structure verification results, acquisition sequence number verification results, time field verification results, numerical verification results, data status verification results, back-substitution verification results, and compensation and re-verification results.

[0040] The data acquisition sequence rule is used to determine whether the acquisition sequence numbers of the same IoT sensor are consecutive. For example, if the previous acquisition sequence number of the same IoT sensor is N, and the acquisition sequence number increment step recorded in the sensor deployment configuration record is 1, then when the current acquisition sequence number is N+1, a consecutive acquisition sequence number marker is generated; when the current acquisition sequence number is greater than N+1, a jump acquisition sequence number marker is generated; when the current acquisition sequence number is equal to N, a duplicate acquisition sequence number marker is generated; and when the current acquisition sequence number is less than N, a backtracking acquisition sequence number marker is generated.

[0041] Sampling time rules are used to determine whether the time field of the same IoT sensor conforms to the sampling interval. For example, if the sampling time interval recorded in the sensor deployment configuration record is 60 seconds, the allowed time deviation is 5 seconds, and the previous time field is 10:00:00, then a time consistency flag is generated when the current time field is between 10:00:55 and 10:01:05; a time backtracking flag is generated when the current time field is earlier than the previous time field and is not within the allowed time deviation range; a sampling time interval abnormality flag is generated when the interval between the current time field and the previous time field does not conform to the sampling time rules; and a sampling time sequence inconsistency flag is generated when the current time field is inconsistent with the sampling order represented by the sampling sequence number.

[0042] The allowable range rule is used to determine whether the calibrated acquired value falls within the permissible range for the corresponding measurement type and unit. For example, when the measurement type is temperature and the unit is Celsius, the allowable range rule recorded in the sensor deployment configuration record or calibration record can be -40 to 85; a normal range flag is generated when the current calibrated acquired value is between -40 and 85; a abnormal range flag is generated when the current calibrated acquired value is less than -40 or greater than 85. When the measurement type is humidity and the unit is percentage, the allowable range rule can be 0 to 100; a normal range flag is generated when the current calibrated acquired value is between 0 and 100; a abnormal range flag is generated when the current calibrated acquired value is less than 0 or greater than 100.

[0043] Numerical variation rules are used to determine whether the change between the current calibrated acquisition value and the previous calibrated acquisition value conforms to the allowable range or pattern of numerical variation between adjacent acquisition times for the corresponding measurement type. For example, when the measurement type is temperature and the sampling interval is 60 seconds, the numerical variation rules recorded in the sensor deployment configuration record can be: the change in the calibrated acquisition value between adjacent acquisition times does not exceed 3; a normal numerical variation marker is generated when the previous calibrated acquisition value is 20 and the current calibrated acquisition value is 21.5; a suspected numerical abrupt change marker is generated when the current calibrated acquisition value is 30. When the measurement type is humidity and the sampling interval is 60 seconds, the numerical variation rules can be: the change in the calibrated acquisition value between adjacent acquisition times does not exceed 10; a suspected numerical abrupt change marker is generated if this range is exceeded.

[0044] Location matching rules are used to determine the correspondence between the installation locations of valid data records acquired by relevant IoT sensors and the current data record to be compensated. For example, if the installation location of the current data record to be compensated is A1, and the installation location of the valid data record acquired by the relevant IoT sensor is also A1 or adjacent to A2, a "proximity" state is generated. If the installation location of the valid data record acquired by the relevant IoT sensor is not adjacent to A1 but belongs to the same measurement object, a "traceable" state is generated. If the correspondence between installation locations cannot be determined, an "uncertain" state is generated. Location matching rules can also record the corresponding ratio between adjacent installation locations. This ratio is used to generate the compensation path and spatial regression state for constraining unilateral historical changes in the valid data records acquired by the relevant IoT sensors.

[0045] The environmental reference data correspondence is used to determine whether there is a correspondence between the environmental reference data and the currently collected data record to be compensated, based on the same time field, installation location identifier, or measurement type. For example, if the current collected data record to be compensated has a measurement type of temperature, an installation location identifier of A1, and a time field of 10:02:00, and the environmental reference data also has a measurement type of temperature, a source location that is the same as or adjacent to A1, and a reference time between 10:01:55 and 10:02:05, then the environmental reference data is determined to correspond to the current collected data record to be compensated, and this is used to generate the environmental back-substitution status during the back-substitution verification. If the environmental reference data has a different measurement type, a non-corresponding source location, or a non-corresponding reference time, it is not used to directly generate an environmental back-substitution pass status.

[0046] After the sequence number verification result and time consistency verification result are written, the time synchronization status flag, environment reference data missing flag, network status data missing flag, historical verification data missing flag, and network status data in the current data collection record are read. The time synchronization status flag indicates whether the time field of this collection was corrected by the time synchronization source. The environment reference data missing flag indicates that the current data collection record did not match the environment reference data. The network status data missing flag indicates that the current data collection record did not obtain the communication detection results during this data transmission process. The historical verification data missing flag indicates that the current data collection record did not read any referable historical data. The network status data indicates the link connectivity status, transmission confirmation status, retransmission status, and signal quality status during this data transmission process.

[0047] After the above states are read, the data status of the currently collected data record is verified. If the time synchronization status flag indicates that a time reference cannot be obtained or time correction cannot be completed, a time synchronization anomaly flag is written to the current collected data record. If an environmental reference data missing flag exists, an environmental reference data missing flag is written to the current collected data record. If a network status data missing flag exists, a network status data missing flag is written to the current collected data record. If a historical verification data missing flag exists, a historical verification data missing flag is written to the current collected data record. If network status data indicates that there is a link interruption, unacknowledged transmission, retransmission, or signal quality that does not meet the communication status rules in the sensor deployment configuration record, a network transmission anomaly flag is written to the current collected data record. The communication status rules are the allowed range of communication statuses recorded in the sensor deployment configuration record.

[0048] After the data status verification result is written, it is determined whether a calibration acquisition value exists in the current data acquisition record. If a calibration acquisition value exists, the allowable range rule recorded in the sensor deployment configuration record or calibration record is read based on the measurement type, measurement unit, and calibration parameter version identifier in the current data acquisition record. The allowable range rule is the range of values ​​that calibration acquisition values ​​are allowed to appear under the corresponding measurement type and measurement unit. If the calibration acquisition value is within the allowable range rule, a normal value range marker is written to the current data acquisition record; if the calibration acquisition value is not within the allowable range rule, a value range exception marker is written to the current data acquisition record. If a missing acquisition marker exists in the current data acquisition record but no calibration acquisition value exists, a missing acquisition value marker is written to the current data acquisition record.

[0049] After the numerical range verification result is written, if the current data record contains a calibrated acquisition value and the previous data record for the same IoT sensor contains a previously saved data record with a normal numerical range, a numerical change consistency verification is performed between the current and previous calibrated acquisition values ​​based on the numerical change rules recorded in the sensor deployment configuration record or calibration record. The numerical change rules define the allowed range or pattern of numerical change for the corresponding measurement type between adjacent acquisition times. If the change between the current and previous calibrated acquisition values ​​conforms to the numerical change rules, a "normal numerical change" flag is written to the current data record; if the change does not conform to the numerical change rules, a "suspected numerical change" flag is written to the current data record. If the previous calibrated acquisition value or the numerical change rules are not obtained, a "numerical change not determined" flag is written to the current data record.

[0050] After the above verification flags are written, the initial anomaly flags for the current acquired data record are formed based on the field structure anomaly flags, integrity verification failure flags, continuous summary verification failure flags, acquisition sequence number jump flags, acquisition sequence number duplicate flags, acquisition sequence number rollback flags, time rollback flags, sampling time interval anomaly flags, acquisition timing inconsistency flags, time synchronization anomaly flags, missing environmental reference data flags, missing network status data flags, missing historical verification data flags, network transmission anomaly flags, missing acquisition value flags, numerical range anomaly flags, and suspected numerical mutation flags already written in the current acquired data record. The initial anomaly flags are a set of verification results written into the current acquired data record, recording the anomalies of the current acquired data record in terms of field structure, integrity, continuous summary, acquisition sequence number, time field, data status, numerical range, and numerical changes.

[0051] In step three, the currently acquired data records with initial anomaly markers written in step two are read, and the currently acquired data records with initial anomaly markers are identified as the current acquired data records to be identified. The current acquired data records to be identified are the acquired data records whose anomaly types need to be determined in this step. Anomaly type is a classification result of the source and manifestation of anomalies in the current acquired data records to be identified, including field structure anomaly types, integrity anomaly types, continuous summary anomaly types, missing acquired values ​​types, acquired sequence number anomaly types, time field anomaly types, network transmission anomaly types, numerical range anomaly types, suspected numerical mutation types, sensor acquisition anomaly types, measured object status anomaly types, and insufficient data status types.

[0052] After the current data record to be identified is read, the system first reads the field structure anomaly flag, integrity verification failure flag, and continuous summary verification failure flag. If a field structure anomaly flag exists, the anomaly type of the current data record is written as a field structure anomaly type. A field structure anomaly type indicates that the current data record contains missing fields, incorrect field order, incorrect field encoding format, or incorrect numerical representation. If an integrity verification failure flag exists, the anomaly type of the current data record is written as an integrity anomaly type. An integrity anomaly type indicates that the text to be verified in the current data record does not correspond to the integrity verification value. If a continuous summary verification failure flag exists, the anomaly type of the current data record is written as a continuous summary anomaly type. A continuous summary anomaly type indicates that the summary chain relationship between the current data record and a previously verified and saved data record on the same IoT sensor does not correspond.

[0053] If field structure anomalies, integrity anomalies, or continuous summary anomalies have already been written, continue reading the sensor identifier, acquisition sequence number, and time fields from the current data record to be identified, and retain other initial anomaly markers already written to the current data record. If an integrity anomaly or continuous summary anomaly exists in the current data record to be identified, the calibration acquisition value in that record will not be considered a valid acquisition value for statistical testing and spatiotemporal prediction fusion identification. Valid acquisition values ​​refer to calibration acquisition values ​​from data records with normal field structure, passed integrity verification, and passed continuous summary verification.

[0054] After writing the above-mentioned anomaly types, read the missing acquisition value markers from the current data record to be identified. If a missing acquisition value marker exists, write the anomaly type of the current data record to be identified as the missing acquisition value type. The missing acquisition value type indicates that no sensor output was read in the current data record to be identified, and no original acquisition value or calibration acquisition value was formed. If the current data record to be identified also has a network transmission anomaly marker, write both the missing acquisition value type and the network transmission anomaly type into the current data record to be identified; if the current data record to be identified does not have a network transmission anomaly marker, only the missing acquisition value type is written.

[0055] After identifying the missing data type, the system reads the data acquisition sequence number jump marker, duplicate data acquisition sequence number marker, and backtracking data acquisition sequence number marker from the current data acquisition record to be identified. It also reads the previous data acquisition record from the same IoT sensor that has been saved and has a normal field structure, based on the sensor identifier. If a data acquisition sequence number jump marker exists, the anomaly type of the current data acquisition record to be identified is written as a suspected missing data acquisition sequence number type. A suspected missing data acquisition sequence number type indicates that there is an unseen data acquisition sequence number between the current data acquisition record and the previous data acquisition sequence number. If a duplicate data acquisition sequence number marker exists, the anomaly type of the current data acquisition record to be identified is written as a duplicate data acquisition sequence number type. A duplicate data acquisition sequence number type indicates that the current data acquisition sequence number is the same as a data acquisition sequence number already saved by the same IoT sensor. If a backtracking data acquisition sequence number marker exists, the anomaly type of the current data acquisition record to be identified is written as a backtracking data acquisition sequence number type. A backtracking data acquisition sequence number type indicates that the current data acquisition sequence number is less than the data acquisition sequence number in the previous data acquisition record from the same IoT sensor that has a normal field structure.

[0056] After writing the acquisition sequence number anomaly type, the system reads the time rollback flag, sampling interval anomaly flag, and acquisition timing inconsistency flag from the current data record to be identified. Based on the sensor identifier, it then reads the previous time field from the previous saved data record with a normal field structure from the same IoT sensor. If a time rollback flag exists, the anomaly type of the current data record to be identified is written as the time field rollback type. The time field rollback type indicates that the time field of the current data record to be identified is earlier than the previous time field and is not within the allowable time deviation range. If a sampling interval anomaly flag exists, the anomaly type of the current data record to be identified is written as the sampling interval anomaly type. The sampling interval anomaly type indicates that the interval between the time field and the previous time field of the current data record to be identified does not conform to the sampling time rules in the sensor deployment configuration record. If an acquisition timing inconsistency flag exists, the anomaly type of the current data record to be identified is written as the acquisition timing inconsistency type. The acquisition timing inconsistency type indicates that the acquisition order represented by the acquisition sequence number in the current data record to be identified is inconsistent with the time order represented by the time field.

[0057] After the sequence number and time field anomaly types are written, the network transmission anomaly flag and network status data in the current data record to be identified are read. If the network status data shows link interruption, unacknowledged transmission, retransmission, or signal quality not meeting the communication status rules in the sensor deployment configuration record, the anomaly type of the current data record to be identified is written as the network transmission anomaly type. The network transmission anomaly type indicates that the current data record to be identified has a communication status anomaly during transmission. If the current data record to be identified also has a suspected missing sequence number, a duplicate sequence number, a time field rollback, or an inconsistent acquisition timing, the network transmission anomaly type is written together with the corresponding sequence number anomaly type or time field anomaly type into the current data record to be identified.

[0058] After writing the network transmission anomaly type, the system reads the time synchronization anomaly flag, environmental reference data missing flag, network status data missing flag, and historical verification data missing flag from the current data record to be identified. If a time synchronization anomaly flag exists, the data status of the current data record to be identified is written as "Time Synchronization Insufficient." This indicates that the time field in the current data record has not been corrected according to the time synchronization source. If an environmental reference data missing flag exists, the data status of the current data record to be identified is written as "Environmental Reference Data Insufficient." This indicates that the current data record to be identified has not obtained environmental reference data that matches the time field, installation location identifier, and measurement type. If a network status data missing flag exists, the data status of the current data record to be identified is written as "Network Status Data Insufficient." This indicates that the current data record to be identified has not obtained communication detection results during this data transmission process. If a historical verification data missing flag exists, the data status of the current data record to be identified is written as "Historical Verification Data Insufficient." This indicates that the current data record to be identified has not obtained any referable historical data. If the current data record to be identified only has the above-mentioned data insufficiency type and no other abnormal types, the abnormal type of the current data record to be identified will be written as the data insufficiency type.

[0059] After the data status is written, it is determined whether a calibration acquisition value exists in the current data record to be identified. If no calibration acquisition value exists and there are missing acquisition value types, numerical range identification, statistical testing, and spatiotemporal prediction fusion identification are not performed. If a calibration acquisition value exists and the current data record to be identified does not have integrity anomaly types, continuous summary anomaly types, or field structure anomaly types, the numerical range anomaly markers and suspected numerical mutation markers in the current data record to be identified are read, and the measurement type, measurement unit, installation location identifier, and time field in the current data record to be identified are also read.

[0060] After the numerical range anomaly flag is read, the permissible range rules recorded in the sensor deployment configuration record or calibration record are read according to the measurement type, measurement unit, and calibration parameter version identifier. If the current data record to be identified has a numerical range anomaly flag, and the calibrated acquisition value is not within the permissible range rules, the anomaly type of the current data record to be identified is written as the numerical range anomaly type. The numerical range anomaly type indicates that the calibrated acquisition value in the current data record to be identified exceeds the allowable numerical range for the corresponding measurement type and measurement unit.

[0061] After an abnormal numerical range is written, or if a suspected numerical abrupt change is detected in the current data record to be identified, the system reads historical valid data records of the same IoT sensor, relevant valid data records of related IoT sensors, and environmental reference data based on the sensor identifier, measurement type, installation location identifier, and time field. Historical valid data records are data records of the same IoT sensor that were saved before the current time field, have a normal field structure, pass integrity verification, pass continuous summary verification, and have a normal numerical range. Relevant valid data records of related IoT sensors are data records that have the same measurement type, are adjacent to the current data record to be identified, or are related to the measurement object, and have a normal field structure, pass integrity verification, pass continuous summary verification, and have a normal numerical range. Environmental reference data consists of independent reference sensor data, standard measuring instrument data, or externally trusted measurement records obtained in step one by matching the time field, installation location identifier, and measurement type.

[0062] After the historical valid data acquisition records are read, the continuous calibration acquisition values ​​prior to the current time field are extracted in chronological order. Then, based on the numerical change rules recorded in the sensor deployment configuration record or calibration record, a rule verification is performed between the current calibration acquisition value and the continuous calibration acquisition values ​​in the historical valid data acquisition records. Rule verification refers to determining whether the current calibration acquisition value conforms to the acquisition constraints of the corresponding measurement type based on the allowable range rules, numerical change rules, and sampling time rules. If the current calibration acquisition value does not conform to the allowable range rules or the numerical change rules, the numerical range anomaly type or the suspected numerical mutation type is retained; if the current calibration acquisition value conforms to both the allowable range rules and the numerical change rules, the parts of the numerical range anomaly type or the suspected numerical mutation type that are inconsistent with the rule verification results are deleted.

[0063] After rule verification is completed, statistical checks are performed on the continuous calibration values ​​in the historical valid data collection records. Statistical checks involve calculating the historical center value and historical fluctuation range based on the continuous calibration values ​​of the same IoT sensor before the current time field, and determining whether the current calibration value deviates from the historical center value and historical fluctuation range. The historical center value is the representative value of the continuous calibration values ​​in the historical valid data collection records, and the historical fluctuation range is the allowable range of variation of the continuous calibration values ​​around the historical center value in the historical valid data collection records; the historical center value and historical fluctuation range are obtained according to the statistical methods recorded in the sensor deployment configuration record or calibration record. If the current calibration value does not deviate from the historical center value and historical fluctuation range, a historical statistical consistency flag is written into the current data collection record to be identified; if the current calibration value deviates from the historical center value and historical fluctuation range, a historical statistical deviation flag is written into the current data collection record to be identified.

[0064] After historical statistical consistency markers or historical statistical deviation markers are written, the relevant valid data acquisition records from IoT sensors and environmental reference data are read, and spatiotemporal prediction fusion identification is performed on the current calibration acquisition value. Spatiotemporal prediction fusion identification refers to simultaneously using historical valid data acquisition records in the time dimension, relevant valid data acquisition records from IoT sensors in the spatial dimension, and environmental reference data to form a reference acquisition value corresponding to the current time field, and then comparing the current calibration acquisition value with the reference acquisition value. The reference acquisition value is obtained according to the prediction method recorded in the sensor deployment configuration record or calibration record; the prediction method includes time trend prediction based on historical valid data acquisition records, spatial correlation prediction based on relevant valid data acquisition records from IoT sensors, and reference comparison prediction based on environmental reference data. When multiple prediction sources exist, the participation degree of each prediction source is determined based on the data availability status, installation location distance, time matching degree, and historical error records, forming a reference acquisition value. Historical error records are records of differences between previously saved prediction reference values ​​and actual calibrated acquisition values ​​that have passed verification.

[0065] After the reference acquisition value is formed, the current calibrated acquisition value is compared with the reference acquisition value. If the difference between the current calibrated acquisition value and the reference acquisition value is within the predicted allowable difference range recorded in the sensor deployment configuration record or calibration record, a spatiotemporal prediction consistency flag is written to the current acquisition data record to be identified. If the difference between the current calibrated acquisition value and the reference acquisition value is not within the predicted allowable difference range, a spatiotemporal prediction deviation flag is written to the current acquisition data record to be identified. The predicted allowable difference range is the range of differences allowed between the current calibrated acquisition value and the reference acquisition value during the prediction identification process for the corresponding measurement type.

[0066] When both historical statistical deviation markers and spatiotemporal prediction deviation markers exist, and neither the relevant IoT sensor valid acquisition data records nor the environmental reference data show the same trend, the anomaly type of the currently identified acquisition data record is written as the sensor acquisition anomaly type. The sensor acquisition anomaly type indicates that the calibrated acquisition value in the currently identified acquisition data record deviates from the historical change state of the same IoT sensor and is not supported by relevant IoT sensor valid acquisition data records and environmental reference data.

[0067] If both historical statistical deviation markers and spatiotemporal prediction deviation markers exist, and the relevant IoT sensor data records or environmental reference data show the same trend as the current calibrated data, the anomaly type of the current data record to be identified is written as the measured object state anomaly type. The measured object state anomaly type indicates that the calibrated data value in the current data record to be identified has undergone an abnormal change, and this abnormal change is consistent with the relevant IoT sensor data records or environmental reference data within the same time range.

[0068] When historical statistical consistency markers exist while spatiotemporal prediction deviation markers exist, the spatial matching relationship is reassessed based on the relevant valid data records acquired by IoT sensors, environmental reference data, and installation location identifiers. The spatial matching relationship is the correspondence between the installation location of the IoT sensor corresponding to the currently identified data record and the installation locations of the relevant IoT sensors and the source locations of the environmental reference data. If the spatial matching relationship does not meet the location matching rules in the sensor deployment configuration record, the data status of the currently identified data record is written as a spatial reference insufficiency state. A spatial reference insufficiency state indicates that the location correspondence between the relevant valid data records acquired by IoT sensors or the environmental reference data used for spatiotemporal prediction fusion identification and the currently identified data record is insufficient. If the spatial matching relationship meets the location matching rules, the anomaly type of the currently identified data record is written as a sensor acquisition anomaly type.

[0069] When historical statistical deviation markers exist while spatiotemporal prediction consistency markers exist, the anomaly type of the currently identified data record is written as the anomaly type of the measured object's state. The spatiotemporal prediction consistency marker indicates that the current calibrated acquisition value is consistent with the reference acquisition value formed by the effective acquisition data record of the relevant IoT sensor or environmental reference data; in this case, although the current calibrated acquisition value deviates from the historical change state of the same IoT sensor, it is consistent with the spatial correlation data or environmental reference data within the same time range.

[0070] When multiple anomaly types have been written into the current data record to be identified, the anomaly type results are generated in the following order: field structure anomaly type, integrity anomaly type, continuous summary anomaly type, missing acquired value type, network transmission anomaly type, acquisition sequence number anomaly type, time field anomaly type, numerical range anomaly type, sensor acquisition anomaly type, measured object status anomaly type, and insufficient data status type. The anomaly type results are the classification results written into the current data record to be identified, including the anomaly type corresponding to the current data record, the data source involved in the identification, the rule source involved in the identification, the range of historical valid data records, the range of relevant IoT sensor valid data records, the environmental reference data source, and the data status.

[0071] In step four, the currently identified data records with anomaly type results from step three are read, and those requiring numerical supplementation, numerical substitution, sequence correction, or record removal are identified as the currently compensated data records. The currently compensated data records are the data records processed in this step based on the anomaly type results. Anomaly compensation refers to the corresponding processing of missing values, abnormal values, duplicate records, or time-series anomaly records in the currently compensated data records according to the anomaly type results. Compensated data refers to the data records or processing results that have undergone anomaly compensation and have been verified again after compensation.

[0072] After the current data record to be compensated is read, the sensor identifier, acquisition sequence number, time field, calibration acquisition value or acquisition missing identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, environmental reference data or environmental reference data missing identifier, network status data or network status data missing identifier, historical data reference information or historical verification data missing identifier, and the anomaly type result written in step three are read and processed in the following order: field structure anomaly type, integrity anomaly type, continuous summary anomaly type, acquisition value missing type, network transmission anomaly type, acquisition sequence number anomaly type, time field anomaly type, numerical range anomaly type, sensor acquisition anomaly type, measured object status anomaly type, and data status insufficiency type.

[0073] After the anomaly type result is read, it is determined whether the current data record to be compensated has the basic fields required for compensation positioning. The basic fields for compensation positioning include sensor identifier, acquisition sequence number or time field, measurement type, and installation location identifier. The sensor identifier is used to determine the IoT sensor to which the current data record to be compensated belongs; the acquisition sequence number or time field is used to determine the position of the current data record in the acquisition sequence; the measurement type is used to determine the category of physical quantity that needs compensation; and the installation location identifier is used to match relevant valid IoT sensor data records and environmental reference data. If any of the sensor identifier, measurement type, or installation location identifier is missing, or if the acquisition sequence number and time field are both unusable, the current data record to be compensated is written to the uncompensable record. An uncompensable record refers to a data record that does not generate a compensation value due to insufficient basic fields required for compensation positioning.

[0074] It should be noted that when the same IoT sensor continuously exhibits missing, incomplete, or continuous summary anomalies in its data collection, there may already be compensated data records generated by these anomaly compensations in the data collection records before and after the current data record to be compensated. Although the compensated data records can be verified after compensation, their calibrated data values ​​are not directly measured by the sensing element, but are calculated from historical valid data records, valid data records from relevant IoT sensors, or environmental reference data.

[0075] At this point, if adjacent data records are used directly to generate new compensation values, the compensation value generated in the previous compensation may be used again in the subsequent compensation. The closer the current data record to be compensated is to the middle of the continuous anomaly interval, the fewer original valid data records can be directly referenced, and the higher the proportion of the compensation data record participating in the compensation. The continuous anomaly interval refers to the range of data records from which the same IoT sensor continuously produces anomaly results in adjacent acquisition sequence numbers or adjacent time fields. The original valid data records refer to data records whose calibration acquisition values ​​are directly measured by the sensing element, and whose field structure is normal, integrity verification passes, continuous summary verification passes, and numerical range is normal.

[0076] Meanwhile, the retransmission status, unacknowledged transmission status, and signal quality status in network status data may dynamically change within continuous abnormal intervals. Environmental reference data may also be missing in some time fields, and the continuity of historical verification data will decrease as the length of the abnormal interval increases. Therefore, the compensation basis data is not fixed and reliable. The influence of historical valid data records, relevant valid data records from IoT sensors, environmental reference data, and compensation data records on the current compensation value will vary depending on the location of the continuous abnormal interval, the completeness of the data source, the network transmission status, and the proportion of compensation data records involved.

[0077] In this situation, if the compensation value is generated according to the fixed compensation calculation method, the compensation value is likely to shift successively within the continuous abnormal interval; if the original valid data collection records and the compensation data collection records are not distinguished, the compensation result is likely to be repeatedly involved in the calculation along the collection sequence; if the participation degree of the compensation basis data is not adjusted according to the network transmission status, the missing status of environmental reference data and the continuity of historical verification data, distorted compensation data is likely to be generated when the compensation basis is insufficient or the compensation basis is occupied by continuous compensation results.

[0078] Therefore, in this embodiment, as Figure 2As shown, when the current data record to be compensated has the basic fields used for compensation positioning, a continuous anomaly interval is determined based on the sensor identifier, acquisition sequence number, time field, measurement type, and installation location identifier. A continuous anomaly interval refers to the range of data records from the same IoT sensor where consecutive anomaly type results occur in adjacent acquisition sequence numbers or adjacent time fields. To determine a continuous anomaly interval, starting from the current data record to be compensated, data records from the same IoT sensor are read before and after the current acquisition sequence number. Reading stops when a data record without anomaly type results is encountered, and whose field structure is normal, integrity verification passes, and continuous summary verification passes. The data records from which consecutive anomaly type results occur before reading stops are determined as the continuous anomaly interval.

[0079] After determining the continuous anomaly interval, the interval position status of the current data record to be compensated within the continuous anomaly interval is determined. The interval position status refers to the state of the current data record to be compensated located at the beginning, middle, or end of the continuous anomaly interval. If the acquisition sequence number of the current data record to be compensated is the same as the beginning acquisition sequence number of the continuous anomaly interval, it is written as the interval beginning status; if the acquisition sequence number of the current data record to be compensated is between the beginning and end acquisition sequence numbers of the continuous anomaly interval, it is written as the interval middle status; if the acquisition sequence number of the current data record to be compensated is the same as the end acquisition sequence number of the continuous anomaly interval, it is written as the interval end status. If the continuous anomaly interval contains only one record of the current data record to be compensated, it is written as a single-point anomaly status.

[0080] After writing the interval position status, candidate compensation basis data is read according to the sensor identifier, acquisition sequence number, time field, measurement type, and installation location identifier. Candidate compensation basis data includes data that may participate in generating compensation values ​​or determining processing methods, including original valid acquisition data records, compensated acquisition data records, relevant IoT sensor valid acquisition data records, and environmental reference data. Original valid acquisition data records are those already saved by the same IoT sensor, directly measured by the sensing element to form calibrated acquisition values, with normal field structure, passed integrity verification, passed continuous summary verification, and normal numerical range. Compensated acquisition data records are those formed after previous anomaly compensation for abnormal results and passed subsequent verification. Relevant IoT sensor valid acquisition data records are those with the same measurement type, adjacent installation location, or related measurement object as the current acquisition data record to be compensated, and with normal field structure, passed integrity verification, passed continuous summary verification, and normal numerical range. Environmental reference data are independent reference sensor data, standard measuring instrument data, or externally trusted measurement records obtained in step one by matching the time field, installation location identifier, and measurement type.

[0081] After the candidate compensation basis data is read, a source attribute is written for each candidate compensation basis data. The source attribute indicates how the candidate compensation basis data was formed. If the candidate compensation basis data is a record of original valid collected data, a direct measurement source attribute is written; if the candidate compensation basis data is a record of compensated collected data, a compensation formation source attribute is written; if the candidate compensation basis data is a record of valid collected data from relevant IoT sensors, a relevant sensor source attribute is written; if the candidate compensation basis data is environmental reference data, an environmental reference source attribute is written. After the source attribute is written, the time field, collection sequence number, installation location identifier, anomaly type result, compensation basis data range, network status data, environmental reference data missing identifier, and historical verification data missing identifier corresponding to the candidate compensation basis data are read.

[0082] After the source attribute is written, the number of compensation transfer layers is calculated for the candidate compensation basis data. The number of compensation transfer layers refers to the number of times the calibration acquisition value or reference value in the candidate compensation basis data undergoes compensation processing from the direct measurement result of the sensing element to the current candidate compensation basis data. If the candidate compensation basis data is an original valid acquisition data record, the number of compensation transfer layers is recorded as no compensation transfer. If the candidate compensation basis data is a compensated acquisition data record, the compensation basis data range in the compensated acquisition data record is read; if the compensation basis data range only contains original valid acquisition data records, relevant IoT sensor valid acquisition data records, or environmental reference data, the number of compensation transfer layers is recorded as one compensation transfer; if the compensation basis data range contains other compensated acquisition data records, one more is added to the compensation transfer layer number of the other compensated acquisition data records to obtain the compensation transfer layer number of the current candidate compensation basis data. If the number of compensation transfer layers cannot be determined, the state of unknown compensation transfer layer is written.

[0083] After writing the compensation transmission layer number, the original measurement support state is determined for the candidate compensation basis data. The original measurement support state indicates whether the candidate compensation basis data can be traced back to the state of the acquired data record formed by direct measurement by the sensing element. If the candidate compensation basis data is an original valid acquired data record, it is written to the direct support state. If the candidate compensation basis data is a compensation acquired data record and its compensation basis data range includes the original valid acquired data record, it is written to the indirect support state. If the candidate compensation basis data is a compensation acquired data record and its compensation basis data range does not include the original valid acquired data record, it is written to the no direct measurement support state. If the candidate compensation basis data is a valid acquired data record from a relevant IoT sensor or environmental reference data, it is written to either the direct support state or the indirect support state depending on whether the data itself originates from a direct measurement record.

[0084] After the original measurement support status is written, the time proximity status is determined based on the time field of the candidate compensation basis data and the time field of the current data record to be compensated. The time proximity status indicates the closeness of the candidate compensation basis data and the current data record to be compensated in terms of acquisition time. The time proximity status is determined according to the sampling time rules in the sensor deployment configuration record; if the interval between the time field of the candidate compensation basis data and the time field of the current data record to be compensated conforms to the sampling time rules, the time proximity status is written; if it does not conform to the sampling time rules but the order can be determined according to the sampling time rules, the time traceability status is written; if the order cannot be determined according to the sampling time rules, the time uncertainty status is written.

[0085] After the time proximity status is written, the location proximity status is determined based on the installation location identifiers of the candidate compensation basis data and the current data record to be compensated. The location proximity status indicates the degree of correspondence between the candidate compensation basis data and the current data record to be compensated in terms of their installation locations. The location proximity status is determined according to the location matching rules in the sensor deployment configuration record; if the installation location identifier of the candidate compensation basis data is the same as or adjacent to the installation location identifier of the current data record to be compensated, the location proximity status is written; if the installation location identifier of the candidate compensation basis data and the installation location identifier of the current data record to be compensated have a measurement object correspondence but are not adjacent, the location traceability status is written; if the installation location correspondence cannot be determined, the location uncertainty status is written.

[0086] After writing the proximity status, the transmission disturbance status is determined based on the network status data within the continuous abnormal interval. The transmission disturbance status represents the impact of the network transmission process on the recording order and availability of the collected data within the continuous abnormal interval. If there are no link interruptions, unacknowledged transmissions, retransmissions, or signal quality issues that do not meet the communication status rules within the continuous abnormal interval, a stable transmission status is written. If there are retransmissions, unacknowledged transmissions, or signal quality issues that do not meet the communication status rules within the continuous abnormal interval, but the collection sequence number and time field still correspond, a traceable transmission disturbance status is written. If there are link interruptions or the collection sequence number and time field do not correspond within the continuous abnormal interval, a non-traceable transmission disturbance status is written.

[0087] After the transmission disturbance status is written, the environmental coverage status is determined based on the acquisition of environmental reference data within the continuous anomaly interval. The environmental coverage status indicates whether the environmental reference data can cover the corresponding time field and installation location identifier of the currently collected data record to be compensated within the continuous anomaly interval. If the environmental reference data covers both the corresponding time field and installation location identifier of the currently collected data record to be compensated, a sufficient environmental coverage status is written; if the environmental reference data does not cover the corresponding time field of the currently collected data record to be compensated but covers adjacent time fields within the continuous anomaly interval, an adjacent environmental coverage status is written; if the environmental reference data is missing within the continuous anomaly interval, an missing environmental coverage status is written.

[0088] After the environmental coverage status is written, the historical continuous status is determined based on the original valid acquisition data records before and after the continuous anomaly interval. The historical continuous status indicates whether there are acquisition data records that can serve as the original measurement support for the same IoT sensor before and after the continuous anomaly interval. If there are original valid acquisition data records before and after the continuous anomaly interval, the historical before-and-after continuous status is written; if there are original valid acquisition data records only before the continuous anomaly interval or only after the continuous anomaly interval, the historical one-sided continuous status is written; if there are no original valid acquisition data records before and after the continuous anomaly interval, the historical insufficient continuous status is written.

[0089] After writing the compensation transmission layer number, original measurement support status, time proximity status, location proximity status, transmission disturbance status, environmental coverage status, and historical continuity status, the admission status of the candidate compensation basis data is determined. The admission status indicates whether the candidate compensation basis data is allowed to participate in the current compensation process. If the candidate compensation basis data exhibits at least two of the following states: unknown compensation transmission layer, no direct measurement support, uncertain time, and uncertain location, it is written as an admission-only status. If the candidate compensation basis data is a compensation acquisition data record, and the current data record to be compensated is in the middle of the interval, and the historical continuity status is insufficient, it is written as an admission-only status. If the candidate compensation basis data is a compensation acquisition data record, and the compensation basis data range of this compensation acquisition data record already includes other compensation acquisition data records within the current continuous anomaly interval, it is written as a restricted admission status. If the candidate compensation basis data does not fall into any of the above categories, it is written as an admission status. The restricted admission status indicates that the candidate compensation basis data can participate in compensation trend judgment, but is not directly used as the numerical source for generating compensation values.

[0090] After the access status is written, the compensation basis data range is formed based on the access status. The compensation basis data range refers to the range of candidate compensation basis data participating in the current compensation process, including the range of original valid collected data records, the range of relevant IoT sensor valid collected data records, the range of environmental reference data sources, and the range of restricted access compensation collected data records. The range of original valid collected data records is represented by the range of collection sequence numbers or time fields of the data records collected by the same IoT sensor participating in the compensation. The range of relevant IoT sensor valid collected data records is represented by the relevant IoT sensor identifiers, the range of collection sequence numbers, or the range of time fields participating in the compensation. The range of environmental reference data sources is represented by the environmental reference data source identifiers and the range of time fields participating in the compensation. The range of restricted access compensation collected data records is represented by the record identifiers of compensation collected data records that only participate in the compensation trend judgment.

[0091] If the anomaly type results of the current data record to be compensated include integrity anomalies or continuous summary anomalies, the calibration acquisition values ​​in the current data record to be compensated will not be used to generate compensation values. If there are original valid data records based on the access status, relevant valid IoT sensor data records, or environmental reference data, compensation values ​​are generated based on the sensor identifier, acquisition sequence number, time field, measurement type, and installation location identifier of the current data record to be compensated. The compensation value is a calibration acquisition value used to replace missing or abnormal values ​​in the current data record to be compensated. When generating compensation values, first, valid calibration acquisition values ​​before and after the continuous anomaly interval are obtained from the original valid data records, and the position of the current data record to be compensated between these valid calibration acquisition values ​​is determined according to the sampling time rule; then, valid calibration acquisition values ​​of the same measurement type and adjacent installation location are obtained from the relevant valid IoT sensor data records, and the positional correspondence between this valid calibration acquisition value and the current data record to be compensated is determined according to the position matching rule; finally, reference values ​​from the environmental reference data corresponding to the time field, installation location identifier, and measurement type of the current data record to be compensated are read. Compensation data records that are in a restricted access state are only used to determine whether the direction of numerical change within a continuous abnormal interval is consistent with the original valid data records, relevant IoT sensor valid data records, or environmental reference data, and do not directly provide numerical values ​​for generating compensation values.

[0092] Before generating compensation values, a compensation path is determined based on the interval location status, transmission disturbance status, environmental coverage status, and historical continuity status. The compensation path is the combination of data sources used when generating compensation values. If the current data record to be compensated is at the beginning or end of the interval, and the historical continuity status is consecutive, the compensation path primarily uses the original valid data records. If the current data record to be compensated is in the middle of the interval, and the environmental coverage status is sufficient or adjacent, the compensation path uses environmental reference data to correct the trend of changes in the original valid data records. If the current data record to be compensated is in the middle of the interval, and the historical continuity status is a single-sided continuous state, and there are related IoT sensor valid data records in close proximity, the compensation path uses related IoT sensor valid data records to constrain single-sided historical changes. If the transmission disturbance status is untraceable, the compensation path is not determined solely by the time field; instead, the position of the current data record to be compensated within the same IoT sensor acquisition sequence is determined simultaneously based on the acquisition sequence number and the continuous summary chain value.

[0093] After the compensation path is determined, candidate compensation values ​​are generated according to the compensation path. Candidate compensation values ​​are temporary compensation values ​​obtained based on a compensation path. When using a compensation path primarily based on original valid data records, the change position corresponding to the current data record to be compensated is determined based on the original valid data records before and after the continuous abnormal interval, the acquisition sequence number, and the time field, and candidate compensation values ​​are generated. When using a compensation path that corrects the change trend of original valid data records using environmental reference data, an interval change trend is first formed based on the original valid data records before and after the continuous abnormal interval, and then the interval change trend is corrected based on the change direction of the environmental reference data within the same time range, and candidate compensation values ​​are generated. When using a compensation path that constrains unilateral historical changes using relevant IoT sensor valid data records, a basic change trend is formed based on the unilateral original valid data records, and the extension range of the basic change trend is limited based on the change direction and positional correspondence of relevant IoT sensor valid data records at adjacent installation locations, and candidate compensation values ​​are generated. When multiple candidate compensation values ​​exist, the compensation path, compensation basis data range, source attribute, and basis access status corresponding to each candidate compensation value are recorded.

[0094] In this embodiment, the generation of candidate compensation values ​​according to the compensation path can be performed in the following manner. The following example illustrates the process of generating candidate compensation values. In practical applications, the sensor deployment configuration record or calibration record can record the corresponding calculation method according to the measurement type, measurement unit, sampling time rules, allowable range rules, numerical variation rules, location matching rules, and the correspondence between environmental reference data.

[0095] When using a compensation path primarily based on original valid acquired data records, the system reads both the original valid acquired data records before and after the consecutive abnormal intervals. The original valid acquired data records before the consecutive abnormal intervals contain the preceding acquisition sequence number, the preceding time field, and the preceding valid calibration acquisition value. The original valid acquired data records after the consecutive abnormal intervals contain the following acquisition sequence number, the following time field, and the following valid calibration acquisition value. The currently acquired data record to be compensated contains the current acquisition sequence number and the current time field. When the current acquisition sequence number can be positioned between the preceding and following acquisition sequences, the change position corresponding to the current acquisition data record to be compensated is generated based on the ratio of the current acquisition sequence number to the preceding and following acquisition sequences. When the current time field can be positioned between the preceding and following time fields, the change position corresponding to the current acquisition data record to be compensated is generated based on the ratio of the current time field to the preceding and following time fields. When both the current acquisition sequence number and the current time field can be positioned and are consistent, a candidate compensation value is generated based on the change position determined by both. When both the current acquisition sequence number and the current time field can be positioned but are inconsistent, the position of the current acquisition data record to be compensated within the same IoT sensor acquisition sequence is determined by combining the continuous summary chain value, and a candidate compensation value is generated based on this position. If the candidate compensation value is within the allowable range rules and the change between it and the preceding and following valid calibration acquisition values ​​conforms to the numerical change rules, the candidate compensation value is retained; if it does not conform to the allowable range rules or the numerical change rules, the value is not considered as an optional candidate compensation value.

[0096] For example, the acquisition sequence number of the original valid acquisition data record before the continuous abnormal interval is 100, the time field is 10:00:00, and the valid calibration acquisition value is 20; the acquisition sequence number of the original valid acquisition data record after the continuous abnormal interval is 104, the time field is 10:04:00, and the valid calibration acquisition value is 24; the acquisition sequence number of the current acquisition data record to be compensated is 102, and the time field is 10:02:00. The current acquisition data record to be compensated is in the middle position between the preceding and following acquisition sequence numbers, and also in the middle position between the preceding and following time fields, so the generated candidate compensation value is 22. If the allowable range rule is -40 to 85, and the value change rule is that the change in calibration acquisition value between adjacent acquisition times does not exceed 3, then this candidate compensation value is determined through the pre-judgment corresponding to the time back-substitution state.

[0097] When using environmental reference data to correct the trend of changes in original valid acquired data records, a first candidate compensation value is generated first, based on the compensation path primarily composed of the original valid acquired data records. Then, environmental reference data corresponding to the original valid acquired data records before the continuous anomaly interval, the environmental reference data corresponding to the current acquired data record to be compensated, and the environmental reference data corresponding to the original valid acquired data records after the continuous anomaly interval are read. If the direction of change of the environmental reference data within the same time range is consistent with the direction of change from the preceding valid calibration acquired value to the following valid calibration acquired value, the first candidate compensation value is corrected according to the proportion of change of the environmental reference data within that time range, generating a second candidate compensation value. If the direction of change of the environmental reference data within the same time range is inconsistent with the direction of change from the preceding valid calibration acquired value to the following valid calibration acquired value, the environmental reference data does not directly provide the value for generating the candidate compensation value; it is only used for subsequent environmental back-substitution status judgment. After generating the second candidate compensation value, it is used as the candidate compensation value, and the compensation path, compensation basis data range, source attribute, and basis access status corresponding to this candidate compensation value are recorded.

[0098] For example, if the effective calibration acquisition value on the front side is 20 and the effective calibration acquisition value on the back side is 24, the first candidate compensation value is generated as 22 according to the compensation path based on the original effective acquisition data record. The environmental reference data corresponding to the front time field is 19.8, the environmental reference data corresponding to the current time field is 21.9, and the environmental reference data corresponding to the back time field is 24.0. The environmental reference data shows an upward trend within the same time range, and is consistent with the direction of change from the effective calibration acquisition value on the front side to the effective calibration acquisition value on the back side. Therefore, the direction of change of the environmental reference data is used to correct the first candidate compensation value, generating a second candidate compensation value. If the second candidate compensation value is within the allowable range rules and the correspondence between it and the environmental reference data is consistent with the correspondence of the environmental reference data, then the environmental back-substitution pass status is written in the back-substitution verification.

[0099] When using relevant IoT sensor data records to constrain the compensation path for unilateral historical changes, the system reads original valid data records from before or after consecutive abnormal intervals, and also reads relevant IoT sensor data records from the same time range, measurement type, and installation location. If only original valid data records exist before the consecutive abnormal interval, the system uses the valid calibration values ​​in these records as a basis, and according to the corresponding proportions recorded in the matching rules for the direction and position of change in the current time field relative to the preceding time field, limits the extension range of the basic change trend and generates candidate compensation values. If only original valid data records exist after the consecutive abnormal interval, the system uses the valid calibration values ​​in these records as a basis, and according to the corresponding proportions recorded in the matching rules for the direction and position of change in the current time field relative to the following time field, reversely limits the extension range of the basic change trend and generates candidate compensation values. If the positional change between the candidate compensation value and the relevant IoT sensor data records conforms to the positional matching rules, a spatial back-substitution pass status is written in the back-substitution verification.

[0100] For example, there is an original valid data acquisition record before the continuous abnormal interval, with acquisition sequence number 100, time field 10:00:00, and valid calibration acquisition value of 20; the current data acquisition record to be compensated has acquisition sequence number 101 and time field 10:01:00. The valid calibration acquisition value of the relevant IoT sensor data acquisition record at 10:00:00 is 19, and the valid calibration acquisition value at 10:01:00 is 20, and the location matching rule records that there is a close proximity between the relevant IoT sensor data acquisition record and the current data acquisition record to be compensated. Based on the changing direction of the relevant IoT sensor data acquisition record, it is determined that the calibration acquisition value corresponding to the current data acquisition record to be compensated should increase relative to 20; if the candidate compensation value generated according to the location matching rule is 21, and this candidate compensation value meets the allowable range rule and the numerical change rule, then 21 is taken as the candidate compensation value, and the time back-substitution state, spatial back-substitution state, and environmental back-substitution state are further judged in the back-substitution verification.

[0101] When restricted access compensation data is used to determine the direction of numerical change within a continuous abnormal interval, only the upward, downward, or stable direction of the restricted access compensation data relative to the original valid data records before and after it, the valid data records of relevant IoT sensors, or the environmental reference data is read. The calibration data in the restricted access compensation data is not read as the direct source of the candidate compensation value. If the direction of numerical change of the candidate compensation value generated by the access compensation data is consistent with the direction of numerical change represented by the restricted access compensation data, the candidate compensation value is retained for back-substitution verification; if they are inconsistent, the candidate compensation value is not retained, or it is only written to the record corresponding to the pending state.

[0102] After candidate compensation values ​​are formed, they undergo back-substitution verification. Back-substitution verification involves temporarily writing the candidate compensation value to the current location of the data record to be compensated, and checking its consistency with the data records collected before and after the continuous anomaly interval, the relevant valid data records from IoT sensors, and the environmental reference data, according to sampling time rules, numerical change rules, allowable range rules, location matching rules, and the correspondence with environmental reference data. If the candidate compensation value conforms to the allowable range rules, and the change between it and the original valid data records before and after the continuous anomaly interval conforms to the numerical change rules, it is written to the time back-substitution status. If the location change between the candidate compensation value and the relevant valid data records from IoT sensors conforms to the location matching rules, it is written to the spatial back-substitution status. If the correspondence between the candidate compensation value and the environmental reference data conforms to the sensor deployment configuration record or calibration record, it is written to the environmental back-substitution status.

[0103] After back-substitution verification, the candidate compensation values ​​are subjected to propagation suppression processing. Propagation suppression processing refers to restricting the compensation result from repeatedly participating in calculations within continuous abnormal intervals based on the participation of the compensation collection data records on which the candidate compensation value is based. If the compensation data range of a candidate compensation value does not contain compensation collection data records, it is written to a state without propagation suppression. If the compensation data range of a candidate compensation value contains compensation collection data records based on restricted access states, but these records only participate in the determination of the direction of change, it is written to a state of weak propagation suppression. If the compensation data range of a candidate compensation value contains compensation collection data records that directly provide numerical values, it is written to a state of strong propagation suppression, and this candidate compensation value is removed from the list of available candidate compensation values.

[0104] After the transmission suppression processing is completed, the selected compensation state is determined for the candidate compensation values ​​that were not eliminated. The selected compensation state indicates whether a candidate compensation value is determined as the current compensation value. If a candidate compensation value has a time-backward substitution pass state, a spatial-backward substitution pass state, and an environmental-backward substitution pass state simultaneously, the candidate compensation value is written into the priority selection state. If a candidate compensation value has a time-backward substitution pass state and a spatial-backward substitution pass state under environmental coverage loss conditions, the candidate compensation value is written into the conditional selection state. If a candidate compensation value has only a single substitution pass state, the candidate compensation value is written into the pending state. If there is no priority selection state or conditional selection state, the current data to be compensated is written into the uncompensable record, along with the reason for insufficient compensation basis. When there are multiple candidate compensation values ​​with priority selection states or conditional selection states, the candidate compensation value with the lowest number of compensation transmission layers, priority of original measurement support state, priority of time proximity state, priority of location proximity state, and weaker transmission suppression state is selected as the compensation value.

[0105] For example, the continuous abnormal intervals of the same IoT sensor include three currently uncompensated data records: collection sequence number 101, collection sequence number 102, and collection sequence number 103. The data record corresponding to collection sequence number 100 is the original valid data record with a valid calibration value of 20; the data record corresponding to collection sequence number 104 is also the original valid data record with a valid calibration value of 24. The current uncompensated data record corresponding to collection sequence number 101 is at the beginning of the interval, the current uncompensated data record corresponding to collection sequence number 102 is in the middle of the interval, and the current uncompensated data record corresponding to collection sequence number 103 is at the end of the interval.

[0106] When generating candidate compensation values ​​for the current data record to be compensated corresponding to acquisition sequence number 101, the compensation basis data range only includes the original valid acquisition data record corresponding to acquisition sequence number 100, the original valid acquisition data record corresponding to acquisition sequence number 104, and the corresponding environmental reference data, but does not include the compensation acquisition data record. Therefore, a state without transmission suppression is generated. If the candidate compensation value is 21 and meets the allowable range rules, numerical change rules, position matching rules, and environmental reference data correspondence, then a time back-substitution pass state, a spatial back-substitution pass state, and an environmental back-substitution pass state are generated, and the candidate compensation value is written into the preferred selection state and determined as the compensation value. The compensation value, compensation basis data range, interval position status, compensation transmission layer number, original measurement support status, transmission disturbance status, environmental coverage status, historical continuity status, compensation path, and corresponding anomaly type are written into the compensation acquisition data record.

[0107] When generating candidate compensation values ​​for the current data record to be compensated corresponding to acquisition sequence number 102, if the compensation basis data range of the candidate compensation value includes the compensation acquisition data record corresponding to acquisition sequence number 101, and this compensation acquisition data record is only in a restricted access state and is only used to determine the direction of numerical change, then a weak propagation suppression state is generated. In the weak propagation suppression state, the compensation value in the compensation acquisition data record corresponding to acquisition sequence number 101 is not read as a direct numerical source; only the direction of numerical change represented by the compensation acquisition data record corresponding to acquisition sequence number 101 is determined to be consistent with the original valid acquisition data record, the relevant valid acquisition data record from the IoT sensor, or the environmental reference data. If the candidate compensation value meets the back-substitution verification and is not eliminated by the strong propagation suppression state, it can be determined as the compensation value based on the selected compensation state.

[0108] When generating candidate compensation values ​​for the current data record to be compensated corresponding to collection sequence number 103, if the compensation basis data range of the candidate compensation value includes the compensation collection data record corresponding to collection sequence number 101 or collection sequence number 102, and this compensation collection data record directly provides the value for generating the candidate compensation value, then a strong propagation suppression state is generated, and the candidate compensation value is removed from the selectable candidate compensation values. Removed candidate compensation values ​​do not enter the compensation selection state and are not written into the compensation value list. If the remaining candidate compensation values ​​do not have a priority selection state or a conditional selection state, then the current data record to be compensated corresponding to collection sequence number 103 is written into the uncompensable record, along with the reason for insufficient compensation basis and the range of compensation basis data already obtained. Through the above processing, the repeated propagation of compensation values ​​can be avoided by directly using the previous compensation value as the numerical source of the next compensation value within a continuous abnormal interval.

[0109] If any anomaly type exists in the current data records to be compensated, indicating a missing acquisition value, the missing acquisition location is determined based on the acquisition sequence number and time field corresponding to the missing acquisition value type. The missing acquisition location refers to the acquisition sequence number or time position in the current data records to be compensated that should contain a calibration acquisition value but did not. If the missing acquisition location is within a continuous anomaly interval, a compensation path is determined based on the interval location status, transmission disturbance status, environmental coverage status, and historical continuity status. Candidate compensation values ​​are then generated according to this compensation path, back-substitution verification is performed, transmission suppression processing is executed, and the selected compensation status is determined. If the missing acquisition location is not within a continuous anomaly interval, and there are valid original acquisition data records before and after it, candidate compensation values ​​are generated according to the time interpolation method recorded in the sensor deployment configuration record or calibration record, and back-substitution verification is performed on the candidate compensation values. The time interpolation method refers to the method of calculating the value corresponding to the missing acquisition location based on the valid calibration acquisition values ​​before and after the missing acquisition location and the corresponding time field.

[0110] If any of the anomaly types in the current data records to be compensated contain numerical range anomalies or sensor acquisition anomalies, the calibrated acquisition values ​​in the current data records will not be used to generate compensation values. Instead, candidate compensation values ​​will be generated based on the compensation basis data range. Candidate compensation values ​​for numerical range anomalies are constrained by the allowable range rules and the original valid acquisition data records before and after the continuous anomaly interval. Candidate compensation values ​​for sensor acquisition anomalies are constrained by the consistency relationship between the original valid acquisition data records, relevant IoT sensor valid acquisition data records, and environmental reference data. After candidate compensation values ​​are generated, back-substitution verification and transmission suppression processing are performed. Candidate compensation values ​​that pass back-substitution verification and are not eliminated by strong transmission suppression are determined as compensation values. After generating compensation values, the compensation value, compensation basis data range, interval location status, number of compensation transmission layers, original measurement support status, transmission disturbance status, environmental coverage status, historical continuity status, compensation path, and corresponding anomaly type are written into the compensation acquisition data record.

[0111] If the anomaly type results of the current data collection record to be compensated contain a state anomaly type of the measured object, no alternative compensation value will be generated, and the current data collection record to be compensated will be written into the retention record. A state anomaly type of the measured object indicates that although the current calibration collection value has undergone an abnormal change, this abnormal change is consistent with the valid data collection records of relevant IoT sensors or environmental reference data. In this case, the calibration collection value in the current data collection record to be compensated will be retained, and the state anomaly type of the measured object and the data source used for the judgment will be written into the record.

[0112] If the only anomaly type in the current data record to be compensated is insufficient data status, the calibration acquisition value, acquisition sequence number, time field, and data status of the current data record to be compensated are read. Insufficient data status indicates that the current data record to be compensated has insufficient time synchronization, insufficient environmental reference data, insufficient network status data, insufficient historical verification data, or insufficient spatial reference, but no other anomaly types are formed. If the field structure of the current data record to be compensated is normal, integrity verification passes, continuous summary verification passes, the acquisition sequence number conforms to the acquisition order rules, the time field conforms to the sampling time rules, and the calibration acquisition value conforms to the allowable range rules, no alternative compensation value is generated, and the current data record to be compensated is written to the reserved record. If the above conditions are not met, the corresponding processing method is determined according to the verification marks that also exist in the current data record to be compensated; if the corresponding processing method cannot be determined, the current data record to be compensated is written to the uncompensable record.

[0113] When a compensation data acquisition record needs to be generated, the record identifier of the current data acquisition record to be compensated is written into the compensation data acquisition record. The record identifier is a marker used to distinguish different data acquisition records, and it consists of at least one of the following: sensor identifier, acquisition sequence number, time field, and integrity check value. The compensation data acquisition record contains the sensor identifier, compensation acquisition sequence number, compensation time field, compensation value, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, anomaly type result, compensation basis data range, compensation method identifier, and compensation source identifier. The compensation acquisition sequence number is the acquisition sequence number corresponding to the compensation data acquisition record. The compensation time field is the time field corresponding to the compensation data acquisition record. The compensation method identifier indicates that the compensation value is generated using time interpolation, spatial reference, environmental reference, or a combination thereof. The compensation source identifier indicates the source of the compensation value: historical valid data acquisition records, relevant valid IoT sensor data acquisition records, or environmental reference data.

[0114] After the fields are written into the compensation data collection record, the fields requiring integrity protection in the compensation data collection record are standardized and concatenated according to the fixed field names, fixed field order, fixed encoding format, and fixed numerical representation rules adopted in step one, resulting in the compensation collection text to be authenticated. The compensation collection text to be authenticated is the set of fields in the compensation data collection record that require integrity protection. The integrity key bound to the IoT sensor is read based on the sensor identifier, and the same key-based message authentication algorithm as in step one is used to calculate the compensation integrity verification value in the compensation collection text. The compensation integrity verification value is written into the compensation data collection record. When using a digital signature, the same compensation collection text to be authenticated is used as the signature object, and the signature result is written into the compensation data collection record as the compensation integrity verification value.

[0115] After the integrity verification value is written, the previous continuous summary chain value is read based on the previous data record or compensation data record whose output order has been determined for the same IoT sensor. The previous data record or compensation data record whose output order has been determined is the record that precedes the current compensation data record in the same IoT sensor output sequence. The previous continuous summary chain value, the summary value of the compensation data to be authenticated, and the integrity verification value are concatenated in a fixed order, and a summary is calculated to obtain the compensation continuous summary chain value, which is then written to the compensation data record.

[0116] After the compensated data collection records are formed, a post-compensation verification is performed on them. This post-compensation verification involves re-verifying the compensated data collection records according to the checks performed in step two: field existence verification, field format verification, integrity verification, continuous summary verification, collection sequence number continuity verification, time consistency verification, data status verification, numerical range verification, and numerical change consistency verification. If a compensated data collection record passes the post-compensation verification, it is written into the compensated data. If a compensated data collection record fails the post-compensation verification, it is written into the uncompensable record, and a verification mark indicating that it failed the post-compensation verification is added.

[0117] After the retained records are created, the following checks are performed on them: field existence, field format, integrity, continuous summary, collection sequence number continuity, time consistency, numerical range, and numerical change consistency. Retained records that pass these checks are written into the compensation data. Retained records that fail these checks are written into the non-compensable records, and a check mark indicating that the checks failed is added.

[0118] After the duplicate removal results, time-series rearrangement results, uncompensable records, and compensation data are generated, the output order is determined according to the sensor identifier, acquisition sequence number, and time field of the same IoT sensor. The compensation data includes compensated acquisition data records that have passed compensation and subsequent verification, retained records that have passed verification, and time-series rearrangement results that have passed time-series rearrangement verification. The duplicate removal results include the record identifier of the removed acquisition data record and the reason for duplicate removal. The uncompensable records include the record identifier of the currently uncompensated acquisition data record, the anomaly type result, the reason for uncompensation, and the range of compensation data already obtained.

[0119] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0120] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0121] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0122] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0124] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for compensating for anomalies in the integrity verification of IoT sensor data acquisition, characterized in that, include: Retrieve the current data records to be compensated; Determine the continuous abnormal interval based on the currently collected data records to be compensated; Generate an interval position status based on the position of the currently collected data to be compensated within the continuous abnormal interval; Read the candidate compensation basis data according to the interval position status, and generate the compensation admission result according to the candidate compensation basis data; The data on the basis for access compensation shall be retained based on the aforementioned compensation access results; The compensation path is determined based on the interval position status and the admission compensation basis data, and candidate compensation values ​​are generated according to the compensation path. Perform back-substitution verification and transitivity suppression processing on the candidate compensation values ​​to obtain back-substitution verification results and transitivity suppression results; The compensation value is determined based on the back-substitution verification result and the transmission suppression result, and the compensation value is written into the compensation acquisition data record.

2. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 1, characterized in that: The step of determining a continuous abnormal interval based on the sensor identifier, the acquisition sequence number, and the time field, and generating an interval position status based on the position of the currently collected data to be compensated within the continuous abnormal interval, includes: Read adjacent data collection records of the same IoT sensor based on the sensor identifier; Based on the acquisition sequence number or the time field, adjacent acquisition data records that are continuous with the current acquisition data record to be compensated and have abnormal results are merged into the continuous abnormal interval. When an adjacent data record that has not been written with an abnormal type result and whose field structure is normal, integrity verification passes, and continuous summary verification passes is read, the expansion of the continuous abnormal interval is stopped. Based on the start position, middle position, end position, or single point position of the currently collected data to be compensated within the continuous abnormal interval, the corresponding interval position status is generated.

3. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 2, characterized in that: The step of reading candidate compensation basis data based on the interval position status and generating compensation admission results based on the candidate compensation basis data includes: Based on the interval position status, read the original valid data collection records, the compensated data collection records, the relevant valid data collection records of IoT sensors, and the environmental reference data, and use them as the candidate compensation basis data; Source attributes are generated based on the formation source of the candidate compensation basis data; The number of compensation transmission layers is generated based on the range of the compensation basis data of the candidate compensation basis data, and the original measurement support status is generated based on whether the candidate compensation basis data can be traced back to the original valid data acquisition record. Based on the time field and installation location identifier of the candidate compensation data, generate time proximity status and location proximity status; Based on the network status data, environmental reference data acquisition status, and original valid data record distribution within the continuous abnormal interval, the transmission disturbance status, environmental coverage status, and historical continuous status are generated. Based on the source attribute, the number of compensation transmission layers, the original measurement support status, the time proximity status, the location proximity status, the transmission disturbance status, the environmental coverage status, and the historical continuity status, the candidate compensation basis data is marked as admitted, restricted admitted, or not admitted, and the compensation admission result is generated.

4. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 3, characterized in that: The step of determining the compensation path based on the interval position status and the admission compensation basis data, and generating candidate compensation values ​​according to the compensation path, includes: Based on the compensation access results, the candidate compensation basis data marked as access is determined as the access compensation basis data, and the candidate compensation basis data marked as restricted access is determined as the restricted access compensation basis data. Based on the interval location status, the transmission disturbance status, the environmental coverage status, and the historical continuity status, the compensation path is determined from the compensation path based on the original effective data acquisition records, the compensation path for correcting the original change trend with environmental reference data, and the compensation path for constraining the one-sided historical change with the effective data acquisition records of relevant IoT sensors. When the transmission disturbance state is a transmission disturbance untraceable state, the position of the current data to be compensated in the same IoT sensor acquisition sequence is determined according to the acquisition sequence number and continuous summary chain value, and the compensation path is corrected according to the position. According to the compensation path, extract valid calibration collection values ​​or reference values ​​from the access compensation basis data; The restricted access compensation basis data is used to determine the direction of numerical change within the continuous abnormal interval, and the restricted access compensation basis data is prohibited from directly providing numerical values ​​for generating candidate compensation values. The candidate compensation value is generated based on the effective calibration acquisition value or the reference value and the direction of the value change.

5. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 4, characterized in that: The step of performing back-substitution verification and transitivity suppression processing on the candidate compensation values ​​to obtain back-substitution verification results and transitivity suppression results, and determining the compensation value based on the back-substitution verification results and the transitivity suppression results, includes: The candidate compensation value is temporarily written into the position corresponding to the current data record to be compensated; Based on the allowable range rules, numerical change rules, position matching rules, and the correspondence of environmental reference data, the temporarily written candidate compensation values ​​are verified to generate time-backward state, spatial-backward state, and environmental-backward state, and the time-backward state, spatial-backward state, and environmental-backward state are used as the backward verification results. Based on whether the compensation basis data range corresponding to the candidate compensation value contains compensation collection data records, a state of no transmission inhibition, a state of weak transmission inhibition, or a state of strong transmission inhibition is generated, and the state of no transmission inhibition, the state of weak transmission inhibition, or the state of strong transmission inhibition is taken as the transmission inhibition result. When the transmission suppression result is a strong transmission suppression state, the corresponding candidate compensation value is removed. Based on the back-substitution verification results and the propagation suppression results corresponding to the candidate compensation values ​​that have not been eliminated, a compensation selection state is generated; Based on the selected compensation status, the compensation value is determined from the candidate compensation values ​​that have never been eliminated.

6. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 1, characterized in that: Before acquiring the current data record to be compensated, the method further includes: A temporary record of the data collected this time is created, and the sensor identifier corresponding to the IoT sensor that performed the data collection is written into the temporary record of the data collected this time. Read the output of the sensing element of the IoT sensor to generate the original collected value, or generate a collection missing indicator when the output of the sensing element is not read; Based on the sensor identifier, the deployment configuration record, deployment location record, and calibration record are read to generate the measurement type, measurement unit, installation location identifier, and calibration parameter version identifier, and calibration acquisition values ​​are generated based on the original acquisition values; A time field is generated based on the local timestamp and the reference time provided by the time synchronization source, and a collection sequence number is generated based on the sensor identifier; Based on the time field, the installation location identifier, and the measurement type, environmental reference data is matched, and network status data and historical data reference information are read; The sensor identifier, acquisition sequence number, time field, calibration acquisition value or acquisition missing identifier, measurement type, measurement unit, installation location identifier, calibration parameter version identifier, environmental reference data, network status data, and historical data reference information are concatenated according to fixed field rules to obtain the acquisition text to be authenticated. The integrity check value is obtained by calculating the integrity check value based on the integrity key of the collected text to be authenticated; The current continuous digest chain value is generated based on the previous continuous digest chain value, the digest value of the text to be authenticated, and the integrity verification value. The integrity check value and the current continuous summary chain value are written into the temporary record of the current data collection to form the current data collection record.

7. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 6, characterized in that: After forming the data record for this collection, the method further includes: Read the data record collected in this instance and determine it as the current data record collected. Perform field structure validation on the currently collected data record according to the fixed field rules, and generate field structure validation results; The text to be verified is regenerated based on the currently collected data record, and an integrity verification result is generated based on the text to be verified and the integrity verification value. A continuous summary verification result is generated based on the previous continuous summary chain value, the summary value of the text to be verified, and the integrity verification value. The collection sequence number is verified according to the collection sequence rules, and a collection sequence number verification result is generated. The time field is validated according to the sampling time rules, and a time field validation result is generated. A numerical verification result is generated based on the calibration acquisition value or the acquisition missing identifier, allowable range rules, and numerical change rules; Based on the environmental reference data, the network status data, and the historical data reference information, a data status verification result is generated; An initial anomaly marker is formed based on the field structure verification result, the integrity verification result, the continuous summary verification result, the collection sequence number verification result, the time field verification result, the numerical value verification result, and the data status verification result.

8. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 7, characterized in that: After forming the initial anomaly marker, the method further includes: Read the currently collected data record that has been written with the initial anomaly mark, and determine the currently collected data record with the initial anomaly mark as the current collected data record to be identified; An anomaly type is generated based on the field structure verification result, the integrity verification result, the continuous summary verification result, the collection sequence number verification result, the time field verification result, the numerical value verification result, and the data status verification result. When the anomaly type involves a change in calibration acquisition value, read historical valid acquisition data records, relevant IoT sensor valid acquisition data records, and environmental reference data, and correct the anomaly type based on the historical valid acquisition data records, the relevant IoT sensor valid acquisition data records, and the environmental reference data; An anomaly type result is generated based on the anomaly type, the data source involved in the identification, and the data status. Based on the anomaly type results, the current data records to be identified that require numerical supplementation, numerical substitution, order correction, or record removal are determined as the current data records to be compensated. Based on the sensor identifier, acquisition sequence number or time field, measurement type and installation location identifier in the current data record to be compensated, determine whether the current data record to be compensated has the basic fields for compensation positioning.

9. The method for compensating for anomalies in IoT sensor data acquisition integrity verification according to claim 1, characterized in that: After writing the compensation value into the compensation acquisition data record, the method further includes: Write the record identifier of the currently collected data record to be compensated into the compensation collected data record; According to fixed field rules, the fields in the compensation data collection records that require integrity protection are concatenated to obtain the compensation collection text to be authenticated; The compensation integrity verification value is calculated based on the integrity key and written into the compensation collection data record. A compensation continuous digest chain value is generated based on the previous continuous digest chain value, the digest value of the compensation text to be authenticated and collected, and the compensation integrity verification value, and the compensation continuous digest chain value is written into the compensation collection data record; Perform compensation and re-verification on the compensation collection data records that contain the compensation integrity verification value and the compensation continuous digest chain value to obtain the compensation and re-verification result. Based on the results of the compensation and re-verification, the compensation data collection records are written into the compensation data or the non-compensable records.

Images