A method and system for intelligent data storage, fusion and governance of multi-sensor monitoring data

Abstract

This invention discloses an intelligent data entry and fusion governance method and system for multi-sensor monitoring data. The method includes: configuring a semantic template for each data source; standardizing the original monitoring data and generating an input summary within the original observation time window; performing adjustment calculations within the solution time window to construct a multi-dimensional quality vector containing residuals, stability, completeness, timeliness, and cross-source consistency; generating a governance plan based on the multi-dimensional quality vector; executing governance and encapsulating the governance basis package; configuring a push time window and a recalculation time window; determining whether to distribute governance results through a push gate function; and triggering recalculation upon failure. This invention achieves automatic alignment of heterogeneous data through semantic templates, drives a closed-loop governance orchestration with multi-dimensional quality vectors, and combines cross-source consistency verification, phylogenetic evidence chain tracing, and recalculation mechanisms to effectively improve the automation level and reliability of multi-sensor monitoring data access and fusion governance.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring data processing and fusion governance technology, specifically to a method and system for intelligent storage and fusion governance of multi-sensor monitoring data. Background Technology

[0002] With the development of the Internet of Things (IoT), the Industrial Internet, and automated monitoring technologies, a diverse range of sensor applications is gradually emerging in engineering scenarios such as construction monitoring near subway and high-speed rail lines, foundation pit monitoring, and tunnel structural health monitoring. These applications include manual, semi-automatic, and fully automatic monitoring. Common equipment includes total stations, hydrostatic levels, inclinometers, crack gauges, track gauges, temperature and pressure sensors, and video / image acquisition terminals. Furthermore, differences exist in data formats, sampling frequencies, communication protocols, coordinate references, and quality control rules across different projects and supplier platforms.

[0003] Existing technologies typically employ manual configuration of data sources, manual import of files, or reliance on a single vendor's platform for data access and processing. On the one hand, manual configuration and static scripts struggle to cope with the increasing number of devices, on-site replacements, and frequent debugging, easily leading to missed detections, delayed uploads, or configuration errors. On the other hand, existing threshold judgments are mostly fixed thresholds or empirical rules, making it difficult to dynamically adapt by combining data quality, observation time windows, and solution time windows, resulting in inaccurate anomaly detection and a coexistence of false positives and false negatives. Furthermore, the lack of unified identification and genealogical management among raw data, adjustment results, and correction results makes it difficult to form standardized data assets, thereby causing high operation and maintenance costs and difficulties in cross-project reuse.

[0004] In scenarios requiring high frequency, high precision, and high quality, such as safety monitoring during construction near operational railway lines, monitoring results need to be connected to a third-party monitoring platform in real time according to the monitoring frequency. Missed transmissions, delays, or substandard quality can trigger tiered alarms, affecting construction organization and safety management. To ensure stable, timely, and accurate data uploads, traditional methods often require dedicated personnel to monitor and repeatedly verify the data, resulting in significant production pressure.

[0005] Therefore, there is an urgent need for a monitoring data automatic storage and fusion governance method and system that can automatically discover and quickly access multi-source heterogeneous monitoring devices, drive governance orchestration and push gates with multi-dimensional quality vectors, and form verifiable spectral evidence chains. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent storage, fusion and governance of multi-sensor monitoring data, in order to solve the problems of existing technologies such as discrete horizontal and vertical access links, difficulty in unifying heterogeneous protocols and field semantics, difficulty in dynamically adjusting processing strategies according to operating conditions and quality, difficulty in verifying cross-source consistency, and lack of verifiable lineage evidence in the governance process.

[0007] To achieve the above objectives, the technical solution provided by this invention is: a method for intelligent data storage and fusion management of multi-sensor monitoring data, comprising the following steps: S1: Establish the master data model for the data source, configure a semantic template for each data source and assign a unique data source alias, and configure the original observation time window; S2: Within the original observation time window, the original monitoring data collected from the data source is standardized based on the semantic template to generate standardized monitoring data; the input standard string is generated according to the preset normalization serialization rules, and the input summary is calculated. S3: Configure the solution time window. Within the solution time window, perform adjustment on the standardized monitoring data according to the observation equation type, error model parameters and accuracy level in the semantic template to obtain the solution results; construct a multi-dimensional quality vector from the solution results; map the multi-dimensional quality vector to a comprehensive quality score in a weighted sum manner, and output a credibility classification label based on the comparison results of the comprehensive quality score and the cross-source consistency quality component relative to the threshold set. S4: Generate a governance plan based on the multidimensional quality vector, comprehensive quality score, trust level label and preset rule template set. The governance plan includes a sequence of governance operators and corresponding parameter sets. Perform governance on standardized monitoring data and solution results according to the governance plan to obtain governance results. Encapsulate the governance results into a governance basis package. The governance basis includes an input summary, governance plan, multidimensional quality vector, comprehensive quality score and trust level label. S5: Configure the push time window and the recalculation time window, define the push gate function, and the judgment conditions of the push gate function include the comprehensive quality score and the trust level label; decide whether to distribute the governance results to the outside world within the push time window based on the judgment result of the push gate function. If the judgment fails, the recalculation time window is triggered to perform recalculation. If the judgment is successful, the governance results are distributed to the outside world, and the automatic storage and fusion governance of the original monitoring data is completed.

[0008] To optimize the above technical solution, the specific measures also include: In step S1, the semantic template includes field mapping rules, unit conversion rules, coordinate reference conversion rules, time alignment rules, observation equation type, error model parameters, and accuracy level; step S1 also includes performing uniqueness verification on the unique alias of the data source, binding the unique alias to the semantic template, and identifying and verifying the source of the original monitoring data.

[0009] Furthermore, in steps S1, S3, and S5, the original observation time window, the solution time window, the push time window, and the recalculation time window are configured in a linked manner: the solution time window covers the original observation time window, and the push time window is not earlier than the end time of the solution time window; the acquisition, solution, management, push, and recalculation tasks are linked and scheduled in a finite state machine manner; the state transition of the finite state machine is jointly driven by the comprehensive quality score Q, the credibility classification label C, and the number of missed measurement points L.

[0010] In step S3, the multidimensional quality vector includes: a residual quality component, calculated based on the difference between the adjustment residual and a preset residual threshold; a stability quality component, calculated based on the fluctuation of the displacement sequence or coordinate sequence within the historical window; an integrity quality component, calculated based on the ratio of the number of measured points in the current period to the number of points to be measured in the current period; a timeliness quality component, calculated based on the ratio of data arrival delay to the continuous measurement time of the station; and a cross-source consistency quality component, calculated by normalizing the differences between observation results from multiple data sources for the same measuring point and the same monitoring element within the same time window with the accuracy level in the semantic template.

[0011] In step S4, a governance plan is generated based on the multidimensional quality vector, comprehensive quality score, credibility classification label, and a preset set of rule templates. Governance is then performed on the standardized monitoring data and solution results according to the governance plan to obtain the governance outcome. The specific process is as follows: The platform has a pre-set set of governance rule templates. Each governance rule template includes a trigger predicate, an operator sequence, a parameter set, and a priority. When the multidimensional quality vector, the comprehensive quality score, and the trust level label satisfy the trigger predicate, the corresponding governance rule template is selected and merged according to priority to obtain a governance plan. According to the operator sequence in the governance plan, the standardized monitoring data and the solution results are governed sequentially, and the multidimensional quality vector and the comprehensive quality score are updated after each operator is executed. The abnormal data removed during the execution process, the corrected parameter records, and the intermediate calculation results are used as evidence information and are encapsulated together with the governance plan, the updated multidimensional quality vector, and the comprehensive quality score into a governance basis package.

[0012] Further, in step S4, the governance basis package is indexed by a genealogical key value and associated with the corresponding original monitoring data, standardized monitoring data, solution results, and governance results; the governance basis package also includes the version number of the governance plan, the set of data sources participating in the fusion, the summary of the removed samples, and the fingerprint of key parameters; a genealogical key value is generated for each governance result, and the genealogical key value includes the project identifier, the measurement point identifier, the timestamp, the data type, the governance plan version number, and the input summary.

[0013] In step S2, the semantic template performs standardization processing on the raw monitoring data collected from the data source, specifically: mapping fields from different sources to a unified field set according to field mapping rules, unifying the units according to unit conversion rules, unifying the spatial reference according to coordinate reference conversion rules, and unifying the observation time according to time alignment rules.

[0014] In step S5, the determination result of the push gate function determines whether to distribute the governance results externally in the following way: Define a push gate function, whose judgment conditions include: the overall quality score is not lower than the preset quality threshold, the cross-source consistency quality component is not lower than the preset consistency threshold, and the trust level label is trustworthy.

[0015] Furthermore, when the judgment conditions are met, the push gate function is successful, and the governance results are distributed to external platforms or interfaces within the push time window, and audit logs are recorded. If any judgment condition is not met, the push gate function fails to judge, generates a backfill task, triggers the backfill recalculation time window to execute backfill recalculation, adjusts parameters to recalculate and re-manage, and marks the management results as pending backfill status on the visualization interface, while writing the reason for judgment failure into the management basis package.

[0016] Furthermore, when the push gate function fails to determine the outcome, based on the input summary, governance plan, multidimensional quality vector, and comprehensive quality score in the governance basis package, the parameters of the adjustment solution or the operator sequence of the governance plan are adjusted within the backfill recalculation time window, and the solution and governance are re-executed until the newly generated governance results meet the determination conditions of the push gate function.

[0017] As another important technical solution, this invention also provides a multi-sensor monitoring data intelligent storage and fusion management system, comprising: The semantic template configuration module is used to establish the master data model of the data source, configure the semantic template for each data source and assign a unique data source alias, and configure the original observation time window; The data standardization and summary generation module is used to perform standardization processing on the raw monitoring data collected from the data source based on semantic templates within the original observation time window, and generate standardized monitoring data; it generates the input standard string according to the preset normalization serialization rules and calculates the input summary. The adjustment and quality assessment module is used to configure the adjustment time window. Within the adjustment time window, the standardized monitoring data is adjusted according to the observation equation type, error model parameters and accuracy level in the semantic template to obtain the adjustment results. A multidimensional quality vector is constructed from the adjustment results. The multidimensional quality vector is mapped to a comprehensive quality score in a weighted sum manner. Based on the comparison results of the comprehensive quality score and the cross-source consistency quality component with the threshold set, a credibility classification label is output. The governance execution and basis encapsulation module is used to generate a governance plan based on a multidimensional quality vector, a comprehensive quality score, a trust level label, and a preset set of rule templates. The governance plan includes a sequence of governance operators and a corresponding parameter set. Governance is performed on standardized monitoring data and solution results according to the governance plan to obtain governance results. The governance results are then encapsulated into a governance basis package, which includes an input summary, a governance plan, a multidimensional quality vector, a comprehensive quality score, and a trust level label. The distribution gate and recalculation module are used to configure the push time window and the recalculation time window, and define the push gate function. The judgment conditions of the push gate function include the comprehensive quality score and the trust level label. Based on the judgment result of the push gate function, it is determined whether to distribute the governance results to the outside world within the push time window. If the judgment fails, the recalculation time window is triggered to perform recalculation. If the judgment is successful, the governance results are distributed to the outside world, and the automatic storage and fusion governance of the original monitoring data are completed.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention enables automatic processing of field mapping, unit conversion, coordinate reference transformation, and time alignment by configuring semantic templates for each data source. This eliminates the need for manual script writing or configuration, lowers the barrier to entry, and avoids data errors caused by configuration mistakes.

[0019] This invention uses a multi-dimensional quality vector composed of residuals, stability, integrity, timeliness, and cross-source consistency as the control basis to automatically generate a governance plan and form a closed-loop feedback of data collection, calculation, governance, and push. This enables the governance strategy to be dynamically adjusted according to data quality, improves the stability of the link in high-frequency monitoring scenarios, and enhances the interpretability of the processing.

[0020] This invention uses cross-source consistency as the basis for dynamic weighting and gate control, and achieves adaptive weight allocation during multi-source data fusion. By comparing multiple sources, it helps to distinguish between sensor faults, link anomalies and changes in actual operating conditions, thereby improving the reliability of the fusion results.

[0021] This invention encapsulates input data, processing parameters, intermediate processes, and final output into a reproducible chain of evidence by using governance basis packages and genealogical keys, supporting auditing, problem playback, and accountability tracing in cross-unit collaborative scenarios.

[0022] This invention employs a quality gate control mechanism to regulate the distribution of results, combined with a recalculation mechanism. When results fail to meet standards, recalculation and adjustment are automatically triggered, preventing missed, delayed, or low-quality data from entering the monitoring platform and reducing manual monitoring and maintenance costs.

[0023] This invention uses a finite state machine to coordinate and schedule tasks such as data acquisition, calculation, management, push, and backup. It also utilizes comprehensive quality scoring, trust level labels, and the number of missed detection points to drive state transitions, achieving organic coordination of multiple time windows and automated connection of processing flows, thereby further improving the overall operating efficiency and robustness of the system. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the architecture of the multi-sensor intelligent monitoring data automatic storage and fusion management system of the present invention.

[0025] Figure 2 This is a schematic diagram of the method flow of the present invention.

[0026] Figure 3 This is a schematic diagram illustrating the linkage between time-segmented scheduling and gate push in an embodiment of the present invention.

[0027] Figure 4 This is a schematic diagram of the closed loop of quality vector-driven governance orchestration in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further described in detail below through specific embodiments, but it should not be construed as limiting the scope of the subject matter of the present invention to the following embodiments. All technologies implemented based on the above content of the present invention fall within the scope of the present invention.

[0029] In some implementations, such as Figure 2 As shown, this invention provides a method for intelligent data storage and fusion management of multi-sensor monitoring data, including the following steps: S1: Establish the master data model of the data source, configure a semantic template for each data source and assign a unique data source alias. The semantic template includes field mapping rules, unit conversion rules, coordinate reference transformation rules, time alignment rules, observation equation type, error model parameters and accuracy level, and configure the original observation time window at the same time. A unique alias is used to uniquely identify the data source. It is generated by combining the project code, station code, measuring point code, monitoring element code, equipment code, and channel type, or by using a unique string identifier mapped from it.

[0030] Step S1 also includes: performing uniqueness verification on the unique alias of the data source, binding the unique alias with the semantic template, and identifying and verifying the source of the original monitoring data; if enabled, performing connectivity verification on the connection string and authentication information, and allowing the corresponding data source configuration to be published or take effect after the connectivity verification is passed, so as to avoid data interference caused by name overwriting and misconfiguration.

[0031] Data sources refer to specific acquisition or access units associated with monitoring objects, equipment, stations, points, and monitoring elements, including real-time upload interfaces of equipment, database tables / views, message topics, or file import terminals.

[0032] S2: Within the original observation time window, the original monitoring data collected from the data source is standardized based on the semantic template: fields from different sources are mapped to a unified field set according to the field mapping rules, the units are unified according to the unit conversion rules, the spatial reference is unified according to the coordinate reference transformation rules, and the observation time is unified according to the time alignment rules to generate standardized monitoring data; the input standard string is generated according to the preset normalization serialization rules, and the input summary is calculated. Based on data source aliases and semantic templates, the platform retrieves the corresponding semantic templates according to the data source aliases, automatically loads the protocol adapter, authentication method, connection parameters, channel type and parsing rules, and automatically generates field mapping, time alignment, unit conversion and data entry orchestration tasks.

[0033] In some implementations, standardized monitoring data includes at least the project code, station code, measuring point code, monitoring element code, observation time, period, direction value (HA), slope distance (SD), zenith distance (VA), equipment type, data source alias, batch identifier (batchId), and calibration version. A batch identifier (batchId) is generated for each batch of standardized monitoring data, and the standardized records are sorted by field and formatted according to a standardized serialization rule to obtain the input standardized string. canonicalStr in This is to facilitate the encapsulation and consistency verification of subsequent governance evidence; In some implementations, the normalized serialization rules include at least: sorting according to a preset field order, standardizing field names, fixed-point formatting of numerical values ​​after standardizing units, standardizing time format, null value encoding, Boolean value standardization, and character set standardization, and employing a cryptographic hash function. H (·) Calculate the input summary Enter the digest. in As a fixed-length fingerprint of this batch of standardized input data, it is used for the genealogical association between the original data and subsequent adjustment, governance, and fusion results, duplicate input identification, anti-tampering verification, and governance basis package EP index.

[0034] S3: Configure the solution time window. Within the solution time window, perform adjustment on the standardized monitoring data according to the observation equation type, error model parameters, and accuracy level in the semantic template to obtain the solution results. Construct a multidimensional quality vector for the solution results. The multidimensional quality vector includes residual quality components, stability quality components, integrity quality components, timeliness quality components, and cross-source consistency quality components. Map the multidimensional quality vector to a comprehensive quality score in a weighted sum manner. Based on the comparison results of the comprehensive quality score and the cross-source consistency quality components with the threshold set, output a credibility classification label. In some implementations, the adjustment calculation employs a weighted least squares or robust weighted least squares model to minimize the weighted sum of squares of the residuals of the observation equation; wherein the observation equation is: ; ; ; in, y Let be the observation vector, representing the standardized monitoring data; A The design matrix represents the functional relationship between the observed values ​​and the parameters to be estimated. The observed values ​​refer to the standardized direction values ​​HA, slant distance SD, zenith distance VA, elevation, displacement increment, or other monitoring elements. The parameters to be estimated refer to the coordinates, elevation, displacement, or deformation parameters of the measuring points obtained through adjustment calculations. x The vector of parameters to be estimated; e The observation error vector refers to the set of deviations between each observed value and the theoretical value obtained by substituting the parameter to be estimated into the observation equation; Let be the mathematical expectation of the error vector; Let be the covariance matrix of the error vector; The observation error covariance matrix; The corresponding weighted least squares adjustment solution is: ; ; in, The parameter estimates obtained from the adjustment solution represent the values ​​based on the observed vectors. y The estimated results of the coordinates, elevation, displacement, deformation or other monitoring elements of the measuring points obtained after adjustment calculation; P The weight matrix is ​​determined by the inverse of the covariance matrix, and the weights are determined a priori by the precision level in the semantic template. Design matrix A The transpose of .

[0035] Residual mass component q resIt is calculated based on the difference between the adjustment residual and the preset residual threshold, and the expression is: ; Stability mass component q stab It is calculated based on the fluctuation of the displacement sequence or coordinate sequence within the historical window, and the expression is: ; Integrity mass component q comp It is calculated based on the ratio of the number of measured points in the current period to the number of points that should be measured in the current period, and the expression is: q comp = M / N ; Timeliness quality component q time It is calculated based on the ratio of data arrival delay to the station's continuous measurement time, and the expression is: ; in, u For residuals; u th The residual threshold; CV The coefficient of variation represents the displacement sequence within the historical window; CV th The stability threshold is the upper limit allowed to characterize the fluctuation of displacement or coordinate sequences within a historical window. It can be preset according to the monitoring element type, engineering stage, or project configuration parameters. Δt For data arrival delay; sampling max The duration of continuous measurement at the station.

[0036] In some implementations, the cross-source consistency quality component is calculated by normalizing the differences between observations from multiple data sources within the same time window for the same measuring point and the same monitoring element, and the accuracy level in the semantic template. The expression is: ; ; in, For the first i Data source results For reference purposes, this is a historical reliable source or fused median. Determined by accuracy level and historical residual statistics. D Indicates displacement or cumulative displacement. X t Indicates the first t The displacement value or cumulative displacement value corresponding to the current data source. X0 indicates the reference result for the same monitoring element at the same measuring point. The reference result may be the initial benchmark value, the historical reliable value, or the reference fusion value of the same group of results.

[0037] In some implementations, the trust rating label C includes at least three levels: trustworthy, doubtful, and invalid; when the overall quality score Q ≥ 1 and cross-source consistency quality component ≥ c1 When C is reliable, then C is trustworthy; when 2≤Q< 1 or c2 ≤Cross-source consistency quality component< c1 When Q < 0, C is suspicious; when Q < 0, C is suspicious. 2 or cross-source consistent quality component < c2 In this case, C is invalid; where 1. 2. c1 , c2 The threshold set is preset; when C is invalid, the time window for recalculation is entered.

[0038] S4: Generate a governance plan based on the multidimensional quality vector, comprehensive quality score, trust level label, and a preset set of rule templates. The governance plan includes a sequence of governance operators and a corresponding parameter set. Perform statistical filtering, rate fitting correction, and multi-source dynamic weighting fusion on the standardized monitoring data and solution results according to the governance plan to obtain the governance results. Encapsulate the governance results into a governance basis package, which includes an input summary, governance plan, multidimensional quality vector, comprehensive quality score, and trust level label. In some implementations, such as Figure 4 As shown, the platform has a pre-set set of governance rule templates. Each governance rule template includes a trigger predicate, an operator sequence, a parameter set, and a priority. When the multidimensional quality vector, the comprehensive quality score, and the trust level label satisfy the trigger predicate, and when the Boolean expression corresponding to the trigger predicate evaluates to true, the rule template is considered to meet the triggering condition. The Boolean expression consists of the multidimensional quality vector, the comprehensive quality score, the trust level label, and the number of missed detection points. L When combining threshold sets, the corresponding governance rule template is selected and the solutions are obtained by merging them according to priority to obtain the governance plan. G=(O,P, ver) The platform selects a set of rule templates whose triggering conditions are met. R Then sort by priority p rj Sort the operator sequences for each rule template from highest to lowest. O jThe processes are sequentially merged, and duplicate operators are deduplicated. Conflicting parameters are either prioritized or determined according to preset conflict rules to arrive at the final governance plan. G operator sequences in O and parameter set P According to the operator sequence in the governance plan, the standardized monitoring data and solution results are governed sequentially, and the multidimensional quality vector and comprehensive quality score are updated after each operator is executed. The abnormal data removed during the execution process, the corrected parameter records, and the intermediate calculation results are used as evidence information and are encapsulated together with the governance plan, the updated multidimensional quality vector, and the comprehensive quality score into the governance basis package.

[0039] Statistical filtering selects different filter families in the rule template, such as the three-standard-deviation criterion, Hampel filtering, or quantile truncation, and writes the summary of the excluded sample set into the governance basis package EP; the rate fitting can adopt piecewise rate fitting: the window length is adaptively selected within the sliding window, the window is divided into segment A and segment B, the average displacement and rate are calculated respectively, and the fitting value of the current period is estimated based on the previous period results and time intervals, and the window length is jointly determined by the stability quality component and the timeliness quality component.

[0040] Furthermore, in the multi-source fusion stage, the platform calculates the overall quality score Q of each source and the cross-source consistency quality component q. cons Determine the fusion weights and obtain the fusion result. D fuse Exponentially normalized weights are used: ; ; in, , These are configurable parameters, determined by the project phase context. Indicates the first The weighting coefficients of each data source in multi-source fusion computing; Indicates the first The cross-source consistency quality component corresponding to each data source is used to characterize the degree of consistency between the results of that data source and the results of other data sources under the same measurement point and the same monitoring element. When cross-source consistency verification fails (e.g., q...), cons Below the threshold cons If the trusted label C is invalid, enter the recalculation time window for recalculation or manual review.

[0041] In some implementations, the governance basis package also includes a governance plan version number, used to identify the rule template version and support traceability and reproduction, and the set of data sources participating in the fusion. This refers to the set of all data source identifiers participating in the current multi-source fusion calculation, such as data source aliases, device numbers or channel numbers, and the summary of removed samples. It also refers to the summary information of samples removed during the governance or fusion process due to anomalies, missing measurements, exceeding limits, or substandard quality, such as sample identifiers, removal reasons, and corresponding summary values, as well as key parameter fingerprints. Finally, it refers to the unique summary identifier calculated after the key parameters used in this governance and fusion are serialized according to predetermined rules, used for parameter traceability and consistency verification. A genealogical key value is generated for each governance result. This genealogical key value includes the project identifier, measurement point identifier, timestamp, data type, governance plan version number, and input summary. A fixed-length key value is calculated using the hash function H(·), with the expression: ; in, p Indicates the project identifier; pt Indicates the measurement point identification; t Represents a timestamp; type Indicates the data type; ver indicates the governance plan version number.

[0042] The governance basis package uses phylogenetic keys as indexes to establish a one-to-one association with the corresponding original monitoring data, standardized monitoring data, solution results, and governance results.

[0043] S5: Configure the push time window and the backfill recalculation time window, and define the push gate function. The judgment conditions of the push gate function include the comprehensive quality score and the trust level label generated based on the multi-dimensional quality vector. Based on the judgment result of the push gate function, decide whether to distribute the governance results to the outside world within the push time window. If the judgment fails, trigger the backfill recalculation time window to perform backfill recalculation. If the judgment is successful, distribute the governance results to the outside world and complete the automatic storage and fusion governance of the original monitoring data.

[0044] In some implementations, a push gate function is defined, and its judgment conditions include: the overall quality score is not lower than a preset quality threshold, the cross-source consistency quality component is not lower than a preset consistency threshold, and the trust level label is trustworthy.

[0045] When the judgment conditions are met, the push gate function is successful, and the governance results are distributed to external platforms or interfaces within the push time window, and the audit log is recorded. If any judgment condition is not met, the push gate function fails to judge, generates a backfill task, triggers the backfill recalculation time window to execute backfill recalculation, adjusts parameters to recalculate and re-manage, and marks the management results as pending backfill status on the visualization interface, while writing the reason for judgment failure into the management basis package.

[0046] When the push gate function fails to determine the problem, based on the input summary, governance plan, multidimensional quality vector and comprehensive quality score in the governance basis package, the parameters of the adjustment solution or the operator sequence of the governance plan are adjusted within the backfill recalculation time window, and the solution and governance are re-executed until the newly generated governance results meet the determination conditions of the push gate function.

[0047] In some implementations, such as Figure 3 As shown, the original observation time window, solution time window, push time window, and recalculation time window are configured in a linked manner: the solution time window covers the original observation time window, and the push time window is no earlier than the end time of the solution time window; the acquisition, solution, management, push, and recalculation tasks are linked and scheduled in a finite state machine manner; the state transition of the finite state machine is driven by the comprehensive quality score, the credibility classification label, and the number of missed measurement points.

[0048] Number of missed measurement points L for: ; in, N This represents the number of measurement points to be measured in the current period. M This represents the number of measured points in the current period.

[0049] State transition refers to the process by which a finite state machine switches from one processing state to the next processing state based on the current data quality and time window conditions.

[0050] When the data collection state ends and M When the value is >0, the system enters the solution state; when the solution state is completed and C≠invalid, the system enters the governance state; when the governance state is completed and the push gate function Gate is successfully determined, the system enters the push state; otherwise, the system enters the replenishment state.

[0051] In some implementations, the platform provides alarm notification services, pushing anomaly classification results, missed detection results, and handling suggestions via SMS, email, enterprise IM, or work order system; at the same time, it provides standardized interface services, such as RESTful API, message subscription, or database shared views, to meet the multi-role linkage of construction units, supervision units, and operation units.

[0052] As another important technical solution, this invention also provides a multi-sensor monitoring data intelligent storage and fusion management system, comprising: The semantic template configuration module is used to establish the master data model of the data source, configure the semantic template for each data source and assign a unique data source alias, and configure the original observation time window; The data standardization and summary generation module is used to perform standardization processing on the raw monitoring data collected from the data source based on semantic templates within the original observation time window, and generate standardized monitoring data; it generates the input standard string according to the preset normalization serialization rules and calculates the input summary. The adjustment and quality assessment module is used to configure the adjustment time window. Within the adjustment time window, the standardized monitoring data is adjusted according to the observation equation type, error model parameters and accuracy level in the semantic template to obtain the adjustment results. A multidimensional quality vector is constructed from the adjustment results. The multidimensional quality vector is mapped to a comprehensive quality score in a weighted sum manner. Based on the comparison results of the comprehensive quality score and the cross-source consistency quality component with the threshold set, a credibility classification label is output. The governance execution and basis encapsulation module is used to generate a governance plan based on a multidimensional quality vector, a comprehensive quality score, a trust level label, and a preset set of rule templates. The governance plan includes a sequence of governance operators and a corresponding parameter set. Governance is performed on standardized monitoring data and solution results according to the governance plan to obtain governance results. The governance results are then encapsulated into a governance basis package, which includes an input summary, a governance plan, a multidimensional quality vector, a comprehensive quality score, and a trust level label. The distribution gate and recalculation module are used to configure the push time window and the recalculation time window, and define the push gate function. The judgment conditions of the push gate function include the comprehensive quality score and the trust level label. Based on the judgment result of the push gate function, it is determined whether to distribute the governance results to the outside world within the push time window. If the judgment fails, the recalculation time window is triggered to perform recalculation. If the judgment is successful, the governance results are distributed to the outside world, and the automatic storage and fusion governance of the original monitoring data are completed.

[0053] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent substitutions, and improvements made by those skilled in the art to the above embodiments without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. A multi-element sensing monitoring data intelligent warehousing and fusion management method, characterized in that, The method comprises the following steps: S1: establishing a master data model of a data source, configuring a semantic template for each data source and assigning a unique data source alias, and configuring an original observation time window; S2: performing standardization processing on the original monitoring data collected from the data source based on the semantic template within the original observation time window to generate standardized monitoring data; generating an input specification string according to a preset normalization serialization rule, and calculating an input summary; S3: configuring a calculation time window, performing adjustment calculation on the standardized monitoring data according to the observation equation type, error model parameter and precision level in the semantic template within the calculation time window to obtain calculation results, and constructing a multi-dimensional quality vector for the calculation results; mapping the multi-dimensional quality vector into a comprehensive quality score in a weighted sum manner, and outputting a trusted classification label based on a comparison result of the comprehensive quality score and a cross-source consistency quality component relative to a threshold set; S4: generating a management plan according to the multi-dimensional quality vector, the comprehensive quality score, the trusted classification label and a preset rule template set, wherein the management plan comprises a management operator sequence and a corresponding parameter set; performing management on the standardized monitoring data and the calculation results according to the management plan to obtain management results, and packaging a management basis package for the management results, wherein the management basis comprises the input summary, the management plan, the multi-dimensional quality vector, the comprehensive quality score and the trusted classification label; S5: configuring a push time window and a backfill recalculation time window, defining a push gate function, wherein the determination conditions of the push gate function include the comprehensive quality score and the trusted classification label; determining whether to distribute the management results externally within the push time window according to the determination result of the push gate function, triggering the backfill recalculation time window to perform backfill recalculation when the determination fails, distributing the management results externally after the determination succeeds, and completing the automatic warehousing and fusion management of the original monitoring data. 2.The multi-element sensing monitoring data intelligent warehousing and fusion management method of claim 1, wherein In step S1, the semantic template includes field mapping rules, unit conversion rules, coordinate reference conversion rules, time alignment rules, observation equation types, error model parameters and precision levels; step S1 further comprises performing uniqueness verification on the unique alias of the data source, binding the unique alias with the semantic template, and identifying and verifying the source of the original monitoring data.

3. The multi-element sensing monitoring data intelligent warehousing and fusion management method according to claim 1, characterized in that: In steps S1, S3 and S5, the original observation time window, the calculation time window, the push time window and the backfill recalculation time window are configured in a linked manner: the calculation time window covers the original observation time window, and the push time window is not earlier than the end time of the calculation time window; the acquisition, calculation, management, push and backfill tasks are scheduled in a linked manner in a finite state machine manner; the state transition of the finite state machine is driven by the comprehensive quality score Q, the trusted classification label C and the number of missing points L.

4. The multi-element sensing monitoring data intelligent warehousing and fusion management method according to claim 1, characterized in that: In step S3, the multi-dimensional quality vector includes: a residual quality component calculated based on a difference between the adjustment residual and a preset residual threshold; a stability quality component calculated based on fluctuation of the displacement sequence or the coordinate sequence within the historical window; an integrity quality component calculated based on a ratio of the current measured point number to the current expected measured point number; a timeliness quality component calculated based on a ratio of the data arrival time delay to the station continuous measurement time; and a cross-source consistency quality component calculated based on a difference between observation results of multiple data sources of the same monitoring element at the same time window and the accuracy level in the semantic template.

5. The multi-element sensing monitoring data intelligent warehousing and fusion management method according to claim 1, characterized in that: In step S4, the management plan is generated according to the multi-dimensional quality vector, the comprehensive quality score, the trusted classification label and the preset rule template set, the standardized monitoring data and the calculation result are executed according to the management plan, and the management result is obtained, and the specific process is as follows: The platform is preset with a management rule template set, each management rule template includes a trigger predicate, an operator sequence, a parameter set and a priority; when the multi-dimensional quality vector, the comprehensive quality score and the trusted classification label meet the trigger predicate, the corresponding management rule template is selected and solved according to the priority to obtain the management plan; the standardized monitoring data and the calculation result are executed according to the operator sequence in the management plan, and the multi-dimensional quality vector and the comprehensive quality score are updated after each operator is executed; The abnormal data, the corrected parameters and the intermediate calculation results excluded in the execution process are taken as evidence information, and are packaged into a management basis package together with the management plan, the updated multi-dimensional quality vector and the comprehensive quality score.

6. The multi-element sensing monitoring data intelligent warehousing and fusion management method according to claim 1, characterized in that: In step S4, the management basis package is indexed by a pedigree key value, and is associated with the corresponding original monitoring data, the standardized monitoring data, the calculation result and the management result; the management basis package further includes a version number of the management plan, a data source set participating in the fusion, an excluded sample abstract and a key parameter fingerprint; A pedigree key value is generated for each piece of management result, and the pedigree key value includes a project identifier, a monitoring point identifier, a timestamp, a data type, a management plan version number and an input abstract.

7. The multi-element sensing monitoring data intelligent warehousing and fusion management method according to claim 1, characterized in that: In step S2, the original monitoring data collected from the data source is standardized according to the semantic template, specifically: different sources are mapped to a unified field set according to a field mapping rule, the dimensions are unified according to a unit conversion rule, the space reference is unified according to a coordinate reference conversion rule, and the observation time is unified according to a time alignment rule.

8. The multi-element sensing monitoring data intelligent warehousing and fusion management method according to claim 1, characterized in that: In step S5, whether the management result is distributed externally is determined by the following method: The push gate function is defined, and the determination conditions include: the comprehensive quality score is not lower than the preset quality threshold, the cross-source consistency quality component is not lower than the preset consistency threshold, and the trust classification label is trusted; When the determination conditions are met, the push gate function is determined to be successful, the governance results are distributed to the external platform or interface within the push time window, and the audit log is recorded; When any of the determination conditions is not met, the push gate function is determined to fail, a backfill task is generated, a backfill recalculation time window is triggered to perform backfill recalculation, and the governance results are marked as a backfill state in the visual interface, and the determination failure reason is written into the governance basis package.

9. The multi-element sensing monitoring data intelligent warehouse and fusion governance method according to claim 1, characterized in that: When the push gate function is determined to fail, based on the input summary, the governance plan, the multi-dimensional quality vector and the comprehensive quality score in the governance basis package, the parameters of the adjustment solution or the operator sequence of the governance plan are adjusted within the backfill recalculation time window, and the solution and governance are re-executed until the newly generated governance results meet the determination conditions of the push gate function.

10. A multi-element sensing monitoring data intelligent warehousing and fusion management system, characterized in that, It includes: The semantic template configuration module is used to establish the main data model of the data source, configure the semantic template for each data source and assign a unique data source alias, and configure the original observation time window; The data standardization and summary generation module is used to perform standardization processing on the original monitoring data collected from the data source based on the semantic template within the original observation time window to generate standardized monitoring data; input specification strings are generated according to the preset normalization serialization rules, and input summaries are calculated; The adjustment solution and quality evaluation module is used to configure a solution time window, perform adjustment solution on the standardized monitoring data according to the observation equation type, error model parameter and precision level in the semantic template within the solution time window to obtain a solution result, and construct a multi-dimensional quality vector for the solution result; The multi-dimensional quality vector is mapped to a comprehensive quality score in the form of weighted sum, and a trust classification label is output based on the comparison results of the comprehensive quality score and the cross-source consistency quality component with respect to the threshold set; The governance execution and basis packaging module is used to generate a governance plan according to the multi-dimensional quality vector, the comprehensive quality score, the trust classification label and the preset rule template set, wherein the governance plan includes a governance operator sequence and a corresponding parameter set; the standardized monitoring data and the solution result are executed according to the governance plan to obtain governance results, and a governance basis package is packaged for the governance results, wherein the governance basis includes an input summary, a governance plan, a multi-dimensional quality vector, a comprehensive quality score and a trust classification label; The distribution gate and backfill recalculation module is used to configure a push time window and a backfill recalculation time window, define a push gate function, and determine whether to distribute the governance results to the outside within the push time window according to the determination result of the push gate function; when the determination fails, the backfill recalculation time window is triggered to perform backfill recalculation, and when the determination succeeds, the governance results are distributed to the outside, and the automatic warehouse and fusion governance of the original monitoring data are completed.

Images