Multi-source heterogeneous gas tunnel safety monitoring data cleaning and fusion method and system

Abstract

The present application relates to big data resource service technical field, specifically to multi-source heterogeneous gas tunnel safety monitoring data cleaning fusion method and system, including the following steps: collecting terminal number, monitoring area, sampling time, sending serial number, gas concentration and state record, chain is built according to terminal number and sending serial number, determine the link deviation amount of residence reference, correct sampling time and gather according to area, calculate concentration difference in combination with air flow direction, generate dynamic threshold to exclude abnormal records, assess sampling credibility and weighted fusion concentration.In the present application, the transmission process and monitoring scene are connected through link residence correction and regional space-time collection, spatial difference and state anomaly are identified through air flow direction constraint and dynamic threshold filtering, credibility weighting weakens the influence of retransmission, time deviation, signal attenuation and spatial anomaly, reduces the delay, repetition, drift data involved in fusion, improves the regional concentration stability and early warning output reliability.

Description

Technical Field

[0001] This invention relates to the field of big data resource service technology, and in particular to a method and system for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data. Background Technology

[0002] The field of big data resource service technology involves database organization, distributed storage, index management, data querying, and resource scheduling for massive amounts of data. Among these, the traditional method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data involves writing gas concentration, environmental parameters, equipment status, and alarm records generated by different monitoring terminals within the tunnel into a database or cloud database. A data index is then established using data tables, timestamps, sensor numbers, and monitoring area fields. On the server side, record deduplication, field normalization, abnormal record filtering, cross-table joins, and the fusion results are then stored in the database.

[0003] Existing technologies receive tunnel monitoring records using databases or cloud databases, relying on timestamps, sensor numbers, and monitoring area fields to build indexes. On the server side, deduplication, field normalization, anomaly filtering, cross-table association, and database entry are performed. However, during operation, it is difficult to map the differences in transmission sequence number, retransmission count, arrival time, and sampling time in a timely manner. Terminal self-checks and power supply fluctuations are easily mixed with sudden changes in actual concentration. Differences between the upwind and downwind sides caused by wind direction lack regional verification. Low signal records and spatial anomaly records may be involved in fusion, and early warning results are easily interfered with by delayed, duplicate, and drifting data. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data. This method enables gas monitoring data generated by different monitoring terminals in the tunnel to be cleaned, corrected, and fused based on link dwell characteristics, regional airflow relationships, and sampling reliability, even when there are transmission delays, retransmissions, abnormal terminal status, power supply abnormalities, signal fluctuations, and spatial differences on the upwind and downwind sides. This results in regional fusion concentration and early warning results corresponding to the monitoring area and collection window.

[0005] To achieve the above objectives, this invention provides a method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data, comprising: acquiring records containing terminal number, monitoring area, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, retransmission count, signal strength, and airflow direction; establishing a link according to the terminal number and transmission sequence number, calculating the dwell time between the arrival time and the sampling time, using the record with the continuous transmission sequence number and the lowest retransmission count as the dwell benchmark to obtain the link deviation; correcting the sampling time according to the dwell benchmark, aggregating the data according to the monitoring area and the corrected time, and matching the upwind and downwind records with the airflow direction to calculate the regional concentration difference; and generating a data set using the median and dispersion of the regional concentration difference. A dynamic threshold is used to filter out records with abnormal self-inspection status, abnormal power supply status, excessive link deviation, and abnormal regional concentration, forming a queue to be fused. The sampling reliability of the records in the queue to be fused is evaluated, and the reliability coefficient is normalized according to the signal strength, and retransmission, time deviation, and spatial abnormal attenuation are superimposed. Records in the same queue below the reliability threshold are removed, and the corresponding gas concentrations are weighted and fused using the sampling reliability as the weight to generate a regional fused concentration. The record with the highest contribution is determined according to the weight contribution of each record to the regional fused concentration. When the regional fused concentration exceeds the warning threshold, a warning result containing the monitoring area, collection window, regional fused concentration, and the record with the highest contribution is output.

[0006] During the data acquisition process, the original messages sent by the monitoring terminal through the direct reporting link or the relay reporting link can be obtained. The header and payload fields of the original messages are parsed, and the parsed terminal number, the sending sequence number, the sampling time, the gas concentration, the self-test status, the power supply status, the number of retransmissions, and the signal strength are written into the original record. The arrival time is taken as the receiving time of the original message written by the receiving gateway to obtain the first record. Then, the monitoring area and installation orientation corresponding to the terminal number in the terminal configuration table are obtained, and the airflow direction corresponding to the first record is determined by combining the airflow direction configuration table. The monitoring area and the airflow direction are added to the first record to generate the record to be established.

[0007] As a configuration verification for the record to be established, the version time of the terminal configuration table and the version time of the airflow direction configuration table are obtained. The sampling time is compared with the effective time period of the terminal configuration table and the effective time period of the airflow direction configuration table. When the sampling time does not fall within the effective time period of the terminal configuration table or the effective time period of the airflow direction configuration table, the corresponding monitoring area field or airflow direction field is marked as a configuration missing state, and the configuration missing state is written to the record to be established. When the sampling time falls within the effective time period of both the terminal configuration table and the effective time period of the airflow direction configuration table, the monitoring area and the airflow direction are determined according to the configuration item with the closest version time and not later than the sampling time. The monitoring area, the airflow direction, and the corresponding configuration matching result are written to the record to be established, generating a record to be established with completed configuration verification.

[0008] In the link establishment and link deviation calculation, the records to be established under the same terminal number are obtained and sorted by the sending sequence number. It is determined whether the sending sequence numbers of two adjacent records to be established are consecutive, and the records with consecutive sending sequence numbers and non-inverted sampling times are grouped into the same transmission chain to generate a link record sequence. The time difference between the arrival time and the sampling time of each record in the link record sequence is calculated, and the time difference is determined as the dwell time. Records with consecutive sending sequence numbers, the lowest number of retransmissions, and dwell time in a stable range are selected from the link record sequence to generate a dwell reference record. The dwell time of each record in the link record sequence is compared with the dwell time of the dwell reference record, and the time deviation obtained from the comparison is determined as the link deviation.

[0009] In the sampling time correction and regional concentration difference calculation, the dwell time and dwell reference recorded in the same transmission chain are obtained. The time offset direction and time offset magnitude of the recorded dwell time relative to the dwell reference are calculated. The sampling time is corrected by callback and forward correction according to the time offset direction and time offset magnitude to generate a correction time. The corrected records are grouped into time windows according to the monitoring area and the correction time. Records that are in the same regional grouping window and whose airflow directions correspond to each other are assigned to regional pairing groups. Upwind and downwind records are determined in the regional pairing groups. The gas concentrations of the upwind and downwind records are compared, and the difference in gas concentrations on both sides is determined as the regional concentration difference. The regional pairing group can be determined by matching multiple records that have completed time correction in the same monitoring area with a preset grouping time window. It is formed when the airflow direction and terminal installation orientation of the candidate pairing records form an upwind and downwind correspondence, and the difference in the correction time of the two candidate pairing records does not exceed the pairing time deviation threshold.

[0010] When forming the queue to be merged, the regional concentration difference generated within a continuous aggregation window in the same monitoring area is obtained, invalid pairing groups that failed to match the upwind and downwind records are removed, the median and dispersion of the remaining regional concentration differences are calculated, and the dynamic threshold of the regional concentration difference is determined based on the median and the dispersion; the self-test status, power supply status, link deviation, and regional concentration difference of each record are compared with the status qualification condition, the link deviation threshold, and the dynamic threshold of the regional concentration difference, respectively, and records that do not meet any comparison condition are filtered out, thus generating the queue to be merged.

[0011] When assessing the sampling reliability, the signal strength, retransmission count, link deviation, and regional concentration difference of each record in the queue to be fused are obtained. The signal strength is matched with the terminal communication quality interval table, and the reliability coefficient is determined according to the matched communication quality interval. The retransmission attenuation is determined based on the retransmission count, the time deviation attenuation is determined based on the proximity of the link deviation to the link deviation threshold, and the spatial anomaly attenuation is determined based on the proximity of the regional concentration difference to the regional concentration difference dynamic threshold. The retransmission attenuation, time deviation attenuation, and spatial anomaly attenuation are deducted sequentially from the reliability coefficient to generate the sampling reliability.

[0012] When generating the regional fusion concentration and the highest contribution record, the queue records to be fused within the same monitoring area and the same aggregation window are obtained. The sampling confidence of each record is compared with the confidence threshold, and records with sampling confidence lower than the confidence threshold are removed to obtain qualified fusion records. The concentration fusion weight is determined according to the sampling confidence of the qualified fusion records, and the corresponding gas concentration is weighted and summarized using the concentration fusion weight to generate the regional fusion concentration. The weight contribution of each qualified fusion record to the regional fusion concentration is compared, and the qualified fusion record with the highest weight contribution is determined as the highest contribution record. When the regional fusion concentration exceeds the warning threshold, a warning result is output.

[0013] This invention also provides a multi-source heterogeneous gas tunnel safety monitoring data cleaning and fusion system. The system is used to implement the above method and includes: a data acquisition module for acquiring records containing terminal number, monitoring area, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, retransmission count, signal strength, and airflow direction; a link establishment and link deviation calculation module for establishing a link according to the terminal number and transmission sequence number, calculating the dwell time between the arrival time and sampling time, using the record with the consecutive transmission sequence number and the lowest retransmission count as the dwell benchmark to obtain the link deviation; a sampling time correction and regional concentration difference calculation module for correcting the sampling time according to the dwell benchmark, aggregating data according to the monitoring area and correction time, and matching upwind and downwind records with the airflow direction to calculate the regional concentration difference; and a dynamic threshold anomaly screening module. The system is used to generate a dynamic threshold based on the median and dispersion of the regional concentration difference, and to filter out records with abnormal self-inspection status, abnormal power supply status, excessive link deviation, and abnormal regional concentration difference, forming a queue to be fused; a sampling reliability assessment module is used to assess the sampling reliability of the records in the queue to be fused, normalize it to a reliability coefficient based on the signal strength, and superimpose retransmission, time deviation, and spatial abnormal attenuation; a weighted fusion and early warning output module is used to remove records in the same area queue that are lower than the reliability threshold, perform weighted fusion of the corresponding gas concentrations with the sampling reliability as the weight, generate a regional fused concentration, and determine the highest contribution record based on the weight contribution of each record to the regional fused concentration; when the regional fused concentration exceeds the early warning threshold, an early warning result including the monitoring area, aggregation window, regional fused concentration, and the highest contribution record is output.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, a link relationship is established based on the terminal number and the transmission sequence number. The dwell time between the arrival time and the sampling time is converted into a link deviation. The dwell benchmark is determined by records with continuous transmission and low retransmission. After correcting the sampling time, the records are collected according to the monitoring area and the correction time. The concentration difference between the upwind and downwind sides is compared in combination with the wind direction. A dynamic threshold is formed by the median and dispersion of the regional concentration difference. Records with self-test anomalies, power supply anomalies, excessive link deviation, and regional difference anomalies are excluded simultaneously. The reliability assessment incorporates signal strength, retransmission, time deviation, and spatial anomalies into the attenuation weight. Low reliability records are removed when merging records in the same area. High reliability records contribute significantly to the gas concentration. The early warning output can reduce the impact of delay, repetition, drift, and spatial error. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the main process of the multi-source heterogeneous gas tunnel safety monitoring data cleaning and fusion method of the present invention; Figure 2 This is a diagram showing the data flow of the original message parsing and configuration verification in this invention. Figure 3 This is the logic diagram for link dwell correction and region pairing in this invention; Figure 4 This is the logic diagram for the dynamic threshold screening and sampling reliability assessment of the present invention; Figure 5 This is a data flow diagram of the system module collaboration and early warning output of the present invention. Detailed Implementation

[0017] The following describes feasible implementation methods and systems for cleaning and fusing safety monitoring data of multi-source heterogeneous gas tunnels, in conjunction with the technical solutions described in this invention. The following embodiments are based on the fields, steps, modules, and processing relationships already recorded in the scope of protection text, invention content, and application materials. Terms such as terminal number, monitoring section, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, retransmission count, signal strength, airflow direction, dwell time, dwell reference, link deviation, correction time, section concentration difference, dynamic limit of section concentration difference, queue to be fused, reliability coefficient, sampling reliability, section fused concentration, highest contribution record, and early warning result are used consistently. Subsequent... Figures 1 to 5 This is used to illustrate the main process, local data flow, logical judgment, status transmission, and system module collaboration relationships in the embodiments.

[0018] Example 1

[0019] Please see Figures 1 to 5 This embodiment provides a method for cleaning and fusing safety monitoring data of multi-source heterogeneous gas tunnels. The method takes the gas monitoring records reported by the monitoring terminal in the tunnel as input and the section fusion concentration and early warning results generated according to the monitoring section and collection window as output. Through link dwell correction, airflow pairing, dynamic limit screening and sampling credibility weighted fusion, a traceable data cleaning and fusion process is formed.

[0020] S1: Obtain a record containing the terminal number, monitoring section, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, retransmission count, signal strength, and airflow direction. The input for this step is the original message sent by the monitoring terminal via a direct reporting link or a relay reporting link. The execution targets are the header field, payload field, and the reception time written by the receiving gateway of the original message. After the original message enters the receiving gateway, a parsable message object is first formed. Then, the parsing process reads the terminal number, transmission sequence number, sampling time, gas concentration, self-test status, power supply status, retransmission count, and signal strength. The arrival time is taken as the reception time written by the receiving gateway, resulting in the first record. The first record has not yet completed the supplementation of the monitoring section and airflow direction and does not directly enter the link establishment process.

[0021] S111: Parse the header and payload fields of the original message. The header field is used to obtain the terminal number and transmission sequence number, and the payload field is used to obtain the sampling time, gas concentration, self-test status, power supply status, retransmission count, and signal strength. After parsing, the above fields and arrival time are written together into the original record to form the first record. If the same original message arrives multiple times due to the relay reporting link, an independent record is formed with the arrival time, retransmission count, and signal strength corresponding to each message. Subsequent processing is based on the continuity of the transmission sequence number, the number of retransmissions, and the link deviation.

[0022] S112, supplement the first record according to the terminal configuration table and the airflow direction configuration table. The terminal configuration table provides the monitoring section and installation orientation corresponding to the terminal number, and the airflow direction configuration table provides the airflow direction corresponding to the sampling time. During processing, first locate the configuration item in the terminal configuration table according to the terminal number, and then determine the airflow direction corresponding to the first record by combining the installation orientation and the airflow direction configuration table. Add the monitoring section and airflow direction to the first record to generate the link to be established record. The output position of the link to be established record is used in the subsequent link establishment process, and its monitoring section and airflow direction are used as the data basis for collection and pairing with the upwind and downwind sides.

[0023] S113, Perform configuration timeliness verification on the records to be established. Obtain the version time of the terminal configuration table and the version time of the airflow direction configuration table, and compare the sampling time with the effective time periods of the terminal configuration table and the airflow direction configuration table, respectively. When the sampling time does not fall within the effective time period of the terminal configuration table, mark the monitoring segment field as having a configuration missing status; when the sampling time does not fall within the effective time period of the airflow direction configuration table, mark the airflow direction field as having a configuration missing status. The configuration missing status is written into the subsequent processing chain along with the records to be established, and is used to identify records that cannot complete pairing or spatial verification during status verification and queue formation.

[0024] S114, when the sampling time falls within the effective time period of both the terminal configuration table and the airflow direction configuration table, the monitoring segment and airflow direction are determined according to the configuration item with the closest version time and no later than the sampling time, and the monitoring segment, airflow direction, and configuration matching result are written into the link establishment record. The configuration matching result is used to indicate that the source of the segment and direction of the record has been confirmed by the configuration version corresponding to the sampling time. The subsequent link establishment process receives the link establishment record that has completed configuration verification as well as the link establishment record with a configuration missing status.

[0025] S2 establishes a link based on the terminal number and transmission sequence number, calculates the dwell time between the arrival time and the sampling time, and uses the record with the consecutive transmission sequence number and the lowest retransmission count as the dwell reference to obtain the link deviation. The input of this step is the record to be established generated in S1, the processing object is the sequence of records sorted by transmission sequence number under the same terminal number, and the output is the transmission link, dwell time, dwell reference record, and link deviation.

[0026] S211: Obtain the records to be established for a link under the same terminal number and sort them according to their transmission sequence number. After sorting, determine whether the transmission sequence numbers of two adjacent records to be established for a link are consecutive, and also determine whether the sampling time has not been reversed. Records with consecutive transmission sequence numbers and non-reversed sampling times are grouped into the same transmission link, generating a link record sequence; when the transmission sequence numbers are not consecutive or the sampling time is reversed, the current transmission link is disconnected at that position, and subsequent records re-enter a new transmission link for evaluation. Through this process, retransmission records, out-of-order arrival records, and link discontinuity records will not be mixed in the same continuous link.

[0027] S212, calculate the time difference between the arrival time and the sampling time of each record in the link record sequence, and determine this time difference as the dwell time. The dwell time represents the link lag state between the formation of the record and its reception at the gateway. Subsequently, select records from the link record sequence that have consecutive transmission sequence numbers, the lowest number of retransmissions, and dwell times within a stable range to generate dwell reference records. The dwell reference records serve as the reference object for time correction and link deviation comparison within the same transmission link; if a record has a high number of retransmissions or its dwell time deviates from the stable range, it is not used as a dwell reference record.

[0028] S213, compare the dwell time of each record in the link record sequence with the dwell time of the reference record, and determine the time deviation obtained from the comparison as the link deviation. The link deviation is written into the subsequent processing chain along with the corresponding record, serving as input data for sampling time correction, dynamic limit filtering, and sampling confidence decay. If a record lacks an arrival time or sampling time and cannot form a dwell time, then the record cannot form a valid link deviation and will be processed as data that does not meet the link verification requirements.

[0029] S3: Correct the sampling time according to the residence benchmark, aggregate the data according to the monitoring section and correction time, and match the upwind and downwind records with the airflow direction to calculate the section concentration difference. The inputs of this step are transmission chain, residence time, residence benchmark, link deviation, monitoring section, airflow direction and gas concentration, and the outputs are correction time, section pairing group and section concentration difference.

[0030] S311: Obtain the dwell time and dwell reference recorded in the same transmission chain, and determine the time offset direction and magnitude of the recorded dwell time relative to the dwell reference. If the recorded dwell time lags behind the dwell reference, the sampling time is corrected by retrospective adjustment according to the offset magnitude; if the recorded dwell time advances relative to the dwell reference, the sampling time is corrected by forward adjustment according to the offset magnitude. The corrected time is recorded as the corrected time and replaces the original sampling time in subsequent aggregation. This correction does not change the original sampling time field; it only generates the corrected time as the time reference required for fusion processing.

[0031] S312, records that have completed time correction are grouped into time windows according to monitoring segments and correction times. During processing, multiple records within the same monitoring segment are matched with preset grouping time windows. Records falling within the same preset grouping time window but with different terminal numbers are selected as candidate pairing records. The correction time difference of the candidate pairing records is also compared with the pairing time deviation limit. Only records that do not exceed the pairing time deviation limit are allowed to enter the pairing judgment.

[0032] S313, Obtain the wind direction and terminal installation orientation corresponding to the candidate pairing records. When two candidate pairing records form an upwind and downwind correspondence in the wind direction, and the correction time difference between the two candidate pairing records does not exceed the pairing time deviation limit, the two candidate pairing records are determined as a segment pairing group, and the upwind and downwind records are determined within the segment pairing group. If the wind direction field is in a missing configuration state, or the two records cannot form an upwind and downwind correspondence, then the candidate pairing record group does not form a valid segment pairing group.

[0033] S314. Compare the gas concentrations recorded on the upwind and downwind sides, and determine the difference in gas concentration between the two sides as the segment concentration difference. The segment concentration difference, along with the corresponding segment pairing group, monitoring segment, and collection window, is written into the subsequent dynamic limit anomaly screening process. This difference result is used to reflect the spatial concentration differences affected by wind direction within the same monitoring segment and the same collection window, enabling the subsequent screening process to identify abnormal records by combining segmented spatial relationships.

[0034] S4 generates dynamic limits based on the median and dispersion of the segment concentration difference, filtering out records with abnormal self-inspection status, abnormal power supply status, excessive link deviation, and abnormal segment concentration differences, forming a queue to be merged. The inputs to this step are the segment concentration difference within the continuous aggregation window, the self-inspection status, power supply status, link deviation, and segment concentration difference of each record, and the outputs are the dynamic limits of the segment concentration difference and the queue to be merged.

[0035] S411: Obtain the segment concentration difference generated within the continuous aggregation window for the same monitoring segment, and remove invalid pairings that failed to match upwind and downwind records. The remaining segment concentration difference serves as the data basis for generating dynamic limits. Calculate the median and dispersion of the remaining segment concentration difference, and determine the dynamic limits for the segment concentration difference based on the median and dispersion. The median is used to express the concentration state of the segment concentration difference within the continuous aggregation window, and the dispersion is used to express the fluctuation state of the segment concentration difference. Together, they form the anomaly judgment boundary that changes with the monitoring segment and aggregation window.

[0036] S412, compare the self-test status of each record with the status qualification conditions. Records with abnormal self-test status are not included in the fusion queue. Abnormal self-test status includes records where the monitoring terminal fails the self-test, the status field does not meet the qualification conditions, or the self-test status field is missing and the status judgment cannot be completed. This process is used to prevent the gas concentration generated when the terminal itself detects an abnormality from directly participating in the fusion.

[0037] S413, compare the power supply status of each record with the status qualification conditions. Records with abnormal power supply status are not included in the fusion queue. Abnormal power supply status includes records where the power supply status does not meet the qualification conditions, power supply fluctuations cause abnormal status fields, or the power supply status field is missing and status judgment cannot be completed. After records with abnormal power supply status are filtered out, the subsequent fusion process only accepts records whose power supply status meets the basic sampling conditions.

[0038] S414 compares the link deviation with the link deviation limit and the segment concentration difference with the segment concentration difference dynamic limit. Records with link deviations exceeding the limit are not included in the fusion queue; records with abnormal segment concentration differences are not included in the fusion queue. If a record fails to generate a valid link deviation or cannot be attributed to a valid segment concentration difference, the record is not considered a qualified fusion input. The records retained after the above comparisons of status, link, and spatial differences generate the fusion queue and are organized according to monitoring segments and collection windows.

[0039] S5 assesses the sampling reliability of the records in the queue to be merged, normalizing them to a reliability coefficient based on signal strength, and then adding retransmission, time deviation, and spatial anomaly attenuation. The inputs to this step are the signal strength, number of retransmissions, link deviation, and segment concentration difference of each record in the queue to be merged, and the output is the sampling reliability of each record.

[0040] S511, the signal strength is matched against the terminal communication quality interval table, and the confidence coefficient is determined according to the matched communication quality interval. The terminal communication quality interval table is used to map the signal strength to a confidence coefficient that can participate in the sampling confidence assessment. Different signal strength intervals correspond to different communication quality states. If the signal strength field is missing, or the signal strength cannot match the terminal communication quality interval table, the record cannot obtain a valid confidence coefficient and will not be included in the subsequent weighted fusion.

[0041] S512 determines the retransmission attenuation based on the number of retransmissions. A higher retransmission count indicates a more significant impact of retransmissions on the record corresponding to the same transmission sequence number during transmission; the sampling reliability of the corresponding record should be reduced by the retransmission attenuation. The retransmission attenuation is determined based on the record's own retransmission count, without incorporating undisclosed external evaluation data.

[0042] S513 determines the time deviation attenuation amount based on how close the link deviation amount is to the link deviation limit. The closer the link deviation amount is to the link deviation limit, the closer the time reliability of the record is to the abnormal boundary, even though it has not been screened out, and the time deviation attenuation amount is deducted accordingly. The input of this attenuation amount is the link deviation amount generated in S2 and the link deviation limit value used in S4, and the output is the sampling reliability applied to the same record.

[0043] S514 determines the spatial anomaly attenuation based on the proximity of the segment concentration difference to the dynamic limit of the segment concentration difference. The closer the segment concentration difference is to the dynamic limit, the closer the spatial difference state is to the anomaly boundary, even though the record was not screened out, and the spatial anomaly attenuation is deducted accordingly. Subsequently, the confidence coefficient is subtracted sequentially from the retransmission attenuation, time deviation attenuation, and spatial anomaly attenuation to generate the sampling confidence. The sampling confidence is bound to the corresponding record and serves as the input for subsequent confidence limit comparison and concentration fusion weight generation.

[0044] S6. Remove records in the same area queue that are below the confidence limit. Weight the corresponding gas concentrations using sampling confidence as the weight to generate the segment fusion concentration. Determine the record with the highest contribution based on the weighted contribution of each record to the segment fusion concentration. The input to this step is the records in the queue to be fused within the same monitoring segment and the same aggregation window, along with their sampling confidence. The output is the qualified fusion record, the segment fusion concentration, and the record with the highest contribution.

[0045] S611: Obtain the records to be fused within the same monitoring segment and the same collection window, and compare the sampling confidence of each record with the confidence limit. Records with a sampling confidence lower than the confidence limit are discarded, and the remaining records form qualified fusion records. If all records within the same collection window are lower than the confidence limit, then the collection window does not generate a valid segment fusion concentration, and the early warning judgment process does not use the low-confidence records in this window as a direct basis.

[0046] S612, the concentration fusion weight is determined according to the sampling reliability of qualified fusion records, and the corresponding gas concentrations are weighted and summarized using the concentration fusion weight to generate the segment fusion concentration. The concentration fusion weight is derived from the sampling reliability relationship between qualified fusion records within the same aggregation window; records with higher reliability contribute more to the segment fusion concentration than records with lower reliability. This weighted fusion process uses gas concentration as the fusion object, sampling reliability as the weight source, and the output result is the segment fusion concentration corresponding to the monitoring segment and aggregation window.

[0047] S613, compare the weighted contributions of each qualified fusion record to the fusion concentration of the section, and determine the qualified fusion record with the highest weighted contribution as the highest contribution record. The highest contribution record must retain at least its terminal number, monitoring section, calibration time, gas concentration, sampling reliability, and corresponding weighted contribution relationship for subsequent early warning result traceability. By determining the highest contribution record, the early warning output can not only provide the fusion concentration of the section, but also locate the source of the record that has the greatest impact on the fusion result.

[0048] S7: When the fusion concentration of a segment exceeds the warning limit, output a warning result including the monitoring segment, the aggregation window, the segment fusion concentration, and the highest contribution record. The inputs to this step are the segment fusion concentration, the highest contribution record, the monitoring segment, and the aggregation window generated in S6. The judgment object is the comparison between the segment fusion concentration and the warning limit. When the segment fusion concentration exceeds the warning limit, a warning result is generated; the warning result includes at least the monitoring segment, the aggregation window, the segment fusion concentration, and the highest contribution record. This warning result can be displayed, recorded, or processed in conjunction with subsequent receiving ends based on its field content. If the segment fusion concentration does not exceed the warning limit, the aggregation window does not output a warning result, but the segment fusion concentration and its corresponding qualified fusion record can still be used as input records for subsequent continuous aggregation window processing.

[0049] Through the continuous processing from S1 to S7 described above, after the original message enters, the record is sequentially formed into the first record, the record to be established, the transmission link, the dwell time, the link deviation, the correction time, the segment pairing group, the segment concentration difference, the dynamic limit of the segment concentration difference, the queue to be fused, the sampling confidence level, the segment fusion concentration, and the early warning result. Each intermediate data is generated from the preceding fields or the results of the preceding processing and is received by the subsequent steps, forming a complete data flow closed loop from sampling record to segment early warning.

[0050] Example 2

[0051] Please see Figure 1 and Figure 5 This embodiment provides a data cleaning and fusion system for safety monitoring of multi-source heterogeneous gas tunnels. The difference between this embodiment and Embodiment 1 is that this embodiment uses system modules to carry the processing chain described in Embodiment 1. The modules transmit data in the order of record input, link processing, time correction, anomaly screening, credibility assessment, weighted fusion, and early warning output; its input and output fields, judgment conditions, intermediate results, and output results are consistent with those of Embodiment 1.

[0052] The system includes a data acquisition module, a link establishment and link deviation calculation module, a sampling time correction and section concentration difference calculation module, a dynamic limit anomaly screening module, a sampling reliability assessment module, and a weighted fusion and early warning output module. The data acquisition module receives raw messages sent by the monitoring terminal through a direct reporting link or a relay reporting link, parses the header and payload fields, and combines this with the reception time written by the receiving gateway to form the first record. The data acquisition module also reads the terminal configuration table and the airflow direction configuration table, supplements the monitoring section and airflow direction according to the configuration version corresponding to the sampling time, and outputs the record to be established. If the monitoring section field or airflow direction field fails the configuration timeliness verification, the data acquisition module marks the corresponding field as missing and outputs this status along with the record to be established.

[0053] After receiving the records to be established, the link establishment and link deviation calculation module establishes a transmission link according to the terminal number and transmission sequence number, determines the continuity of the transmission sequence number and whether the sampling time is reversed, and calculates the dwell time within the same transmission link. This module determines the dwell reference record from records with continuous transmission sequence numbers, the lowest retransmission count, and a stable dwell time range. It then compares the dwell time of each record with the dwell time of the dwell reference record to obtain the link deviation. The link deviation is output by the module and is sent to the sampling time correction and segment concentration difference calculation module for correction time generation, and to the dynamic limit anomaly screening module and sampling reliability assessment module for link verification and time deviation attenuation.

[0054] The sampling time correction and section concentration difference calculation module receives the transmission link, residence time, residence reference, link deviation, monitoring section, and airflow direction. Based on the offset direction and magnitude of the recorded residence time relative to the residence reference, this module performs callback and forward correction on the sampling time to generate a correction time. Subsequently, it aggregates data into time windows according to the monitoring section and correction time, and matches upwind and downwind records with airflow direction and terminal installation orientation. Records that satisfy the airflow direction correspondence and whose correction time difference does not exceed the pairing time deviation limit form a section pairing group. The gas concentration difference between the upwind and downwind records within the section pairing group is output as the section concentration difference.

[0055] The dynamic limit anomaly screening module receives the concentration difference of a monitoring segment within a continuous aggregation window and removes invalid pairings that fail to match upwind and downwind records. It then determines the dynamic limit of the segment concentration difference based on the median and dispersion of the remaining segment concentration differences. This module also receives recorded self-test status, power supply status, link deviation, and segment concentration difference, and compares them with the status compliance conditions, link deviation limits, and dynamic limits of the segment concentration difference, respectively. Records with abnormal self-test status, abnormal power supply status, excessive link deviation, or abnormal segment concentration differences are not included in the fusion queue; records that meet the comparison conditions are output as part of the fusion queue according to the monitoring segment and aggregation window.

[0056] The sampling reliability assessment module receives the signal strength, retransmission count, link deviation, and segment concentration difference for each record in the fusion queue. This module first matches the signal strength against the terminal communication quality interval table to determine the reliability coefficient. Then, it determines the retransmission attenuation based on the retransmission count, the time deviation attenuation based on the proximity of the link deviation to the link deviation limit, and the spatial anomaly attenuation based on the proximity of the segment concentration difference to the segment concentration difference dynamic limit. The reliability coefficient is subtracted from the attenuation values ​​to generate the sampling reliability, which is then bound to the corresponding record and output to the weighted fusion and early warning output module.

[0057] The weighted fusion and early warning output module receives records from the same monitoring segment and collection window, along with their sampling reliability. This module compares the sampling reliability with a reliability limit, discarding records below the limit to obtain qualified fusion records. It then determines the concentration fusion weight based on the sampling reliability of the qualified fusion records and performs a weighted aggregation of the corresponding gas concentrations to generate the segment fusion concentration. The module further compares the weighted contributions of each qualified fusion record to the segment fusion concentration, identifying the qualified fusion record with the highest weighted contribution as the highest contribution record. When the segment fusion concentration exceeds the early warning limit, the weighted fusion and early warning output module outputs an early warning result containing the monitoring segment, collection window, segment fusion concentration, and the highest contribution record. In this system, data outputs from each module serve as inputs for subsequent modules, and abnormal states, missing states, and limit comparison results are all recorded traceably within their respective modules.

[0058] The field states, processing order, judgment conditions, parameter sources, module collaboration, and carrier inheritance relationships described in the above embodiments are used to illustrate the possible implementations of the present invention and should not be limited to the specific embodiments listed. Without departing from the technical solutions described in the present invention and the scope supported by the application materials, any equivalent substitutions, equivalent modifications, equivalent combinations, order adjustments, corresponding module replacements, equivalent changes in field names, equivalent inheritance of the executing entity, or equivalent changes in the carrier form that can be conceived by those skilled in the art should fall within the scope of protection of this patent; however, they should not be extended to unclaimed topics, nor should the substantive correspondence of the technical objects be changed through name changes.

Claims

1. A method for cleaning and fusing safety monitoring data of multi-source heterogeneous gas tunnels, characterized in that, include: Acquire records containing terminal number, monitoring area, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, number of retransmissions, signal strength, and airflow direction; Establish a link based on the terminal number and the sending sequence number, calculate the dwell time between the arrival time and the sampling time, and use the record with the consecutive sending sequence number and the lowest number of retransmissions as the dwell benchmark to obtain the link deviation. The sampling time is corrected according to the dwell time, and the data is collected according to the monitoring area and the correction time. The upwind and downwind side records are matched with the wind direction to calculate the regional concentration difference. A dynamic threshold is generated using the median and dispersion of the regional concentration difference to filter out records of abnormal self-inspection status, abnormal power supply status, excessive link deviation, and abnormal regional concentration difference, forming a queue to be fused. The sampling reliability of the queue records to be merged is evaluated, and the reliability coefficient is normalized according to the signal strength, and retransmission, time deviation and spatial abnormal attenuation are superimposed. Records in the same queue that are below the confidence threshold are removed. The corresponding gas concentrations are weighted and fused using the sampling confidence as the weight to generate a regional fused concentration. The record with the highest contribution is determined based on the weight contribution of each record to the regional fused concentration. When the regional fusion concentration exceeds the warning threshold, a warning result is output, which includes the monitoring area, the aggregation window, the regional fusion concentration, and the record of the highest contribution.

2. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 1, characterized in that, The steps to obtain records containing terminal number, monitoring area, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, number of retransmissions, signal strength, and airflow direction include: The system acquires the original message sent by the monitoring terminal through the direct reporting link or the relay reporting link, parses the header field and payload field of the original message, and writes the parsed terminal number, the sending sequence number, the sampling time, the gas concentration, the self-test status, the power supply status, the number of retransmissions and the signal strength into the original record. The system uses the reception time of the original message written by the receiving gateway as the arrival time to obtain the first record. Obtain the monitoring area and installation orientation corresponding to the terminal number in the terminal configuration table, determine the airflow direction corresponding to the first record in conjunction with the airflow direction configuration table, and add the monitoring area and the airflow direction to the first record to generate a record to be established.

3. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 2, characterized in that, The process of generating the chain record to be built includes: Obtain the version time of the terminal configuration table and the version time of the airflow direction configuration table, and compare the sampling time with the effective time period of the terminal configuration table and the effective time period of the airflow direction configuration table respectively; When the sampling time does not fall within the effective time period of the terminal configuration table or the effective time period of the airflow direction configuration table, the corresponding monitoring area field or airflow direction field is marked as a configuration missing state, and the configuration missing state is written into the chain to be built record. When the sampling time falls within the effective time period of both the terminal configuration table and the airflow direction configuration table, the monitoring area and the airflow direction are determined according to the configuration item whose version time is closest to and no later than the sampling time. The monitoring area, the airflow direction, and the corresponding configuration matching result are written into the chain-to-be-built record to generate a chain-to-be-built record with completed configuration verification.

4. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 1, characterized in that, The steps of establishing a link based on the terminal number and the transmission sequence number and obtaining the link offset include: Obtain the records of the chain to be established under the same terminal number, sorted by the sending sequence number, determine whether the sending sequence numbers of two adjacent records of the chain to be established are consecutive, and group the records with consecutive sending sequence numbers and non-inverted sampling times into the same transmission chain to generate a link record sequence. Calculate the time difference between the arrival time and the sampling time of each record in the link record sequence, determine the time difference as the dwell time, and select records from the link record sequence that have consecutive sending sequence numbers, the lowest number of retransmissions, and a dwell time in a stable range to generate a dwell benchmark record; The dwell time of each record in the link record sequence is compared with the dwell time of the dwell reference record, and the time deviation obtained from the comparison is determined as the link deviation.

5. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 1, characterized in that, The steps of correcting the sampling time according to the residence benchmark and calculating the concentration difference in the region include: Obtain the dwell time and dwell reference recorded in the same transmission chain, calculate the time offset direction and time offset magnitude of the recorded dwell time relative to the dwell reference, and perform callback and forward correction on the sampling time according to the time offset direction and the time offset magnitude to generate a correction time; According to the monitoring area and the correction time, the records that have completed the correction are collected by time window. Records that are in the same area collection window and whose wind direction corresponds to each other are assigned to the area pairing group. The upwind and downwind records are determined in the area pairing group. By comparing the gas concentrations recorded on the upwind side and the gas concentrations recorded on the downwind side, the difference in gas concentrations between the two sides is determined as the regional concentration difference.

6. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 5, characterized in that, The process for determining the region pairing groups includes: Multiple records that have completed time correction within the same monitoring area are obtained. The correction times of the multiple records are matched with preset collection time windows respectively. Records that fall within the same preset collection time window and have different terminal numbers are identified as candidate pairing records. Obtain the wind direction and terminal installation orientation corresponding to the candidate pairing record. When two candidate pairing records form an upwind and downwind correspondence in the wind direction, and the correction time difference between the two candidate pairing records does not exceed the pairing time deviation threshold, the two candidate pairing records are determined as the regional pairing group.

7. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 1, characterized in that, The steps of generating a dynamic threshold and forming the fusion queue based on the median and dispersion of the concentration difference in the region include: The concentration difference of the region generated in the same monitoring area within a continuous collection window is obtained, invalid pairing groups that failed to match the records on the upwind and downwind sides are removed, the median and dispersion of the remaining regional concentration differences are calculated, and the dynamic threshold of the regional concentration difference is determined based on the median and the dispersion. The self-test status, power supply status, link deviation, and regional concentration difference of each record are compared with the status qualification condition, the link deviation threshold, and the regional concentration difference dynamic threshold, respectively. Records that do not meet any of the comparison conditions are filtered out, and the queue to be fused is generated.

8. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 1, characterized in that, The steps for assessing the sampling confidence of the records in the queue to be fused include: The signal strength, retransmission count, link deviation, and regional concentration difference of each record in the queue to be merged are obtained. The signal strength is matched with the terminal communication quality interval table, and the confidence coefficient is determined according to the matched communication quality interval. The retransmission attenuation is determined based on the number of retransmissions, the time deviation attenuation is determined based on the proximity of the link deviation to the link deviation threshold, and the spatial anomaly attenuation is determined based on the proximity of the regional concentration difference to the regional concentration difference dynamic threshold. The sampling confidence is generated by successively subtracting the retransmission attenuation, the time deviation attenuation, and the spatial anomaly attenuation from the confidence coefficient.

9. The method for cleaning and fusing multi-source heterogeneous gas tunnel safety monitoring data according to claim 1, characterized in that, The steps of removing records below the confidence threshold and generating the region fusion concentration and the highest contribution record include: Obtain the queue records to be fused within the same monitoring area and the same collection window, compare the sampling confidence of each record with the confidence threshold, and remove records whose sampling confidence is lower than the confidence threshold to obtain qualified fusion records; The concentration fusion weight is determined according to the sampling reliability of the qualified fusion record, and the corresponding gas concentration is weighted and summarized using the concentration fusion weight to generate the regional fusion concentration; The weighted contribution of each qualified fusion record to the fusion concentration of the region is compared, and the qualified fusion record with the highest weighted contribution is determined as the highest contribution record. When the fusion concentration of the region exceeds the warning threshold, a warning result is output.

10. A multi-source heterogeneous gas tunnel safety monitoring data cleaning and fusion system, characterized in that, The system is used to implement the method according to any one of claims 1-9, comprising: The data acquisition module acquires records including terminal number, monitoring area, sampling time, transmission sequence number, gas concentration, self-test status, power supply status, arrival time, number of retransmissions, signal strength, and airflow direction. The link establishment and link deviation calculation module establishes a link according to the terminal number and the sending sequence number, calculates the dwell time between the arrival time and the sampling time, and uses the record with the continuous sending sequence number and the lowest number of retransmissions as the dwell benchmark to obtain the link deviation. The sampling time correction and regional concentration difference calculation module corrects the sampling time according to the dwelling benchmark, collects data according to the monitoring area and correction time, and combines the upwind and downwind side records with the wind direction to calculate the regional concentration difference. The dynamic threshold anomaly screening module generates a dynamic threshold based on the median and dispersion of the regional concentration difference, and screens out records of self-inspection status anomalies, power supply status anomalies, link deviation exceeding limits, and regional concentration difference anomalies, forming a queue to be fused. The sampling reliability assessment module assesses the sampling reliability of the queue records to be fused, normalizes the signal strength into a reliability coefficient, and superimposes retransmission, time deviation and spatial abnormal attenuation. The weighted fusion and early warning output module removes records in the same area queue that are below the confidence threshold, performs weighted fusion on the corresponding gas concentrations using the sampling confidence as the weight, generates a regional fusion concentration, and determines the record with the highest contribution based on the weight contribution of each record to the regional fusion concentration; when the regional fusion concentration exceeds the early warning threshold, it outputs an early warning result that includes the monitoring area, the aggregation window, the regional fusion concentration, and the record with the highest contribution.

Images