Infrastructure monitoring and detection multi-modal data fusion processing method and system

Abstract

The invention provides an infrastructure monitoring and detection multi-modal data fusion processing method and system, and the method comprises the steps: obtaining the operation state data of an infrastructure in a target region; preprocessing the operation state data of the infrastructure of the target area to obtain intermediate data; transmitting the intermediate data to a database or a cloud, and performing data fusion on the intermediate data to obtain target data; and storing the target data in a pre-constructed risk state library according to the data type. According to the invention, the data can be fused, the accuracy and integrity of the data are improved, and the problems of incomplete urban underground infrastructure monitoring and poor data fusion processing effect in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for fusion processing of multimodal data for infrastructure monitoring and detection. Background Art

[0002] Underground infrastructure (such as tunnels, rails, overhead lines, etc.) is characterized by long distances, large scales, and large volumes, and its operating status is difficult to be monitored comprehensively, in real time, and accurately. Traditional monitoring methods have problems such as limited coverage, insufficient accuracy, and poor real-time performance, which make it difficult to meet the needs of safe operation and maintenance of modern urban underground infrastructure. How to achieve comprehensive, real-time, and high-precision monitoring of large-scale infrastructure structural deformation, foreign body intrusion, and other conditions is the primary problem in ensuring the safe operation of urban underground infrastructure.

[0003] The underground environment is complex and changeable. There are conditions such as no satellite positioning, low light, and long-distance constraints. In addition, the detection task needs to be completed within a limited time window (such as a "short skylight" condition), which places extremely high demands on the accuracy, efficiency, and reliability of the detection equipment. How to develop equipment that can efficiently and accurately detect infrastructure defects such as tunnels, contact networks, and tracks in complex underground environments, and realize millimeter-level integrated dynamic inspections, is a key issue in improving the efficiency of urban underground infrastructure operation and maintenance.

[0004] In addition, a large amount of multi-source heterogeneous data will be generated during the monitoring and inspection of the operation status of urban underground infrastructure. These data are discontinuous in time and space and inconsistent in modality, which brings great challenges to data fusion. How to develop multi-source data fusion technology for infrastructure service status to comprehensively evaluate the operation status of underground infrastructure is an important issue to improve the level of intelligent operation and maintenance of urban underground infrastructure.

[0005] There is no effective solution to the problems of incomplete monitoring of urban underground infrastructure and poor data fusion processing in existing related technologies. Summary of the invention

[0006] The present invention provides a method and system for fusion processing of multimodal data for infrastructure monitoring and detection, which are used to solve the defects of incomplete monitoring of urban underground infrastructure and poor data fusion processing effect existing in the prior art.

[0007] In a first aspect, the present invention provides a method for fusion processing of multimodal data for infrastructure monitoring and detection, comprising: Obtaining the operating status data of the infrastructure in the target area; Preprocessing the operation status data of the infrastructure in the target area to obtain intermediate data; The intermediate data is transmitted to a database or a cloud, and data fusion is performed on the intermediate data to obtain target data; The target data is stored in a pre-built risk status library according to data type.

[0008] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, the operating status data of the infrastructure in the target area is obtained, including: Conducting full-time and full-area comprehensive monitoring of the infrastructure in the target area, and conducting comprehensive and integrated inspections of the infrastructure in the target area, to obtain the operating status data.

[0009] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, the infrastructure of the target area is comprehensively monitored all the time and in all areas, including: The infrastructure in the target area is subjected to distributed optical fiber monitoring of microstructures, real-time monitoring of large-scale structural deformation, and voiceprint recognition monitoring of foreign object intrusion.

[0010] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, micro-structure distributed optical fiber monitoring is performed on the infrastructure of the target area, including: Converting the state change of the infrastructure in the target area into an optical signal change through micro-nano optical fiber; Simultaneously transmit a number of optical signals with different frequencies, wavelengths or modulation modes on the same optical fiber to monitor the health status of the infrastructure in the target area; The optical fiber sensing signals of the operating status of the infrastructure in the target area are processed and analyzed through parallel computing of a graphics processor.

[0011] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, large-scale structural deformation of infrastructure in the target area is monitored in real time, including: In-situ temperature compensation is performed through double grating optical fiber, and strain sensitization is performed using an I-shaped sensitivity-enhancing tunnel co-operating sensor structure.

[0012] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, foreign body intrusion voiceprint recognition monitoring is performed on the infrastructure in the target area, including: Performing long-distance distributed acoustic detection of infrastructure in the target area through microstructured optical fibers; Through AI-assisted voiceprint recognition algorithm, foreign intrusion into the infrastructure of the target area is identified.

[0013] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, a comprehensive and integrated inspection of the infrastructure in the target area is performed, including: A rail damage and high-speed moving magnetization signal mapping model is established, and energy correlation and synchronous noise modal decomposition damage reconstruction algorithm are used to perform longitudinal and transverse core damage detection on the surface of the rail in the target area; The inspection robots perform multi-mode mutual calibration positioning and multi-robot parent-child collaborative detection on the target area.

[0014] According to a multi-modal data fusion processing method for infrastructure monitoring and detection provided by the present invention, the operation status data of the infrastructure in the target area is preprocessed to obtain intermediate data, including: The operating status data of the infrastructure in the target area is cleaned, compressed and standardized to obtain the intermediate data.

[0015] According to a method for fusion processing of multimodal data for infrastructure monitoring and detection provided by the present invention, data fusion is performed on the intermediate data, including: Merging the intermediate data from different sources into a data set; estimating deviations between measured values ​​and actual values ​​in the intermediate data; Identify and exclude non-target data points in the intermediate data according to preset rules; the non-target data points are data points in the intermediate data that do not conform to the preset rules; Identifying a central point of data having deviation or noise in the intermediate data; The identified center point is corrected.

[0016] In a second aspect, the present invention further provides an infrastructure monitoring and detection multimodal data fusion processing system, characterized in that it includes: The data perception layer is used to obtain the operating status data of the infrastructure in the target area; A preprocessing layer, used to preprocess the operation status data of the infrastructure in the target area to obtain intermediate data; A transport layer, used to transmit the intermediate data to a database or a cloud; A fusion layer, used for fusing the intermediate data to obtain target data; The fusion knowledge base is used to store the target data; the fusion knowledge base includes several risk status bases.

[0017] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multimodal data fusion processing method for infrastructure monitoring and detection as described in the first aspect above is implemented.

[0018] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the infrastructure monitoring and detection multimodal data fusion processing method as described in the first aspect above.

[0019] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the infrastructure monitoring and detection multimodal data fusion processing method as described in the first aspect above.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The multimodal data fusion processing method for infrastructure monitoring and detection provided by the present invention obtains the operating status data of these infrastructures through sensors or other monitoring and detection equipment. Then the operating status data is preprocessed to improve the validity of the data and obtain intermediate data. The intermediate data is then transmitted to a database or cloud for storage. Since the source of the intermediate data may come from different sensors or data sources, in this method, the intermediate data is fused to improve the accuracy and completeness of the data, which solves the problems of incomplete urban underground infrastructure monitoring and poor data fusion processing effect in the existing related technologies.

[0021] 2. The present invention develops long-distance, high-sensitivity, wide-band distributed optical fiber monitoring of microstructures, real-time monitoring of large-scale structural deformation, and high-precision identification technology and equipment for foreign object intrusion soundprints, thereby solving the problem of comprehensive and real-time monitoring of long linear and large-scale spatial operating status, and realizing real-time identification of large-scale infrastructure structural deformation and other status, and sudden risks such as foreign object intrusion.

[0022] 3. The present invention proposes a 6D model underground infrastructure operation status multimodal data fusion method. Constructing a 6D model of multi-source monitoring and inspection data of infrastructure can correct the multi-channel data deviation of different monitoring and inspection equipment, thereby solving the problem of operating status data fusion under massive multi-source heterogeneous data.

[0023] 4. The present invention integrates six inherent core databases, namely, tunnel structure status database, foreign body intrusion status database, flood and fire status database, contact network track status database, rail structure status database and limit status database, by integrating the knowledge base. In addition, it also includes an expandable database to achieve efficient management of different operating status data of urban underground infrastructure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of the infrastructure monitoring and detection multimodal data fusion processing method provided by the present invention; Figure 2 is a schematic diagram of the line of sight flow of a 6D model in an embodiment of the present invention; Figure 3 It is a structural block diagram of the infrastructure monitoring and detection multimodal data fusion processing system provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The present invention provides a method for fusion processing of multi-modal data for infrastructure monitoring and detection. Figure 1 is a flow chart of the infrastructure monitoring and detection multimodal data fusion processing method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step S101, obtaining operation status data of infrastructure in a target area; Step S102, preprocessing the operation status data of the infrastructure in the target area to obtain intermediate data; Step S103, transmitting the intermediate data to a database or cloud, and performing data fusion on the intermediate data to obtain target data; Step S104: storing the target data in a pre-built risk status library according to the data type.

[0028] In this method, for the infrastructure in the target area, especially the underground facilities, the operating status data of these infrastructures are first obtained through sensors or other monitoring and detection equipment. Then the operating status data is preprocessed to improve the validity of the data and obtain intermediate data. The intermediate data is then transmitted to the database or cloud for storage. Furthermore, the data can be encrypted during the transmission process to ensure the security of the data transmission process. Since the source of the intermediate data may come from different sensors or data sources, in this method, the intermediate data is fused to improve the accuracy and completeness of the data, solving the problems of incomplete monitoring of urban underground infrastructure and poor data fusion processing in the existing related technologies. The target data obtained after fusion is stored in a pre-constructed risk status library, and the types of risk status libraries are divided according to the data type, such as tunnel structure status library, foreign body intrusion status library, fire and flood status library, contact network rail status library, rail structure status library and limit status library.

[0029] In some of the embodiments, obtaining the operating status data of the infrastructure in the target area includes: conducting full-time and full-area comprehensive monitoring of the infrastructure in the target area, and conducting comprehensive and integrated inspections of the infrastructure in the target area to obtain the operating status data.

[0030] Specifically, comprehensive monitoring of the infrastructure in the target area is carried out all the time and in all areas, including: distributed optical fiber monitoring of the microstructure of the infrastructure in the target area, real-time monitoring of the deformation of the large-scale structure, and voiceprint recognition monitoring of foreign object intrusion.

[0031] In this embodiment, microstructure distributed optical fiber monitoring is performed on the infrastructure of the target area, including: converting the state changes of the infrastructure of the target area into optical signal changes through micro-nano optical fibers; simultaneously transmitting a number of optical signals with different frequencies, different wavelengths or different modulation methods on the same optical fiber to monitor the health status of the infrastructure of the target area; and processing and analyzing the optical fiber sensing signals of the operating status of the infrastructure of the target area through parallel computing of a graphics processor.

[0032] For example, long-distance, high-sensitivity, broadband microstructure distributed optical fiber monitoring of infrastructure in the target area mainly improves the sensitivity, capacity and response performance of optical fiber sensing through three technologies: microstructure integrated sensitivity enhancement, multi-domain multiplexing capacity enhancement and parallel solution speed increase. Among them, the microstructure integrated sensitivity enhancement technology enhances the sensitivity of optical fiber sensors by designing and integrating micro-nano optical fibers with special geometric shapes, which can convert the tiny stress, strain and other state changes of underground infrastructure such as tunnel structures into significant optical signal changes, and capture more accurate operation status data; the multi-domain multiplexing capacity enhancement technology transmits multiple optical signals of different frequencies, different wavelengths or different modulation methods on the same optical fiber at the same time, so as to achieve comprehensive monitoring of the health status of long and large tunnel structures, rail structures, etc.; the parallel solution speed increase technology uses graphics processing unit (GPU) parallel computing to accelerate the processing and analysis of the measured infrastructure operation status optical fiber sensing signals. When dealing with emergencies such as track damage and foreign body intrusion, its rapid response capability is crucial to taking timely measures to reduce losses.

[0033] In addition, in this microstructure distributed optical fiber monitoring technology, coherent detection technology is used to extract the phase, frequency and other parameter information of the optical signal. The main formula of coherent detection technology is:

[0034] in, Represents the reference light wave output by the detector p With signal light wave s The difference frequency signal, represents the optical frequency difference, represents the phase difference, β represents the scale factor, represents the amplitude of the reference light wave, represents the amplitude of the signal light wave, t Indicates time. Difference frequency signal The frequency of the optical signal is much lower than that of the original optical signal, so it is easier to process and detect. Since direct detection technology can only detect the intensity of the signal, coherent detection technology has stronger detection capabilities and can restore the optical signal. In addition, coherent detection has a higher signal-to-noise ratio and a lower minimum detectable power. The main formula is as follows:

[0035]

[0036] in, represents the signal-to-noise ratio of coherent detection, represents the signal-to-noise ratio of direct detection, represents the minimum detectable power for coherent detection, represents the minimum detectable power for direct detection, is the quantum efficiency of the detector, represents the photon flux of the incident signal, is Planck's constant, represents the frequency of the optical signal, Represents the bandwidth of the detection system. Therefore, coherent detection technology can be used to detect the small vibration state parameters of subway tunnel rails and contact network rails.

[0037] Real-time monitoring of large-scale structural deformation of infrastructure in the target area, including: in-situ temperature compensation through dual grating optical fiber, and strain sensitization using I-shaped tunnel co-operating sensor structure. In addition, real-time monitoring of tunnel segment deformation, failure analysis of reinforcement steel rings, and identification of voiding degree are achieved based on defect-based structural intrinsic vibration coupling modulation. Among them, the principle formula of the I-shaped tunnel conformal sensor is as follows:

[0038] in, K represents the strain gain factor, which indicates the relationship between the sensor strain and the fiber Bragg grating strain; represents the strain of the sensor, represents the strain of the fiber Bragg grating, represents the strain of the material used in the fiber Bragg grating, l A represents the original length of the fiber Bragg grating, l B represents the original length of the sensor part connected to the FBG, h A and h B represents the thickness of the two materials, S A and S B represents the cross-sectional area of ​​the fiber Bragg grating part and the connected part; E A and E B represent the Young's modulus of the material of the fiber grating part and the connecting part respectively.

[0039] Conduct voiceprint recognition and monitoring of foreign body intrusion on the infrastructure of the target area, including: long-distance distributed acoustic wave detection of the infrastructure of the target area through microstructured optical fiber; and identify foreign body intrusion on the infrastructure of the target area through AI-assisted voiceprint recognition algorithm.

[0040] Exemplarily, the high-precision foreign body intrusion soundprint recognition technology mainly uses microstructured optical fiber to achieve long-distance distributed high-fidelity sound wave detection, and uses AI-assisted soundprint recognition algorithm to achieve high-precision recognition of foreign body intrusion in underground facilities. AI-assisted soundprint recognition technology pre-processes the collected sound wave signals and extracts characteristic parameters such as the signal's wavelet information entropy and spectrum characteristics. It then further performs PCA principal component analysis on the characteristic parameters and constructs a multi-dimensional characteristic parameter matrix. Based on the multi-dimensional characteristic parameter matrix, the neural network model is trained and optimized to ultimately achieve sound source recognition and positioning.

[0041] Conduct comprehensive and integrated inspections on the infrastructure in the target area, including: establishing a mapping model between rail damage and high-speed mobile magnetization signals, using energy correlation and synchronous noise modal decomposition damage reconstruction algorithms to conduct vertical and horizontal nuclear damage detection on the surface of the rails in the target area; and using inspection robots to conduct multi-mode mutual calibration and positioning of the target area and collaborative detection by multi-robot parent-child groups.

[0042] Specifically, the comprehensive integrated inspection part mainly includes rail surface-shallow layer damage high-speed composite electromagnetic detection technology and multi-mode mutual calibration positioning, multi-robot collaborative integrated inspection technology. Rail surface-shallow layer damage high-speed composite electromagnetic detection technology establishes a rail damage and high-speed mobile magnetization signal mapping model, and adopts energy correlation and synchronous noise mode decomposition damage reconstruction algorithm to solve the problem of shallow rail vertical and horizontal nuclear damage detection, which can realize efficient detection of surface damage and shallow surface damage such as rail surface abrasion and peeling. Multi-mode mutual calibration positioning, multi-robot collaborative integrated inspection technology uses network tunnel limit and track integrated high-precision integrated inspection robot to realize multi-mode mutual calibration positioning and multi-robot parent-child group collaborative operation technology in underground satellite-free positioning environment, and dynamic integrated comprehensive rapid detection.

[0043] In some of the embodiments, the operating status data of the infrastructure in the target area is preprocessed to obtain intermediate data, including: performing data cleaning, data compression and standardization on the operating status data of the infrastructure in the target area to obtain intermediate data.

[0044] In this embodiment, the operation status data is first cleaned, incomplete data and erroneous data need to be filtered, and the missing and erroneous content is fed back to the client, and the data is reorganized and uploaded according to the protocol or written into the data warehouse after the client completes it; data compression and standardization are to improve data transmission and storage efficiency, and the data is compressed and standardized. In addition, during the data transmission process, the pre-processed intermediate data is mainly transmitted to the data center or cloud using wired / wireless communication networks, and high-speed and reliable data transmission is supported.

[0045] In order to merge or map massive data sets from non-homogeneous sources into a unified data set, in some embodiments, data fusion is performed on the intermediate data, including: merging intermediate data from different sources into a data set; estimating the deviation between the measured value and the actual value in the intermediate data; identifying and excluding non-target data points in the intermediate data according to preset rules; non-target data points are data points in the intermediate data that do not comply with the preset rules; identifying the center point of the data with deviation or noise in the intermediate data; and correcting the identified center point.

[0046] In this embodiment, the intermediate data is associated, matched, and fused through the 6D model algorithm to extract deeper information and features. The 6D model mainly includes six data processing methods, namely multi-channel fusion, data depth mining and pattern recognition, reliable spatial deviation estimation, bad criterion matching, reliable offset center identification, and center space correction.

[0047] Among them, multi-channel fusion combines information from different sensors or data sources (usually with different physical properties or representations) into a single, consistently represented data set. Reliable spatial deviation estimation is used to estimate the deviation between the measured value and the actual value, while considering the reliability of the deviation. Bad criterion matching can identify and exclude data points that do not meet the requirements based on preset thresholds or rules. Reliable offset center identification can identify reliable and representative center points in data with deviations or noise, thereby reducing the impact of deviations on the results. Center space correction can correct the center point of the data to further improve the reliability, comprehensiveness and accuracy of the data.

[0048] Figure 2 FIG. 1 is a schematic diagram of the line of sight flow of the 6D model in the embodiment of the present invention. When the massive multi-source heterogeneous monitoring data of the operation status of the underground infrastructure is fused and processed, the specific implementation process of the 6D model is as follows: Figure 2As shown in the figure, the specific steps are as follows: First, collect real-time monitoring data and regular inspection data (structural health monitoring data (such as stress, strain), environmental monitoring data (such as temperature, humidity, gas concentration), and images, videos and text descriptions recorded by inspection personnel) from various monitoring devices (such as monitoring equipment, video surveillance systems, inspection robots, etc.) from underground infrastructure in different cities (such as subway tunnels, drainage pipes, integrated pipe corridors, etc.). Use data fusion algorithms (such as weighted average, Kalman filtering, Bayesian network, etc.) to pre-process data from different devices and channels (data cleaning, denoising, time synchronization, etc.), and then integrate these data into a unified data format or model. Then use machine learning, data mining and other technologies to conduct in-depth analysis of the fused data set. By extracting key features and identifying normal and abnormal patterns, it provides important information and clues for subsequent deviation estimation and bad data detection. Then, based on data mining and pattern recognition, statistical analysis methods (such as mean, standard deviation, confidence interval, etc.) and machine learning techniques (such as cluster analysis, anomaly detection algorithm) are used to evaluate the degree of deviation between the mining results and the actual operating status of the underground infrastructure. This includes identifying outliers, noise, and spatial deviations caused by factors such as sensor location and accuracy in the data. Then, the mined data is checked item by item according to the preset data quality standards or criteria (data range, consistency, completeness, etc.). Data points that do not meet the rules are marked, isolated, or deleted. After excluding bad data points, cluster analysis, principal component analysis and other methods are used to extract representative center points or eigenvalues ​​from the remaining data. These center points can better reflect the actual situation of the operation status of the underground infrastructure. At the same time, domain knowledge and expert experience are used to verify and adjust the identification results. Finally, according to the characteristics of the operation status of the underground infrastructure and the spatial distribution characteristics of the monitoring data, spatial interpolation, data smoothing and other technologies are used to correct the center points. At the same time, considering the mutual influence and constraint relationship between different data sources, the correction results are comprehensively evaluated and adjusted to eliminate or reduce errors caused by deviations and noise. This helps to improve data quality and the accuracy of analysis results.

[0049] Through the above steps, the comprehensive, accurate and efficient fusion processing of massive multi-source heterogeneous monitoring data of the operation status of urban underground infrastructure can be achieved, providing a reliable data foundation for subsequent modules. The mathematical expression of the 6D model is as follows:

[0050]

[0051] Assume that n Monitoring devices for underground infrastructure in different cities, each device generates data of D, then the data after multi-channel fusion is , is the data fusion function; after fusion, the data set Perform feature extraction and pattern recognition on it to obtain key features and normal mode With exception mode , Represents data mining and pattern recognition functions; is the deviation between the fused data and the actual operating status, is the deviation estimation function; Represents the data quality standard, and the data points that do not meet the standard are D clean , represents a bad criteria matching algorithm; represents a representative center point or eigenvalue, stands for cluster analysis or principal component analysis algorithm; is the final high-quality data, is a characteristic of the operating state of underground infrastructure. Represents spatial interpolation and data smoothing algorithms.

[0052] In this method, the fusion knowledge base module is the core component of the multimodal data processing system for monitoring and detecting the operation status of urban underground infrastructure. Its design reflects a high degree of systematization and specialization. The module uses an intelligent classification algorithm to integrate six core databases, namely, the tunnel structure status database, the foreign body intrusion status database, the flood and fire status database, the contact network track status database, the rail structure status database, and the limit status database, to achieve efficient management of different operation status data of urban underground infrastructure. Among them, the tunnel structure status database focuses on collecting and analyzing key parameters such as cracks, water seepage, and settlement deformation on the tunnel wall, providing solid data support for evaluating the health of the tunnel structure. The foreign body intrusion status database focuses on collecting key status parameters such as unauthorized personnel or object intrusion in the tunnel. The flood and fire status database mainly collects key indicators such as water level, humidity, temperature, and smoke concentration. The contact network track status database focuses on the management of the operation status parameters of the power transmission system and the track system, including the contact network suspension status, gauge change, and roadbed status. The rail structure status database focuses on the collection and management of subtle features such as wear, cracks, and geometric size changes of the rails. The limit state library manages the spatial restriction information about the operation of equipment, vehicles and other objects in the tunnel. In addition, this module is also flexible in scalability and can add new databases or optimize existing functions according to actual needs to adapt to the increasingly complex development needs of urban underground infrastructure. The mathematical model of the fusion knowledge base is as follows:

[0053] in,K E represents the fusion knowledge data set, Represents 6 large data subsets, The weights of the six data subsets, namely, the inherent tunnel structure status library, the foreign body intrusion status library, the flood and fire status library, the contact network track status library, the rail structure status library and the limit status library; Used to control the importance or priority of each data subset in the fusion knowledge base; It is an expandable item. When new operation status monitoring data of urban underground infrastructure appears, It represents the new running status data subset, The weight coefficient is determined by the dynamic weight adjustment method. First, the number of occurrences of risk state parameters in each subset (such as tunnel structure, foreign body intrusion, flood and fire, etc.) in the recent period (the latest time window) is counted through the database and standardized as frequency. Assume that in a time window, the number of occurrences of the risk state parameter of the i-th subset is , then its frequency is:

[0054] in, n Indicates the number of subsets, the minimum number of subsets is 6; the frequency of each subset As its weight Finally, through continuous monitoring of short-term data, the weights are adjusted dynamically to reflect the latest risk status parameters.

[0055] The present invention also provides an infrastructure monitoring and detection multimodal data fusion processing system. The infrastructure monitoring and detection multimodal data fusion processing system provided by the present invention is described below. The infrastructure monitoring and detection multimodal data fusion processing system described below and the infrastructure monitoring and detection multimodal data fusion processing method described above can be referenced to each other. Figure 3 is a structural block diagram of the infrastructure monitoring and detection multimodal data fusion processing system provided by the present invention, such as Figure 3 As shown, the system includes: The data perception layer is used to obtain the operating status data of the infrastructure in the target area; The preprocessing layer is used to preprocess the operating status data of the infrastructure in the target area to obtain intermediate data; The transport layer is used to transfer intermediate data to the database or cloud; The fusion layer is used to fuse the intermediate data to obtain the target data; The fusion knowledge base is used to store target data; the fusion knowledge base includes several risk status bases.

[0056] In this system, for the infrastructure in the target area, especially the underground facilities, the data perception layer first obtains the operating status data of these infrastructures through sensors or other monitoring and detection equipment. Then the preprocessing layer preprocesses the operating status data to improve the validity of the data and obtain intermediate data. The transmission layer then transmits the intermediate data to the database or cloud for storage. Furthermore, the data can be encrypted during the transmission process to ensure the security of the data transmission process. Since the source of the intermediate data may come from different sensors or data sources, in this system, the fusion layer fuses the intermediate data to improve the accuracy and completeness of the data, solving the problems of incomplete monitoring of urban underground infrastructure and poor data fusion processing in the existing related technologies. The target data obtained after fusion is stored in a pre-constructed risk status library. The types of risk status libraries are divided according to the data type, such as tunnel structure status library, foreign body intrusion status library, fire and flood status library, contact network track status library, rail structure status library and limit status library.

[0057] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 communicate with each other through the communication bus 404. The processor 401 may call the logic instructions in the memory 403 to execute the infrastructure monitoring and detection multimodal data fusion processing method, which includes: Obtaining the operating status data of the infrastructure in the target area; Preprocessing the operation status data of the infrastructure in the target area to obtain intermediate data; The intermediate data is transferred to the database or cloud, and data fusion is performed on the intermediate data to obtain the target data; Store target data into a pre-built risk status library according to data type.

[0058] In addition, the logic instructions in the above-mentioned memory 403 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0059] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the infrastructure monitoring and detection multimodal data fusion processing method provided by the above methods, the method includes: Obtaining the operating status data of the infrastructure in the target area; Preprocessing the operation status data of the infrastructure in the target area to obtain intermediate data; The intermediate data is transferred to the database or cloud, and data fusion is performed on the intermediate data to obtain the target data; Store target data into a pre-built risk status library according to data type.

[0060] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the infrastructure monitoring and detection multimodal data fusion processing method provided by the above methods, the method comprising: Obtaining the operating status data of the infrastructure in the target area; Preprocessing the operation status data of the infrastructure in the target area to obtain intermediate data; The intermediate data is transferred to the database or cloud, and data fusion is performed on the intermediate data to obtain the target data; Store target data into a pre-built risk status library according to data type.

[0061] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal data fusion processing method for infrastructure monitoring and detection, characterized in that: include: Obtaining the operating status data of the infrastructure in the target area; Preprocessing the operation status data of the infrastructure in the target area to obtain intermediate data; The intermediate data is transmitted to a database or a cloud, and data fusion is performed on the intermediate data to obtain target data; Storing the target data in a pre-built risk status library according to data type; Performing data fusion on the intermediate data includes: Merging the intermediate data from different sources into a data set; estimating deviations between measured values ​​and actual values ​​in the intermediate data; Identify and exclude non-target data points in the intermediate data according to preset rules; the non-target data points are data points in the intermediate data that do not conform to the preset rules; Identifying a central point of data having deviation or noise in the intermediate data; The identified center point is corrected.

2. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 1 is characterized in that: Obtain the operating status data of the infrastructure in the target area, including: Conducting full-time and full-area comprehensive monitoring of the infrastructure in the target area, and conducting comprehensive and integrated inspections of the infrastructure in the target area, to obtain the operating status data.

3. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 2 is characterized in that: Conduct full-time and comprehensive monitoring of the infrastructure in the target area, including: The infrastructure in the target area is subjected to distributed optical fiber monitoring of microstructures, real-time monitoring of large-scale structural deformation, and voiceprint recognition monitoring of foreign object intrusion.

4. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 3 is characterized in that: Conduct microstructure distributed fiber optic monitoring of the infrastructure in the target area, including: Converting the state change of the infrastructure in the target area into an optical signal change through micro-nano optical fiber; Simultaneously transmit a number of optical signals with different frequencies, wavelengths or modulation modes on the same optical fiber to monitor the health status of the infrastructure in the target area; The optical fiber sensing signals of the operating status of the infrastructure in the target area are processed and analyzed through parallel computing of a graphics processor.

5. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 3 is characterized in that: Conduct real-time monitoring of large-scale structural deformation of the infrastructure in the target area, including: In-situ temperature compensation is performed through double grating optical fiber, and strain sensitization is performed using an I-shaped sensitivity-enhancing tunnel co-operating sensor structure.

6. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 3 is characterized in that: Conduct voiceprint recognition monitoring of foreign body intrusion on the infrastructure of the target area, including: Performing long-distance distributed acoustic detection of infrastructure in the target area through microstructured optical fibers; Through AI-assisted voiceprint recognition algorithm, foreign intrusion into the infrastructure of the target area is identified.

7. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 3 is characterized in that: Conduct a comprehensive and integrated inspection of the infrastructure in the target area, including: A rail damage and high-speed moving magnetization signal mapping model is established, and energy correlation and synchronous noise modal decomposition damage reconstruction algorithm are used to perform longitudinal and transverse core damage detection on the surface of the rail in the target area; The inspection robots perform multi-mode mutual calibration positioning and multi-robot parent-child collaborative detection on the target area.

8. The infrastructure monitoring and detection multimodal data fusion processing method according to claim 1 is characterized in that: Preprocessing the operating status data of the infrastructure in the target area to obtain intermediate data includes: The operating status data of the infrastructure in the target area is cleaned, compressed and standardized to obtain the intermediate data.

9. A multimodal data fusion processing system for infrastructure monitoring and detection, characterized in that: include: The data perception layer is used to obtain the operating status data of the infrastructure in the target area; A preprocessing layer, used to preprocess the operation status data of the infrastructure in the target area to obtain intermediate data; A transport layer, used to transmit the intermediate data to a database or a cloud; A fusion layer, used for fusing the intermediate data to obtain target data; The fusion knowledge base is used to store the target data; the fusion knowledge base includes several risk status bases.

Images