A computer vision-based change detection method, system, device, and medium

Abstract

The application provides a computer vision-based change detection method, system, device and medium, and relates to the technical field of structural health monitoring. The application collects images of a structure to be measured with targets through a visual measuring instrument, identifies and constructs image coordinate groups of the targets; then jointly solves all coordinates to obtain three-dimensional positions of the targets in a unified coordinate system, and further calculates multi-dimensional key parameters reflecting the overall and local states of the structure; then inputs these parameters and environmental data into a space-time propagation model, constructs a structure graph with key parts as nodes according to physical topology; then simulates the space-time propagation process of the structure state on the graph to determine the evolution mode; finally, determines the safety level of the structure according to the mode and executes early warning, which can improve the foresight and accuracy of structure safety early warning.

Description

Technical Field

[0001] This application relates to the field of structural health monitoring technology, and in particular to a change detection method, system, device and medium based on computer vision. Background Technology

[0002] Computer vision-based change detection methods have shown broad application prospects in the field of infrastructure health monitoring in recent years. By performing non-contact measurements on the surface of structures, this method can continuously perceive key parameters such as displacement, deformation, and vibration of complex structures such as bridges, dams, and ancient buildings, providing important data support for structural safety assessment and early warning.

[0003] Existing technologies include various vision-based structural monitoring schemes. These schemes acquire three-dimensional deformation information of the structure by placing targets on the structural surface, acquiring images using one or more cameras, and combining sub-pixel localization with multi-view geometric calculations. These methods typically achieve displacement measurement accuracy at the millimeter or even sub-millimeter level. Some schemes also incorporate environmental data compensation and multi-sensor fusion techniques to improve the robustness and reliability of the monitoring.

[0004] However, in practical engineering applications, the aforementioned methods mostly focus on the independent extraction and analysis of local deformation parameters, lacking systematic modeling of the overall structural state evolution. Furthermore, the dynamic correlation between monitoring data and environmental factors and structural topology has not been fully revealed, leading to assessments of structural safety status relying heavily on static threshold judgments, making it difficult to achieve early identification and trend prediction from local anomalies to overall risks. Therefore, existing technologies suffer from insufficient characterization of the overall structural safety state evolution pattern and limited early warning foresight. Summary of the Invention

[0005] The purpose of this application is to provide a change detection method, system, device, and medium based on computer vision to solve the problems of poor foresight and low accuracy in structural safety early warning in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a change detection method based on computer vision, comprising:

[0007] Original images of multiple key parts of the structure under test are acquired using a visual measurement instrument, and the original images carry target information.

[0008] The same target information is identified in multiple original images, and a set of image point coordinates corresponding to each target information is constructed based on the corresponding coordinates of the same target information in the multiple original images.

[0009] Based on the key parameters of the vision measuring instrument, the coordinates of the image points corresponding to all target information are jointly calculated to obtain the three-dimensional spatial coordinates of each target information in a unified coordinate system. Based on the three-dimensional spatial coordinates, the measured values ​​of various key parameters are calculated. The measured values ​​are used to reflect the overall state of each key part and the structure under test.

[0010] The measured values ​​of various key parameters and the acquired environmental data are input into a preset spatiotemporal propagation model, so that the spatiotemporal propagation model can construct a structural graph with each key part as a node and the relationship between each key part as an edge according to the physical topology of the structure under test.

[0011] The changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process are simulated on the structural diagram to determine the evolution mode of the structural state.

[0012] Based on the evolution pattern of the structural state, the safety level of the structure under test is determined, and corresponding early warnings are issued based on the safety level.

[0013] Optionally, the step of inputting the measured values ​​of multiple key parameters and the acquired environmental data into a preset spatiotemporal propagation model, so as to construct a structural graph with each key component as a node and the relationships between the key components as edges based on the physical topology of the structure under test through the spatiotemporal propagation model, includes:

[0014] By using the environment-structure response decoupler in the spatiotemporal propagation model, the systematic influence caused by the environmental data is removed from the measured values ​​of the various key parameters, resulting in a first abnormal index sequence reflecting potential structural damage.

[0015] Acquire auxiliary sensor data for key components, and extract a second abnormal indicator sequence from the auxiliary sensor data. The equipment used for acquiring the auxiliary sensor data is different from a visual measurement instrument.

[0016] For each key part, multi-source evidence fusion is performed based on the first abnormal indicator sequence and the second abnormal indicator sequence to perform consistency verification and generate an initial abnormal state value;

[0017] Based on the engineering design drawings or actual 3D point cloud model of the structure under test, the physical topology of the structure under test is determined. The physical topology includes the spatial location of each key part and the physical connection relationship between key components.

[0018] Each key component is defined as a node, and graph edges are established between pairs of nodes with physical connections to form a structural graph.

[0019] Secondly, this application provides a computer vision-based change detection system, including:

[0020] The acquisition module is used to acquire original images of multiple key parts of the structure under test through a vision measuring instrument. The original images carry target information.

[0021] The recognition module is used to identify the same target information in multiple original images, and construct a set of image point coordinates corresponding to each target information based on the corresponding coordinates of the same target information in the multiple original images.

[0022] The calculation module is used to jointly calculate the coordinates of image points corresponding to all target information according to the key parameters of the vision measuring instrument, so as to calculate the three-dimensional spatial coordinates of each target information in a unified coordinate system, and calculate the measurement values ​​of various key parameters based on the three-dimensional spatial coordinates. The measurement values ​​are used to reflect the overall state of each key part and the structure under test.

[0023] The input module is used to input the measured values ​​of various key parameters and the acquired environmental data into a preset spatiotemporal propagation model, so that the spatiotemporal propagation model can construct a structural graph with each key part as a node and the relationship between each key part as an edge according to the physical topology of the structure under test.

[0024] The determination module is used to simulate the changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process on the structural diagram, so as to determine the evolution mode of the structural state.

[0025] The judgment module is used to determine the safety level of the structure under test based on the evolution pattern of the structural state, and to issue a corresponding warning based on the safety level.

[0026] Thirdly, this application provides an electronic device, comprising:

[0027] Memory, used to store computer programs;

[0028] A processor is configured to execute the computer program to implement the steps of a computer vision-based change detection method as described in the first aspect above.

[0029] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of a computer vision-based change detection method as described in the first aspect above.

[0030] This application provides a computer vision-based change detection method, which has the following advantages:

[0031] First, by acquiring the original image carrying the target and identifying and constructing the coordinates of the image points, a reliable data foundation is established for subsequent calculations, ensuring the accurate locking and tracking of the monitored target. Then, based on the parameters of the vision measuring instrument, the coordinates of multiple points are jointly calculated and three-dimensional spatial coordinates under a unified coordinate system are obtained, thereby realizing the accurate mapping from two-dimensional image to three-dimensional spatial deformation. At the same time, by calculating a variety of key parameters, the local and overall state of the structure can be comprehensively characterized.

[0032] Next, key parameters and environmental data are input into the spatiotemporal propagation model, and a structural diagram is constructed based on the physical topology. This allows the monitoring data to be organically combined with the inherent connection relationship of the structure, providing a realistic physical background for state simulation. Then, the spatiotemporal propagation process of the state is simulated on the structural diagram, and the evolution mode is determined, which can reveal the dynamic diffusion law and potential development path of anomalies in the structure. Finally, the safety level is determined based on the model and an early warning is issued, thereby upgrading monitoring from static parameter comparison to intelligent decision-making based on dynamic evolution process, enhancing the accuracy and timeliness of early warning.

[0033] Furthermore, this application eliminates systematic environmental influences through environment-structure response decoupling, then integrates anomaly indicators from visual and auxiliary sensors and performs consistency verification to generate more reliable initial anomaly state values. Subsequently, the physical topology is determined based on design drawings or 3D models, and a structural graph is constructed with key components as nodes and physical connections as edges, providing a realistic physical framework for anomaly propagation simulation. Therefore, this application improves the robustness of initial anomaly state determination and the engineering fit of structural graph construction through multi-source data fusion and physical topology combination, laying a solid foundation for subsequent accurate simulation of anomaly propagation. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart illustrating a change detection method based on computer vision provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram illustrating a specific implementation of a change detection method based on computer vision, provided in an embodiment of this application.

[0037] Figure 3 This is a schematic diagram of the structure of a computer vision-based change detection system provided in an embodiment of this application. Detailed Implementation

[0038] To address the problems existing in the prior art, this application provides a change detection method based on computer vision. The core of this method is as follows: First, the three-dimensional deformation parameters of each key part of the structure in a unified coordinate system are calculated through multi-view visual measurement; then, these multi-dimensional parameters and environmental data are input into a preset model, and a structural graph with parts as nodes and connections as edges is constructed based on the actual physical connection relationship of the structure; then, the propagation process of abnormal states between nodes is simulated on the graph, thereby revealing the dynamic evolution pattern of the overall state of the structure, and safety level judgment and early warning are performed based on this pattern.

[0039] Therefore, this method achieves a transformation from "point-based alarm" to "networked situational simulation" by dynamically incorporating discrete measurement points into a topological network that reflects the actual mechanical transmission path. This fundamentally solves the problem of insufficient early warning foresight caused by the lack of spatiotemporal correlation analysis of structural states in the background technology, and improves the systematicness and predictability of structural safety risk assessment.

[0040] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] The core of this application is to provide a change detection method based on computer vision, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0042] Step 101: Acquire original images of multiple key parts of the structure under test using a visual measurement instrument. The original images carry target information.

[0043] In step 101, the visual measuring instrument is a high-precision imaging device used to acquire images of the structure under test from a distance without contact. The visual measuring instrument in this application is based on a low-light image sensor and has the ability to work stably under all-weather conditions such as day, night, rain, and fog.

[0044] Critical parts refer to specific locations in a structure that have a significant impact on overall safety, such as the supports of a bridge, the connection points of arch ribs, or the joints of a dam body. It should be understood that critical parts can be set in different scenarios such as bridges, dams, and ancient buildings, which will not be elaborated here.

[0045] A target is a marker with specific optical characteristics that is pre-fixed to these key parts. Its image information can be used for subsequent precise positioning and tracking. Carrying target information means that the original image contains clear images of these targets.

[0046] In this embodiment of the application, the visual measuring instrument is first deployed at an observation position that can cover multiple key parts of the structure to be measured. Then, the device is started to acquire images of the key parts where the targets are set, thereby obtaining a series of raw images containing the optical features of these targets.

[0047] Step 102: Identify the same target information in multiple original images, and construct a set of image point coordinates corresponding to each target information based on the corresponding coordinates of the same target information in the multiple original images.

[0048] In step 102, the same target information refers to the image feature information corresponding to the same physical target in different original images, such as the specific coded pattern or geometric shape of the target; the corresponding coordinates refer to the two-dimensional pixel position of the target information in the image coordinate system corresponding to each original image; the image point coordinates are a set of ordered coordinate data formed by combining the two-dimensional pixel positions of the same target information identified in different original images according to the time sequence or viewpoint relationship.

[0049] In this embodiment, each original image is first preprocessed, for example, by using an image enhancement algorithm to improve the contrast between the target region and the background; then, using a template matching or feature descriptor-based method, the pixel regions of all targets are located in each image, and their encoded patterns are decoded to obtain the identification information of each target; finally, the identification information obtained from decoding in different images is compared, and targets with the same identification information are determined to be the same target information, thus completing the target association across images.

[0050] After completing the target association, the pixel center coordinates of each associated target information in all the original images in which it appears are extracted. Then, these two-dimensional coordinates are organized into an ordered list according to a preset order, such as the image acquisition timestamp order or the visual measurement instrument number order. This list is a set of image point coordinates corresponding to the target information. This process establishes an image position file for each physical target at different viewpoints or at different times.

[0051] In this embodiment, after step 102, the following process is also included:

[0052] A1: Obtain angular velocity and linear acceleration data in real time from the inertial measurement unit built into each vision measuring instrument.

[0053] In step A1, the inertial measurement unit is a sensor module integrated inside the vision measuring instrument. This module is used to measure the angular velocity and linear acceleration data of the device itself in three-dimensional space. The angular velocity data reflects the speed of rotation of the device around each axis of its own coordinate system, and the linear acceleration data reflects the speed of linear motion of the device along each axis of its own coordinate system.

[0054] In this embodiment of the application, when the visual measuring instrument performs image acquisition according to step 101, its built-in inertial measurement unit is started synchronously and continuously samples at a rate higher than the image acquisition frequency, recording the angular velocity and linear acceleration data of the visual measuring instrument at the moment of image acquisition exposure in real time, providing raw observation values ​​for subsequent motion compensation.

[0055] In practical applications, a vision measurement instrument deployed at a bridge monitoring point records the triaxial angular velocity data at the exposure moment of a captured image frame using its built-in inertial measurement unit. Radius per second, triaxial acceleration data are meters per square second.

[0056] A2: By integrating the angular velocity and the linear acceleration, the attitude change and position offset of the vision measuring instrument at the time of image acquisition relative to the initial calibration time can be calculated.

[0057] In step A2, the attitude change refers to the three-dimensional angular rotation change of the vision measuring instrument from the initial calibration time to the current image acquisition time, which is usually represented by a rotation matrix or quaternion; the position offset refers to the three-dimensional spatial translation change of the vision measuring instrument from the initial calibration time to the current image acquisition time, which is represented by a three-dimensional vector.

[0058] In this embodiment of the application, the pose of the visual measuring instrument at the initial calibration time is first used as the reference zero point. Then, the series of angular velocity data from the initial time to the current image acquisition time obtained in step A1 is integrated over time to calculate the rotation angle of the visual measuring instrument at the current time relative to the initial time, and then converted into the attitude change amount. At the same time, a series of linear acceleration data within the same time period are double integrated and separated in combination with the influence of gravitational acceleration to calculate the translation distance of the visual measuring instrument at the current time relative to the initial time, that is, the position offset.

[0059] In practical applications, assuming the initial calibration attitude of the vision measuring instrument is an identity matrix and the initial position is the origin of the coordinate system, the rotation matrix corresponding to the attitude change at the current moment is calculated by integrating the angular velocity and linear acceleration data obtained by sampling at 100 Hz within 0.1 seconds from the initial moment to the current image acquisition moment. The final solution is {[0.9998, 0.002, -0.001], [-0.002, 0.9999, 0.0005], [0.001, -0.0005, 0.9999]}, and the position offset is [0.003, 0.001, -0.0005] meters.

[0060] A3: Based on the pinhole imaging model and the geometric relationship of rigid body motion, a coordinate offset model caused by the motion of the vision measuring instrument is constructed. The coordinate offset model is used to describe the image point coordinate offset caused by the motion of the camera itself in the vision measuring instrument.

[0061] In step A3, the pinhole imaging model is a mathematical model describing the coordinate relationship between a three-dimensional point projected onto a two-dimensional image plane through a camera lens. This application embodiment does not limit the specific structural parameter design of the model. For example, the internal structure of the pinhole imaging model mainly includes focal length, principal point coordinates, and lens distortion parameters. The focal length describes the distance from the image sensor to the optical center of the lens, the principal point coordinates identify the intersection of the optical axis and the image plane, and the lens distortion parameters are used to correct the radial and tangential deformation caused by the lens. Furthermore, this application does not specifically limit the structural design of the layers and other components used in the internal structure of the pinhole imaging model, and can set them according to the actual situation.

[0062] The geometric relationship of rigid body motion describes the mathematical expression of the position and orientation changes of the visual measuring instrument as a rigid body in three-dimensional space. The embodiments of this application do not limit the mathematical expression of this relationship.

[0063] The coordinate offset model is a mathematical function whose inputs are the attitude change and position offset obtained from step A2, as well as the initial coordinates of a three-dimensional point. The output is the pixel coordinate change caused by camera motion after the three-dimensional point is projected onto the image plane. Furthermore, this application does not impose specific limitations on the structural design and parameter design of the internal structure of the coordinate offset model, and can set them accordingly based on the actual situation.

[0064] In this embodiment, a pinhole imaging model of the visual measuring instrument camera is first established, which includes intrinsic parameters such as the camera's focal length and principal point coordinates. Then, the attitude change and position offset calculated in step A2 are regarded as a rigid body motion transformation of the camera coordinate system relative to the world coordinate system. Finally, combining the pinhole imaging projection formula and the coordinate transformation principle, it is derived how the projection coordinates of a fixed spatial target on the image plane will change when the camera undergoes this rigid body motion, thereby constructing a mathematical model describing the coordinate offset.

[0065] In practical applications, assuming the three-dimensional spatial coordinates of a target are [X, Y, Z] in the world coordinate system, and the initial extrinsic parameter matrix of the camera is... ,in, It is a 3×3 rotation matrix. The translation vector is 3×1. The initial extrinsic parameter matrix of the camera is used to characterize the pose relationship between the camera and the world coordinate system at the initial moment. Its intrinsic parameter matrix of the pinhole imaging model is K. Then the initial image point coordinates are... Through formula The calculation shows that after the camera moves, the new extrinsic parameter matrix is... ,in The new image point coordinates can be calculated from the initial extrinsic parameter matrix and the change calculated in step A2. The coordinate offset model describes the process from... arrive offset .

[0066] A4: Use the coordinate offset model to reverse correct the coordinates of a set of image points to eliminate the pseudo-deformation caused by the movement of the vision measuring instrument itself, and obtain the corrected coordinates of a set of image points.

[0067] In step A4, the coordinate offset model can predict the theoretical offset of image point coordinates based on camera motion information and perform reverse compensation; pseudo-deformation refers to changes in image point coordinates that are not caused by the actual deformation of the measured structure, but by the slight movement of the visual measuring instrument itself during image acquisition.

[0068] In this embodiment of the application, the coordinate offset model constructed in step A3 is first used to calculate the theoretical offset of the image point coordinates corresponding to each target based on the camera's attitude change and position offset calculated in step A2 at the current image acquisition time. Then, the original image point coordinates constructed in step 102 are subtracted from the calculated theoretical offset to offset the influence of camera motion, and finally a set of corrected image point coordinates that only reflect the actual deformation of the structure are obtained.

[0069] In practical applications, continuing the example in step 1022, the original image point coordinates of the "A01" target obtained in step 102 are {[1024, 768], [980, 820]}. After calculations in steps A2 and A3, it is found that due to a slight rotation and translation of the camera when acquiring the second image, the coordinates of the "A01" target have a pseudo offset of [5, 3] pixels. Through reverse correction in step A4, [5, 3] is subtracted from the coordinates [980, 820] in the second image to obtain the corrected coordinates [975, 817]. Thus, the corrected set of image point coordinates of the "A01" target is updated to {[1024, 768], [975, 817]}.

[0070] This application constructs an image coordinate sequence by identifying and associating the same target information in multiple images, and simultaneously collects the inertial motion data of the vision measurement instrument itself. This allows for modeling and elimination of the interference of camera motion on the coordinates, thus achieving accurate extraction and purification of target image position data and laying a reliable data foundation for subsequent high-precision 3D calculation.

[0071] Step 103: Based on the key parameters of the vision measuring instrument, jointly calculate the coordinates of the image points corresponding to all target information to calculate the three-dimensional spatial coordinates of each target information in a unified coordinate system, and calculate the measurement values ​​of various key parameters based on the three-dimensional spatial coordinates. The measurement values ​​are used to reflect the overall state of each key part and the structure under test.

[0072] Among them, the key parameters of the vision measurement instrument refer to the intrinsic parameters describing the imaging geometry of the camera and the extrinsic parameters describing the position and attitude of the camera in space. Intrinsic parameters include focal length and principal point coordinates. The unified coordinate system is a predefined three-dimensional world coordinate system used to describe the spatial position of all targets and structural components. The three-dimensional spatial coordinates are the three-dimensional position data of the target in this world coordinate system. The measured values ​​of various key parameters are a series of physical quantities used to quantify the structural state by further calculating the three-dimensional spatial coordinates of the target.

[0073] In this embodiment, step 103 includes the following process:

[0074] Step 1031: Using a multi-view stereo geometry algorithm, perform cross-view matching on targets with the same coded information in images acquired by different vision measuring instruments at the same time stamp, and determine a set of image coordinate pairs corresponding to each target.

[0075] In step 1031, an image coordinate pair refers to a coordinate combination consisting of the coordinates of image points belonging to the target extracted from images captured by different vision measuring instruments at the same time for the same physical target.

[0076] In this embodiment of the application, multiple images collected at the same time stamp are first obtained from different vision measuring instruments, along with the target information and coordinates identified in each image. Then, the encoded information carried by each target is compared, and the target information from different images but with the same encoding is associated, thereby determining a set of image coordinates composed of multiple viewpoints for each physical target as a set of image coordinate pairs.

[0077] Step 1032: Based on the pre-calibrated intrinsic and extrinsic parameter matrices of each vision measuring instrument, the least squares bundle adjustment algorithm is used to globally optimize the image coordinate pairs to calculate the three-dimensional spatial coordinates of each target in a unified world coordinate system.

[0078] In step 1032, the intrinsic parameter matrix describes the internal imaging characteristics of the vision measurement instrument camera, and the extrinsic parameter matrix describes the position and orientation of the camera in the world coordinate system. The least squares bundle adjustment algorithm is a method for simultaneously optimizing the calculation of all camera parameters and three-dimensional point coordinates by minimizing the sum of squared errors between the observed values ​​and the theoretical projection values. This embodiment does not limit the specific expression of the algorithm, and can be set accordingly according to the actual situation.

[0079] In this embodiment of the application, this step uses all the image coordinate pairs obtained in step 1031 as observation inputs, and combines the intrinsic parameter matrix and initial extrinsic parameter matrix of each vision measurement instrument to establish a set of projection geometric equations between all target 3D points and all camera image coordinates. Then, the least squares bundle adjustment algorithm is used to iteratively optimize all unknown parameters, and finally solves the 3D spatial coordinates of each target in the unified world coordinate system that minimizes the overall projection error.

[0080] Step 1033: Calculate the spatial pose parameters of the corresponding key part based on the three-dimensional spatial coordinates of multiple targets on the same key part. The spatial pose parameters include at least two of displacement, settlement, tilt, and deflection. At the same time, extract the vibration spectrum characteristics of the key part through frequency domain analysis based on the coordinate sequence formed by the three-dimensional spatial coordinates of the same target at different time points.

[0081] In step 1033, the spatial pose parameter is a physical quantity used to describe the position and angular state of the key parts of the structure in space; the vibration spectrum feature is characteristic data that reflects the distribution of vibration energy of the structure at different frequencies at that part after performing frequency domain analysis such as Fourier transform on the sequence of target position changes over time.

[0082] In this embodiment, the three-dimensional spatial coordinates of all targets belonging to the same key part are first extracted. By calculating the distance, angle or fitted plane equation and other geometric relationships between these coordinate points, the displacement, settlement, tilt or deflection value of the part is derived. At the same time, for each target, its three-dimensional coordinates at multiple consecutive time points are collected to form a time series. The frequency domain transformation of the series is performed to analyze its frequency components and extract the main vibration frequency and amplitude characteristics.

[0083] Step 1034: The spatial pose parameters and the vibration spectrum characteristics are used together as the measured values ​​of multiple key parameters reflecting the overall state of each key part and the structure under test.

[0084] In step 1034, the measured values ​​of various key parameters constitute a multi-dimensional set of indicators that integrates structural spatial deformation and dynamic response.

[0085] This application obtains the three-dimensional coordinates of the target by jointly solving multi-view image data, and extracts multi-dimensional deformation and vibration parameters from it, realizing a comprehensive, three-dimensional quantitative assessment of the overall and local state of the structure.

[0086] Step 104: Input the measured values ​​of various key parameters and the acquired environmental data into the preset spatiotemporal propagation model, so as to construct a structural graph with each key part as a node and the relationship between the key parts as an edge according to the physical topology of the structure under test through the spatiotemporal propagation model.

[0087] Among them, environmental data refers to data on external influencing factors related to the state of the structure under test, such as temperature, humidity, wind speed, rainfall, and traffic load.

[0088] The spatiotemporal propagation model is a mathematical model that can describe the diffusion and changes of anomalies or damage in a structural physical network over time and in relation to their spatial location. The structure of the spatiotemporal propagation model can be designed as a graph dynamics system with multi-layer processing. This embodiment does not impose specific limitations on the specific design of the structure and sub-modules of the model, and can be set accordingly according to the actual situation.

[0089] A specific example is as follows: The model first includes an environment-structure response decoupler to filter out systematic components caused by environmental data from the input visual measurement key parameters, generating a first anomaly index sequence; secondly, the model has a multi-source data fusion engine, which receives the first anomaly index sequence, a second anomaly index sequence from auxiliary sensors, and a third anomaly index sequence characterizing environmental sensitivity, and performs evidence fusion and consistency verification through a built-in Bayesian network to calculate a quantified initial anomaly state value for each key component; subsequently, the model also has a physical topology mapping layer, which maps each key component to a graph node and the physical connection relationship between components to graph edges based on the structural design drawings or 3D model, thereby generating a structural graph that reflects the real connection network of the structure.

[0090] Physical topology refers to the actual physical connections and spatial relative positions of the components of the structure under test; a structural graph is a graph theory model in which key parts are abstracted as nodes, and the physical connections between key parts are abstracted as edges connecting nodes, thus transforming the complex physical structure into a network graph that is easy to calculate and analyze.

[0091] In this embodiment, such as Figure 2 As shown, step 104 includes the following process:

[0092] Step 1041: Using the environment-structure response decoupler in the spatiotemporal propagation model, remove the systematic influence caused by the environmental data from the measured values ​​of the various key parameters to obtain a first abnormal index sequence reflecting potential structural damage.

[0093] In step 1041, the environment-structure response decoupler is a functional module in the spatiotemporal propagation model, used to separate the normal structural response caused by environmental factors from the possible structural damage response; the systematic influence quantity refers to the structural parameter change components that are regular due to environmental factors; the first abnormal index sequence is the time series data reflecting the potential abnormality or damage of the structure itself after removing the systematic environmental influence.

[0094] In this embodiment, firstly, key parameter measurements over a long time series and synchronized environmental data are collected. Then, a statistical relationship model between the environmental data and each key parameter is established using regression analysis or machine learning methods. This model can predict the normal response value of the structure under specific environmental conditions. Next, the actual measured values ​​are compared with the normal response values ​​predicted by the model. The residual part can be regarded as the first abnormal indicator sequence that is not affected by the periodicity of the environment. The machine learning method can be algorithms such as random forest, gradient boosting tree, or support vector machine.

[0095] In practical applications, for the deflection monitoring values ​​of a key part of a bridge, 30 consecutive days of data, along with ambient temperature data, are input into a decoupler. The decoupler fits a model of the relationship between temperature and deflection through linear regression. The model predicts that the normal deflection at the average temperature of the day is 15 mm, while the actual monitored value is 16.5 mm. The difference of 1.5 mm is taken as the first abnormal indicator value for that day. The 30 differences generated over 30 consecutive days constitute the first abnormal indicator sequence for that part, for example, the sequence is {0.8, 0.5, 1.0, ..., 1.5} mm.

[0096] Step 1042: Obtain auxiliary sensor data for key parts, and extract a second abnormal indicator sequence from the auxiliary sensor data. The device used for the auxiliary sensor data is different from the visual measurement instrument.

[0097] In step 1042, the auxiliary sensor data is data collected by other types of sensors, such as strain gauges, accelerometers, or inclinometers, installed at key locations; the second anomaly index sequence is another time-series data sequence extracted from these auxiliary sensor data that reflects the abnormal state of the structure.

[0098] In this embodiment of the application, raw data is read from auxiliary sensors deployed at various key locations, and data processing methods similar to or adapted to the characteristics of the sensors, such as thresholding, trend analysis, or modal analysis, are used to identify and extract signals or features that characterize the abnormality from the raw data, forming a second abnormal indicator sequence that is time-aligned with the first abnormal indicator sequence.

[0099] In practical applications, strain gauges are deployed at key parts of the bridge. The daily maximum strain change is extracted from the strain gauge data and compared with its own historical baseline to obtain the daily strain anomaly index, thereby forming a second anomaly index sequence that is synchronized with the first anomaly index sequence in time. For example, the sequence is {10,8,12,...,15} microstrain.

[0100] Step 1043: For each key part, perform multi-source evidence fusion based on the first abnormal indicator sequence and the second abnormal indicator sequence to perform consistency verification and generate an initial abnormal state value.

[0101] The initial abnormal state value is a comprehensive value generated after multi-source evidence fusion and credibility weighting to describe the current degree of abnormality of the key part. It will serve as the initial value of each node in the spatiotemporal propagation model.

[0102] Step 1043 may specifically include the following steps:

[0103] B1: Using the latest state values ​​of the first abnormal indicator sequence, the second abnormal indicator sequence, and the third abnormal indicator sequence derived from environmental data that represents the sensitivity of the corresponding key parts to environmental changes as observation inputs, a Bayesian network model is constructed with the goal of inferring the true damage state of the structure.

[0104] In step B1, the third anomaly index sequence is a time series data reflecting the changes in the sensitivity of key parts to environmental factors, and its latest state value characterizes the current environmental sensitivity; the Bayesian network model is an inference model based on probabilistic graph theory, which consists of nodes representing variables and directed edges representing the probabilistic dependencies between variables.

[0105] In this embodiment, for each key part, a hidden node representing its "true damage state" is first defined. This node has several discrete states, such as "normal", "minor damage", and "severe damage". Then, three observation nodes are defined, representing the latest state values ​​of a first anomaly index from visual measurement, a second anomaly index from an auxiliary sensor, and a third anomaly index characterizing environmental sensitivity. Finally, based on domain knowledge or historical data, directed connections are established between the observation nodes and the hidden nodes to represent the influence of observational evidence on the inference of the true damage state.

[0106] In practical applications, a simple Bayesian network model is constructed for the aforementioned key parts of the bridge. This model contains one hidden node "damage state" and three observation nodes "visual deflection anomaly", "strain anomaly", and "temperature sensitivity".

[0107] B2: Inject a prior probability distribution and a conditional probability table into the Bayesian network model, wherein the prior probability distribution is set based on historical damage statistics, and the conditional probability table is established based on the historical performance and calibration data of various sensors.

[0108] In step B2, the prior probability distribution is the probability estimate of the "true damage state" node being at each level in the absence of current observation evidence; the conditional probability table defines the probability that each observation node takes a different value given the "true damage state".

[0109] In this embodiment, based on the historical maintenance records and damage statistics of the structure or specific part, a priori probability distribution is set for the "true damage state" node. For example, the probability of "normal" is 0.85, the probability of "minor damage" is 0.12, and the probability of "severe damage" is 0.03. At the same time, based on the data of the visual measuring instrument and auxiliary sensors in historical calibration and performance testing, the probability of different abnormal levels of each observation index under each true damage state is statistically analyzed, thereby filling the conditional probability table of each observation node.

[0110] In practical applications, prior probabilities are injected into the Bayesian network of the aforementioned bridge sections: P(normal) = 0.85, P(minor damage) = 0.12, P(severe damage) = 0.03; and a conditional probability table is set. For example, when the true state is "minor damage", the probability of observing "visual deflection anomaly" as "high" is 0.7, as "medium" is 0.25, and as "low" is 0.05.

[0111] B3: Input the latest state values ​​of each abnormal indicator sequence as observation evidence into the Bayesian network model, run the inference algorithm, and calculate the joint posterior probability distribution of the corresponding key parts at different preset damage levels.

[0112] In step B3, the latest state value of each abnormal indicator sequence refers to the specific value of the first, second, and third abnormal indicator sequences at the most recent sampling time; the joint posterior probability distribution refers to the latest probability estimate of the "true damage state" node at each preset damage level after all current observation evidence has been input.

[0113] In this embodiment, the current visual deflection anomaly value, such as 1.5 mm, strain anomaly value, such as 15 microstrain, and temperature sensitivity value, such as 0.1 mm / degree Celsius, are converted into discrete states corresponding to the observation nodes in a Bayesian network and used as hard evidence input to the model. Then, inference algorithms such as variable elimination or belief propagation are used to calculate the probabilities that the key part is in "normal", "slightly damaged", and "severely damaged" under the current evidence, which are 0.2, 0.65, and 0.15, respectively.

[0114] B4: Based on the joint posterior probability distribution, quantify the degree of conflict between each observational evidence, and use the value of the maximum probability value in the posterior probability distribution as the confidence level of this fusion result.

[0115] In step B4, the degree of conflict is a quantitative indicator that measures the consistency between different observational evidences. The degree of conflict is high when the evidence points to inconsistencies. The confidence level is a measure of the credibility of the fusion result.

[0116] In the embodiments of this application, the degree of conflict can be quantified by calculating the Kullback-Leibler divergence between pieces of evidence or other metrics; at the same time, the maximum value in the joint posterior probability distribution is taken as the confidence level of this fusion. The higher the value, the stronger the certainty of the fusion result.

[0117] B5: Sum all the probability values ​​belonging to the preset abnormality level in the joint posterior probability distribution to obtain the comprehensive abnormality probability value of the key part.

[0118] In step B5, the preset abnormal level usually refers to all damage levels other than the "normal" level, such as "minor injury" and "severe injury"; the comprehensive abnormal probability value is the sum of the probabilities that the part is in any abnormal state.

[0119] In this embodiment of the application, the probability of "minor injury" (0.65) and the probability of "serious injury" (0.15) in the joint posterior probability distribution are added together to obtain a comprehensive abnormality probability value of 0.80 for this part.

[0120] B6: The product of the comprehensive anomaly probability value and the confidence level is used as the initial anomaly state value for the corresponding key component to be input into the spatiotemporal propagation model.

[0121] In this embodiment of the application, the overall abnormality probability value of the part, 0.80, is multiplied by its confidence level, 0.65, to obtain the initial abnormality value of the part, 0.52.

[0122] Step 1044: Based on the engineering design drawings or actual 3D point cloud model of the structure under test, determine the physical topology of the structure under test. The physical topology includes the spatial location of each key part and the physical connection relationship between key components.

[0123] In step 1044, the physical connection relationship refers to the actual mechanical connection method between structural components, such as welding, bolting, hinge, or rigid connection.

[0124] In this embodiment of the application, the three-dimensional spatial coordinates of each key part and the connection type and topological relationship between the components to which these parts belong are obtained by parsing the CAD design drawings of the structure or by identifying and extracting the structural components from the obtained three-dimensional laser point cloud model of the structure.

[0125] In practical applications, for a simply supported beam bridge, the two piers and the main beam are identified as key components according to its design drawings. The physical topology is as follows: the two ends of the main beam are connected to the tops of the two piers through supports, forming a chain connection relationship of "pier A-main beam-pier B".

[0126] Step 1045: Define each key part as a node, and establish graph edges between node pairs with physical connections to form a structure graph.

[0127] In this embodiment of the application, each key monitoring part is abstracted as a graph node according to the physical topology determined in step 1044; then, for each pair of key parts with direct physical connection, an undirected graph edge is established between their corresponding nodes; finally, all nodes and edges constitute the structural graph of the structure to be tested.

[0128] In practical applications, based on the topology of the simply supported beam bridge described above, a structural diagram containing three nodes is formed: node 1 (pier A), node 2 (main beam), and node 3 (pier B); an edge is established between node 1 and node 2, and an edge is established between node 2 and node 3.

[0129] This application generates a reliable initial abnormal state by fusing multi-source monitoring data and decoupling environmental influences, and constructs a structural diagram based on the real physical topology, providing a high-confidence starting point and a propagation framework that conforms to the laws of mechanics for subsequent simulation of the spatiotemporal propagation of anomalies, thus realizing the transformation from discrete data perception to structural networked state representation.

[0130] Step 105: Simulate the changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process on the structural diagram to determine the evolution pattern of the structural state.

[0131] Among them, the information on the change of structural state in the spatiotemporal propagation process refers to the data on how the abnormal state value spreads and evolves from one node to adjacent nodes over time along the edges of the structural graph; the evolution pattern is the regular characteristics of how the abnormal state starts, spreads, converges or dissipates, which are identified from the simulated time series data.

[0132] In this embodiment, step 105 includes the following process:

[0133] Step 1051: Based on the physical connection relationship of each graph edge in the structure graph, initialize the mechanical influence coefficient of each graph edge.

[0134] In step 1051, the mechanical influence coefficient is a numerical value that quantifies the inherent influence of the physical connection relationship represented by the edge of the graph when transmitting abnormal states. This value is preset based on the connection type and component properties.

[0135] In this embodiment of the application, based on the actual physical connection relationship corresponding to each side of the structural diagram, such as welding, bolting or hinge, and the material and cross-sectional properties of the connecting components, an initial mechanical influence coefficient is assigned to each side. This coefficient reflects the basic value of the connection's ability to transmit stress or deformation under normal conditions.

[0136] Step 1052: Define a dynamic influence function, which takes the initial abnormal state values ​​of the two end nodes of the graph edge as input and outputs a dynamic amplification factor that is non-linearly related to the input value.

[0137] In step 1052, the dynamic amplification factor is a coefficient that changes with the input abnormal state value and is used to characterize the nonlinear effects that may be amplified or suppressed during the propagation of the abnormal state.

[0138] In this embodiment, the dynamic influence function is designed as a nonlinear function, such as a piecewise function or an exponential function. It should be noted that the specific expression used for this function is not specifically limited in this embodiment. It can be set according to the actual situation. Its input is the larger or average value of the initial abnormal state values ​​of the nodes at both ends of the connecting edge, and the output is a factor greater than or equal to 1. When the input abnormal state value is higher, the output dynamic amplification factor is larger, so as to simulate the nonlinear enhancement of the influence on adjacent parts when the damage intensifies.

[0139] Step 1053: At the start of the simulation, the initial dynamic amplification factor of each graph edge is calculated based on the initial abnormal state value of each node through the dynamic influence function. The mechanical influence coefficient of each graph edge is multiplied by the corresponding initial dynamic amplification factor to obtain the initial time-varying edge weight.

[0140] In step 1053, the initial time-varying edge weight is a value obtained by combining the inherent properties of the connection with the dynamic influence of the current abnormal state, which describes the actual weight of the edge in the abnormal propagation at the current moment.

[0141] In this embodiment of the application, at the start of the simulation, the initial abnormal state values ​​of each node calculated in step 1043 are substituted into the dynamic influence function to calculate an initial dynamic amplification factor for each graph edge. Then, the factor is multiplied by the mechanical influence coefficient of the edge, and the product is the initial time-varying edge weight of the edge. This weight determines the initial intensity of the abnormal state propagating along the edge at the current moment.

[0142] Step 1054: Construct a time-varying matrix from the initial time-varying edge weights of all node pairs, where the matrix elements corresponding to node pairs with no physical connection have a value of zero.

[0143] In step 1054, the time-varying matrix is ​​a two-dimensional matrix whose rows and columns correspond to all nodes in the structure graph. The element value in the i-th row and j-th column of the matrix represents the time-varying edge weight of the edge from node i to node j. If there is no edge between two nodes, the element value is zero.

[0144] In this embodiment of the application, a zero matrix is ​​created according to the node order in the structure graph. Then, all node pairs with physical connections are traversed, and the initial time-varying edge weights of the corresponding edges calculated in step 1053 are filled into the corresponding positions of the matrix to form an initial time-varying matrix. This matrix quantitatively characterizes the propagation capability of the abnormal state on each path in the structure network at the beginning of the simulation.

[0145] Step 1055: Input the structure diagram, the initial abnormal state values ​​of each node, and the time-varying matrix into the graph dynamics simulator.

[0146] In step 1055, the structure of the graph dynamics simulator can be designed as an iterative computation unit that includes a state initialization module, a neighborhood propagation computation module, a state update engine, and a matrix dynamic update module.

[0147] A specific example is as follows: The state initialization module of the simulator first receives the structure diagram generated in step 104, the initial abnormal state values ​​of each node, and the time-varying matrix as input;

[0148] Its neighborhood propagation calculation module uses a time-varying matrix to calculate the weighted sum of the abnormal state values ​​of all its neighboring nodes and the corresponding edge weights for each node, which is then used as the input for the propagation term;

[0149] The state update engine embeds a nonlinear dynamic equation, which in each iteration step comprehensively processes for each node a propagation term from the neighborhood propagation calculation module, a negative decay term proportional to the node's current state, and a positive autocatalytic term triggered when the node's state exceeds a preset threshold, thereby calculating the new abnormal state value of the node in the next time step.

[0150] The matrix dynamic update module recalculates the dynamic amplification factor of each edge and updates the weight elements of the time-varying matrix based on the updated state of all nodes through the dynamic influence function. The entire simulator executes the state update and matrix update process in a loop until the preset simulation duration or state convergence condition is reached, and finally outputs the abnormal state change sequence of all nodes after multiple iterations.

[0151] Step 1056: The graph dynamics simulator performs iterative calculations based on the nonlinear dynamic equation, which includes the following components: a propagation term of a weighted neighborhood anomalous state based on a time-varying matrix, a decay term of the node's own anomalous state, and a self-catalytic term simulating the propagation of damage instability.

[0152] In step 1056, the nonlinear dynamic equation is a mathematical model describing how the abnormal state value of each node changes over time. It should be noted that the specific form of the nonlinear dynamic equation is not specifically limited in the embodiments of this application, and can be set accordingly according to the actual situation.

[0153] The propagation term of the weighted neighborhood anomaly describes the process by which the anomaly of neighboring nodes affects the current node through time-varying edge weights; the decay term describes the trend of the node's own anomaly naturally mitigating over time; and the autocatalytic term describes the nonlinear phenomenon that when the node's anomaly exceeds a certain threshold, its own deterioration rate will accelerate.

[0154] In this embodiment, after receiving input, the graph dynamics simulator establishes a state update equation for each node. At each simulation time step, for each node, it first calculates the weighted sum of the abnormal state values ​​of all its neighboring nodes multiplied by the corresponding edge weights as the propagation input, then subtracts a decay amount proportional to the current state value of the node, and finally adds an autocatalytic increment triggered when the state of the node exceeds a preset threshold and nonlinearly related to its current state value. The state values ​​of all nodes at the next time step are iteratively calculated through this equation.

[0155] Step 1057: At each iteration time step, recalculate the dynamic amplification factor of each graph edge based on the abnormal state values ​​of each node in the previous time step, and update the time-varying matrix.

[0156] In step 1057, updating the time-varying matrix refers to dynamically adjusting the edge weights according to the changes in node states during the simulation process, so as to reflect the characteristics of anomaly propagation capability as the state changes.

[0157] In this embodiment of the application, after completing the state update of a time step, steps 1052 and 1053 are re-executed using the updated abnormal state values ​​of each node. That is, based on the new state values, the new dynamic amplification factor of each graph edge is calculated through the dynamic influence function, and multiplied with the fixed mechanical influence coefficient to obtain the new time-varying edge weight of the time step. The corresponding elements in the time-varying matrix are updated with this weight to provide updated propagation weights for the iterative calculation of the next time step.

[0158] Step 1058: When the preset stopping iteration condition is reached, multiple sets of future node abnormal state change sequences are obtained. The evolution pattern is obtained by analyzing the multiple sets of future node abnormal state change sequences.

[0159] In step 1058, the preset stopping iteration condition can be reaching the set future simulation duration, or all node states tending to stabilize, or the state of a certain key node exceeding the danger threshold; multiple sets of future node abnormal state change sequences record the trajectory of the abnormal state value of each node changing over time during the simulation period.

[0160] In this embodiment of the application, the simulator continuously iterates and calculates until the stopping condition is met. At this time, the state value sequence of each node throughout the simulation time is output. By analyzing these sequences, the starting point of the abnormal state, the main propagation path, the convergence area, whether instability acceleration occurs, and whether it eventually tends to stabilize can be identified. The combination of these features constitutes the determined abnormal evolution pattern.

[0161] This application achieves dynamic simulation of the propagation path and evolution trend of abnormal states in a structure by introducing dynamic edge weights and nonlinear dynamic equations on the structural graph. This enables the prediction of the overall risk evolution pattern of the structure from local monitoring data, thereby improving the foresight of safety early warning.

[0162] Step 106: Determine the safety level of the structure under test based on the evolution pattern of the structural state, and issue a corresponding warning based on the safety level.

[0163] Among them, the safety level is a qualitative or quantitative classification that describes the immediate degree of danger of a structure, based on a comparison of current monitoring data with preset standards; the early warning is a notification issued to management personnel based on the safety level and risk prediction results, which includes different levels of urgency and handling suggestions.

[0164] In this embodiment, step 106 includes the following process:

[0165] Step 1061: Compare the values ​​of the latest N key parameters at the latest time points with the corresponding preset multi-level safety thresholds, and determine the safety level of the structure under test based on the comparison results.

[0166] In step 1061, the multi-level safety threshold is a numerical limit set for each key parameter at multiple levels, such as "normal threshold", "early warning threshold" and "alarm threshold", which is used to divide different safety state intervals. In this embodiment of the application, the numerical value of the preset multi-level safety threshold is not specifically limited, but can be set according to the actual situation.

[0167] In this embodiment, the latest measurement values ​​of each key parameter are first extracted over a recent period. Then, the measurement value of each parameter is compared with its corresponding multi-level security thresholds one by one to determine which threshold range each parameter is currently in. Next, according to a set of predefined rules, such as taking the worst security level among all parameters or weighting and summing the security levels of each parameter, the current security level of the entire structure under test is finally determined, such as "Level 1 Security", "Level 2 Warning", or "Level 3 Alarm".

[0168] Step 1062: Input the evolution pattern, current environmental temperature and humidity and load data, and the trend characteristics of the various key parameters into a pre-trained risk probability prediction model. The risk probability prediction model predicts the probability that the parameter values ​​of the structure under test will exceed preset multi-level safety thresholds within at least one specified monitoring period in the future.

[0169] In step 1062, the risk probability prediction model is a mathematical model trained based on machine learning algorithms. It can learn the complex relationship between environmental factors, parameter trends and evolution patterns and future risk occurrence based on historical data, and output the probability value of exceeding the limit risk within a specified future time period. Furthermore, this application does not impose specific limitations on the structural design of the layers and other components used in the internal structure of the risk probability prediction model, and can set them according to the actual situation.

[0170] In this embodiment, the model uses a large dataset containing historical evolution pattern sequences, environmental data, parameter trends, and labels indicating whether future exceedance events will occur during the training phase to establish a mapping relationship from input features to future risk probabilities. During the application phase, the evolution pattern features obtained in step 105, real-time acquired environmental temperature, humidity, and load data, as well as short-term change trend features of key parameters are input into the model. The model calculates and outputs the probability value of structural parameters exceeding various safety thresholds in one or more future time periods, such as "the probability of exceeding the warning threshold within the next 24 hours is 0.3".

[0171] Step 1063: Combine the security level, the exceeded security threshold, and the corresponding risk probability value, and determine the comprehensive risk level of the structure under test according to the predefined risk decision mapping rules.

[0172] In step 1063, the risk decision mapping rule is a set of logical rules or decision tables that combine the current security status with the probability of future risks for a higher-level comprehensive evaluation.

[0173] In this embodiment of the application, the current security level obtained in step 1061, the probability of future exceeding the limit risk obtained in step 1062, and their corresponding threshold levels are used as inputs. A comprehensive judgment is made according to a preset mapping rule. For example, even if the current security level is "Level 1 Security", if the probability of exceeding the "alarm threshold" is very high in the next 24 hours, the comprehensive risk level may be upgraded to "High Risk". Conversely, if the current level is "Level 2 Warning" but the probability of future risk is very low, the comprehensive risk level may be assessed as "Medium Risk". Through this combined judgment, a comprehensive risk level that considers both the current state and future trends is finally determined.

[0174] Step 1064: Based on the comprehensive risk level, trigger the corresponding early warning response level and automatically generate an early warning report containing the risk location, key parameter status, risk prediction information, and handling suggestions.

[0175] In step 1064, the early warning response level is a different response action set according to the comprehensive risk level, such as "attention", "preparedness" and "immediate action"; the early warning report is a structured document or message that summarizes all key information related to this risk.

[0176] In this embodiment of the application, based on the comprehensive risk level determined in step 1063, the preset early warning response level is automatically matched and triggered, and the report generation process is started at the same time. The report generator will extract relevant risk location information, the detailed status of each key parameter at present, and the prediction conclusions obtained by the risk probability prediction model from the database, and combine them with the standard handling plan corresponding to the risk level to automatically generate a complete early warning report, and send it to relevant management personnel through the designated communication channel.

[0177] This application achieves a closed loop from risk perception to decision support by integrating the assessment of the current state of the structure with the prediction of future risk probabilities, and triggering graded early warning responses and generating detailed handling reports based on the comprehensive risk level. This improves the accuracy of structural safety early warning and the timeliness and effectiveness of emergency response.

[0178] Figure 3 This application provides a schematic diagram of a specific implementation of a computer vision-based change detection system, with reference to... Figure 3 The system may include:

[0179] The acquisition module 31 is used to acquire original images of multiple key parts of the structure under test through a vision measuring instrument, and the original images carry target information.

[0180] The recognition module 32 is used to identify the same target information in multiple original images, and construct a set of image point coordinates corresponding to each target information based on the corresponding coordinates of the same target information in the multiple original images.

[0181] The calculation module 33 is used to jointly calculate the coordinates of the image points corresponding to all target information according to the key parameters of the vision measuring instrument, so as to calculate the three-dimensional spatial coordinates of each target information in a unified coordinate system, and calculate the measurement values ​​of various key parameters based on the three-dimensional spatial coordinates. The measurement values ​​are used to reflect the overall state of each key part and the structure under test.

[0182] The input module 34 is used to input the measured values ​​of various key parameters and the acquired environmental data into a preset spatiotemporal propagation model, so as to construct a structural graph with each key part as a node and the relationship between the key parts as an edge according to the physical topology of the structure under test through the spatiotemporal propagation model.

[0183] The determination module 35 is used to simulate the changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process on the structural diagram, so as to determine the evolution mode of the structural state.

[0184] The judgment module 36 is used to determine the safety level of the structure under test based on the evolution pattern of the structural state, and to issue a corresponding warning based on the safety level.

[0185] This application provides a computer vision-based change detection system for implementing the aforementioned computer vision-based change detection method. Therefore, specific implementation methods of the computer vision-based change detection system can be found in the previous section on the embodiment of the computer vision-based change detection method. The specific implementation methods can be referred to the descriptions of the corresponding embodiments, which will not be repeated here.

[0186] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described computer vision-based change detection methods.

[0187] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described computer vision-based change detection methods.

[0188] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0189] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the computer vision-based change detection method.

[0190] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0191] The foregoing has provided a detailed description of a computer vision-based change detection method, system, device, and medium. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A change detection method based on computer vision, characterized in that, include: Original images of multiple key parts of the structure under test are acquired using a visual measurement instrument, and the original images carry target information. The same target information is identified in multiple original images, and a set of image point coordinates corresponding to each target information is constructed based on the corresponding coordinates of the same target information in the multiple original images. Based on the key parameters of the vision measuring instrument, the coordinates of the image points corresponding to all target information are jointly calculated to obtain the three-dimensional spatial coordinates of each target information in a unified coordinate system. Based on the three-dimensional spatial coordinates, the measurement values ​​of various key parameters are calculated. The measurement values ​​are used to reflect the overall state of each key part and the structure under test. The measured values ​​of multiple key parameters and the acquired environmental data are input into a preset spatiotemporal propagation model, so that the spatiotemporal propagation model can construct a structural graph with each key part as a node and the relationship between each key part as an edge according to the physical topology of the structure under test. The changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process are simulated on the structural diagram to determine the evolution mode of the structural state. Based on the evolution pattern of the structural state, the safety level of the structure under test is determined, and a corresponding early warning is issued based on the safety level.

2. The method according to claim 1, characterized in that, The step involves inputting the measured values ​​of multiple key parameters and acquired environmental data into a preset spatiotemporal propagation model. Based on the physical topology of the structure under test, the spatiotemporal propagation model constructs a structural graph with key components as nodes and the relationships between these key components as edges. This includes: By using the environment-structure response decoupler in the spatiotemporal propagation model, the systematic influence caused by the environmental data is removed from the measured values ​​of the various key parameters, resulting in a first abnormal index sequence reflecting potential structural damage. Acquire auxiliary sensor data for key components, and extract a second abnormal indicator sequence from the auxiliary sensor data. The equipment used for acquiring the auxiliary sensor data is different from a visual measurement instrument. For each key part, multi-source evidence fusion is performed based on the first abnormal indicator sequence and the second abnormal indicator sequence to perform consistency verification and generate an initial abnormal state value; Based on the engineering design drawings or actual 3D point cloud model of the structure under test, the physical topology of the structure under test is determined. The physical topology includes the spatial location of each key part and the physical connection relationship between key components. Each key component is defined as a node, and graph edges are established between pairs of nodes with physical connections to form a structural graph.

3. The method according to claim 2, characterized in that, For each key component, multi-source evidence fusion is performed based on the first anomaly indicator sequence and the second anomaly indicator sequence to conduct consistency verification and generate an initial anomaly state value, including: Using the latest state values ​​of the first abnormal indicator sequence, the second abnormal indicator sequence, and the third abnormal indicator sequence derived from environmental data, which represents the sensitivity of the corresponding key parts to environmental changes, as observation inputs, a Bayesian network model is constructed with the goal of inferring the true damage state of the structure. A prior probability distribution and a conditional probability table are injected into the Bayesian network model, wherein the prior probability distribution is set based on historical damage statistics, and the conditional probability table is established based on the historical performance and calibration data of various sensors. The latest state values ​​of each abnormal indicator sequence are used as observation evidence and input into the Bayesian network model. The inference algorithm is run to calculate the joint posterior probability distribution of the corresponding key parts at different preset damage levels. Based on the joint posterior probability distribution, the degree of conflict between each observation evidence is quantified, and the value of the maximum probability value in the posterior probability distribution is used as the confidence level of this fusion result. The combined anomaly probability value of the key part is obtained by summing all the probability values ​​belonging to the preset anomaly level in the joint posterior probability distribution. The product of the comprehensive anomaly probability value and the confidence level is used as the initial anomaly state value for the corresponding key component to be input into the spatiotemporal propagation model.

4. The method according to claim 1, characterized in that, The step of simulating the changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process on the structural diagram to determine the evolution pattern of the structural state includes: Based on the physical connection relationship of each graph edge in the structure graph, initialize the mechanical influence coefficient of each graph edge; Define a dynamic influence function, which takes the initial abnormal state values ​​of the two end nodes of the graph edge as input and outputs a dynamic amplification factor that has a non-linear relationship with the input value. At the start of the simulation, the initial dynamic amplification factor of each graph edge is calculated based on the initial abnormal state value of each node through the dynamic influence function. The mechanical influence coefficient of each graph edge is multiplied by the corresponding initial dynamic amplification factor to obtain the initial time-varying edge weight. Construct a time-varying matrix from the initial time-varying edge weights of all node pairs, where the matrix elements corresponding to node pairs with no physical connection have a value of zero; The structure diagram, the initial abnormal state values ​​of each node, and the time-varying matrix are input into the graph dynamics simulator; The graph dynamics simulator performs iterative calculations based on a nonlinear dynamic equation, which includes the following components: a propagation term of a weighted neighborhood anomalous state based on a time-varying matrix, a decay term of the node's own anomalous state, and a self-catalytic term simulating the propagation of damage instability. At each iteration time step, the dynamic amplification factor of each graph edge is recalculated based on the abnormal state values ​​of each node in the previous time step, and the time-varying matrix is ​​updated. When the preset stopping iteration condition is reached, multiple sets of future node abnormal state change sequences are obtained. By analyzing these multiple sets of future node abnormal state change sequences, the evolution pattern is obtained.

5. The method according to claim 1, characterized in that, The method involves jointly calculating the image point coordinates corresponding to all target information based on the key parameters of the visual measuring instrument, thereby calculating the three-dimensional spatial coordinates of each target information in a unified coordinate system. Based on these three-dimensional spatial coordinates, measurement values ​​of various key parameters are calculated. These measurement values ​​reflect the overall state of each key component and the structure under test, including: Using a multi-view stereo geometry algorithm, cross-view matching is performed on targets with the same coded information in images acquired by different vision measuring instruments at the same time stamp, to determine a set of image coordinate pairs corresponding to each target; Based on the pre-calibrated intrinsic and extrinsic parameter matrices of each vision measuring instrument, the least squares bundle adjustment algorithm is used to globally optimize the image coordinate pairs in order to calculate the three-dimensional spatial coordinates of each target in a unified world coordinate system. Based on the three-dimensional spatial coordinates of multiple targets on the same key part, the spatial pose parameters of the corresponding key part are calculated. The spatial pose parameters include at least two of displacement, settlement, tilt and deflection. At the same time, based on the coordinate sequence formed by the three-dimensional spatial coordinates of the same target at different time points, the vibration spectrum characteristics of the key part are extracted by frequency domain analysis. The spatial pose parameters and the vibration spectrum characteristics are used together as measured values ​​of multiple key parameters reflecting the overall state of each key part and the structure under test.

6. The method according to claim 1, characterized in that, After constructing a set of image point coordinates corresponding to each target information based on the corresponding coordinates of the same target information in the multiple original images, the method further includes: Acquire angular velocity and linear acceleration data in real time from the inertial measurement units built into each vision measurement instrument; By integrating the angular velocity and the linear acceleration, the attitude change and position offset of the vision measuring instrument at the time of image acquisition relative to the initial calibration time can be calculated. Based on the pinhole imaging model and the geometric relationship of rigid body motion, a coordinate offset model caused by the motion of the vision measuring instrument is constructed. The coordinate offset model is used to describe the image point coordinate offset caused by the motion of the camera itself in the vision measuring instrument. The coordinate offset model is used to reverse the coordinates of a set of image points to eliminate the spurious deformation caused by the movement of the vision measuring instrument itself, and the corrected coordinates of a set of image points are obtained.

7. The method according to claim 1, characterized in that, The step of determining the safety level of the structure under test based on the evolution pattern of the structural state, and issuing corresponding early warnings based on the safety level, includes: The values ​​of the latest N key parameters at the latest time points are compared with the corresponding preset multi-level safety thresholds, and the safety level of the structure under test is determined based on the comparison results. The evolution pattern, current environmental temperature and humidity and load data, and the trend characteristics of the various key parameters are input into a pre-trained risk probability prediction model; the risk probability prediction model predicts the risk probability that the parameter values ​​of the structure under test will exceed the preset multi-level safety thresholds within at least one specified monitoring period in the future. The security level, the exceeded security threshold, and the corresponding risk probability value are combined, and the comprehensive risk level of the structure under test is determined according to the predefined risk decision mapping rules. Based on the comprehensive risk level, the corresponding early warning response level is triggered, and an early warning report containing the risk location, key parameter status, risk prediction information, and handling suggestions is automatically generated.

8. A change detection system based on computer vision, characterized in that, include: The acquisition module is used to acquire original images of multiple key parts of the structure under test through a vision measuring instrument. The original images carry target information. The recognition module is used to identify the same target information in multiple original images, and construct a set of image point coordinates corresponding to each target information based on the corresponding coordinates of the same target information in the multiple original images. The calculation module is used to jointly calculate the coordinates of image points corresponding to all target information according to the key parameters of the vision measuring instrument, so as to calculate the three-dimensional spatial coordinates of each target information in a unified coordinate system, and calculate the measurement values ​​of various key parameters based on the three-dimensional spatial coordinates. The measurement values ​​are used to reflect the overall state of each key part and the structure under test. The input module is used to input the measured values ​​of various key parameters and the acquired environmental data into a preset spatiotemporal propagation model, so as to construct a structural graph with each key part as a node and the relationship between each key part as an edge according to the physical topology of the structure under test through the spatiotemporal propagation model. The determination module is used to simulate the changes in the structural state corresponding to the measured values ​​of the key parameters during the spatiotemporal propagation process on the structural diagram, so as to determine the evolution mode of the structural state. The judgment module is used to determine the safety level of the structure under test based on the evolution pattern of the structural state, and to issue a corresponding warning based on the safety level.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of a computer vision-based change detection method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables a change detection method based on computer vision as described in any one of claims 1 to 7.

Images