Method for identifying and early warning abnormal behaviors of pedestrians on bridge based on intelligent monitoring

Abstract

The invention discloses an on-bridge pedestrian abnormal behavior identification and early warning method based on intelligent monitoring, and relates to the technical field of bridge engineering, and the method comprises the steps: carrying out the registration of a current visible light image and a current infrared image, and obtaining a registration dual-mode image based on a current image coordinate system; considering current bridge deformation to establish a current coordinate mapping conversion model for mapping pixel coordinate points in the current image coordinate system to physical coordinate points of a current bridge physical space coordinate system; obtaining the current three-dimensional actual coordinates of the pedestrian; recognizing abnormal behaviors of the pedestrians on the bridge based on the current three-dimensional actual coordinates of the pedestrians and acquiring current dynamic risk score values of the pedestrians; and determining the current risk level of the pedestrian according to the current dynamic risk score value of the pedestrian and a preset risk threshold value to realize multi-level linkage protection response. The method provided by the invention solves the technical problem in the prior art that the current actual coordinate of the pedestrian is difficult to accurately obtain due to environmental interference, and the abnormal recognition of the pedestrian on the bridge is inaccurate.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge engineering, and in particular to a method for identifying and warning abnormal behavior of pedestrians on a bridge based on intelligent monitoring. Background Art

[0002] As an important transportation infrastructure, the safety of bridges is related to public safety and social stability. Although modern bridges are usually equipped with passive protection facilities such as pedestrian walkways and guardrails on both sides, such facilities can only provide basic protection functions and cannot effectively deal with deliberate climbing and overtaking, resulting in safety incidents such as jumping off bridges. Existing bridge safety monitoring mainly relies on manual intervention, including witnesses calling the police or security personnel checking the monitoring in real time. This passive monitoring mode has obvious time delays, making it difficult to achieve timely intervention.

[0003] Existing monitoring technologies face multiple challenges: on the one hand, traditional systems lack the ability to adapt to the special characteristics of bridge environments (such as large spans, vibrations, climate change, etc.), and it is difficult to accurately obtain pedestrian spatial location information in complex environments; on the other hand, the effect of single-modal perception technology is significantly reduced in low-visibility environments, and the data dimension is limited, which cannot support the recognition of complex behavior patterns. More importantly, existing technologies generally lack closed-loop solutions that integrate monitoring, identification, early warning and protection, and cannot implement differentiated protection strategies according to different risk levels.

[0004] Therefore, there is an urgent need to develop a bridge pedestrian safety monitoring and protection system that can realize accurate coordinate acquisition, multimodal information fusion, intelligent behavior recognition and graded protection response under complex environmental conditions, so as to achieve early identification and active intervention of potential dangerous behaviors on the bridge, and effectively improve the bridge safety protection capabilities. Summary of the invention

[0005] The main purpose of the present invention is to provide a method for identifying and warning abnormal behavior of pedestrians on a bridge based on intelligent monitoring, aiming to solve the technical problem in the prior art that it is difficult to accurately obtain the actual coordinates of pedestrians under environmental interference, resulting in inaccurate identification of abnormal behavior.

[0006] To achieve the above object, the present invention provides a method for identifying and warning abnormal behavior of pedestrians on a bridge based on intelligent monitoring, comprising the steps of: S10, registering a current visible light image and a current infrared image to obtain a registered dual-modal image based on a current image coordinate system; S20, considering the current deformation of the bridge, establishing a current coordinate mapping transformation model that realizes mapping of pixel coordinate points in the current image coordinate system to physical coordinate points in the current bridge physical space coordinate system; S30, obtaining the current three-dimensional actual coordinates of the pedestrian in the physical space coordinate system of the bridge ( ), specifically including the steps: S31. Extract the pedestrian targets of each pedestrian from the registered bimodal images and determine the center coordinates of the pedestrian target boxes; S32. Map the center coordinates of the pedestrian target boxes to the bridge physical space coordinate system through the current coordinate mapping conversion model to obtain the current mapped physical horizontal coordinates of the pedestrians; S33. Based on the previous actual physical horizontal coordinates of the pedestrians, obtain the current predicted physical horizontal coordinates of the pedestrians through motion prediction; S34. Based on the current mapped physical horizontal coordinates of the pedestrians and the current predicted physical horizontal coordinates of the pedestrians, perform weighted fusion to obtain the current actual physical horizontal coordinates of the pedestrians ( , ); S35. Use the binocular disparity image combined with the IMU pitch angle to compensate for the installation error to obtain the current elevation coordinates of the pedestrians, and obtain the real-time elevation physical coordinates of the pedestrians ; S40. Identify abnormal behaviors of pedestrians on the bridge based on the current three-dimensional actual coordinates of the pedestrians; S50. Calculate and obtain the current dynamic risk score value of the pedestrians based on the multi-factor fusion model; S60. Determine the current risk level of the pedestrians according to the current dynamic risk score value of the pedestrians and the preset risk threshold; S70. Implement a multi-level linkage protection response based on the current risk level of the pedestrians.

[0007] Further, step S10 specifically includes: S11. Process the current visible light image by using the vibration-wind load dynamic Gaussian filtering and edge protection weight method to obtain the current visible light filtered compensation image; S12. Use the material adaptive calibration model to dynamically compensate for environmental interference based on the thermodynamic parameters of steel and concrete respectively, and calibrate the current infrared image to obtain the infrared calibration image; S13. Based on the elastic registration model, adopt the deformation feedback elastic registration strategy, and combine the real-time deformation data of the bridge to optimize the registration matrix for fusion to obtain the registered bimodal image, where the registered bimodal image and the registered bimodal image share the same image coordinate system and retain their respective feature information.

[0008] Further, in step S11, use the formula to calculate the dynamic Gaussian filter kernel parameters at time t, where, and are the current vibration amplitude and the current vibration frequency at time t obtained through the vibration sensor respectively, is the current wind speed at time t obtained through the environmental sensor, is the vibration matching coefficient, is the wind load correction coefficient, and the value range of k is , The value range of .

[0009] Furthermore, in step S12, the functional expression of the material adaptive calibration model is , where represents the calibrated infrared temperature value, are the model coefficients related to the current temperature, current wind speed, and current humidity respectively, and are the current wind speed and current humidity obtained through the environmental sensor respectively, is the environmental temperature difference between the current temperature and the calibration reference temperature, is the original infrared temperature value.

[0010] Furthermore, in step S13, the functional expression of the elastic registration model is: where is the elastic deformation risk assessment coefficient, and the value range is 0 - 1, is the number of reference points, is the confidence weight of the feature point i, satisfying , is the current state coordinate, is the reference state coordinate, is the square of the spatial distance between the current position and the reference position of the monitoring point, is the deformation sensitivity parameter.

[0011] Furthermore, in step S20, at least 8 CAD physical feature points ( ) are extracted from the bridge design CAD model as matching reference points for coordinate mapping with the registered bimodal image. At the same time, the dynamic Gaussian filter kernel parameter and the thermal expansion coefficient are incorporated into the coordinate transformation process as dynamic deformation compensation factors for calculation, and the homography transformation matrix H for realizing the coordinate mapping between the image coordinate system and the bridge physical space coordinate system is obtained.

[0012] Furthermore, the functional expression of the homography transformation matrix H is: where + , is the real-time deformation offset, is the deformation compensation weight, and are the horizontal scaling and rotation parameters of the image respectively. For a concrete bridge, they need to be adjusted according to the curvature of the bridge deck, and are the perspective transformation coefficients, and are the translation parameters respectively, is the instantaneous value of the acceleration component of the bridge in the X direction at time s, is the instantaneous value of the dynamic Gaussian filter kernel parameter during the integration process, is the instantaneous value of the current vibration frequency during the integration process, is the differential of the integration variable, is the bridge span, ( ) are the center coordinates of the pedestrian target box.

[0013] Furthermore, step S40 includes: S41, constructing a position feature vector and the current regional risk label of the pedestrian, , where is the current regional risk label of the pedestrian. The current regional risk label of the pedestrian is used to characterize the type of risk area where the pedestrian is currently located. The current regional risk label of the pedestrian includes a first regional label, a second regional label, and a third regional label. Among them, based on the current three-dimensional actual coordinates of the pedestrian ( ), the current stress state of the bridge and the current traffic flow of the bridge are dynamically determined for the current regional risk label of the pedestrian.

[0014] Furthermore, the formula is used to dynamically generate the current regional risk label of the pedestrian. Among them, represents the distance from the pedestrian to the guardrail, , represents the distance from the pedestrian to the left guardrail, represents the distance from the pedestrian to the right guardrail, is the safety distance, , is the current Mises stress of the bridge, is the yield strength of the steel, is the compressive strength of the concrete, is the detection of incursion into the carriageway; , if the traffic flow Q in a time period ≤ 2000 vehicles / hour, then determine the incursion determination time of 5 s. If the traffic flow Q in a time period > 2000 vehicles / hour, then determine the incursion determination time is 3s, is the Z - coordinate of the left - hand lane,[ is the Z - coordinate of the right - hand lane.[

[0015] Furthermore, step S40 further includes S42. According to the motion feature vector , the attitude feature vector and the temperature feature vector to obtain the fused feature vector , , where , , are the position feature weight, the attitude feature weight, and the temperature feature weight respectively; Use a time - series sliding window to smooth the fused feature vector to obtain the multi - dimensional feature vector ; where is the Gaussian weight, the window size is 5, and the multi - dimensional feature vector is used as the input data for the current dynamic risk score value of the pedestrian.[

[0016] Furthermore, the pedestrian motion feature vector , is used to characterize the horizontal plane behavior dynamics, , is the current horizontal speed of the pedestrian, is the horizontally - accelerated speed after low - pass filtering, is the horizontal plane trajectory curvature; is used to characterize the elevation plane behavior dynamics, , is the current vertical speed of the pedestrian, is the vertical acceleration, is the height change amount relative to the previous moment.[

[0017] Furthermore, in step S50, use the formula to calculate and obtain the current dynamic risk score value of the pedestrian, where is the current attitude feature score of the pedestrian, is the correction coefficient of the pedestrian's stay - time dimension, is the group - effect correction coefficient, is the regional risk coefficient corresponding to the current physical area label of the pedestrian, is the pedestrian motion trend coefficient, is the thermodynamic risk factor; where the current attitude feature score of the pedestrian Obtained based on the behavior type label, and the behavior type label is composed of a multi-dimensional feature vector , the current actual physical horizontal coordinate of the pedestrian ( ), and the pedestrian motion feature vector ; The correction coefficient of the pedestrian stay time dimension Obtained based on the pedestrian motion feature vector ; The group effect correction coefficient Obtained based on the interactive pedestrian quantity data; The regional risk coefficient Obtained based on the current physical area label of the pedestrian; The pedestrian motion trend coefficient Based on the pedestrian moving direction, the pose feature vector , and the pedestrian motion feature vector ; The thermodynamic risk factor Obtained based on the pedestrian temperature feature vector ;

[0018] It can be understood that in the solution of the present invention, the correlation indexes are determined by the expert library evaluation method and / or the historical data analysis method and / or the normalization processing method as , , , , and ; the importance scores of each correlation index are processed according to expert experience to obtain a judgment matrix; the weight values of each evaluation index are calculated according to the judgment matrix, and consistency check is performed to obtain the final weight values in the specific states of each correlation index.

[0019] Furthermore, the behavior type label includes a climbing behavior pose, a crossing behavior pose, a lingering behavior pose, a rapid running behavior pose, and a normal walking pose; if the current pose of the pedestrian is a climbing behavior pose, the current pose feature score of the pedestrian is determined to be 0.7; if the current pose of the pedestrian is a crossing behavior pose, the current pose feature score of the pedestrian is determined to be 0.6; if the current pose of the pedestrian is a lingering behavior pose, the current pose feature score of the pedestrian is determined to be 0.5; if the current pose of the pedestrian is a rapid running behavior pose, the current pose feature score of the pedestrian is determined to be 0.4, and the current horizontal speed of the pedestrian in the rapid running behavior pose ; if the current pose of the pedestrian is a normal walking pose, the current pose feature score of the pedestrian is determined to be 0.2, and the current horizontal speed of the pedestrian in the normal walking pose is 0 ; and / or using the formula: Calculate the time dimension correction coefficient , where is the action duration determined based on the pedestrian motion feature vector ; and / or determine the number of interacting pedestrians based on the multi-target tracking module, and quantify the amplification effect of the group effect on the risk; if in the single-person state, determine ; if in the two-person state, determine ; if in the state of more than two people, determine ; and / or obtain the regional risk coefficient according to the regional label mapping; if it is the first regional label, determine the regional risk coefficient ; if it is the second regional label, determine the regional risk coefficient ; if it is the third regional label, determine the regional risk coefficient and / or if and the coordinate moves towards the guardrail, then determine the motion trend coefficient ; if and the coordinate moves away from the guardrail, then determine the motion trend coefficient ; If | | ≤ , then determine the motion trend coefficient ; and / or determine the thermodynamic abnormal state based on the temperature feature vector ; where is the preset signal for temperature anomaly alarm.

[0020] Furthermore, in step S60, S61, compare the current dynamic risk score value of the pedestrian with the preset risk threshold, and the preset risk threshold includes the first risk critical point value 1.0 and the second risk critical point value 1.5; if > 1.5, then determine it as the high-risk level; if 1.0 ≤ ≤ 1.5, then determine it as the medium-risk level; if < 1.0, then determine it as the low-risk level; S62, display the determined risk level in real time on the monitoring interface and update it to the risk analysis database.

[0021] Furthermore, in step S70, a multi-level linkage protection response is implemented based on the risk level, which specifically includes: S71, establishing a three-level linkage early warning mechanism to achieve a gradient early warning response from the site to the remote: S72, implementing differential protection measures according to different risk levels. For low risk levels, a low risk response is carried out; for medium risk levels, a medium risk response is carried out; for high risk levels, a high risk response is carried out; Among them, during a low risk response, on-site audible and visual warnings are triggered; during a medium risk response, on-site audible and visual warnings are triggered and the remote monitoring center warning is activated; during a high risk response, the emergency linkage mechanism is activated.

[0022] Compared with the prior art, the method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring provided by the present invention has the following beneficial effects: The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring provided by the present invention overcomes complex environmental interference through a dual-modal fusion technology (visible light / infrared) and a bridge dynamic deformation adaptive model, and realizes mapping the center coordinates of the pedestrian target box in the registered dual-modal image from the current image coordinate system to the current bridge physical space coordinate system to obtain the current mapped physical horizontal coordinate of the pedestrian; then based on the previous actual physical horizontal coordinate of the pedestrian, the current predicted physical horizontal coordinate of the pedestrian is obtained through motion prediction; finally, the current actual physical horizontal coordinate of the pedestrian is obtained through weighted fusion of the current mapped physical horizontal coordinate of the pedestrian and the current predicted physical horizontal coordinate of the pedestrian. In the solution of the present invention, the influence of the bridge environment is fully considered and the current actual physical horizontal coordinate of the pedestrian is determined based on the previous actual physical horizontal coordinate of the pedestrian; and binocular disparity images are used in combination with the IMU pitch angle to compensate for the installation error to obtain the current elevation coordinate of the pedestrian, and the real-time elevation physical coordinate of the pedestrian is obtained; finally, the current three-dimensional actual coordinate of the pedestrian in the bridge physical space coordinate system is obtained; and after determining the current three-dimensional actual coordinate of the pedestrian, the abnormal behaviors of the pedestrians on the bridge are identified and the current dynamic risk score value of the pedestrians is obtained; finally, a multi-level linkage protection response is implemented based on the current risk level of the pedestrian; compared with the traditional technology, the present invention not only solves the problem of inaccurate acquisition of pedestrian coordinates under environmental interference, but also realizes all-weather and full-process intelligent monitoring and active protection, significantly improving the intelligent level and warning efficiency of bridge safety protection. Description of the Drawings

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0024] Figure 1 It is a schematic flow chart of a method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring in an embodiment of the present invention; Figure 2 It is a schematic flow chart of obtaining registered bimodal images in an embodiment of the present invention; Figure 3 It is a schematic flow chart of obtaining the current three-dimensional actual coordinates of a pedestrian in an embodiment of the present invention.

[0025] The realization of the purpose, functional characteristics and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0028] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0029] In addition, the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions appears to be contradictory or unable to be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0030] Please refer to the attached Figure 1 , Figure 2 and Figure 3, the present invention provides a method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring, including the following steps: S10, registering the current visible light image and the current infrared image to obtain a registered dual-modal image based on the current image coordinate system; S20, considering the current bridge deformation, establishing a current coordinate mapping conversion model for mapping the pixel coordinate points in the current image coordinate system to the physical coordinate points in the current bridge physical space coordinate system; S30, obtaining the current three-dimensional actual coordinates of the pedestrian in the bridge physical space coordinate system ( ), specifically including the steps: S31, extracting the pedestrian targets of each pedestrian from the registered dual-modal image and determining the center coordinates of the pedestrian target boxes; S32, mapping the center coordinates of the pedestrian target boxes to the bridge physical space coordinate system through the current coordinate mapping conversion model to obtain the current mapped physical horizontal coordinates of the pedestrian; S33, obtaining the current predicted physical horizontal coordinates of the pedestrian based on motion prediction according to the previous actual physical horizontal coordinates of the pedestrian; S34, obtaining the current actual physical horizontal coordinates of the pedestrian through weighted fusion based on the current mapped physical horizontal coordinates of the pedestrian and the current predicted physical horizontal coordinates of the pedestrian ( , ); S35, obtaining the current elevation coordinates of the pedestrian by using the binocular disparity image combined with the IMU pitch angle to compensate for the installation error, and obtaining the real-time elevation physical coordinates of the pedestrian ; S40, identifying the abnormal behaviors of pedestrians on the bridge based on the current three-dimensional actual coordinates of the pedestrian; S50, calculating and obtaining the current dynamic risk score value of the pedestrian based on the multi-factor fusion model; S60, determining the current risk level of the pedestrian according to the current dynamic risk score value of the pedestrian and the preset risk threshold; S70, realizing a multi-level linkage protection response based on the current risk level of the pedestrian.

[0031] A method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring, through dual-modal fusion technology (visible light / infrared) and a bridge dynamic deformation adaptive model, overcomes complex environmental interference, and realizes mapping the center coordinates of the pedestrian target box in the registered dual-modal image from the current image coordinate system to the current bridge physical space coordinate system to obtain the current mapped physical horizontal coordinate of the pedestrian; then based on the previous actual physical horizontal coordinate of the pedestrian, the current predicted physical horizontal coordinate of the pedestrian is obtained through motion prediction; finally, the current actual physical horizontal coordinate of the pedestrian is obtained through weighted fusion of the current mapped physical horizontal coordinate of the pedestrian and the current predicted physical horizontal coordinate of the pedestrian. In the solution of the present invention, the influence of the bridge environment is fully considered and the current actual physical horizontal coordinate of the pedestrian is determined based on the previous actual physical horizontal coordinate of the pedestrian; and binocular disparity images are used in combination with the IMU pitch angle to compensate for the installation error to obtain the current elevation coordinate of the pedestrian, and the real-time elevation physical coordinate of the pedestrian is obtained; finally, the current three-dimensional actual coordinate of the pedestrian in the bridge physical space coordinate system is obtained; and after determining the current three-dimensional actual coordinate of the pedestrian, the abnormal behaviors of the pedestrians on the bridge are identified and the current dynamic risk score value of the pedestrians is obtained; finally, a multi-level linkage protection response is realized based on the current risk level of the pedestrian; compared with the traditional technology, the present invention not only solves the problem of inaccurate acquisition of pedestrian coordinates under environmental interference, but also realizes all-weather and full-process intelligent monitoring and active protection, significantly improving the intelligent level and warning efficiency of bridge safety protection.

[0032] Further, step S10 specifically includes: S11, processing the current visible light image by using vibration-wind load dynamic Gaussian filtering and edge protection weight method to obtain the current visible light filtered compensation image.

[0033] It can be understood that in the solution of the present invention, vibration-wind load coupled dynamic filtering is realized. This method breaks through the limitations of traditional static Gaussian filtering. By establishing an accurate mathematical model of vibration frequency, amplitude and wind load action, real-time adaptive adjustment of filtering parameters is realized; the system input includes the original visible light image , vibration sensor data (frequency at time t , amplitude ), and environmental parameters (wind speed at time t , light intensity ), and the output is a visible light image that has been accurately filtered and retains structural edge features; an innovative bridge structure-oriented edge protection mechanism is introduced, and adaptive enhancement of edges with different bridge type features is realized through a direction correlation function, effectively solving the problem of edge blurring of traditional image processing algorithms under bridge vibration and complex environmental illumination.

[0034] S12. Using the material adaptive calibration model, based on the respective thermodynamic parameters of steel and concrete, dynamically compensate for environmental interference, and calibrate the current infrared image to obtain an infrared calibrated image.

[0035] It can be understood that in the solution of the present invention, a material adaptive calibration model is developed, breaking through the technical barrier that traditional infrared image processing ignores the differences in material thermal properties, realizing the refined differentiation processing of steel structures and concrete structures, and accurately compensating for the differential effects of environmental factors on different materials by establishing a multi-physical field coupling model including temperature, wind speed, and humidity. The system input includes the original infrared image , the identification of bridge material types (steel, concrete), and multi-dimensional environmental sensing data (temperature at time t , wind speed , humidity ), and outputs the calibrated infrared image, greatly improving the temperature measurement accuracy and providing a high-precision basis for subsequent temperature anomaly detection.

[0036] S13. Based on the elastic registration model, adopt the deformation feedback elastic registration strategy, and optimize the registration matrix for fusion in combination with the real-time deformation data of the bridge to obtain a registered bimodal image. Among them, the registered bimodal image and the registered bimodal image share the same image coordinate system and retain their respective feature information.

[0037] It can be understood that in the solution of the present invention, a deformation feedback elastic registration technology is constructed, innovatively taking the real-time deformation data of the bridge as the key input variable in the registration process, and realizing the dynamic registration of bimodal images through the elastic registration model. The system dynamically optimizes the elastic deformation radius (determined by the real-time monitoring data of the bridge) and outputs the accurately registered bimodal image. This technology solves the problem of accuracy attenuation of traditional registration algorithms under the long-term deformation conditions of the bridge and realizes stable registration during the long-term monitoring process.

[0038] It can be understood that in the solution of the present invention, by constructing a complete data preprocessing optimization closed-loop, a parameter mutual feedback mechanism is formed among the processing modules. For example, the dynamic filtering parameters are transmitted to the coordinate positioning module to optimize the deformation compensation parameters, and the elastic deformation radius is fed back to update the homography matrix. This multi-level parameter self-adaptation and feedback optimization mechanism ensure the stability of the system under the condition of drastic changes in the bridge environment.

[0039] Furthermore, in step S11, use the formula to calculate the dynamic Gaussian filter kernel parameters at time t, and adaptively adjust the smoothing intensity according to the bridge vibration and wind load conditions, so as to suppress the image blur caused by bridge vibration while retaining the structural edge features. Among them, are the current vibration amplitude and current vibration frequency at time t obtained through the vibration sensor, respectively, is the current wind speed at time t obtained through the environmental sensor, is the vibration matching coefficient, is the wind load correction coefficient, and the value range of k is , The value range of is . Among them, is the vibration matching coefficient, taking a high value for cable-stayed bridges and a low value for beam bridges; is the wind load correction coefficient, calibrated by wind tunnel tests.

[0040] Furthermore, in step S11, the formula is used to calculate the image edge protection weight, where is the edge sensitivity factor, represents the image gradient intensity, is the direction correlation function (direction weight function). Introducing the direction weight function , the edges of key structures such as guardrails and cables are strengthened, , and θ is the angle between the image gradient direction and the direction of the bridge main structure.

[0041] Furthermore, when it is perpendicular to the guardrail bridge structure, 89 ≤ θ ≤ 91°, and when it is perpendicular to the cable bridge structure, 17 ≤ θ ≤ 75°. Preferably, when strengthening the vertical guardrail, θ ≈ 90°.

[0042] Furthermore, in step S12, the functional expression of the material adaptive calibration model is: , represents the calibrated infrared temperature value, that is, the temperature value of the pixel point (x, y) at time t after environmental factor compensation, are the model coefficients related to the current temperature, current wind speed, and current humidity respectively, and are the current wind speed and current humidity obtained through the environmental sensor respectively, is the environmental temperature difference formed between the current temperature and the calibration reference temperature, is the original infrared temperature value. Preferably, for steel bridges, , , ; for concrete bridges, , , .

[0043] Furthermore, in step S13, the functional expression of the elastic registration model is: Among them, is the elastic deformation risk assessment coefficient, which is used to measure the impact degree of the bridge elastic deformation state on pedestrian safety. The value range is 0-1, and the larger the value, the higher the risk; is the number of reference points, which is the total number of bridge characteristic points or sensor nodes used to calculate the deformation influence; is the confidence weight of feature point i. For example, the weight of the pylon anchorage point is 0.3, and the weight of the guardrail joint point is 0.2, satisfying ; is the current state coordinate, is the reference state coordinate, is the square of the spatial distance between the current position and the reference position of the monitoring point, is the deformation sensitivity parameter.

[0044] Furthermore, , is the elastic deformation radius determined according to the bridge design specification; the deformation compensation parameter (the bridge deformation compensation amount comes from the feedback of the coordinate positioning module), and the parameters of the registration model are updated in real time through , where represents the initial state value of the deformation sensitivity parameter in the elastic registration model, is the adjustment coefficient, which represents the influence degree of the deformation on the registration sensitivity, forming a closed-loop feedback mechanism, enabling the registration process to adapt to the deformation state of the bridge changing with time, and improving the registration stability during the long-term monitoring process.

[0045] In a specific implementation example of the solution of the present invention, for the problem of multi-source data interference in the bridge monitoring scenario, the present solution proposes a hierarchical adaptive preprocessing framework, which solves the deficiencies of traditional methods in aspects such as noise suppression, light compensation, and spatio-temporal registration through dynamic parameter adjustment and dual-modal collaborative processing. The output data directly drives the subsequent coordinate positioning and feature extraction modules to form a closed-loop optimization link. The innovation lies in vibration-wind load coupling filtering, material adaptive temperature calibration, and deformation feedback elastic registration, significantly improving the data quality and system robustness.

[0046] ​Specifically, for the problem of data acquisition in complex environments in bridge monitoring scenarios, the present invention implements an adaptive signal preprocessing method, which mainly solves the impacts of bridge vibration, wind load interference, and deformation on image quality and registration accuracy. Specifically, the monitoring system of a certain bridge consists of multiple groups of visible light cameras and infrared thermal imagers. During the actual deployment process, the system first conducts a multi-parameter characteristic analysis of the bridge to determine the initial configuration of the preprocessing parameters applicable to this type of bridge. In the preprocessing link of visible light images, the system dynamically adjusts the filtering kernel parameters according to the real-time collected vibration and wind force data, making the filtering intensity match the current state of the bridge, and ensuring that the structural edge features can be retained when the bridge vibrates greatly. In practical applications, the system can automatically adapt to environmental changes when the wind speed changes, maintaining the clarity of image edges. Especially, the edge retention effect of key structures such as bridge cables and guardrails is remarkable. In the preprocessing link of infrared images, the system adopts different temperature calibration parameters for steel structure and concrete structure areas according to the composite material structure characteristics of the bridge. When the environmental temperature changes, the system can perform dynamic compensation according to the differences in material thermal characteristics, significantly improving the reliability of temperature data and providing an accurate basis for subsequent analysis of the temperature characteristics of abnormal pedestrian behaviors. In the dual-modal spatio-temporal registration link, the system selects feature points on the bridge that are not easily affected by deformation as registration reference points, including fixed structural features such as bridge tower anchor points and guardrail joints. The system determines the deformation sensitivity parameters according to the bridge design specifications and dynamically updates the registration model parameters through a deformation feedback mechanism. When the bridge deforms due to load changes, the system can maintain high-precision registration, and the registration error is significantly lower than that of traditional methods. Through the above three-stage preprocessing process, this embodiment realizes high-quality acquisition and processing of bridge monitoring data, and maintains stable processing performance under various weather conditions and bridge operating states. The dual-modal images processed by the system provide high-quality data input for the subsequent pedestrian coordinate positioning and behavior recognition modules, significantly improving the operating reliability and recognition accuracy of the entire monitoring system.

[0047] Further, in step S20, at least 8 CAD physical feature points ( ) are extracted from the bridge design CAD model as matching reference points to register the dual-modal images and register the dual-modal images for coordinate mapping. At the same time, the vibration parameters (i.e., the dynamic Gaussian filtering kernel parameters ) and the coefficient of thermal expansion are incorporated into the coordinate conversion process as dynamic deformation compensation factors for calculation, and the homography transformation matrix H for realizing the coordinate mapping between the image coordinate system and the bridge physical space coordinate system is obtained.

[0048] Further, the functional expression of the homography transformation matrix H is: Among them, + , is the real-time deformation offset, is the deformation compensation weight, and are the horizontal scaling and rotation parameters of the image respectively. For a concrete bridge, it needs to be adjusted according to the bridge deck curvature. and are the perspective transformation coefficients, and are the translation parameters, corresponding to the offset of the starting point coordinates of the bridge deck. Optionally, : Adjust according to the bridge vibration frequency ; take a high value for high-frequency vibration; is the instantaneous value of the acceleration component of the bridge in the X direction at time s, is the instantaneous value of the dynamic Gaussian filter kernel parameter during the integration process, is the instantaneous value of the current vibration frequency during the integration process, is the differential of the integration variable, is the bridge span Specifically, represents the parameter value at a specific moment , represents a function of the integration variable τ during the integration process. Further, in step S12, there is also a step of using verification feature points for mapping verification. Randomly select 5 non-calibrated points to verify the error. If the average deviation then trigger the re-acquisition of feature points and the update of matrix H.

[0049] Optionally, for a newly built bridge, the system preferentially uses homography transformation (CAD-driven mode) to achieve high-precision close-range mapping. In practical applications, the system first extracts at least 8 feature points from the bridge CAD design model as matching reference points. These feature points usually include obvious and stable positions such as bridge towers, guardrail connection points, and cable fixing points. The system matches these CAD physical feature points with the registered bimodal images, and at the same time introduces vibration parameters and thermal expansion coefficients as dynamic deformation compensation factors, and calculates the homography matrix H by minimizing the reprojection error. During the calculation of homography transformation, the system dynamically processes the bridge deformation factors, incorporates the real-time data collected by the vibration sensor and the material thermal expansion coefficient α into the calculation, so that the coordinate transformation process can adapt to the elastic deformation of the bridge under different environmental conditions. After each mapping is completed, the system automatically selects 5 non-calibrated verification points to check the mapping accuracy. When the average deviation exceeds the preset threshold, it triggers the re-acquisition of feature points and the update of the matrix.

[0050] Further, in step S20, three key feature points of the guardrail are located by Beidou / GPS to construct a satellite relative coordinate system, and continuous positioning is achieved by combining visual inertial odometry (VIO) technology to obtain the affine transformation parameters for realizing the coordinate mapping between the image coordinate system and the bridge physical space coordinate system; the longitude and latitude coordinates of 3 key feature points of the guardrail are collected and converted into UTM plane coordinates ( ); the affine transformation parameters ( ) are calculated. For old bridges or scenarios where accurate CAD data cannot be obtained, the system uses affine transformation (satellite-driven mode) to achieve coordinate mapping. In this mode, the system accurately locates 3 key feature points on the bridge through a Beidou / GPS receiver, and converts the obtained longitude and latitude coordinates into physical coordinates in the UTM plane coordinate system. The system constructs a satellite relative coordinate system based on these feature points, and at the same time combines visual inertial odometry (VIO) technology to achieve continuous positioning and calculates the affine transformation parameter matrix. During VIO continuous positioning, the visual front end extracts the guardrail seam as the VIO waypoint, the descriptor uses the BRIEF algorithm, and the IMU pre-integration ， tight coupling optimization is the Huber robust kernel function to suppress the mis-matching of feature points; is the IMU noise covariance matrix, which is determined through calibration experiments. is the angular velocity measurement value at time t, is the rotation matrix at time t, is the acceleration measurement value at time t, the visual observation value, representing the pixel coordinates of the feature points in the image, is the function for projecting a point in three-dimensional space onto the two-dimensional image plane, is the waypoint coordinate, representing the position of the feature point in three-dimensional space, is the residual of the IMU.

[0051] In a specific embodiment of the present invention, the homography transformation (CAD-driven mode) is mainly used for high-precision close-range mapping of new bridges, and the affine transformation (satellite-driven mode) is mainly used for old bridges or GPS correction scenarios, and the system adaptively selects the conversion method according to the bridge type. The coordinate conversion module provides a spatial mapping reference; the target detection result locates the pedestrian pixel position; the IMU and binocular data respectively compensate for motion blur and provide height information to form a three-dimensional positioning closed loop.

[0052] Understandably, in the solution of the present invention, a dual-mode dynamic coordinate system is innovatively constructed, which solves the positioning drift problem of traditional fixed coordinate mapping under the conditions of bridge structure deformation, temperature change and vibration environment. The system has developed two sets of scenario-customized coordinate conversion mechanisms for different bridge types, and realized millimeter-level positioning accuracy through a sensor-driven dynamic adjustment strategy. For newly built bridges, the system constructs an accurate structure-coordinate mapping relationship based on the CAD model, breaking through the technical limitation that traditional methods ignore the deformation during the service period of the bridge. The system inputs include registered bimodal images, CAD model feature point coordinates and multi-source sensor data, and innovatively integrates vibration parameters and the thermal expansion coefficient α into the calculation process of the homography transformation matrix H to dynamically compensate the coordinate drift caused. The system also implements a feature point optimization strategy, intelligently selects anti-deformation feature points (such as tower anchor points) according to the bridge structure characteristics, and avoids areas vulnerable to load influence (such as bridge deck expansion joints), ensuring the long-term stability of coordinate conversion. For existing bridges, the system has developed an adaptive coordinate system with tight coupling of satellite positioning and visual inertial odometer (VIO), which solves the problem of the failure of traditional single positioning methods in complex bridge environments. The system inputs include satellite positioning data, IMU data ((acceleration , angular velocity )) and visual feature points. A satellite relative coordinate system is established by identifying three key feature points of the guardrail. At the same time, an innovative bridge structure feature enhanced VIO algorithm is developed, using the guardrail as a visual road sign, with the descriptor using the BRIEF algorithm, and the backend is tightly coupled and optimized through IMU pre-integration and visual features to achieve continuous positioning in areas where satellite signals are blocked (such as under the bridge tower). The system automatically switches the main positioning source according to the satellite signal strength S_GNSS. When S_GNSS > 45 dB, satellite data is preferentially used to solve the problem of positioning continuity in complex bridge environments. The system also innovatively implements a dynamic mapping mechanism for dangerous areas, breaking through the limitations of traditional fixed electronic fences. By fusing pedestrian real-time coordinates, bridge structure database (guardrail / lane boundary coordinates) and stress sensor data, the system constructs a safety threshold model of stress-flow coupling. This model can dynamically adjust the safety distance threshold according to the local stress state of the bridge and traffic flow, automatically expand the range of the dangerous area when heavy vehicles pass, and shorten the vehicle lane intrusion determination time from 5 s to 3 s during peak hours (traffic flow Q > 2000 vehicles / h), solving the problem of insufficient adaptability of traditional fixed area division under variable load conditions.

[0053] Furthermore, in step S32, the center coordinates of the pedestrian target box ( ) are mapped to the bridge physical space coordinate system through the coordinate mapping conversion model to obtain the current mapped physical horizontal coordinates of the pedestrian ( , ), .

[0054] Further, in step S33, based on the IMU data, the current predicted physical horizontal position of the pedestrian is obtained by motion prediction according to the previous actual physical horizontal coordinate of the pedestrian based on the IMU data . , where ; , and represent the velocity and acceleration in the x direction respectively, and represent the velocity and acceleration in the z direction respectively, represents the time interval.

[0055] Further, by using the current mapped physical horizontal coordinates of the pedestrian ( , ) and the current predicted physical horizontal position of the pedestrian , a weighted fusion process is adopted to obtain the current actual physical horizontal coordinates of the pedestrian ( , ); where ; , is the detection confidence, , is the velocity sensitivity coefficient, , is the velocity threshold for distinguishing walking and running.

[0056] Further, the binocular parallax combined with the IMU pitch angle is used to compensate for the installation error to obtain the current elevation coordinate of the pedestrian, and the real-time elevation physical coordinate of the pedestrian , ; where is the original height value calculated by the binocular parallax algorithm, is the pitch angle measured by the IMU, is the camera installation height compensation value. Optionally, binocular ranging , tilt compensation , : camera installation height deviation (calibration value); : pitch angle (rad) output by the IMU in real time.

[0057] In a specific embodiment of the present invention, in order to break through the accuracy bottleneck of traditional single-vision positioning in the environments of bridge vibration and light change, a high-precision tracking of the three-dimensional coordinates of pedestrians is achieved through a multi-source fusion positioning mechanism. The system customizes a vision-inertial collaborative optimization algorithm for the scenario of bridge pedestrian monitoring, effectively solving the problem of unstable positioning caused by high-speed movement, light change and bridge vibration. Specifically, in S31, an improved YOLOv5 model is used for pedestrian target detection, and the bounding box regression term in the loss function is optimized for the bridge scenario, so as to reduce the interference of complex bridge structures (such as cables, guardrails) on pedestrian detection. The input of the system is a registered bimodal image, and the output is the center coordinates of the pedestrian target box and the detection confidence. In S32 - S34, vision-IMU adaptive fusion positioning is achieved, innovatively solving the positioning deviation problem caused by visual blur in the high-speed movement state. The system first calculates the horizontal position of the pedestrian ( , ) through the homography matrix H, and at the same time realizes motion prediction based on the IMU acceleration data. The core innovation of the system is to propose a speed-sensitive dynamic weight fusion algorithm, which calculates the visual data weight in real time. When the pedestrian speed exceeds 2 m / s, the system automatically reduces the visual weight to below 0.4 and increases the IMU weight, thus effectively suppressing the positioning jump in the high-speed movement scenario. The system also introduces a bridge structure constraint mechanism, which uses the guardrail position to limit the Z coordinate range and automatically excludes positioning outliers. In S35, binocular disparity and IMU pitch angle compensation technologies are used to achieve vertical positioning, breaking through the technical limitations of traditional vertical measurement that ignores camera installation errors and bridge inclination. The system realizes tilt compensation through the formula. The system also introduces a moving average filtering process (window size N = 5), effectively suppressing the height measurement fluctuation caused by bridge deck vibration, and reducing the height measurement error from ±0.5 m of the traditional method to ±0.1 m. The whole step forms a complete multi-source fusion three-dimensional positioning system, and there is a strict data link between each module: the trajectory data is fed back to the IMU integration parameter optimization module to suppress integration drift; the height data is transmitted to the behavior analysis module for fall risk assessment; the position label is used to trigger the re-calibration of the coordinate system. The system solves the problem of insufficient adaptability of traditional positioning methods in the complex dynamic environment of bridges, providing a stable and accurate spatial positioning basis for subsequent behavior analysis.

[0058] In the present invention, the process of calculating the real-time coordinates of pedestrians based on multi-source data fusion is as follows: First, the system obtains visible light and infrared registered images sharing the same coordinate system through the dual-modal image preprocessing and registration module, and establishes a coordinate mapping and transformation model in combination with the dynamic deformation data of the bridge. The real-time coordinate calculation of pedestrians adopts a combined strategy of "visual measurement - inertial prediction - weighted fusion", effectively overcoming the limitations of a single data source in a complex bridge environment. In the target detection stage, the system identifies pedestrian targets from the registered dual-modal images, and extracts the center coordinates of each pedestrian target box based on a multi-feature matching algorithm of deep learning ( ). This algorithm enhances the detection ability for partially occluded targets through an attention mechanism, and at the same time fuses infrared thermal features to improve the detection accuracy under night and adverse weather conditions. In the coordinate mapping stage, the system maps the center coordinates of the pedestrian target box to the physical space coordinate system of the bridge through the pre-established homography transformation matrix H, obtaining the current mapped physical horizontal coordinates of the pedestrian ( , ). The transformation matrix H is obtained by minimizing the reprojection error between CAD feature points and actual image feature points, and is updated in real time to adapt to the dynamic deformation of the bridge. For the structural micro-deformation caused by vibration and temperature changes, the system introduces a dynamic correction term for real-time compensation. In the motion prediction stage, the system models the motion state of pedestrians based on the data of the inertial measurement unit. The system uses the actual physical horizontal coordinates of the pedestrians obtained at the previous moment ( , Z ) as the reference point, and combines the extracted speed and acceleration parameters to predict the current physical position . To reduce the interference of bridge vibration on acceleration data, the system uses frequency domain analysis to identify and filter out the noise components close to the natural frequency of the bridge. In the data fusion stage, the system dynamically adjusts the weight ratio of the visual measurement result and the inertial prediction result according to the characteristics of the current scene. When the lighting conditions are good and the target speed is low, the visual measurement weight is higher; when the target motion speed increases or the lighting conditions deteriorate, the inertial prediction weight adaptively increases. The weight calculation function design takes into account the non-linear effects of target detection confidence and speed factors to ensure the optimal fusion effect under various conditions. Finally, the system obtains the elevation coordinates of pedestrians through binocular parallax ranging technology combined with the IMU pitch angle data. Since the installation height error of the camera and lens distortion will cause elevation measurement deviation, the system introduces an installation error correction term and pitch angle compensation to achieve high-precision vertical positioning, and finally outputs the complete three-dimensional real-time physical coordinates of pedestrians ( ), providing accurate spatial location information for subsequent behavior recognition and risk assessment. This implementation effectively solves the problem of pedestrian positioning in the bridge environment through a multi-source data fusion strategy, achieving stable coordinate calculation under the presence of interference factors such as vibration, light changes, and structural deformation, and meeting the technical requirements for pedestrian abnormal behavior recognition.

[0059] Further, step S40 includes S41, constructing a position feature vector based on the pedestrian's current three-dimensional actual coordinates and the pedestrian's current area risk label , , where is the pedestrian's current area risk label, and the pedestrian's current area risk label is used to characterize the type of risk area where the pedestrian is currently located. The pedestrian's current area risk label includes a red (first) area label (hazard area label), a yellow (second) area label (critical area label), and a green (third) area label (safe area label). Among them, based on the pedestrian's current three-dimensional actual coordinates ( ), the current stress state of the bridge and the current traffic flow dynamics of the bridge to determine the pedestrian's current area risk label.

[0060] Further, use the formula to dynamically generate the pedestrian's current area risk label, where, represents the distance from the pedestrian to the guardrail, , represents the distance from the pedestrian to the left guardrail, represents the distance from the pedestrian to the right guardrail, is the safety distance, , is the current Mises stress of the bridge, is the yield strength of the steel, is the compressive strength of the concrete, is the detection of incursion into the carriageway; , if the traffic flow Q in the time period ≤ 2000 vehicles / hour, then determine the incursion determination time of the carriageway as 5s, if the traffic flow Q in the time period > 2000 vehicles / hour, then determine the incursion determination time of the carriageway as 3s, is the Z - coordinate of the left carriageway, is the Z - coordinate of the right carriageway.

[0061] Further, to implement risk-driven dynamic grid division, a risk-driven dynamic grid division mechanism is adopted to dynamically adjust the spatial division granularity according to the regional risk level: a high-precision grid of 0.1m×0.1m is used for the red (first) area label, a medium-precision grid of 0.3m×0.3m is used for the yellow (second) area label, and a standard grid of 0.5m×0.5m is used for the green (third) area label, so as to achieve refined monitoring and positioning of high-risk areas.

[0062] Further, step S40 further includes S42, extracting motion feature vectors , pose feature vectors and temperature feature vectors ; Adopt a risk-adaptive dynamic fusion strategy, adjust the multi-modal feature weights in real time according to the risk level, and output a fused feature vector , ; Among them, , , are the position feature weight, pose feature weight and temperature feature weight respectively; Use a time-series sliding window to smooth the fused feature vector to suppress instantaneous noise and obtain a multi-dimensional feature vector ; Among them, is the Gaussian weight, the window size is 5, and the multi-dimensional feature vector is used as the input data for the current dynamic risk score value of the pedestrian.

[0063] Optionally, is the weight coefficient in the time-series sliding window, specifically the Gaussian weight applied to the time dimension. The time-series sliding window refers to selecting a fixed-length time window (5 time points in this application) on a continuous time series. As time goes by, the window slides forward continuously. When processing the data at the current moment, the historical data within the window is considered at the same time. The Gaussian weight means that within this window, different weight coefficients are assigned to the data at different time points. These weight coefficients follow the characteristics of the Gaussian distribution (normal distribution): the data at the current moment (t) obtains the highest weight; the weight of the data at the previous moment (t-1) is the second; the weights of the data at earlier moments (t-2, t-3, t-4) gradually decrease. This weight configuration makes the feature data at the current moment receive the highest attention, and the influence of historical data weakens gradually according to the time distance, thus realizing the smoothing of time-series data while retaining the main feature information of the current state.

[0064] Further, extract the motion feature vector , the pose feature vector and the temperature feature vector . Extracting the motion feature vector specifically includes: by fusing IMU data and historical coordinate sequences, quantifying the pedestrian's motion state, and constructing a motion feature vector, , where is the current speed of the pedestrian, and the calculation formula is: , is the acceleration processed by a 5Hz low-pass filter, is the trajectory curvature, , is used to identify abnormal horizontal behaviors such as staying and running. , is the vertical speed, , is the vertical acceleration, , is the change amount relative to the previous moment, , is used to identify high-risk vertical abnormal behaviors such as climbing and crossing. Extracting the pose feature vector specifically includes: based on the lightweight HRNet network, extracting human key points from visible and infrared images, constructing a skeleton model, and performing dual-modal complementary fusion; wherein, , is the key point fusion coordinate, , is the visible light confidence, is the overall confidence of pose recognition; Extracting the temperature feature vector specifically includes: by analyzing the infrared temperature distribution, detecting the pedestrian's abnormal high-temperature area and dynamically adjusting the threshold in combination with the material characteristics to construct the temperature feature vector , , where, is the preset signal for temperature anomaly alarm, ; is the material adaptive threshold, , , is the average environmental temperature, is the temperature distribution entropy used to quantify the temperature distribution uniformity, is the current temperature value. The motion feature vector provides the input data for the regional risk coefficient and the dwell time dimension correction coefficient , the multi-dimensional feature vector​ For calculation , For calculation , For calculation .

[0065] Furthermore, based on the multi-dimensional feature vector , the current actual physical horizontal coordinate of the pedestrian ( ), the horizontal motion feature vector and the vertical motion feature vector , determine the behavior type label, and determine the current posture feature score of the pedestrian according to the behavior type label ; if the current posture of the pedestrian is a climbing behavior posture (vertical speed continues for more than 2 seconds, and the elevation change ), then determine the current posture feature score of the pedestrian to be 0.7; if the current posture of the pedestrian is a striding behavior posture (vertical speed is positive first and then negative, the vertical acceleration conforms to the striding feature curve, and the elevation change shows an "ascending - plateau - descending" pattern), then determine the current posture feature score of the pedestrian to be 0.6 points; if the current posture of the pedestrian is a lingering behavior posture (horizontal speed continues for more than 30 seconds, and the vertical speed ), then determine the current posture feature score of the pedestrian to be 0.5 points; if the current posture of the pedestrian is a rapid running behavior posture horizontal speed , and the horizontal acceleration is significant), then determine the current posture feature score of the pedestrian to be 0.4 points; if the current posture of the pedestrian is a normal walking posture (horizontal speed , the trajectory curvature is stable, and the vertical direction parameters , ), then determine the current posture feature score of the pedestrian to be 0.2 points.

[0066] It can be understood that the current actual physical horizontal coordinate of the pedestrian ( ): provides the accurate position information of the pedestrian in the physical space coordinate system of the bridge, where the change rate and cumulative change amount are the key parameters for judging abnormal behaviors in the vertical direction, and the time series changes of : This vector contains the current speed of the pedestrian , the acceleration processed by a 5 Hz low-pass filter and the trajectory curvature and other key kinematic parameters. It is used to determine whether the pedestrian's motion state exceeds a preset threshold , reflects the motion acceleration characteristics and can effectively identify the spatial characteristics of non-linear motion patterns. The multi-dimensional feature vector : After being smoothed by a time-series sliding window, this vector effectively integrates the pose feature vector , the temperature feature vector and the position feature vector information. Among them, the pose feature provides the key point position relationship through the skeleton model, the temperature feature provides the abnormal hot spot distribution information, and the position feature is associated with the regional risk level, jointly constituting the multi-modal representation of the behavior feature.

[0067] Further, the formula is used to calculate the time dimension correction coefficient , where is the action duration determined based on the pedestrian motion feature vector .

[0068] Further, based on the multi-object tracking module, the number of interacting pedestrians is determined to quantify the amplification effect of the group effect on the risk; if it is a single-person state, then is determined; if it is a two-person state, then is determined; if it is a state with more than two people, then is determined.

[0069] Further, the regional risk coefficient is obtained according to the regional label mapping; if it is a red regional label, then the regional risk coefficient ; if it is a yellow (second) regional label, then the regional risk coefficient ; if it is a green (third) regional label, then the regional risk coefficient Further, if and the coordinate moves towards the guardrail, then the motion trend coefficient is determined; if and the coordinate moves away from the guardrail, then the motion trend coefficient is determined; if | | ≤ , then the motion trend coefficient is determined.

[0070] Further, the thermodynamic abnormal state is determined based on the temperature feature vector ​ , wherein, is a preset signal for abnormal temperature alarm.

[0071] In a specific embodiment, when the trio climbs in the red (first) area for 90 seconds, then , , , , , , , corresponding to a high-risk warning; in another embodiment, when a single person climbs in the red (first) area for 120 seconds, then , , , , 0, , , corresponding to a medium-risk warning; in yet another embodiment, when a single person runs rapidly in the yellow (second) area for 60 seconds, then , , , , , , , corresponding to a low-risk warning; the system updates the risk score every 1 second to achieve real-time assessment of the risk of abnormal behaviors. At the same time, according to the actual warning effect, the basic score and various coefficients are dynamically adjusted to ensure that the warning level of the system is consistent with the actual risk level and improve the accuracy of risk assessment. Through this multi-dimensional risk assessment mechanism, both misjudgment of normal behaviors is avoided and real high-risk behaviors can be discovered and warned in a timely manner.

[0072] In specific implementation, the solution of the present invention innovatively constructs a multi-dimensional feature collaborative analysis framework, and through a deep spatio-temporal model and a risk adaptive fusion strategy, solves the problems of insufficient stability and accuracy of traditional behavior recognition methods in complex bridge environments. S41, realizes risk-driven dynamic grid division and location feature construction, breaking through the technical limitation that traditional fixed-resolution grids cannot adapt to the requirements of different risk areas. The system adaptively adjusts the feature extraction granularity according to the three-dimensional coordinates of pedestrians and area labels: the red (first) area adopts a high-precision grid of 0.1m × 0.1m, the yellow (second) area adopts a medium-precision grid of 0.3m × 0.3m, and the green area adopts a standard grid of 0.5m × 0.5m. This multi-scale feature extraction strategy not only ensures the positioning accuracy of high-risk areas but also significantly reduces the overall computational load. The system outputs a location feature vector , wherein It includes risk markers based on the stress state, which directly drive the weight allocation of the subsequent fusion module. S42, a multi-dimensional feature vector system is constructed, and the system quantifies behavioral features from multiple aspects such as the pedestrian's historical trajectory, acceleration, and posture. Motion feature vector Captures the motion pattern by extracting the pedestrian's speed, filtered acceleration (using a 5Hz low-pass filter to suppress bridge vibration interference), and trajectory curvature; Posture feature vector Extracts key points from the bimodal image through lightweight HRNet and performs weighted fusion to achieve light adaptability; Temperature feature vector Provides an auxiliary judgment basis by analyzing the infrared temperature distribution. The core innovation of the system lies in realizing risk-adaptive dynamic feature fusion , where the fusion weights are adjusted in real time according to the risk level. The position and posture features are prioritized in high-risk areas, the computational load is reduced in low-risk areas, and transient noise is suppressed by introducing a time-series sliding window filter, significantly improving the detection stability. The system deploys a spatio-temporal attention network (BTSA-Net) optimized for bridge scenarios, innovatively fusing bridge structure constraints and pedestrian behavior analysis. This network uses a spatio-temporal separable convolution structure to extract short-term motion features (window length 1.5s), and strengthens the feature weights of key areas such as guardrails and roadway boundaries through a spatial attention mechanism; Innovatively introduces a trajectory morphology encoder to input the historical trajectory sequence into the LSTM network to generate a trajectory morphology encoding , which is used to identify abnormal patterns such as climbing and crossing; Developed a thermodynamics-behavior correlation matrix to establish a mapping relationship between infrared temperature features and behavioral features, improving the behavioral recognition accuracy in low-light scenarios. The system output includes behavioral classification results and risk scoring parameters. Risk scoring model Realizes a dynamic scoring mechanism with multi-factor coupling, where is the basic risk score value, is the regional risk coefficient, is the motion trend coefficient, is the time dimension correction coefficient, is the number of interacting people coefficient, is the thermodynamics risk factor. This model breaks through the limitations of traditional single-dimensional evaluation and realizes the precise quantification of high-risk behaviors such as climbing ( =0.7), crossing ( =0.6), and long-term stay ( =0.5). The system innovatively introduces a scene adaptability optimization mechanism. When the false alarm rate > 20% in 10 consecutive warnings, it automatically triggers the gradient descent optimization of the feature fusion weights α', β', γ' to achieve the continuous evolution of the system.

[0073] Understandably, traditional methods do not consider the interference of bridge vibration on IMU data, resulting in the accumulation of acceleration errors. The present invention quantifies the pedestrian motion state (speed, acceleration, trajectory curvature) by fusing IMU data with historical coordinate sequences, providing data support for identifying abnormal behaviors (such as falling, running). When extracting motion features, the input data includes historical coordinate sequences (time window of 1.5 seconds) and the raw IMU acceleration ; vibration noise suppression is adopted, and the acceleration error is greatly reduced by 5Hz low-pass filtering; real-time curvature calculation, and subtle trajectory changes are captured through differential operations and other means.

[0074] Among them, the speed calculation formula is , is the sampling period, synchronized with the coordinate positioning module; is the instantaneous speed, with the unit of m / s. The acceleration filtering formula is: is the cut-off frequency, filtering out bridge vibration noise (typical vibration frequency 0.5 - 5Hz).

[0075] The trajectory curvature formula is , is the trajectory curvature, used to identify abnormal behaviors such as climbing and crossing.

[0076] The motion feature vector is output to the behavior analysis module for motion anomaly detection (such as climbing, crossing). Based on the lightweight HRNet network, human key points are extracted from visible light and infrared images to construct a skeleton model. In low-light scenarios, infrared data supplements the key point information, avoiding pose missing detection caused by insufficient light in traditional methods. The key point fusion formula is: is the key point fusion coordinate, is the human key point coordinate extracted from the visible light image, , visible light confidence (normalized light intensity); is the key point coordinate output by the infrared branch.

[0077] The pose feature vector is output to the visualization system for real-time pose rendering and out-of-bounds warning.

[0078] Further, in step S60: S61, compare the current dynamic risk score value R(t) of the pedestrian with a preset risk threshold. If R(t) > 1.5, it is determined as a high-risk level; if 1.0 ≤ R(t) ≤ 1.5, it is determined as a medium-risk level; if R(t) < 1.0, it is determined as a low-risk level. S62, display the determined risk level on the monitoring interface in real time and update it to the risk analysis database. S63, establish a closed-loop self-optimization mechanism to achieve continuous evolution of the system; perform dynamic calibration of parameters. When the false alarm rate in 10 consecutive warnings > 20%, automatically trigger the gradient descent optimization of the feature fusion weights α', β', γ'; implement incremental learning of the model, perform incremental learning on the spatio-temporal attention network and the risk event trigger model, and update the parameters of the fully connected layer.

[0079] Further, establish a closed-loop self-optimization mechanism to achieve continuous evolution of the system; perform dynamic calibration of parameters. When the false alarm rate in 10 consecutive warnings > 20%, automatically trigger the gradient descent optimization of the feature fusion weights α', β', γ'; implement incremental learning of the model, inject new behavior patterns (such as climbing, crossing) into the spatio-temporal attention network through the online learning interface, perform incremental learning on the high-risk event trigger model, and update the parameters of the fully connected layer; the regional dynamic calibration mechanism synchronizes the regional division results to the calculation of the risk scoring module in real time to ensure that the regional risk coefficient is consistent with the actual state of the bridge.

[0080] Further, in step S70, implement multi-level linkage protection responses based on the risk level, specifically including: S71, establish a three-level linkage warning mechanism to achieve a gradient warning response from the site to the remote end; S72, execute differential protection measures according to different risk levels. For low-risk levels, perform low-risk responses; for medium-risk levels, perform medium-risk responses; for high-risk levels, perform high-risk responses. Among them, during low-risk responses, trigger on-site audible and visual warnings; during medium-risk responses, trigger on-site audible and visual warnings and activate the remote monitoring center warning; during high-risk responses, activate the emergency linkage mechanism.

[0081] Understandably, the early warning control unit adopts a three - level linkage early warning mechanism, including a on - site early warning module, a remote early warning module, and an emergency linkage module. Among them, the on - site early warning module realizes sound and light warning by integrating intelligent lighting components, and the lighting brightness and warning volume can be adaptively adjusted according to the environment; the remote early warning module transmits early warning information, target images, and behavior characteristics to the monitoring center in real - time through the industrial Ethernet, and adopts a lightweight data compression protocol to greatly reduce bandwidth occupancy; the emergency linkage module automatically establishes a communication connection with relevant emergency rescue departments based on the preset emergency response procedures. The early warning control unit realizes hierarchical response according to the risk level: when the risk level is low, it triggers on - site sound and light early warning to remind the surrounding personnel to pay attention; when the risk level is medium, it simultaneously starts early warning in the remote monitoring center, and the duty personnel conduct real - time monitoring and judgment; when the risk level reaches high, it automatically activates the emergency linkage mechanism to ensure the timely intervention of emergency rescue forces. The system has the functions of automatically recording and statistically analyzing early warning information, can generate early warning event reports, and the emergency response records are fed back to the risk scoring model (sub - claim seven) to optimize the weight of the basic score value, providing data support for subsequent safety management.

[0082] Furthermore, the functional lighting unit adopts an adjustable LED light source. In the normal working mode, the light source brightness can be automatically adjusted within the range of 50 - 300 cd / m², the color rendering index Ra≥80, and the color temperature can be adjusted within the range of 3000K - 6500K to realize the intelligent matching of lighting requirements and ambient light; when the system triggers a safety early warning, the functional lighting unit automatically switches to the early warning mode. Among them, in the low - risk state, the light source flashes at a frequency of 0.5 Hz and the brightness increases to 500 cd / m², in the medium - risk state, it flashes at a frequency of 1 Hz and the brightness increases to 700 cd / m², and in the high - risk state, it flashes at a frequency of 2 Hz and the brightness increases to 1000 cd / m², thus forming a significant visual warning effect.

[0083] Furthermore, the landscape lighting unit adopts an RGB matrix design and is built - in with a variety of lighting scene modes. In the normal working state, the system automatically switches the corresponding lighting effects according to weather conditions, time periods, and special festivals to enhance the bridge landscape value; when a safety early warning is triggered, the landscape lighting automatically switches to the red warning mode and flashes synchronously with the functional lighting to strengthen the early warning effect. During the lighting mode switching process, the system adopts a gradual transition mechanism to avoid light pollution caused by sudden changes, and at the same time reserves a remote control interface to support managers to adjust parameters according to actual needs.

[0084] Understandably, this system constructs a gradient early warning response mechanism from the site to the remote end, innovatively realizes the precise mapping of risk levels and early warning intensities, and solves the technical bottlenecks of single response and poor timeliness of traditional protection systems. The early warning control unit adopts a three-level linkage early warning mechanism and realizes differential responses according to the risk score R(t): when R(t) < 1.0, on-site audible and visual early warnings are triggered; when 1.0 ≤ R(t) ≤ 1.5, the early warning of the remote monitoring center is started simultaneously; when R(t) > 1.5, the emergency linkage mechanism is automatically activated. The system innovatively develops a lightweight data compression protocol, and early warning information, target images, and behavioral characteristics are transmitted to the monitoring center in real time through the industrial Ethernet, with significantly reduced bandwidth occupancy compared with traditional methods. The emergency linkage module automatically establishes communication connections with relevant emergency rescue departments based on the preset emergency response procedures, and the response time is significantly reduced compared with traditional manual notifications. The intelligent lighting system realizes the deep integration of early warning and lighting. The functional lighting unit dynamically adjusts the flashing frequency and brightness according to the risk level in the early warning mode: in the low-risk state, it flashes at a frequency of 0.5 Hz and the brightness is increased to 500 cd / m²; in the medium-risk state, it flashes at a frequency of 1 Hz and the brightness is increased to 700 cd / m²; in the high-risk state, it flashes at a frequency of 2 Hz and the brightness is increased to 1000 cd / m². The landscape lighting unit adopts an RGB matrix design and automatically switches to the red warning mode and flashes synchronously with the functional lighting during early warning, forming a three-dimensional safety early warning network. During the lighting mode switching process of the system, a gradual transition mechanism is adopted to avoid light pollution, and a remote control interface is reserved to support managers to adjust parameters according to actual needs. The system has the functions of automatically recording and statistically analyzing early warning information. The emergency response records are fed back to the risk scoring model to optimize the weight of the R0 basic score value, forming a complete closed-loop of early warning-response-evaluation-optimization. This technical solution of the deep integration of intelligent early warning and lighting not only significantly improves the ability to identify and respond to abnormal behaviors of bridges, but also provides new ideas for the future development of bridge protection systems. Through the above systematic improvements and detailed descriptions, the present invention fully demonstrates the technical innovation and implementation path of high-precision pedestrian behavior recognition and life safety protection in complex bridge environments, providing comprehensive technical guarantees for bridge life safety protection.

[0085] Compared with the prior art, the technical solution adopted by the present invention has at least the following beneficial effects: An intelligent monitoring system using dual-modal fusion realizes all-weather real-time identification and early warning of dangerous behaviors such as climbing and abnormal staying through the collaborative work of a visible light camera and an infrared sensor. It achieves a technological leap from "passive protection" to "active protection + intelligent emergency", significantly improving the protection effect. A complete multi-level linkage early warning mechanism is established, and through three-level responses of on-site early warning, remote early warning, and emergency linkage, it realizes the full-process intelligent management from hidden danger discovery to emergency disposal, improving the overall protection efficiency of the system. Early warning information and behavior data are uploaded in real time, supporting the generation of event reports to assist subsequent safety management decisions. An intelligent lighting system is innovatively integrated. Through the organic combination of functional lighting and landscape lighting, it not only meets the lighting requirements of the bridge deck but also can automatically adjust lighting parameters and modes according to different time periods, weather conditions, and festivals, creating a beautiful night landscape effect. At the same time, it has a hierarchical early warning function and can achieve safety early warning prompts through light effect changes. Through parameter dynamic calibration and model incremental learning (online updating of the behavior pattern library), the closed-loop freedom and dynamic adaptability of the intelligent monitoring system are realized. The system can continuously optimize the algorithm according to the actual use environment, improve the early warning accuracy, and also support the rapid adaptation of new abnormal behavior patterns (such as new climbing actions).

[0086] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An abnormal behavior recognition and warning method for pedestrians on a bridge based on intelligent monitoring, characterized in that, It includes the following steps: S10. Register the current visible light image and the current infrared image to obtain a registered dual-modal image based on the current image coordinate system; S20. Considering the current bridge deformation, establish a current coordinate mapping conversion model for mapping the pixel coordinate points in the current image coordinate system to the physical coordinate points in the current bridge physical space coordinate system; S30. Obtain the current three-dimensional actual coordinates of the pedestrian in the physical space coordinate system of the bridge ( ), which specifically includes the steps: S31. Extract the pedestrian targets of each pedestrian from the registered dual-modal image and determine the center coordinates of the pedestrian target box; S32. Map the center coordinates of the pedestrian target box to the bridge physical space coordinate system through the current coordinate mapping conversion model to obtain the current mapped physical horizontal coordinate of the pedestrian; S33. Based on the previous actual physical horizontal coordinate of the pedestrian, obtain the current predicted physical horizontal coordinate of the pedestrian through motion prediction; S34. The current actual physical horizontal coordinate of the pedestrian is obtained by weighted fusion based on the current mapped physical horizontal coordinate of the pedestrian and the current predicted physical horizontal coordinate of the pedestrian ( , ); In S35, the binocular parallax image is combined with the IMU pitch angle to compensate for the installation error to obtain the current elevation coordinate of the pedestrian, and the real-time elevation physical coordinate of the pedestrian is obtained. ; S40. Identify the abnormal behaviors of pedestrians on the bridge based on the current three-dimensional actual coordinates of the pedestrians; S50. Calculate and obtain the current dynamic risk score value of the pedestrian based on the multi-factor fusion model; S60. Determine the current risk level of the pedestrian according to the current dynamic risk score value of the pedestrian and the preset risk threshold; S70. Implement a multi-level linkage protection response based on the current risk level of the pedestrian.

2. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 1, wherein Step S10 specifically includes: S11. Process the current visible light image by using vibration-wind load dynamic Gaussian filtering and edge protection weight method to obtain the current visible light filtered compensation image; S12. Use the material adaptive calibration model to dynamically compensate for environmental interference based on the respective thermodynamic parameters of steel and concrete, and calibrate the current infrared image to obtain the infrared calibration image; S13. Based on the elastic registration model, adopt the deformation feedback elastic registration strategy, combine the real-time deformation data of the bridge to optimize the registration matrix for fusion, and obtain the registered bimodal image. Among them, the registered bimodal image and the registered bimodal image share the same image coordinate system and retain their respective feature information.

3. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 2, wherein In step S11, the formula is used to calculate the dynamic Gaussian filter kernel parameter at time t, where and are the current vibration amplitude and the current vibration frequency at time t obtained by the vibration sensor respectively, is the current wind speed at time t obtained by the environmental sensor, is the vibration matching coefficient, is the wind load correction coefficient, and the value range of k is , the value range of .

4. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 1, wherein In step S12, the function expression of the material adaptive calibration model is , where represents the calibrated infrared temperature value, are the model coefficients related to the current temperature, current wind speed, and current humidity respectively, and are the current wind speed and current humidity obtained through the environmental sensor respectively, is the environmental temperature difference between the current temperature and the calibration reference temperature, is the original infrared temperature value.

5. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 1, characterized in that, In step S13, the function expression of the elastic registration model is: Among them, is the elastic deformation risk assessment coefficient, and its value range is 0 - 1, is the number of reference points, is the confidence weight of feature point i, satisfying , is the current state coordinate, is the reference state coordinate, is the square of the spatial distance between the current position and the reference position of the monitoring point, is the deformation sensitivity parameter.

6. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 1, wherein, In step S20, at least eight CAD physical feature points are extracted from the bridge design CAD model ( ) as matching reference points for coordinate mapping with the registered bimodal image. At the same time, the dynamic Gaussian filter kernel parameters and the coefficient of thermal expansion are incorporated into the coordinate transformation process as dynamic deformation compensation factors for calculation, and the homography transformation matrix H for realizing the coordinate mapping between the image coordinate system and the bridge physical space coordinate system is obtained.

7. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 6, characterized in that, The function expression of the homography transformation matrix H is: Among them, + , is the real-time deformation offset, is the deformation compensation weight, and are the horizontal scaling and rotation parameters of the image respectively, and the concrete bridge needs to be adjusted according to the bridge deck curvature. and are the perspective transformation coefficients. and are the translation parameters respectively. is the instantaneous value of the acceleration component of the bridge in the X direction at time s. is the instantaneous value of the dynamic Gaussian filter kernel parameter during the integration process. is the instantaneous value of the current vibration frequency during the integration process. is the differential of the integration variable. is the bridge span, ([[]] ) is the center coordinate of the pedestrian target box.

8. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 1, characterized in that, Step S40 includes: S41. Construct a position feature vector based on the current three-dimensional actual coordinates of the pedestrian and the current area risk label of the pedestrian, , where is the current area risk label of the pedestrian, and the current area risk label of the pedestrian is used to characterize the type of risk area where the pedestrian is currently located. The current area risk label of the pedestrian includes a first area label, a second area label, and a third area label. Among them, based on the current three-dimensional actual coordinates of the pedestrian ( ), the current stress state of the bridge and the current traffic flow of the bridge are dynamically determined to obtain the current area risk label of the pedestrian.

9. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 8, wherein, Use the formula Dynamically generate a risk label for the current area of the pedestrian, where, represents the distance from the pedestrian to the guardrail, , represents the distance from the pedestrian to the left guardrail, represents the distance from the pedestrian to the right guardrail, is the safety distance, , is the current Mises stress of the bridge, is the yield strength of the steel, is the compressive strength of the concrete, is the detection of roadway intrusion; , if the traffic flow Q in a time period ≤ 2000 vehicles / hour, then determine the vehicle lane intrusion determination time to be 5 s. If the traffic flow Q in a time period > 2000 vehicles / hour, then determine the vehicle lane intrusion determination time to be 3 s, is the Z - coordinate of the left vehicle lane, is the Z - coordinate of the right vehicle lane.

10. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 8, characterized in that, Step S40 further includes S42, obtaining a fused feature vector according to a motion feature vector , an attitude feature vector and a temperature feature vector , where , , are the position feature weight, the attitude feature weight and the temperature feature weight respectively; Adopt a time-sequential sliding window for the fused feature vector to perform smoothing processing to obtain a multi-dimensional feature vector ; Among them, is the Gaussian weight, the window size is 5, and the multi-dimensional feature vector is used as the input data for the current dynamic risk score value of the pedestrian.

11. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 10, characterized in that, The pedestrian motion feature vector , is used to characterize the horizontal plane behavior dynamics, , is the current speed of the pedestrian in the horizontal direction, is the horizontal acceleration after low-pass filtering, is the curvature of the horizontal plane trajectory; is used to characterize the elevation plane behavior dynamics, , is the current speed of the pedestrian in the vertical direction, is the vertical acceleration, is the height change relative to the previous moment.

12. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 10, wherein, In step S50, the formula Calculate and obtain the current dynamic risk score of the pedestrian, where: is the pedestrian’s current posture feature score, is the correction coefficient of pedestrian dwell time dimension, is the group effect correction coefficient, is the regional risk coefficient corresponding to the current physical area label of the pedestrian, is the pedestrian motion trend coefficient, is the thermodynamic risk factor; Among them, the current posture feature score of the pedestrian is obtained based on the behavior type label, and the behavior type label is composed of a multi-dimensional feature vector , the current actual physical horizontal coordinate of the pedestrian ( ), and the pedestrian motion feature vector is obtained; Pedestrian stay time dimension correction coefficient Based on the pedestrian motion feature vector Obtain; Group effect correction coefficient Obtained based on the data of the number of interacting pedestrians; Regional risk coefficient Obtained based on the current physical area label of the pedestrian; Pedestrian movement trend coefficient Based on the pedestrian movement direction and pose feature vector and the pedestrian motion feature vector Obtained; Thermodynamic risk factor Based on the pedestrian temperature feature vector Obtained.

13. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 12, characterized in that, The described behavior type tags include climbing behavior postures, crossing behavior postures, staying behavior postures, rapid running behavior postures, and normal walking postures; if the current posture of the pedestrian is a climbing behavior posture, the current posture feature score of the pedestrian is determined to be 0.7; if the current posture of the pedestrian is a crossing behavior posture, the current posture feature score of the pedestrian is determined to be 0.6; if the current posture of the pedestrian is a staying behavior posture, the current posture feature score of the pedestrian is determined to be 0.5; if the current posture of the pedestrian is a rapid running behavior posture, the current posture feature score of the pedestrian is determined to be 0.4, and the current horizontal speed of the pedestrian in the rapid running behavior posture ; If the current posture of the pedestrian is a normal walking posture, determine the current posture feature score of the pedestrian is 0.2 points, and the current horizontal speed of the pedestrian in the normal walking posture is 0 ; and / or use the formula: Calculate the time dimension correction coefficient , where is the action duration determined based on the pedestrian motion feature vector ; and / or determine the number of interacting pedestrians based on the multi-object tracking module, and quantify the amplification effect of the group effect on the risk; if in the single-person state, determine ; if in the two-person state, determine ; if in the state of more than two people, determine ; and / or obtain the regional risk coefficient according to the regional label mapping; if it is the first regional label, determine the regional risk coefficient ; if it is the second regional label, determine the regional risk coefficient ; if it is the third regional label, determine the regional risk coefficient And / or if and the coordinates move towards the guardrail, then determine the movement trend coefficient ; If and the coordinates move away from the guardrail, then determine the movement trend coefficient ; If | | ≤ When, then determine the motion trend coefficient ; and / or based on the temperature eigenvector Determine the thermodynamic abnormal state: Among them, is the preset signal for temperature anomaly alarm.

14. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 12, wherein In step S60, in S61, the current dynamic risk score value of the pedestrian is compared with a preset risk threshold, and the preset risk threshold includes a first risk critical point value of 1.0 and a second risk critical point value of 1.5; if > 1.5, it is determined as a high-risk level; If 1.0 ≤ ≤ 1.5, it is determined as a medium risk level; if < 1.0, it is determined as a low risk level; S62. Real-time display the determined risk level on the monitoring interface and update it to the risk analysis database.

15. The method for identifying and warning abnormal behaviors of pedestrians on a bridge based on intelligent monitoring according to claim 13, wherein In step S70, implement a multi-level linkage protection response based on the risk level, specifically including: S71. Establish a three-level linkage early warning mechanism to achieve a gradient early warning response from the site to the remote: S72. Implement differential protection measures according to different risk levels. If it is a low risk level, perform a low risk response; if it is a medium risk level, perform a medium risk response; if it is a high risk level, perform a high risk response; Among them, when the low risk response is triggered, the on-site sound and light warning is triggered. When the medium risk response is triggered, the on-site sound and light warning is triggered and the remote monitoring center warning is started. When the high risk response is triggered, the emergency linkage mechanism is activated.

Images