Method and system for identifying abnormal behaviors of pedestrians on bridge

Abstract

The invention discloses a method and system for identifying abnormal behaviors of pedestrians on a bridge, and relates to the technical field of bridge engineering, and the method comprises the steps: obtaining the center coordinates of a pedestrian target frame of each pedestrian under a bridge physical space coordinate system based on image recognition and coordinate transformation; pedestrian static feature data and pedestrian dynamic feature data are obtained based on the pedestrian target frame center coordinates and inertial measurement data, the pedestrian static feature data comprise pedestrian position feature vectors and the number of interactive pedestrians, and the pedestrian dynamic feature data comprise pedestrian motion feature vectors; and performing pedestrian abnormal behavior identification based on the pedestrian static feature data and the pedestrian dynamic feature data. According to the method provided by the invention, pedestrian abnormal behavior identification is automatically carried out based on the pedestrian static feature data and the pedestrian dynamic feature data, and real-time data support is provided for pedestrian bridge safety early warning.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge engineering, and in particular to a method and system for identifying abnormal behavior of pedestrians on a bridge. Background Art

[0002] As an important transportation infrastructure, the safety of bridges is closely related to public safety and social stability. Although modern bridges are usually equipped with passive protection facilities such as pedestrian walkways and guardrails on both sides, such facilities can only provide basic protection functions and cannot effectively deal with intentional climbing and overtaking, resulting in safety incidents such as jumping from bridges. Existing bridge safety monitoring mainly relies on on-site observation and identification or real-time monitoring to identify pedestrian behavior on bridges, which makes it difficult to automatically identify abnormal behavior of pedestrians on bridges in a timely manner. In view of this, it is necessary to provide a method for identifying abnormal behavior of pedestrians on bridges to achieve real-time action recognition of pedestrians on bridges and provide real-time data support for pedestrian bridge safety warnings. Summary of the Invention

[0003] The main purpose of the present invention is to provide a method and system for identifying abnormal behavior of pedestrians on bridges, aiming to solve the technical problem in the existing technology that existing bridge safety monitoring mainly relies on on-site observation and identification or real-time monitoring and identification to identify the behavior of pedestrians on bridges, and it is difficult to automatically identify the abnormal behavior of pedestrians on bridges in a timely manner.

[0004] To achieve the above object, the present invention provides a method for identifying abnormal behavior of pedestrians on a bridge, comprising the following steps:

[0005] S10, based on image recognition and coordinate conversion, obtain the pedestrian target frame center coordinates (X(t), Y(t), Z(t)) of each pedestrian in the physical space coordinate system of the bridge;

[0006] S20, based on the center coordinates of the pedestrian target frame and the inertial measurement data, obtain the pedestrian static feature data and the pedestrian dynamic feature data, wherein the pedestrian static feature data includes the pedestrian position feature vector F pos and the number of interactive pedestrians. Pedestrian dynamic feature data includes pedestrian motion feature vector F motion ;

[0007] Among them, the pedestrian position feature vector F pos =[X(t), Z(t), Lable(t)], where (X(t), Z(t)) represents the horizontal coordinates of the pedestrian target box center in the bridge physical space coordinate system, and Lable(t) is the pedestrian's current area risk label, which is used to characterize the type of risk area the pedestrian is currently in. The pedestrian's current area risk label includes the first area label, the second area label, and the third area label.

[0008] Among them, the pedestrian motion feature vector Fmotion =[F x , F y ],F x Used to characterize the dynamics of horizontal surface behavior, F x =[v x (t), a xfiltered (t), C(t)], v x (t) is the current horizontal speed of the pedestrian, a xfiltered (t) is the horizontal acceleration after low-pass filtering, C(t) is the horizontal trajectory curvature; F y Used to characterize the dynamic behavior of elevation plane, F y =[v y (t), a y (t), ΔY(t)], v y (t) is the current vertical speed of the pedestrian, a y (t) is the vertical acceleration, ΔY(t) is the height change relative to the previous moment;

[0009] S30: Identify abnormal pedestrian behavior based on the pedestrian static feature data and the pedestrian dynamic feature data.

[0010] Further, S31, a current dynamic risk score of the pedestrian is calculated based on the pedestrian's static feature data and the pedestrian's dynamic feature data using a multi-factor fusion model;

[0011] S32: Determine the current risk level of the pedestrian based on the pedestrian's current dynamic risk score and a preset risk threshold.

[0012] Further, S11, registering the current visible light image and the current infrared image to obtain a registered bimodal image based on the current image coordinate system;

[0013] S12, based on the registered bimodal image and the current coordinate mapping transformation model, the center coordinates of the pedestrian target box of each pedestrian in the physical space coordinate system of the bridge are obtained.

[0014] Furthermore, the pedestrian static feature data also includes the pedestrian posture feature vector F pose ,

[0015] F pose =[Keypoint fused , Conf pose ], Keypoint fused is the key point fusion coordinate,

[0016] Keypoint fused =Keypoint vis ·Conf vis +Keypoint ir·(1-Conf vis ),

[0017] Keypoint vis Keypoint is the coordinate of the pedestrian key point mapped by the infrared image, ir Pedestrian key point coordinates mapped to visible light images, Conf vis is the visible light confidence, Conf vis =L amb (t) / 255,Conf pose The overall confidence of posture recognition.

[0018] Furthermore, the pedestrian static feature data also includes the pedestrian temperature feature vector F temp ,

[0019] F temp =[Alert temp , H temp ], among which, Alert temp It is an abnormal temperature alarm signal.

[0020]

[0021] ΔT th is the material adaptive threshold, μ env (t) = mean(T amb (t)), μ env (t) is the average ambient temperature, H temp is the temperature distribution entropy used to quantify the uniformity of temperature distribution, and T(x, y, t) is the current temperature value.

[0022] Furthermore, the formula R(t)=R0×K is used. t ×K n ×K p ×K v ×K thermal Calculate the current dynamic risk score of the pedestrian, where R0 is the pedestrian's static feature score, K t is the pedestrian stay time dimension correction coefficient, K n is the group effect correction coefficient, K p is the pedestrian area risk coefficient, K v is the pedestrian motion trend coefficient, K thermal is the thermodynamic risk factor;

[0023] Among them, the pedestrian static feature score R0 is obtained based on the behavior type label, and the behavior type label is composed of the multidimensional feature vector F final (t), the pedestrian’s current actual physical horizontal coordinates (X(t), Y(t), Z(t)), and the pedestrian’s motion feature vector Fmotion Get, multidimensional feature vector F final(t) By fusion feature vector F fused(t) Obtained by time series sliding window smoothing, the fusion feature vector F fused(t) By the pedestrian position feature vector F pos , posture feature vector F pose And the pedestrian temperature feature vector F temp Fusion acquisition;

[0024] Pedestrian dwell time dimension correction coefficient K t Based on the pedestrian motion feature vector F motion Get;

[0025] Group effect correction coefficient K n Based on the data acquisition of the number of interactive pedestrians;

[0026] Regional risk factor K p Based on the current physical area label of the pedestrian;

[0027] Pedestrian movement trend coefficient K v Based on the pedestrian's moving direction and posture feature vector F pose , and pedestrian motion feature vector F motion Get;

[0028] Thermodynamic risk factor K thermal Based on the pedestrian temperature feature vector F temp Get.

[0029] Furthermore, the behavior type labels include climbing behavior posture, crossing behavior posture, staying behavior posture, fast running behavior posture and normal walking posture; if the pedestrian's current posture is a climbing behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.7; if the pedestrian's current posture is a crossing behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.6; if the pedestrian's current posture is a staying behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.5; if the pedestrian's current posture is a fast running behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.4, and the pedestrian's current horizontal speed v in the fast running behavior posture is 0. t >2m / s; if the pedestrian's current posture is a normal walking posture, the pedestrian's current posture feature score R0 is determined to be 0.2 points. When the pedestrian is in a normal walking posture, the current horizontal speed of the pedestrian is 0m / s<v t ≤2m / s.

[0030] Furthermore, the pedestrian's current dynamic risk score value R(t) is compared with the preset risk threshold, which includes a first risk critical point value of 1.0 and a second risk critical point value of 1.5; if R(t)>1.5, it is judged as a high risk level; if 1.0≤R(t)≤1.5, it is judged as a medium risk level; if R(t)<1.0, it is judged as a low risk level.

[0031] The present invention also provides a system for identifying and protecting pedestrians from abnormal behavior on a bridge based on intelligent monitoring, comprising a processing device for implementing the steps of the above-mentioned method for identifying pedestrians from abnormal behavior on a bridge.

[0032] The present invention also provides a system for identifying and protecting abnormal behavior of pedestrians on a bridge based on intelligent monitoring, including a processing device.

[0033] Compared with the existing technology, the method for identifying abnormal behavior of pedestrians on a bridge provided by the present invention has the following beneficial effects:

[0034] The present invention provides a method for identifying abnormal behavior of pedestrians on a bridge. After obtaining the coordinates of the pedestrian target frame corresponding to each pedestrian in the physical space coordinate system of the bridge; based on the center coordinates of the pedestrian target frame and inertial measurement data, the pedestrian static feature data and the pedestrian dynamic feature data are obtained. The pedestrian static feature data includes the pedestrian position feature vector F pos and the number of interactive pedestrians. Pedestrian dynamic feature data includes pedestrian motion feature vector F motion , and the pedestrian position feature vector F pos It can characterize the pedestrian’s current horizontal position corresponding to the pedestrian’s current area risk label, the pedestrian motion feature vector F motion F in x Used to identify horizontal motion behaviors such as staying and running, F y Used to identify actions on elevated surfaces such as climbing, crossing, and jumping; finally, based on the pedestrian's static feature data and dynamic feature data, it automatically identifies abnormal pedestrian behavior and provides real-time data support for pedestrian bridge safety warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0036] Figure 1 Schematic diagram of a flow chart of a method for identifying abnormal behavior of pedestrians on a bridge in one embodiment of the present invention;

[0037] Figure 2 FIG. 4 is a flow chart of obtaining and registering dual-modality images in one embodiment of the present invention.

[0038] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0041] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0042] In addition, the descriptions of "first", "second", etc. in the present invention are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0043] Please refer to the attached Figure 1 and Figure 2 ,The present invention provides a method for identifying abnormal behavior of pedestrians on a bridge, S10, based on image recognition and coordinate conversion, obtaining the center coordinates of the pedestrian target frame of each pedestrian in the physical space coordinate system of the bridge;

[0044] S20, based on the center coordinates of the pedestrian target frame and the inertial measurement data, obtain the pedestrian static feature data and the pedestrian dynamic feature data, wherein the pedestrian static feature data includes the pedestrian position feature vector F pos and the number of interactive pedestrians. Pedestrian dynamic feature data includes pedestrian motion feature vector F motion ;

[0045] Among them, the pedestrian position feature vector F pos =[X(t), Z(t), Lable(t)], (X(t), Z(t)) represents the horizontal coordinate of the center of the pedestrian target box in the physical space coordinate system of the bridge, Lable(t) is the pedestrian's current regional risk label, which is used to characterize the type of risk area the pedestrian is currently in. The pedestrian's current regional risk label includes the first regional label (first regional label), the second regional label (yellow regional label), and the third regional label (green regional label);

[0046] Among them, the pedestrian motion feature vector F motion =[F x , F y ], a xfiltered (t),F x =[v x (t), a xfiltered (t), C(t)], v x (t) is the current speed of the pedestrian in the horizontal direction, a xfiltered (t) is the horizontal acceleration after low-pass filtering, C(t) is the horizontal trajectory curvature; F y Used to identify altitude behavior (such as climbing, crossing, jumping, etc.), F y =[v y (t), a y (t), ΔY(t)], v y (t) is the current speed of the pedestrian in the vertical direction, a y (t) is the vertical acceleration, ΔY(t) is the height change relative to the previous moment;

[0047] S30: Identify abnormal pedestrian behavior based on the pedestrian's static feature data and the pedestrian's dynamic feature data.

[0048] The present invention provides a method for identifying abnormal behavior of pedestrians on a bridge. After obtaining the coordinates of the pedestrian target frame corresponding to each pedestrian in the physical space coordinate system of the bridge; based on the center coordinates of the pedestrian target frame and inertial measurement data, the pedestrian static feature data and the pedestrian dynamic feature data are obtained. The pedestrian static feature data includes the pedestrian position feature vector F pos and the number of interactive pedestrians. Pedestrian dynamic feature data includes pedestrian motion feature vector F motion , and the pedestrian position feature vector F pos It can characterize the pedestrian’s current horizontal position corresponding to the pedestrian’s current area risk label, the pedestrian motion feature vector F motion F in x Used to identify horizontal motion behaviors such as staying and running, F yUsed to identify actions on elevated surfaces such as climbing, crossing, and jumping; finally, based on the pedestrian's static feature data and dynamic feature data, it automatically identifies abnormal pedestrian behavior and provides real-time data support for pedestrian bridge safety warnings.

[0049] Furthermore, step S10 includes: S11, registering the current visible light image and the current infrared image to obtain a registered bimodal image based on the current image coordinate system; S12, obtaining the center coordinates of the pedestrian target frame of each pedestrian in the physical space coordinate system of the bridge based on the registered bimodal image and the current coordinate mapping conversion model. Specifically, S11, registering the current visible light image and the current infrared image to obtain a registered bimodal image based on the current image coordinate system; S12, considering the current deformation of the bridge, establishing a current coordinate mapping conversion model that maps pixel coordinate points in the current image coordinate system to physical coordinate points in the current physical space coordinate system of the bridge, or obtaining the current coordinate mapping conversion model based on measurement or image processing; S13, obtaining the pedestrian's current three-dimensional actual coordinates (X(t), Y(t), Z(t)) in the physical space coordinate system of the bridge based on the pedestrian image coordinates and the current coordinate mapping conversion model.

[0050] Optionally, step S13 includes: S131, extracting the pedestrian target of each pedestrian from the registered bimodal image and determining the center coordinates of the pedestrian target frame; S132, mapping the center coordinates of the pedestrian target frame to the physical space coordinate system of the bridge through the current coordinate mapping conversion model to obtain the pedestrian's current mapped physical horizontal coordinates; S133, obtaining the pedestrian's current predicted physical horizontal coordinates based on motion prediction according to the pedestrian's previous actual physical horizontal coordinates; S134, obtaining the pedestrian's current actual physical horizontal coordinates (X(t), Z(t)) based on weighted fusion of the pedestrian's current mapped physical horizontal coordinates and the pedestrian's current predicted physical horizontal coordinates; S135, using binocular parallax images combined with IMU pitch angle compensation installation errors to obtain the current pedestrian's elevation coordinates, and obtain the pedestrian's real-time elevation physical coordinates Y(t).

[0051] Optionally, step S11 specifically includes: S111, using vibration-wind load dynamic Gaussian filtering and edge protection weighting to process the current visible light image to obtain a current visible light filter compensation image.

[0052] It can be understood that the solution of the present invention realizes the vibration-wind load coupled dynamic filtering. This method breaks through the limitations of traditional static Gaussian filtering and realizes real-time adaptive adjustment of filtering parameters by establishing an accurate mathematical model of vibration frequency, amplitude and wind load. The system input includes the original visible light image I vis (x, y, t), vibration sensor data (frequency f at time t) vib (t), amplitude A vib(t)) and environmental parameters (wind speed v at time t) wind (t), light intensity L amb (t)), outputting a visible light image that has been precisely filtered and retains structural edge features; innovatively introducing a bridge structure-guided edge protection mechanism, which achieves adaptive enhancement of the characteristic edges of different bridge types through a directional correlation function, effectively solving the edge blurring problem of traditional image processing algorithms under bridge vibration and complex ambient lighting.

[0053] S112 , using a material adaptive calibration model to dynamically compensate for environmental interference based on respective thermodynamic parameters of steel and concrete, and calibrating the current infrared image to obtain an infrared calibration image.

[0054] It is understandable that the solution of the present invention has developed a material adaptive calibration model, which breaks through the technical barrier of traditional infrared image processing that ignores the differences in material thermal properties, and realizes the refined differentiation of steel structures and concrete structures. By establishing a multi-physics field coupling model including temperature, wind speed, and humidity, it accurately compensates for the differential effects of environmental factors on different materials. The system input includes the original infrared image Bridge material type identification (steel, concrete) and multi-dimensional environmental sensor data (temperature T at time t amb (t), wind speed v wind (t), humidity h humidity (t)), outputs a calibrated infrared image, and the temperature measurement accuracy is greatly improved, providing a high-precision foundation for subsequent temperature anomaly detection.

[0055] S113, based on the elastic registration model, a deformation feedback elastic registration strategy is adopted, and the registration matrix is ​​optimized and fused with the real-time deformation data of the bridge to obtain a registered dual-modal image. and registering dual-modality images Share the same image coordinate system and retain their respective feature information.

[0056] As can be understood, the present invention employs a deformation-feedback elastic registration technique, innovatively incorporating real-time bridge deformation data as a key input variable in the registration process. This allows for dynamic registration of bimodal images through an elastic registration model. The system dynamically optimizes the elastic deformation radius (determined by real-time bridge monitoring data) and outputs precisely registered bimodal images. This technique addresses the accuracy degradation of traditional registration algorithms under long-term bridge deformation conditions, achieving stable registration over extended monitoring periods.

[0057] Optionally, in step S111, the formula Calculate the dynamic Gaussian filter kernel parameters and adaptively adjust the smoothing intensity according to the bridge vibration and wind load conditions to achieve the goal of suppressing image blur caused by bridge vibration while retaining the structural edge features. vib (t) and f vib (t) are the current vibration amplitude and current vibration frequency at time t obtained by the vibration sensor, v wind is the current wind speed at time t, as obtained by the environmental sensor. k is the vibration matching coefficient, and η is the wind load correction factor. The values ​​of k range from 0.8 to 1.2, and η range from 0.3 to 0.7. k is the vibration matching coefficient, with a higher value for cable-stayed bridges and a lower value for beam bridges. η is the wind load correction factor, calibrated by wind tunnel testing.

[0058] Optionally, in step S112, the formula Calculate the image edge protection weight, where λ is the edge sensitivity factor, Represents the image gradient intensity, D structure (θ) is the directional correlation function (directional weight function), and the directional weight function D is introduced structure (θ), strengthen the edges of key structures such as guardrails and cables, D structure (θ)=1+sin 2 (2θ), θ is the angle between the image gradient direction and the main structure direction of the bridge.

[0059] Optionally, in step S112, the function expression of the material adaptive calibration model is:

[0060] T cal (x,y,t)=T raw ·(1+αΔT amb )+βv wind +γh humidity , T cal (x, y, t) represents the calibrated infrared temperature value, that is, the temperature value of the pixel point (x, y) at time t after compensation for environmental factors. α, β, and γ are the model coefficients related to the current temperature, current wind speed, and current humidity, respectively. v wind and h humidity The current wind speed and humidity are obtained by the environmental sensor, ΔT amb is the ambient temperature difference between the current temperature and the calibration reference temperature, T raw is the original infrared temperature value. Preferably, for steel bridges, α = 0.15, β = -0.1, γ = -0.08; for concrete bridges, α = 0.12, β = -0.05, γ = -0.08.

[0061] Furthermore, in step S113, the function expression of the elastic registration model is:

[0062]

[0063] Among them, H elastic is the elastic deformation risk assessment coefficient, which is used to measure the impact of the elastic deformation state of the bridge on pedestrian safety. Its value range is 0-1, and a larger value indicates a higher risk. N is the number of reference points, which is used to calculate the total number of bridge feature points or sensor nodes affected by deformation. i is the confidence weight of feature point i, for example, the weight of the bridge tower anchor point is 0.3, and the weight of the guardrail joint point is 0.2, satisfying p i is the current state coordinate, q i is the reference state coordinate, ||p i -q i || 2 is the square of the spatial distance between the current position of the monitoring point and the reference position, σ def is the deformation sensitivity parameter.

[0064] Furthermore, ΔL max is the elastic deformation radius determined according to the bridge design specification; the deformation compensation parameter ΔP(t) is used (the bridge deformation compensation amount comes from the feedback of the coordinate positioning module), through Update the parameters σ of the registration model in real time def , where σ def (0) represents the initial state value of the deformation sensitivity parameter in the elastic registration model, and k′ is the adjustment coefficient, which represents the degree of influence of deformation on the registration sensitivity. A closed-loop feedback mechanism is formed, which enables the registration process to adapt to the deformation state of the bridge over time and improves the registration stability during long-term monitoring.

[0065] Furthermore, in step S12, at least 8 CAD physical feature points are extracted from the bridge design CAD model. As matching reference points and registering bimodal images and registering dual-modality images Coordinate mapping is performed, and the vibration parameter σ(t) (i.e., the dynamic Gaussian filter kernel parameter σ(t)) and the thermal expansion coefficient α are incorporated into the coordinate transformation process as dynamic deformation compensation factors for calculation. The homography transformation matrix H that realizes the coordinate mapping between the image coordinate system and the bridge physical space coordinate system is obtained.

[0066] Furthermore, the function expression of the homography transformation matrix H is:

[0067]

[0068] in, ΔP(t) is the real-time deformation offset, λ is the deformation compensation weight, H 11 and H 12 The image horizontal scaling and rotation parameters are respectively. The concrete bridge needs to be adjusted according to the curvature of the bridge deck. 31 and H 32 is the perspective transformation coefficient, H 13 and H 23 is the translation parameter, corresponding to the offset of the starting coordinate of the bridge deck. Optionally, λ=0.1~0.5: according to the bridge vibration frequency f vib (t) Adjustment, high frequency vibration takes high value; a x (s) is the instantaneous value of the acceleration component of the bridge in the X direction at time s, σ(τ) is the instantaneous value of the dynamic Gaussian filter kernel parameter during the integration process, and f vib (τ) is the instantaneous value of the current vibration frequency during the integration process, dτ is the differential of the integral variable, and L is the bridge span. Specifically, σ(t) represents the parameter value at a specific time t, and σ(τ) represents the function of the integral variable τ during the integration process.

[0069] Furthermore, step S12 also includes the step of using verification feature points to perform mapping verification, randomly selecting 5 non-calibrated points to verify the error, and if the average deviation ∈ avg If the value is >0.3m, it will trigger the re-collection of feature points and update of matrix H.

[0070] Optionally, for newly built bridges, the system gives priority to homography transformation (CAD driven mode) to achieve high-precision close-range mapping. In actual applications, the system first extracts at least 8 feature points from the bridge CAD design model as matching reference points. These feature points usually include obvious and stable structural positions such as bridge towers, guardrail connection points, and cable fixing points. The system matches these CAD physical feature points with the registered dual-modal image, and introduces vibration parameters and thermal expansion coefficients as dynamic deformation compensation factors, and calculates the homography matrix H by minimizing the reprojection error. During the homography transformation calculation process, the system dynamically processes the bridge deformation factors and converts the real-time data σ collected by the vibration sensor into the real-time data σ t The material's thermal expansion coefficient, α, is incorporated into the calculations, allowing the coordinate transformation process to adapt to the elastic deformation of the bridge under different environmental conditions. After each mapping is completed, the system automatically selects five non-calibrated verification points to verify the mapping accuracy. When the average deviation exceeds a preset threshold, feature point re-collection and matrix update are triggered.

[0071] Furthermore, in step S12, the three key feature points of the guardrail are located using Beidou / GPS to construct a satellite relative coordinate system, and the visual inertial odometry (VIO) technology is combined to achieve continuous positioning, and the affine transformation parameters for the coordinate mapping between the image coordinate system and the bridge physical space coordinate system are obtained; the latitude and longitude coordinates of the three key feature points of the guardrail are collected and converted into UTM plane coordinates. The affine transformation parameters (a, b, c, d, e, f) are calculated. For old bridges or scenes where accurate CAD data cannot be obtained, the system uses affine transformation (satellite drive mode) to achieve coordinate mapping. In this mode, the system accurately locates the three key feature points of the guardrail on the bridge through the Beidou / GPS receiver, and converts the obtained latitude and longitude coordinates into physical coordinates in the UTM plane coordinate system. The system constructs a satellite relative coordinate system based on these feature points, and combines visual inertial odometry (VIO) technology to achieve continuous positioning and calculate the affine transformation parameter matrix. During VIO continuous positioning, the visual front end extracts the guardrail seams as VIO landmark points, the descriptor uses the BRIEF algorithm, and the IMU pre-integration Tightly coupled optimization ρ is the Huber robust kernel function, which suppresses feature point mismatching; Σ IMU is the IMU noise covariance matrix, which is determined by calibration experiments. ω(t) is the angular velocity measurement at time t, R(t) is the rotation matrix at time t, a(t) is the acceleration measurement at time t, and z visual is the visual observation value, which represents the pixel coordinates of the feature points in the image, h proj is the function that projects a point in three-dimensional space onto a two-dimensional image plane, X landmark is the coordinate of the landmark point, indicating the position of the feature point in three-dimensional space, r IMU is the residual error of IMU.

[0072] In one embodiment of the present invention, homography transformation (CAD-driven mode) is primarily used for high-precision, close-range mapping of newly built bridges, while affine transformation (satellite-driven mode) is primarily used for older bridges or GPS calibration scenarios. The system adaptively selects the transformation method based on the bridge type. The coordinate transformation module provides a spatial mapping benchmark; object detection results locate pedestrian pixel positions; and IMU and binocular data compensate for motion blur and provide height information, respectively, forming a closed-loop 3D positioning system.

[0073] It can be understood that the solution of the present invention innovatively constructs a dual-mode dynamic coordinate system to solve the positioning drift problem of traditional fixed coordinate mapping under bridge structure deformation, temperature change and vibration environment. The system has developed two sets of scene-customized coordinate conversion mechanisms for different bridge types, and achieved millimeter-level positioning accuracy through sensor-driven dynamic adjustment strategies. For newly built bridges, the system constructs an accurate structure-coordinate mapping relationship based on the CAD model, breaking through the technical limitations of traditional methods that ignore the deformation of bridges during service. The system input includes the registration of dual-modal images, CAD model feature point coordinates and multi-source sensor data, and innovatively converts the vibration parameter σ tThe thermal expansion coefficient α is integrated into the calculation process of the homography transformation matrix H to dynamically compensate for the coordinate drift caused by ΔP(t). The system also implements a feature point optimization strategy, intelligently selects anti-deformation feature points (such as bridge tower anchor points) according to the structural characteristics of the bridge, avoids areas susceptible to load (such as bridge deck expansion joints), and ensures the long-term stability of coordinate transformation. For existing bridges, the system has developed an adaptive coordinate system that tightly couples satellite positioning with visual inertial odometry (VIO), which solves the failure problem of traditional single positioning methods in complex bridge environments. System inputs include satellite positioning data, IMU data (acceleration a x , a y , a z , angular velocity ω x ,ω y ,ω z )) and visual feature points, a satellite relative coordinate system is established by identifying the three key feature points of the guardrail. At the same time, a bridge structure feature enhancement VIO algorithm is innovatively developed, using the guardrail as a visual landmark, and the descriptor uses the BRIEF algorithm. The back-end is tightly coupled and optimized through IMU pre-integration and visual features to achieve continuous positioning in areas where satellite signals are blocked (such as under the bridge tower). The system automatically switches the main positioning source according to the satellite signal strength S_GNSS. When S_GNSS>45dB, satellite data is used first, which solves the problem of positioning continuity in complex bridge environments. The system also innovatively implements a dynamic mapping mechanism for dangerous areas, breaking through the limitations of traditional fixed electronic fences. By integrating real-time pedestrian coordinates, a bridge structure database (guardrail / lane boundary coordinates) and stress sensor data, the system constructs a stress-flow coupled safety threshold model. The model can dynamically adjust the safety distance threshold according to the local stress state of the bridge and traffic flow, automatically expand the scope of the danger zone when heavy vehicles pass, and shorten the lane intrusion judgment time from 5s to 3s during peak hours (traffic flow Q>2000 vehicles / h), solving the problem of insufficient adaptability of traditional fixed area division under variable load conditions.

[0074] Furthermore, in step S12, the center coordinates (x image ,y image ) is mapped to the physical space coordinate system of the bridge to obtain the pedestrian's current mapped physical horizontal coordinate (X visual , Z visual ),

[0075] Furthermore, in step S12, the pedestrian's current predicted physical horizontal position (X) is obtained by performing motion prediction based on the pedestrian's previous actual physical horizontal coordinate ((X(t-1), Z(t-1)) according to the IMU data. IMU (t), Z IMU (t)), where v x 、a x They represent the velocity and acceleration in the x direction, v z 、a z They represent the velocity and acceleration in the z direction respectively, and Δt represents the time interval.

[0076] Furthermore, the pedestrian's current mapped physical horizontal coordinate (X visual , Z visual ) and the pedestrian’s current predicted physical horizontal position (X IMU (t), Z IMU (t)), and adopt weighted fusion processing to obtain the pedestrian’s current actual physical horizontal coordinates (X(t), Z(t)); where X(t) = w vis ·X visual +w IMU ·X IMU ; Z(t)=w vis ·Z visual +w IMU ·Z IMU ; Conf detect For detection confidence, k = 0.5, k is the speed sensitivity coefficient, v0 = 1.5 m / s, and v0 is the speed threshold for distinguishing walking from running.

[0077] Furthermore, binocular parallax is combined with IMU pitch angle compensation to obtain the current pedestrian elevation coordinates, and the pedestrian's real-time elevation physical coordinates Y(t) are obtained, Y(t) = Y raw ·cos(θ tilt )+Δy install

[0078] Among them, Y raw is the original height value calculated by the binocular disparity algorithm, θ tilt is the pitch angle measured by IMU, Δy install Compensate for camera height. Optionally, binocular distance measurement Tilt compensation T height =Y raw ·cos(θ tilt )+Δy install , Δy install =0.05m: Camera installation height deviation (calibration value); θ tilt : Pitch angle (rad) output by IMU in real time.

[0079] In a specific embodiment of the present invention, in order to break through the accuracy bottleneck of traditional single visual positioning in the environment of bridge vibration and illumination change, high-precision tracking of pedestrian three-dimensional coordinates is achieved through a multi-source fusion positioning mechanism. The system customizes a visual-inertial collaborative optimization algorithm for bridge pedestrian monitoring scenarios, which effectively solves the problem of unstable positioning caused by high-speed motion, illumination changes and bridge vibration. Specifically, S31, an improved YOLOv5 model is used for pedestrian target detection, and the bounding box regression term in the loss function is optimized for bridge scenes, thereby reducing the interference of complex bridge structures (such as cables and guardrails) on pedestrian detection. The system input is a registered bimodal image, and outputs the coordinates of the center of the pedestrian target box and the detection confidence. S32-S34, realizes visual-IMU adaptive fusion positioning, and innovatively solves the positioning deviation problem caused by visual blur under high-speed motion conditions. The system first calculates the horizontal position of the pedestrian (X visual ,Z visual ), and realize motion prediction based on IMU acceleration data. The core innovation of the system lies in the speed-sensitive dynamic weight fusion algorithm. vis Real-time calculation of visual data weights, when the pedestrian speed v (t) When the speed exceeds 2m / s, the system automatically reduces the visual weight to below 0.4 and increases the IMU weight, thereby effectively suppressing positioning jumps in high-speed motion scenes. The system also introduces a bridge structure constraint mechanism, using the guardrail position to limit the Z coordinate range and automatically eliminate positioning abnormal values. S35 uses binocular parallax and IMU pitch angle compensation technology to achieve vertical positioning, breaking through the technical limitations of traditional vertical measurement that ignores camera installation errors and bridge tilt. The system uses Y height =Y raw cos(θ tilt )+Δy install The formula realizes tilt compensation. The system also introduces sliding average filtering processing (window size N=5) to effectively suppress the height measurement fluctuations caused by bridge deck vibration, reducing the height measurement error from ±0.5m of the traditional method to ±0.1m. The entire step forms a complete multi-source fusion three-dimensional positioning system, and there is a strict data link between the modules: the trajectory data is fed back to the IMU integral parameter optimization module to suppress integral drift; the height data is transmitted to the behavior analysis module for fall risk assessment; the position tag is used to trigger the recalibration of the coordinate system. The system solves the problem of insufficient adaptability of traditional positioning methods in the complex dynamic environment of bridges, and provides a stable and accurate spatial positioning foundation for subsequent behavior analysis.

[0080] In the present invention, the process of calculating the real-time coordinates of pedestrians based on multi-source data fusion is as follows: the system first obtains visible light and infrared registered images that share the same coordinate system through the dual-modal image preprocessing and registration module, and establishes a coordinate mapping conversion model in combination with the dynamic deformation data of the bridge. The real-time coordinate calculation of pedestrians adopts a combined strategy of "visual measurement-inertial prediction-weighted fusion", which effectively overcomes the limitations of a single data source in a complex bridge environment. In the target detection stage, the system identifies pedestrian targets from the registered dual-modal images, and extracts the center coordinates (x image ,y image The algorithm enhances the detection capability of partially obscured targets through the attention mechanism, and integrates infrared thermal features to improve the detection accuracy at night and in adverse weather conditions. In the coordinate mapping stage, the system maps the center coordinates of the pedestrian target frame to the physical space coordinate system of the bridge through the pre-established homography transformation matrix H, and obtains the current mapped physical horizontal coordinates of the pedestrian (X visual , Z visual ). The transformation matrix H is obtained by minimizing the reprojection error between the CAD feature points and the actual image feature points, and is updated in real time to adapt to the dynamic deformation of the bridge. For structural micro-deformations caused by vibration and temperature changes, the system introduces a dynamic correction term ΔP(t) for real-time compensation. In the motion prediction stage, the system models the motion state of pedestrians based on inertial measurement unit data. The system uses the actual physical horizontal coordinates (X(t-1), Z(t-1)) of the pedestrian obtained at the last moment as the reference point, combined with the extracted speed and acceleration parameters, to predict the current physical position (X IMU (t), Z IMU (t)). To reduce the interference of bridge vibration on acceleration data, the system uses frequency domain analysis to identify and filter out noise components close to the natural frequency of the bridge. During the data fusion stage, the system dynamically adjusts the weight ratio of visual measurement results and inertial prediction results according to the current scene characteristics. When the lighting conditions are good and the target speed is low, the visual measurement weight w vis Higher; when the target speed increases or the lighting conditions deteriorate, the inertia prediction weight w IMU Adaptive improvement. The weight calculation function design takes into account the nonlinear effects of target detection confidence and speed factors to ensure the best fusion effect under various conditions. Finally, the system obtains the pedestrian elevation coordinates through binocular parallax ranging technology combined with IMU pitch angle data. Since the camera installation height error and lens distortion will cause elevation measurement deviation, the system introduces the installation error correction term Δy installThe system uses a multi-source data fusion strategy to effectively solve the problem of pedestrian positioning in bridge environments. It achieves stable coordinate calculations in the presence of interference factors such as vibration, illumination changes, and structural deformation, meeting the technical requirements for identifying abnormal pedestrian behavior.

[0081] Furthermore, step S30 includes: S31, using a multi-factor fusion model to calculate the pedestrian's current dynamic risk score based on the pedestrian's static feature data and the pedestrian's dynamic feature data; S32, determining the pedestrian's current risk level based on the pedestrian's current dynamic risk score and a preset risk threshold.

[0082] Furthermore, the pedestrian static feature data also includes the pedestrian posture feature vector F pose , pedestrian posture feature vector F pose Used to represent the current posture of the pedestrian,

[0083] F pose =[Keypoint fused , Conf pose ], Keypoint fused is the key point fusion coordinate,

[0084] Keypoint fused =Keypoint vis ·Conf vis +Keypoint ir ·(1-Conf vis ),

[0085] Keypoint vis Keypoint is the coordinate of the key point mapped to the infrared image. ir The key point coordinates of the visible light image mapping, Conf vis is the visible light confidence, Conf vis =L amb (t) / 255,Conf pose The overall confidence of posture recognition.

[0086] Optionally, extract the posture feature vector F pose Specifically, based on the lightweight HRNet network, human body key points are extracted from visible light and infrared images, a skeleton model is constructed, and bimodal complementary fusion is performed to identify the current posture of pedestrians.

[0087] Furthermore, the pedestrian static feature data also includes the pedestrian temperature feature vector F temp ,

[0088] F temp =[Alert temp , H temp ], among which, Alert temp It is an abnormal temperature alarm signal.

[0089]

[0090] ΔT th is the material adaptive threshold, μ env (t) = mean(T amb (t)), μ env (t) is the mean ambient temperature, H temp is the temperature distribution entropy used to quantify the uniformity of temperature distribution, and T(x, y, t) is the current temperature value.

[0091] Furthermore, the formula R(t)=R0×K is used. t ×K n ×K p ×K v ×K thermal Calculate the current dynamic risk score of the pedestrian, where R0 is the pedestrian's static feature score, K t is the pedestrian stay time dimension correction coefficient, K n is the group effect correction coefficient, K p is the pedestrian area risk coefficient, K v is the pedestrian motion trend coefficient, K thermal is the thermodynamic risk factor;

[0092] Among them, the pedestrian static feature score (pedestrian current posture feature score) R0 is obtained based on the behavior type label, and the behavior type label is composed of the multidimensional feature vector F final(t) , the pedestrian's current actual physical horizontal coordinates (X(t), Y(t), Z(t)), the pedestrian's motion feature vector F motion Get, multidimensional feature vector F final(t) By fusion feature vector F fused(t) Obtained by time series sliding window smoothing, the fusion feature vector F fused(t) By the pedestrian position feature vector F pos , posture feature vector F pose And the pedestrian temperature feature vector F temp Fusion acquisition;

[0093] Pedestrian dwell time dimension correction coefficient K t Based on the pedestrian motion feature vector F motion Get;

[0094] Group effect correction coefficient Kn Based on the data acquisition of the number of interactive pedestrians;

[0095] Regional risk factor K p Based on the current physical area label of the pedestrian;

[0096] Pedestrian movement trend coefficient K v Based on the pedestrian's moving direction and posture feature vector F pose , and pedestrian motion feature vector F motion Get;

[0097] Thermodynamic risk factor K thermal Based on the pedestrian temperature feature vector F temp Get.

[0098] It can be understood that in the solution of the present invention, the correlation indicators R0, K t , K n , K p , K v and K thermal ; According to expert experience, the importance scores of each related indicator are processed to obtain the judgment matrix; the weight value of each evaluation indicator is calculated based on the judgment matrix, and a consistency test is performed to obtain the final weight value of each related indicator under the specific state.

[0099] Furthermore, using formula F fused (t)=α'·F pos +β′·F pose +γ′·F temp Calculate the fusion feature vector F at time t fused (t), where α′, β′, and γ′ are the position feature weight, posture feature weight, and temperature feature weight, respectively;

[0100] The fusion feature vector F is fused using a time series sliding window fused (t) Smoothing is performed to obtain the multidimensional feature vector F final (t), the smoothing function expression is: Among them, ω i is Gaussian weight, window size N=5;

[0101] The behavior type labels include climbing behavior posture, crossing behavior posture, staying behavior posture, fast running behavior posture and normal walking posture; if the pedestrian's current posture is climbing behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.7; if the pedestrian's current posture is crossing behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.6; if the pedestrian's current posture is staying behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.5; if the pedestrian's current posture is fast running behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.4, and the pedestrian's current horizontal speed v in the fast running behavior posture is 0. t >2m / s; if the pedestrian's current posture is a normal walking posture, the pedestrian's current posture feature score R0 is determined to be 0.2 points. When the pedestrian is in a normal walking posture, the current horizontal speed of the pedestrian is 0m / s<v t ≤2m / s.

[0102] Optionally, ω i It is the weight coefficient in the time series sliding window, specifically the Gaussian weight applied to the time dimension. The time series sliding window refers to the selection of a time window of fixed length (5 time points in this patent) on a continuous time series, and the window continues to slide forward as time goes by. Each time the data at the current moment is processed, the historical data in the window is also considered. Gaussian weight means that different weight coefficients are assigned to data at different time points within this window. These weight coefficients follow the characteristics of the Gaussian distribution (normal distribution): the data at the current moment (t) obtains the highest weight; the data at the previous moment (t-1) has the second highest weight; the weight of the data at earlier moments (t-2, t-3, t-4) gradually decreases. This weight configuration makes the feature data at the current moment receive the highest attention, while the influence of historical data gradually weakens according to the time distance, thereby achieving smooth processing of the time series data while retaining the main feature information of the current state.

[0103] Optionally, based on the multidimensional feature vector F final (t), the pedestrian’s current actual physical horizontal coordinates (X(t), Y(t), Z(t)), and the horizontal motion feature vector F x =[v(t),a filtered (t), C(t)] and the vertical motion eigenvector F y= [vy(t), ay(t), ΔY(t)] determines the behavior type label, and determines the current posture feature score R0 of the pedestrian according to the behavior type label; if the current posture of the pedestrian is a climbing behavior posture (vertical speed vy(t) > 0.5 m / s for more than 2 seconds continuously, and elevation change ΔY(t) > 0.8 m), then determine the current posture feature score R0 of the pedestrian to be 0.7; if the current posture of the pedestrian is a striding behavior posture (vertical speed vy(t) is positive first and then negative, vertical acceleration ay(t) conforms to the striding feature curve, and the elevation change presents an "ascending - plateau - descending" pattern), then determine the current posture feature score R0 of the pedestrian to be 0.6 points; if the current posture of the pedestrian is a lingering behavior posture (horizontal speed v(t) < 0.3 m / s for more than 30 seconds continuously, and vertical speed |vy(t)| < 0.1 m / s), then determine the current posture feature score R0 of the pedestrian to be 0.5 points; if the current posture of the pedestrian is a rapid running behavior posture (horizontal speed v(t) > 2 m / s, and horizontal acceleration a filtered (t) is significant), then determine the current posture feature score R0 of the pedestrian to be 0.4 points; if the current posture of the pedestrian is a normal walking posture (horizontal speed 0.3 < v(t) < 1.5 m / s, trajectory curvature C(t) is stable, and vertical direction parameters |vy(t)| < 0.2 m / s, |ΔY(t)| < 0.3 m), then determine the current posture feature score R0 of the pedestrian to be 0.2 points.

[0104] Specifically, the current actual physical horizontal coordinates (X(t), Y(t), Z(t)) of the pedestrian: provide the accurate position information of the pedestrian in the physical space coordinate system of the bridge. Among them, the change rate and cumulative change of Y(t) are the key parameters for judging abnormal behaviors in the vertical direction, and the time - series changes of X(t) and Z(t) are used to analyze the horizontal movement trajectory, velocity vector, and relative position relationship with the guardrail of the pedestrian.

[0105] Pedestrian motion feature vector F motion : This vector includes key kinematic parameters such as the current speed v(t) of the pedestrian, the acceleration a filtered (t) after 5Hz low - pass filtering processing, and the trajectory curvature C(t). v(t) is used to determine whether the motion state of the pedestrian exceeds the preset threshold, a filtered (t) reflects the motion acceleration characteristics, and C(t) can effectively identify the spatial characteristics of non - linear motion patterns.

[0106] Multidimensional feature vector F final : After being smoothed by a time - series sliding window, this vector effectively integrates the posture feature vector F pose , temperature feature vector F temp and position feature vector F posAmong them, posture features provide key point position relationships through the skeleton model, temperature features provide abnormal hotspot distribution information, and position features are associated with regional risk levels, together forming a multimodal representation of behavioral characteristics.

[0107] Understandably, the traditional method does not consider the interference of bridge vibration on IMU data, resulting in the accumulation of acceleration errors. The present invention quantifies the pedestrian motion state (speed, acceleration, trajectory curvature) by fusing IMU data with historical coordinate sequences, providing data support for identifying abnormal behaviors (such as falling, running). When extracting motion features, the input data includes historical coordinate sequences. (time window 1.5 seconds) and IMU raw acceleration a x (t); vibration noise suppression and 5Hz low-pass filtering are used to significantly reduce acceleration errors; curvature is calculated in real time, and subtle trajectory changes are captured through differential operations.

[0108] The speed calculation formula is: Δt=0.1s, Δt is the sampling period, which is synchronized with the coordinate positioning module; v(t) is the instantaneous velocity, in m / s. The acceleration filter formula is

[0109] a filtered (t) = Butterworth(a x (t),f cutoff =5Hz)

[0110] f cutoff The cut-off frequency is used to filter out bridge vibration noise (typical vibration frequency is 0.5-5Hz).

[0111] The formula for trajectory curvature is C(t) is the trajectory curvature, which is used to identify abnormal behaviors such as climbing and crossing.

[0112] The motion feature vector is output to the behavior analysis module for motion anomaly detection (such as climbing and leaping). Based on the lightweight HRNet network, human key points are extracted from visible light and infrared images to construct a skeleton model. In low-light scenes, infrared data supplements key point information to avoid posture omissions caused by insufficient light in traditional methods. The key point fusion formula is:

[0113] Keypoint fused =Keypoint vis ·Conf vis +Keypoint ir ·(1-Conf vis )

[0114] Keypoint fused Keypoint is the key point fusion coordinate,vis The coordinates of the key points of the human body extracted from the visible light image, Conf vis =L amb (t) / 255, visible light confidence (normalized light intensity);

[0115] Keypoint ir The key point coordinates output for the infrared branch.

[0116] Posture feature vector F pose =[Keypoint fused , Conf pose ] is output to the visualization system for real-time posture rendering and out-of-bounds warning.

[0117] Furthermore, using the formula Calculate the time dimension correction coefficient K t , where t T is based on the pedestrian motion feature vector F motion Determines how long the action lasts.

[0118] Furthermore, the number of interactive pedestrians is determined based on the multi-target tracking module to quantify the amplification effect of the group effect on the risk; if it is a single person, K is determined. n =1.0; if it is a two-player state, determine K n =1.1; if the number of people is greater than two, then determine K n =1.2.

[0119] Furthermore, using the formula

[0120] Dynamically generate the pedestrian current area risk label, where D guardrail Indicates the distance from the pedestrian to the guardrail. It is the distance from the pedestrian to the left guardrail. Indicates the distance from the pedestrian to the right guardrail, D safe For a safe distance, σ vM is the current Mises stress of the bridge, σ yield is the yield strength of steel, f cu is the compressive strength of concrete,

[0121] InLane(t) is lane intrusion detection,

[0122]

[0123] If the traffic volume Q is less than or equal to 2000 vehicles / hour, the lane intrusion determination time T thIf the traffic volume Q>2000 vehicles / hour, the lane intrusion determination time T th 3s, Z lane_left is the Z coordinate of the left lane, Z lane_right is the Z coordinate of the right lane;

[0124] If it is the first region label, then the regional risk coefficient K p =1.5; if it is the second area label, then the regional risk coefficient K p =1.2; if it is the third regional label, then the regional risk coefficient K p =1.0.

[0125] Furthermore, if v t >0.5m / s and the coordinate Z(t) moves toward the guardrail, then determine the motion trend coefficient K v =1.5; if v t >0.5m / s and the coordinate Z(t) moves away from the guardrail, then determine the motion trend coefficient K v =0.7; if |v t When |≤0.5m / s, determine the motion trend coefficient K v =1.

[0126] Furthermore, based on the temperature feature vector F temp Determine the thermodynamically abnormal state,

[0127]

[0128] Among them, Alert temp It is an abnormal temperature alarm signal.

[0129] In a specific embodiment, in the specific implementation, in Example 1, the three-person team climbs in the red zone for 90 seconds, then R0=0.7, K t =1.13, K n =1.2, K p =1.5, K v =1.5, K thermal =1.0, R = 0.7 × 1.13 × 1.2 × 1.5 × 1.5 × 1.0 = 2.14, corresponding to a high-risk warning; in Example 2, a single person climbs in the red zone for 120 seconds, then R0 = 0.7, K t =1.2, K n =1.0,K p =1.5, K v =1.0,K thermal =1.0, R = 0.7 × 1.2 × 1.0 × 1.5 × 1.0 × 1.0 = 1.26, corresponding to a medium risk warning; in Example 3, a single person runs rapidly in the yellow area for 60 seconds, then R0 = 0.4, Kt =1.07, K n =1.0,K p =1.2, K v =1.0,K thermal =1.0, R = 0.4 × 1.07 × 1.0 × 1.2 × 1.0 × 1.0 = 0.51, corresponding to a low-risk warning. The system updates the risk score every second, enabling real-time assessment of the risk of abnormal behavior. Simultaneously, based on the actual warning effect, the base score and various coefficients are dynamically adjusted to ensure that the system's warning level aligns with the actual risk level, improving the accuracy of risk assessment. This multi-dimensional risk assessment mechanism avoids misjudgment of normal behavior while enabling timely detection and warning of truly high-risk behavior.

[0130] In the solution of the present invention, the system deploys a spatiotemporal attention network (BTSA-Net) optimized for bridge scenarios, innovatively integrating bridge structural constraints with pedestrian behavior analysis. The network uses a spatiotemporal separation convolutional structure to extract short-term motion features (window length 1.5s), and uses a spatial attention mechanism to enhance the feature weights of key areas such as guardrails and lane boundaries; innovatively introduces a trajectory morphology encoder to input the historical trajectory sequence into the LSTM network to generate a trajectory morphology code H traj , used to identify abnormal patterns such as climbing and crossing; a thermodynamic-behavior correlation matrix was developed to map infrared temperature characteristics with behavioral characteristics, improving the accuracy of behavior recognition in low-light scenarios. The system output includes behavior classification results and risk scoring parameters. Risk scoring model R(t) = R0 × K t ×K n ×K p ×K v ×K thermal A dynamic scoring mechanism with multi-factor coupling is implemented.

[0131] Furthermore, the pedestrian's current dynamic risk score R(t) is compared with the preset risk threshold. If RR(t)>1.5, it is judged as a high risk level; if 1.0≤R(t)≤1.5, it is judged as a medium risk level; if R(t)<1.0, it is judged as a low risk level.

[0132] The present invention also provides a system for identifying and protecting pedestrians from abnormal behavior on a bridge based on intelligent monitoring, comprising a processing device for implementing the steps of the above-mentioned method for identifying pedestrians from abnormal behavior on a bridge.

[0133] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for identifying abnormal behavior of pedestrians on a bridge, characterized in that: The steps include: S10, based on image recognition and coordinate conversion, obtain the pedestrian target frame center coordinates (X(t), Y(t), Z(t)) of each pedestrian in the physical space coordinate system of the bridge; S20, obtaining pedestrian static feature data and pedestrian dynamic feature data based on the pedestrian target frame center coordinates and inertial measurement data, wherein the pedestrian static feature data includes the pedestrian position feature vector F pos and the number of interactive pedestrians, the pedestrian dynamic feature data includes the pedestrian motion feature vector F motion ; Among them, the pedestrian position feature vector F pos =[X(t), Z(t), Lable(t)], where (X(t), Z(t)) represents the horizontal coordinate of the center of the pedestrian target box in the physical space coordinate system of the bridge, and Lable(t) is the pedestrian's current area risk label, which is used to characterize the type of risk area the pedestrian is currently in. The pedestrian's current area risk label includes a first area label, a second area label, and a third area label. Among them, the pedestrian motion feature vector F motion =[F x , F y ],F x Used to characterize the dynamics of horizontal surface behavior, F x =[v x (t), a xfiltered (t), C(t)], v x (t) is the current horizontal speed of the pedestrian, a xfiltered (t) is the horizontal acceleration after low-pass filtering, C(t) is the horizontal trajectory curvature; F y Used to characterize the dynamic behavior of elevation plane, F y =[v y (t), a y (t), ΔY(t)], v y (t) is the current vertical speed of the pedestrian, a y (t) is the vertical acceleration, ΔY(t) is the height change relative to the previous moment; S30: Identify abnormal pedestrian behavior based on the pedestrian static feature data and the pedestrian dynamic feature data.

2. The method for identifying abnormal behavior of pedestrians on a bridge according to claim 1, characterized in that: Step S30 includes: S31, using a multi-factor fusion model to calculate and obtain the pedestrian's current dynamic risk score based on the pedestrian's static feature data and the pedestrian's dynamic feature data; S32: Determine the current risk level of the pedestrian based on the pedestrian's current dynamic risk score and a preset risk threshold.

3. The method for identifying abnormal behavior of pedestrians on a bridge according to claim 1, characterized in that: Step S10 includes: S11, registering the current visible light image and the current infrared image to obtain a registered bimodal image based on the current image coordinate system; S12, based on the registered bimodal image and the current coordinate mapping transformation model, the center coordinates of the pedestrian target box of each pedestrian in the physical space coordinate system of the bridge are obtained.

4. The method for identifying abnormal behavior of pedestrians on a bridge according to any one of claims 1 to 3, characterized in that: The pedestrian static feature data also includes a pedestrian posture feature vector F pose , F pose =[Keypoint fused , Conf pose ], Keypoint fused Keypoint is the key point fusion coordinate, fused =Keypoint vis ·Conf vis +Keypoint ir ·(1-Conf vis ), Keypoint vis Keypoint is the coordinate of the pedestrian key point mapped by the infrared image, ir Pedestrian key point coordinates mapped to visible light images, Conf vis is the visible light confidence, Conf vis =L amb (t) / 255,Conf pose The overall confidence of posture recognition.

5. The method for identifying abnormal behavior of pedestrians on a bridge according to claim 4, characterized in that: The pedestrian static feature data also includes the pedestrian temperature feature vector F temp , F temp =[Alert temp , H temp ], among which, Alert temp It is an abnormal temperature alarm signal. ΔT th is the material adaptive threshold, μ env (t)=mean(T amb (t)), μ env (t) is the mean ambient temperature, H temp is the temperature distribution entropy used to quantify the uniformity of temperature distribution, and T(x, y, t) is the current temperature value.

6. The method for identifying abnormal behavior of pedestrians on a bridge according to claim 5, characterized in that: Using the formula R(t)=R0×K t ×K n ×K p ×K v ×K thermal Calculate the current dynamic risk score of the pedestrian, where R0 is the pedestrian's static feature score, K t is the pedestrian stay time dimension correction coefficient, K n is the group effect correction coefficient, K p is the pedestrian area risk coefficient, K v is the pedestrian motion trend coefficient, K thermal is the thermodynamic risk factor; Among them, the pedestrian static feature score R0 is obtained based on the behavior type label, and the behavior type label is composed of the multidimensional feature vector F final(t) , the pedestrian's current actual physical horizontal coordinates (X(t), Y(t), Z(t)), the pedestrian's motion feature vector F motion Get, multidimensional feature vector F final(t) By fusion feature vector F fused(t) Obtained by time series sliding window smoothing, the fusion feature vector F fused(t) By the pedestrian position feature vector F pos , posture feature vector F pose And the pedestrian temperature feature vector F temp Fusion acquisition; Pedestrian dwell time dimension correction coefficient K t Based on the pedestrian motion feature vector F motion Get; Group effect correction coefficient K n Based on the data acquisition of the number of interactive pedestrians; Regional risk factor K p Based on the current physical area label of the pedestrian; Pedestrian movement trend coefficient K v Based on the pedestrian's moving direction and posture feature vector F pose , and pedestrian motion feature vector F motion Get; Thermodynamic risk factor K thermal Based on the pedestrian temperature feature vector F temp Get.

7. The method for identifying abnormal behavior of pedestrians on a bridge according to claim 6, characterized in that: The behavior type labels include climbing behavior posture, crossing behavior posture, staying behavior posture, fast running behavior posture and normal walking posture; if the pedestrian's current posture is a climbing behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.7; if the pedestrian's current posture is a crossing behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.6; if the pedestrian's current posture is a staying behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.5; if the pedestrian's current posture is a fast running behavior posture, the pedestrian's current posture feature score R0 is determined to be 0.4, and the pedestrian's current horizontal speed v in the fast running behavior posture is 0. t >2m / s; if the pedestrian's current posture is a normal walking posture, the pedestrian's current posture feature score R0 is determined to be 0.2 points. When the pedestrian is in a normal walking posture, the current horizontal speed is 0m / s<v t ≤2m / s.

8. The method for identifying abnormal behavior of pedestrians on a bridge according to claim 6, characterized in that: Compare the pedestrian's current dynamic risk score R(t) with a preset risk threshold, which includes a first risk critical point value of 1.0 and a second risk critical point value of 1.5; if R(t)>1.5, it is determined to be a high risk level; If 1.0≤R(t)≤1.5, it is determined to be a medium risk level; If R(t)<1.0, it is judged to be a low risk level.

9. A system for identifying and protecting pedestrians from abnormal behavior on a bridge based on intelligent monitoring, characterized in that: It includes a processing device, which is used to implement the steps of the method for identifying abnormal behavior of pedestrians on a bridge as described in any one of claims 1 to 8.

Images