A security risk identification and early warning system based on video behavior features

Abstract

The application provides a kind of security risk identification and early warning system based on video behavior characteristics, it is related to image recognition technical field, including: equipment motion separation module, for extracting equipment detection frame each structure area motion feature, obtains the equipment body motion trajectory eliminating sling swing interference;Personnel perception deviation calculation module is used to extract personnel face orientation vector and calculate deviation angle with center point displacement direction as reference;Non-perception risk quantization module is based on equipment body trajectory and personnel trajectory to calculate predicted approach rate, and fusion perception deviation angle and scene mask compensation factor, generate pre-judgment non-perception risk intensity index;Risk early warning decision module is used to sort personnel and equipment pairing relationship according to index, mark high-risk interaction event and output early warning record when trend slope breaks through dynamic early warning threshold.The application can realize more accurate risk identification and more timely early warning in complex man-machine cross operation scene such as port.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to a security risk identification and early warning system based on video behavioral characteristics. Background Technology

[0002] With the continuous development of intelligence and digitalization, video-based behavioral feature analysis technology has been widely applied in many fields, such as industrial production safety monitoring, high-risk operation risk assessment, and real-time security in public places. The development of these technologies benefits from the rapid progress of computer vision and deep learning algorithms. By detecting, recognizing, and analyzing the behavior of targets in images, real-time perception of personnel behavior, equipment status, and the risks of their interaction can be achieved. Currently, mainstream technical methods typically include modules such as object detection, object tracking, and action recognition, which are used to construct a deep analysis framework for behavioral features in complex scenarios. The specific implementation of these technologies largely relies on efficient object detection algorithms (such as YOLO and Faster R-CNN based on convolutional neural networks) and action recognition models, which can accurately extract and analyze the motion features and behavioral patterns of targets in videos. For safety risk analysis tasks, existing research has achieved preliminary early warning mechanisms based on abnormal behavior pattern monitoring and introduced motion trajectory prediction and collision risk assessment methods in some industrial fields. However, in complex interactive scenarios, traditional technologies still have significant technical shortcomings in handling the dynamic interaction risks between personnel and large equipment.

[0003] The shortcomings of existing technologies in identifying and warning of risks in personnel-equipment interactions are mainly manifested in the following aspects: First, existing research mainly relies on static assessment models for single-target risks, simply monitoring abnormal behaviors of equipment or personnel, making it difficult to comprehensively capture the collaborative interaction characteristics between equipment and personnel; Second, existing technologies typically assess the risk of interaction relationships based on a single parameter (such as the closest distance), lacking a comprehensive quantitative mechanism for multi-dimensional features, and ignoring the impact of the rigid and flexible motion characteristics of the equipment itself and the perception capabilities of personnel on the probability of risk occurrence; Third, traditional risk warning technologies generally adopt fixed threshold strategies, resulting in a lag in capturing and responding to the cumulative change trends of risks in complex dynamic interaction scenarios, which may lead to untimely warnings or even missed reports of high-risk events. These shortcomings mean that existing technologies still cannot provide reliable technical support for potential interaction risks in scenarios with significant dynamic characteristics (such as port terminals, construction sites, and collaborative operation scenarios of large equipment). Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a security risk identification and early warning system based on video behavioral characteristics. This system can, to some extent, solve the problems of the inability to accurately distinguish between the rigid and flexible motion characteristics of equipment, the lack of quantitative indicators for human perception capabilities, and the difficulty in timely avoiding high-risk interactive events due to delayed risk warnings.

[0005] According to one aspect of the present invention, a security risk identification and early warning system based on video behavioral characteristics is provided, comprising:

[0006] The device motion separation module is used to extract motion features from each structural region of the device detection box in the video frame sequence, and separate rigid and flexible motion components by the coupling difference of the motion features to obtain the motion trajectory of the device body.

[0007] The personnel perception deviation calculation module is used to extract the orientation vector of the facial region of the personnel detection box in the video frame sequence, and calculate the deviation angle with the displacement direction of the center point of the personnel detection box as a reference to obtain the personnel perception deviation angle.

[0008] The non-perceptible risk quantification module is used to calculate and predict the approach rate based on the motion trajectory of the device body and the trajectory of the center point of the personnel detection box, and combine the personnel perception deviation angle and the scene mask compensation factor to obtain a predictive non-perceptible risk intensity index.

[0009] The risk warning decision module is used to sort the personnel and equipment pairing relationships in descending order according to the predictive, imperceptible risk intensity index, trace the risk accumulation trend of the top-ranked pairing relationships, and mark it as a high-risk interaction event and output a warning record when the trend slope exceeds the dynamic warning threshold.

[0010] Furthermore, the device motion separation module includes:

[0011] The structural area division unit is used to vertically divide the inspection frame of the gantry crane into the gantry area, the boom area, and the lifting equipment area.

[0012] The coupling relationship verification unit is used to calculate the time-weighted correlation coefficient between each region, and to confirm the rigid coupling relationship and the flexible decoupling relationship based on the correlation coefficient and the standard deviation of the direction angle difference.

[0013] The motion component separation unit is used to perform a weighted average operation on the motion characteristics of the rigid coupling region to obtain the translational motion vector sequence of the equipment body, and to subtract the translational motion vector sequence from the overall displacement vector sequence of the detection frame to obtain the flexible motion component of the spreader.

[0014] Furthermore, in the coupling relationship verification unit, for the two regions to be analyzed... and Weighted time series correlation coefficient Defined as:

[0015]

[0016] in, For the region With the region Phase delay The time-weighted correlation coefficient under the following conditions For the first Temporal normalization weights of frames, For the region No. The magnitude of the motion vector of the frame. For the region No. ( ) Frame motion vector magnitude, For the set of valid frame indices, For the region The weighted average of the motion vector magnitude over the time window. It is the weighted average of the time-shifted sequence over the valid frames.

[0017] Furthermore, the personnel perception deviation calculation module includes:

[0018] The facial landmark localization unit is used to construct a refined facial landmark localization method that integrates sub-pixel-level peak fitting and multi-scale response weighting, and outputs refined localization coordinates.

[0019] An orientation vector construction unit is used to construct a unit vector from the midpoint of the line connecting the centers of the eyes as the facial reference origin, and to construct a unit vector pointing from the facial reference origin to the tip of the nose as the person's facial orientation vector.

[0020] The deviation angle calculation unit is used to calculate the difference between the direction angle of the vector pointing to the person's face and the direction angle of the vector of unit displacement to obtain the deviation angle perceived by the person.

[0021] Furthermore, in the facial key point localization unit, the first... Detailed positioning coordinates of key points The calculation formula is expressed as:

[0022]

[0023] in, For the first Detailed positioning coordinates of key points This represents the number of scale levels in the feature pyramid. For the first Response weighting coefficients at each scale level For the first The key point is in the first Integer pixel coordinates corresponding to the peak values ​​in the heatmap at each scale level. For the first Coordinate offset vectors obtained by sub-pixel fitting at each scale level.

[0024] Furthermore, the imperceptible risk quantification module includes:

[0025] The trajectory prediction unit is used to input the motion velocity vector, acceleration vector, and historical position coordinate sequence into a pre-trained LSTM trajectory prediction network to predict the future trajectory. The frame contains the coordinate vectors of the personnel or equipment positions.

[0026] The approach rate calculation unit is used to calculate the Euclidean distance between the predicted position coordinates of the personnel and the predicted position coordinates of the equipment to obtain the predicted distance. The predicted approach rate is obtained by subtracting the predicted distance from the current distance and dividing by the length of the future time window.

[0027] The perception attenuation modulation unit is used to calculate the normalized proportion by comparing the absolute value of the perception deviation angle of the person with the half-angle width of the horizontal field of vision of the human eyes. Based on the comparison result between the normalized proportion and the field of vision proportion boundary, the perception attenuation coefficient is calculated. The predicted approach rate is multiplied by the perception attenuation coefficient to obtain the predicted risk base value.

[0028] The scene mask compensation unit is used to calculate the scene mask compensation factor. The scene mask compensation factor is added by one and multiplied with the predicted risk base value to obtain the predictive imperceptible risk intensity index.

[0029] Furthermore, in the trajectory prediction unit, the future trajectory prediction is performed. Frame position coordinate vector The calculation formula is expressed as:

[0030]

[0031] in, For the predicted future The frame's position coordinate vector; This is the position coordinate vector of the current frame; The velocity vector is obtained by fitting the historical trajectory to the current frame. For the predicted frame number index; This refers to the inter-frame time interval. The time scale parameter is used to control the decay characteristics of the velocity term as the prediction duration; The acceleration vector is obtained by differentiating the historical trajectory in the current frame. The field strength coefficient of the equipment influence field; The device's position coordinate vector; The safe operating radius of the equipment; Output the weight matrix for the neural network; To predict the first The hidden state vector that is recursively updated in frames.

[0032] Furthermore, in the scene mask compensation unit, the scene mask compensation factor... The calculation formula is expressed as:

[0033]

[0034] in, As a scene mask compensation factor, To reveal the lead time, which is the difference between the length of the future time window and the expected arrival time, The length of the future time window. The area influence coefficient of the obstruction. The pixel area of ​​the occluding object's boundary region in the current video frame. The total pixel area of ​​the current video frame. is the base of the natural logarithm. This is the spatial distance sensitivity coefficient. The spatial distance from the current location of the person to the intersection of the boundary of the obstruction. This refers to the safe critical distance between the equipment and the boundary of the obstruction.

[0035] Compared with existing technologies, this invention significantly improves the accuracy of extracting equipment motion trajectory features by separating the differences between rigid and flexible motion components through the equipment motion separation module; it introduces personnel perception ability as an important factor in risk assessment for the first time through the quantitative analysis of personnel orientation vectors and deviation angles by the personnel perception deviation calculation module; it achieves accurate quantification of multi-dimensional interactive risks by integrating equipment motion trajectory, personnel perception deviation angles, and scene mask compensation factors through the non-perceptual risk quantification module; and it effectively solves the problem of delayed response to high-risk interactive events in traditional technologies through the dynamic trend analysis and dynamic warning threshold mechanism of the risk warning decision module. This invention enables more comprehensive and accurate risk identification and more timely risk warning in complex dynamic interactive scenarios, providing reliable technical support for reducing the interaction risks between personnel and equipment. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0037] Figure 1 A flowchart of a security risk identification and early warning system based on video behavior features according to an embodiment of the present invention;

[0038] Figure 2 This is a comparison chart showing the impact of the scene mask compensation factor on the risk intensity index according to an embodiment of the present invention. Detailed Implementation

[0039] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0040] As mentioned in the background section, existing technologies have three main problems: First, the inability to effectively distinguish between the rigid and flexible motion characteristics of equipment leads to insufficient accuracy in equipment trajectory modeling, making it difficult to fully quantify risk factors. Second, traditional technologies lack effective quantitative analysis of human perception capabilities, relying solely on location features for static risk assessment and ignoring the impact of subjective perception on interactive safety. Third, most existing risk warning technologies employ fixed threshold strategies, failing to capture the changing characteristics of risk accumulation trends in dynamic interactive scenarios, which can easily lead to missed reports or delayed warnings of high-risk events.

[0041] Figure 1 This is a system block diagram of a security risk identification and early warning system based on video behavior features according to an embodiment of the present invention. Figure 1 and Figure 2 As shown, the security risk identification and early warning system based on video behavior features includes: a device motion separation module, a personnel perception deviation calculation module, an imperceptible risk quantification module, and a risk early warning decision module.

[0042] The device motion separation module is used to extract motion features from each structural region of the device detection box in the video frame sequence, and separate rigid and flexible motion components by the coupling difference of the motion features to obtain the motion trajectory of the device body.

[0043] The device motion separation module includes a structural region division unit, a coupling relationship verification unit, and a motion component separation unit.

[0044] The structural region division unit is used to vertically divide the gantry crane detection frame into a gantry region, a boom region, and a lifting device region, specifically including:

[0045] Fixed surveillance cameras are deployed at key intersections in the port operation area, covering the intersection of the gantry crane's running track and personnel passageways. The cameras capture continuous video streams at a fixed frame rate, and image frames are extracted from the video stream in timestamp order to form a video frame sequence. Each frame in the video frame sequence carries timestamp information, and the time interval between adjacent frames is determined by the reciprocal of the camera's frame rate.

[0046] In the video frame sequence, the detection frame of the gantry crane covers the entire outline of the equipment, including three main areas: the gantry structure, the boom structure, and the spreader suspension system.

[0047] The gantry structure is located at the bottom of the detection frame and is fixed to the track, performing rigid translational motion. The boom structure is located in the upper part of the detection frame and is rigidly connected to the gantry. The spreader suspension system is located at the end of the boom and is flexibly connected by steel wire ropes, generating periodic oscillations due to load eccentricity and lateral wind forces. The detection frame is vertically divided into the bottom gantry area, the upper middle boom area, and the end spreader area, and pixel-level motion information of each area is extracted.

[0048] For each segmented structural region, feature point optical flow vector calculations are performed on the gantry region, boom region, and spreader region. For the gantry region in the current frame, the intersection of the bottom horizontal edge and the two vertical edges of the region is extracted as feature points; for the boom region, the skeletal nodes evenly distributed along the central axis of the region are extracted as feature points; for the spreader region, the connection points between the suspended wire rope and the hook, as well as the four corner vertices of the hook, are extracted as feature points.

[0049] After determining the feature point in the current frame, a search window is constructed within the corresponding structural region of the next adjacent frame, centered on the coordinates of the feature point in the current frame. Within the search window, the difference in grayscale distribution between the feature point and its neighborhood is calculated pixel by pixel. The pixel with the smallest difference is selected as the matching position for that feature point in the next frame. The difference between the pixel coordinates of the feature point in the current frame and the pixel coordinates of the matching position in the next frame is calculated to obtain the displacement vector of the feature point from the current frame to the next frame. The horizontal component of the displacement vector represents the number of pixels displaced laterally, and the vertical component represents the number of pixels displaced vertically. The average displacement vectors of all feature points within the same structural region are calculated using vector addition rules to obtain the overall motion vector of that region. The magnitude of the overall motion vector is obtained by taking the square root of the sum of the squares of the horizontal and vertical components, and the direction angle is obtained by taking the arctangent function of the vertical and horizontal components. The magnitude and direction angle of the overall motion vector are used as the motion characteristics of that structural region between frames.

[0050] The coupling relationship verification unit is used to calculate the time-weighted correlation coefficient between each region, and to confirm the rigid coupling relationship and the flexible decoupling relationship based on the correlation coefficient and the standard deviation of the direction angle difference. Specifically:

[0051] Based on prior knowledge of the crane's mechanical structure, a preliminary assessment of the connection relationships between the three areas is made: the gantry and boom are rigidly connected by bolts or welding, and should exhibit the same translational motion pattern, thus preliminarily determining a rigid connection; the spreader is suspended from the boom end by wire ropes, and its motion includes a swing component independent of the gantry and boom, thus preliminarily determining a flexible suspension decoupling relationship between the spreader and the gantry, and between the spreader and the boom. Based on this, a coupling difference quantification verification is performed on the preliminarily determined connection relationships to ultimately confirm the rigid-flexible coupling properties of each area.

[0052] Furthermore, correlation coefficients were calculated for the motion characteristics of the gantry region, the boom region, and the spreader region, including the correlation coefficients between the gantry and the boom, the spreader and the gantry, and the spreader and the boom.

[0053] First, for the two regions to be analyzed and Timing delay frame count Determine the set of valid frame indexes to participate in the calculation. ={ +1,…, }, number of valid frames = - For each valid frame ∈ The time-series normalized weights are defined as follows:

[0054]

[0055] Computational area Weighted mean and region of motion vector magnitude in valid frames The weighted average of the motion vector amplitudes after time shift is expressed by the formula:

[0056] ,

[0057] in, For the region No. Frame motion vector magnitude, For the region The weighted average of the motion vector magnitude over the time window. For the region No. ( ) Frame motion vector magnitude, It is the weighted average of the time-shifted sequence over the valid frames.

[0058] Then the weighted time series correlation coefficient Defined as:

[0059]

[0060] in, For the region With the region Phase delay The time-weighted correlation coefficient is calculated.

[0061] If the area With the region Time-weighted correlation coefficient between Greater than or equal to the preset high correlation threshold at zero phase delay Then, the standard deviation of the direction angle difference is further calculated, and the formula is expressed as:

[0062]

[0063] in, The weighted standard deviation of the direction angle differences. For the region No. Frame orientation angle and region No. The absolute value of the difference in frame orientation angles. It is the weighted average of the differences in direction angles.

[0064] Based on the initially determined connection type, the corresponding quantitative verification logic is executed: for the gantry and boom initially determined to be rigidly connected, if and Then it is verified that the two are rigidly connected and coupled; if or This indicates that the motion consistency does not meet the expectations of a rigid connection, and is marked as an abnormal state requiring recalibration. For the spreader and gantry, and the spreader and boom, which are initially determined to be in a flexible suspension decoupling relationship, if... Below the preset low relevance threshold If this is verified, it is confirmed that there is no rigid motion coupling between the two, and the relationship is confirmed as flexible decoupling.

[0065] It should be noted that the threshold The setup is as follows: For a sample of lifting equipment with a known rigid connection structure, collect motion vector data of the gantry and boom regions under normal operating conditions, and calculate the time-weighted correlation coefficient under zero phase delay. Statistical analysis was performed on the correlation coefficient values ​​of multiple sets of sample data, and the lower quantile of the correlation coefficient distribution was extracted as the judgment threshold. The lower quantile is the value at the 5th percentile after the sample correlation coefficient sequence is sorted in ascending order;

[0066] threshold The setup is as follows: For a sample of lifting equipment with a known rigid connection structure, collect motion vector direction angle data of the gantry and boom regions under the same operating conditions, and calculate the weighted standard deviation of the direction angle difference. Statistical analysis was performed on the standard deviation values ​​of multiple sets of sample data, and the upper quantile of the standard deviation distribution was extracted as the judgment threshold. The upper quantile is the value located at the 95th percentile position after the sample standard deviation sequence is arranged in ascending order.

[0067] Low correlation threshold The setup is as follows: For a sample of lifting equipment with a known flexible suspension structure, under normal operating conditions, collect motion vector data of the spreader region, gantry region, and boom region, and calculate the time-weighted correlation coefficient under zero phase delay. The upper quantile of the correlation coefficient distribution is extracted as the decision threshold. The upper quantile is the value at the 10th percentile position after the sample correlation coefficient sequence is arranged in ascending order.

[0068] Through the above quantitative verification, it was confirmed that there is a rigid connection between the gantry and the boom, and a flexible suspension decoupling relationship between the spreader and the gantry, and between the spreader and the boom.

[0069] The motion component separation unit is used to perform a weighted average operation on the motion characteristics of the rigid coupling region to obtain the translational motion vector sequence of the equipment body, and to subtract the translational motion vector sequence from the overall displacement vector sequence of the detection frame to obtain the flexible motion component of the spreader. Specifically:

[0070] Based on the acquired rigid coupling motion feature group, a weighted average operation is performed. The motion features of the gantry area and the boom area are assigned weight coefficients according to the pixel area ratio of their respective regions within the detection box. The motion vectors of the gantry area and the boom area are weighted and summed to obtain the translational motion vector sequence of the equipment body. The translational motion vector sequence reflects the actual movement speed and direction of the rigid whole of the gantry and boom along the track.

[0071] Simultaneously, the displacement of the center point of the device detection frame between adjacent frames is extracted as the displacement vector sequence of the entire detection frame. Considering the flexible decoupled motion characteristics of the spreader area, the translational motion vector sequence of the device body is subtracted from the overall displacement vector sequence of the detection frame to obtain the flexible motion component purely contributed by the spreader's oscillation. This flexible motion component is isolated as an independent periodic oscillation signal and does not participate in the cumulative calculation of the device body's motion trajectory.

[0072] The separated translational motion vector sequence of the equipment body is obtained, and a temporal accumulation operation is performed on the translational motion vector sequence. Using the center coordinates of the equipment detection box corresponding to the first frame of the video frame sequence as the trajectory starting point, the translational motion vectors of each frame are sequentially superimposed in a two-dimensional spatial coordinate system. After each superposition, the spatial position coordinates of the equipment body in the current frame are updated. Connecting the spatial position coordinates corresponding to all frames forms a continuous motion trajectory of the equipment body in the port operation area. The acquired motion trajectory eliminates the interference of the periodic oscillation of the spreader on displacement determination, providing a clean reference benchmark for calculating the actual approach speed between personnel and equipment.

[0073] The personnel perception deviation calculation module is used to extract the orientation vector of the facial region of the personnel detection box in the video frame sequence, and calculate the deviation angle with the displacement direction of the center point of the personnel detection box as a reference to obtain the personnel perception deviation angle.

[0074] The personnel perception deviation calculation module includes a facial key point localization unit, an orientation vector construction unit, and a deviation angle calculation unit.

[0075] The facial key point localization unit is used to construct a refined facial key point localization method that integrates sub-pixel-level peak fitting and multi-scale response weighting, and outputs refined localization coordinates, including:

[0076] Based on the acquired video frame sequence, an orientation vector extraction operation is performed on the facial region within the person detection box. For the person detection box, the center points of the eyes, the tip of the nose, and the center point of the chin are located within the detection box using a facial key point detection algorithm. The midpoint of the line connecting the centers of the eyes is taken as the facial reference origin, and the direction of the line connecting the tip of the nose relative to the facial reference origin is defined as the reference direction for the front of the face. Thus, a unit vector pointing from the facial reference origin to the tip of the nose is constructed as the person's facial orientation vector.

[0077] In particular, to address the issue of uneven lighting, angle deviation, and thermal image peak position shift in port operation scenarios, which are caused by long-distance imaging, a refined facial key point localization method is constructed that integrates sub-pixel-level peak fitting and multi-scale response weighting. The formula is expressed as:

[0078]

[0079] in, For the first Detailed positioning coordinates of key points This represents the number of scale levels in the feature pyramid. For the first Response weighting coefficients at each scale level For the first The key point is in the first Integer pixel coordinates corresponding to the peak values ​​in the heatmap at each scale level. For the first Coordinate offset vectors obtained by sub-pixel fitting at each scale level.

[0080] The formula for calculating the response weighting coefficient is as follows:

[0081] in, For the first The key point is in the first Thermographic response intensity at peak locations at each scale level For the first The key point is in the first Standard deviation of heatmap response in the neighborhood of peak location at each scale level For the first The key point is in the first Standard deviation of heatmap response in the neighborhood of peak location at each scale level This is the standard deviation penalty strength parameter.

[0082] The sub-pixel coordinate offset vector is calculated by fitting a quadratic surface to the peak neighborhood, as expressed by the formula:

[0083]

[0084] in, For the first The key point is in the first The inverse of the Hessian matrix of the heatmap response to the peak locations at each scale level. For the first The key point is in the first The heatmap response gradient vector at each scale level peak location.

[0085] Specifically, the positioning process includes:

[0086] Based on the facial regions within the acquired person detection bounding boxes, the cropped facial images are first adjusted to the network input size and pixel values ​​are normalized. Then, they are input into the backbone convolutional network to extract multi-level feature maps and construct... Layer feature pyramid;

[0087] An independent keypoint detection head network is connected to each level of the feature pyramid to generate heatmaps corresponding to the center points of the eyes, the tip of the nose, and the center point of the chin. The response value of each pixel in the heatmap represents the probability density of the presence of the target keypoint at that location.

[0088] against For each keypoint heatmap at each scale level, the integer pixel coordinates corresponding to the maximum heatmap response value are extracted through global maximum search. At this time, Preliminary coordinate detection by the group;

[0089] The gradient vector is then calculated using the central difference method within the neighborhood of integer coordinates. and Hessian matrix Substitute the gradient vector and the inverse Hessian matrix into the subpixel offset correction formula to calculate the subpixel offset vector. Integer coordinates With subpixel offset The coordinates are added together to obtain the refined coordinates for that scale level.

[0090] For each scale level Calculate the standard deviation of the heatmap response values ​​within the integer coordinate neighborhood. And read the peak intensity Substitute the peak intensity and standard deviation into the multi-scale response weighting coefficient formula to calculate the response weighting coefficient for that scale level. ;

[0091] Will Refined coordinates at each scale level are weighted by response coefficients. Substituting the values ​​into the keypoint fine-grained localization fusion formula and performing a weighted summation, we obtain the final keypoint localization coordinates. .

[0092] The orientation vector construction unit is used to construct a unit vector pointing from the facial reference origin to the tip of the nose as the person's facial orientation vector, taking the midpoint of the line connecting the centers of the eyes as the facial reference origin. This includes:

[0093] The facial reference origin is obtained by averaging the center points of the left and right eyes in the positioning coordinates. The direction vector is obtained by subtracting the coordinates of the facial reference origin from the coordinates of the nose tip. The direction vector is then normalized using Euclidean distance to obtain the unit orientation vector. The orientation vector has horizontal and vertical components in the two-dimensional image plane. The horizontal component represents the degree of orientation deflection of the face in the left and right directions, and the vertical component represents the degree of orientation deflection of the face in the pitch direction. The orientation angle of the orientation vector is taken as the orientation angle of the person's face in the current frame.

[0094] The deviation angle calculation unit is used to calculate the difference between the direction angle of the person's face orientation vector and the direction angle of the unit displacement direction vector to obtain the person's perceived deviation angle, including:

[0095] Based on the acquired facial orientation vector of the person, the center point position of the person detection box is tracked and calculated between consecutive video frames. The difference between the center point coordinates of the current frame and the previous frame is calculated to obtain the displacement vector of the person's center point from the previous frame to the current frame. The horizontal component of the displacement vector represents the number of pixels the person moves horizontally, and the vertical component represents the number of pixels the person moves vertically. The displacement vector is normalized to obtain the unit displacement direction vector. The direction angle of this unit displacement direction vector is used as the person's movement direction angle, which represents the actual direction of movement of the person within the port operation area.

[0096] Furthermore, based on the acquired facial orientation vector and the displacement direction vector of the center point of the personnel detection box, the angle between the two vectors is calculated as the personnel perception deviation angle. The difference between the direction angle of the facial orientation vector and the personnel movement direction angle is calculated to obtain the deviation angle of the facial orientation relative to the movement direction. The positive or negative sign of the deviation angle indicates whether the facial orientation is biased to the left or right of the movement direction, and the absolute value of the deviation angle indicates the magnitude of the angle between the facial orientation and the movement direction.

[0097] If the deviation angle is close to zero degrees, it means that the person's face is facing the same direction of movement, the person is facing their own direction of movement, and their visual attention is focused on the front of the path of movement.

[0098] If the deviation angle is far from zero degrees, it indicates that there is a significant deviation between the person's facial orientation and the direction of movement, and that the person's visual attention is not focused on the path ahead, which may indicate a blind spot in the perception of obstacles or equipment movement ahead.

[0099] The deviation angle is defined as the personnel perception deviation angle, which quantifies the degree of attenuation of a person's ability to perceive potential dangers in their direction of travel.

[0100] The non-perceptible risk quantification module is used to calculate and predict the approach rate based on the motion trajectory of the device body and the trajectory of the center point of the personnel detection box. Combined with the personnel perception deviation angle and the scene mask compensation factor, a predictive non-perceptible risk intensity index is obtained.

[0101] The imperceptible risk quantification module includes a trajectory prediction unit, a proximity rate calculation unit, a perception attenuation modulation unit, and a scene mask compensation unit.

[0102] The trajectory prediction unit is used to input the motion velocity vector, acceleration vector, and historical position coordinate sequence into a pre-trained LSTM trajectory prediction network to predict the future trajectory. The frame contains the coordinate vectors of the personnel or equipment positions, including:

[0103] Based on the acquired motion trajectory of the device itself and the trajectory of the center point of the personnel detection box, future state prediction calculations are performed respectively.

[0104] For the trajectory of the center point of the personnel detection box, the personnel position coordinate sequence in the most recent consecutive frames is extracted. The position coordinate sequence is linearly fitted in the horizontal and vertical directions to obtain the velocity components of the personnel in the horizontal and vertical directions. The two velocity components are combined to form the personnel's current motion velocity vector.

[0105] For the motion trajectory of the device body, extract the device position coordinate sequence within the most recent consecutive frames, and perform the same linear fitting operation on the position coordinate sequence to obtain the motion velocity vector of the device at the current moment.

[0106] Furthermore, the position coordinate sequence of several historical frames of the person is extracted, and a second-order central difference operation is performed on the sequence to obtain the acceleration vector of the person at the current moment. Simultaneously, the safe operating radius corresponding to the equipment is queried from the equipment type database. The motion velocity vector, acceleration vector, and historical position coordinate sequence are input into a pre-trained LSTM trajectory prediction network. The historical trajectory features are encoded through the gating mechanism of the LSTM units to generate the hidden state vector of the current frame. .

[0107] When forecasters need to predict the future... When the frame position is determined, the current frame's state is hidden. As the starting point of recursion, the prediction steps range from 1 to... The process unfolds gradually, and in each recursive update step, the hidden state from the previous step and the motion velocity vector are compared. and acceleration vector After splicing, the data is fed into an LSTM unit to generate a new hidden state. After the recursive update, the hidden state is obtained. And by outputting the weight matrix The mapping is used as a neural network position correction term.

[0108] Based on the above parameters, a trajectory prediction network model is constructed to predict the future trajectory. The frame represents the coordinate vector of the personnel or equipment position, expressed by the formula:

[0109]

[0110] in, For the predicted future The frame's position coordinate vector; This is the position coordinate vector of the current frame; The velocity vector is obtained by fitting the historical trajectory to the current frame. For the predicted frame number index; This refers to the inter-frame time interval. The time scale parameter is used to control the decay characteristics of the velocity term as the prediction duration; The acceleration vector is obtained by differentiating the historical trajectory in the current frame. The field strength coefficient of the equipment influence field; The device's position coordinate vector; The safe operating radius of the equipment; Output the weight matrix for the neural network; To predict the first The hidden state vector that is recursively updated in frames.

[0111] It should be noted that the LSTM trajectory prediction network is trained by collecting historical real trajectory samples of personnel and equipment within the port operation area, using continuous previous data... Frame trajectory segments are used as network input and post-processing. Frame trajectory segments are used as output labels to construct the training dataset; the mean squared error loss function is employed, and the gating parameters and output weight matrix of the LSTM unit are adjusted using a time-based backpropagation algorithm. Perform offline training until the loss function value converges to a preset threshold to obtain a pre-trained trajectory prediction network.

[0112] The approach rate calculation unit is used to calculate the Euclidean distance between the predicted position coordinates of the person and the predicted position coordinates of the equipment to obtain the predicted distance, and to subtract the predicted distance from the current distance and divide by the length of the future time window to obtain the predicted approach rate, including:

[0113] Multiplying the device's motion velocity vector by the length of the future time window and adding it to the device's position coordinates in the current frame yields the predicted position coordinates of the device at the end of the future time window. ;

[0114] Calculators predict location coordinates With the predicted location coordinates of the equipment The Euclidean distance between them is used to obtain the predicted distance at the end of the future time window, and the coordinates of the personnel position in the current frame are calculated. With current frame device position coordinates The current distance is obtained by the Euclidean distance between them. The predicted approach rate is obtained by subtracting the predicted distance from the current distance and dividing by the length of the future time window.

[0115] A positive predicted approach rate indicates that the spatial distance between people and equipment will decrease within the future time window, while a negative predicted approach rate indicates that the spatial distance between people and equipment will increase within the future time window.

[0116] The perception attenuation modulation unit is used to calculate the normalized proportion by comparing the absolute value of the person's perception deviation angle with the half-angle width of the horizontal visual field of the human eyes, calculate the perception attenuation coefficient based on the comparison result between the normalized proportion and the visual field proportion boundary, and multiply the predicted approach rate with the perception attenuation coefficient to obtain the predicted risk baseline value, including:

[0117] The modulation effect of the human perception bias angle on the predicted approach rate is specifically:

[0118] First, the absolute value of the perceived deviation angle is compared to the half-angle width of the horizontal visual field of the human eyes to obtain the normalized proportion of the deviation angle relative to the visual field boundary. The half-angle width of the horizontal visual field of the human eyes is a fixed angle parameter determined by the human physiological structure. When the value of the normalized proportion is less than the value of the central visual field proportion boundary, it indicates that the person's face is still within the central visual field, and the value of the normalized proportion is directly used as the perception attenuation coefficient. When the value of the normalized proportion is between the central visual field proportion boundary and the peripheral visual field proportion boundary, it indicates that the person's face is entering the peripheral visual field. In this case, the value of the normalized proportion is multiplied by itself to obtain the square value, and the square value is used as the perception attenuation coefficient. The squaring operation causes the increase of the normalized proportion to be non-linearly amplified within the peripheral visual field. When the normalized percentage exceeds the boundary of the peripheral field of vision, it indicates that the person's face orientation has deviated from the normal field of vision. At this time, no further calculation is performed on the normalized percentage. Instead, the square value corresponding to the boundary of the peripheral field of vision is directly used as the saturation upper limit of the perception attenuation coefficient. The value obtained after normalization and segmented calculation is used as the perception attenuation coefficient of the person's proximity risk.

[0119] Next, the predicted approach rate is multiplied by the perception attenuation coefficient to obtain the basic value of the predicted risk considering the impact of blind spots in human perception.

[0120] The scene mask compensation unit is used to calculate the scene mask compensation factor. The scene mask compensation factor is incremented by one and multiplied by the predicted risk baseline value to obtain a predictive, imperceptible risk intensity index. Specifically:

[0121] If risk indicators are calculated solely based on the current approach speed of personnel and equipment and the current facial orientation of personnel, it is impossible to capture the special dangerous scenario where personnel are about to be suddenly exposed from behind an obstruction into the field of vision of the equipment operator. When personnel are moving behind an obstruction, the equipment operator's line of sight is blocked by the obstruction, making it impossible to observe the personnel's approach behavior in advance. When personnel move to the boundary of the obstruction and are about to be exposed, a new moving target will suddenly appear in the field of vision of the equipment operator. This sudden change in line of sight will require the equipment operator to have additional time to re-establish perception of the spatial environment and judge the movement status and collision risk of the newly appearing target. Compared with the sudden exposure scenario caused by the disappearance of the obstruction during the approach process when personnel are always in a visible state, this has a higher actual danger, but this potential danger cannot be quantified and identified in the risk indicators calculated based on the current state.

[0122] The scene masking compensation factor is introduced to calculate the estimated arrival time of a person moving from the current position to the boundary of the occlusion. It determines whether the person will change from an occluded state to an exposed state within a future time window. Based on the proportion of the exposure lead time in the predicted time window and the scale of the occlusion in the image, the basic risk indicators are amplified and modulated. This allows the pairing of personnel and equipment that are about to be exposed to obtain a higher risk score, so that they can be prioritized in subsequent risk ranking and early warning decisions and trigger early warnings.

[0123] The scene masking compensation factor is constructed based on three core dimensions of the risk of sudden exposure caused by the disappearance of obstructions: the urgency of the exposure time, the intensity of visual interference from the obstruction, and the spatial proximity of people. The formula is expressed as:

[0124]

[0125] in, As a scene mask compensation factor, To reveal the lead time, which is the difference between the length of the future time window and the expected arrival time, The length of the future time window. The area influence coefficient of the obstruction. The pixel area of ​​the occluding object's boundary region in the current video frame. The total pixel area of ​​the current video frame. is the base of the natural logarithm. This is the spatial distance sensitivity coefficient. The spatial distance from the current location of the person to the intersection of the boundary of the obstruction. This refers to the safe critical distance between the equipment and the boundary of the obstruction.

[0126] Based on the constructed scene mask compensation factor, the scene mask compensation factor is increased by one and used as the amplification factor. This factor is then multiplied with the predicted risk base value to obtain the modulated comprehensive risk score.

[0127] When the scene mask compensation factor is equal to zero, that is, when the extended line of the personnel movement trajectory does not intersect with the boundary area of ​​the obstruction or the expected arrival time exceeds the length of the future time window, the magnification factor is equal to one, so that the comprehensive risk score is equal to the predicted risk base value, keeping the original value of the basic risk quantification unchanged.

[0128] When the scene masking compensation factor is greater than zero, meaning that a person will be exposed from behind an obstruction to the equipment operator's field of vision within a future time window, the magnification factor is greater than one, making the comprehensive risk score equal to the predicted risk base value multiplied by this magnification factor. The larger the value of the scene masking compensation factor, the more urgent the exposure time, the stronger the visual interference from the obstruction, or the closer the person is to the boundary of the obstruction. The larger the magnification factor, the greater the increase in the comprehensive risk score relative to the predicted risk base value. When the comprehensive risk scores of all personnel and equipment pairings are sorted in descending order, pairings with a sudden risk of exposure to an obstruction receive higher values ​​and rank higher because their predicted risk base values ​​are magnified by the magnification factor. Thus, under the condition of limited early warning resources, the early warning response mechanism for this pairing is triggered first.

[0129] like Figure 2As shown, to verify the actual effect of scene masking compensation factor on improving risk intensity index, typical scene video sequences of personnel approaching a running gantry crane from behind a container stack in a port operation area were collected, and the trend of risk intensity index changes before and after the introduction of scene masking compensation was compared and analyzed. If the personnel are located behind an obstruction at frame 0, outside the operating radius of the equipment, and then move towards the equipment at a constant speed along the edge of the obstruction, the personnel will be behind the obstruction and about to be exposed to the equipment's field of vision between frames 30 and 50. At this time, the influence of the obstruction reaches its peak, and the amplification effect of scene masking compensation factor on the predicted risk baseline value is most significant. After introducing scene masking compensation, the risk intensity index reaches 0.92 at frame 40, exceeding the preset warning threshold of 0.8. This is 40 frames earlier than the situation where the threshold of 0.85 is only reached at frame 80 without compensation, providing sufficient response time window for on-site personnel to avoid the obstruction and for equipment to decelerate. The comparison curves show that the risk index curve without scene mask compensation has a gentle upward trend, and the warning condition is only reached when personnel approach the boundary of the equipment's operating range. However, the risk index curve with compensation shows a significant increase in slope in the frame interval where the influence of obstructions is high. This change characteristic has a strong correlation with the obstruction effect of obstructions on personnel's perception ability. This verifies that the scene mask compensation mechanism can effectively quantify the amplification effect of environmental obstruction factors on collision risk and significantly improve the warning lead time and reliability of the system in complex port scenarios.

[0130] The risk warning decision module is used to sort the personnel and equipment pairing relationships in descending order according to the predictive non-perceptible risk intensity index, trace back the risk accumulation trend of the top-ranked pairing relationships, and mark it as a high-risk interaction event when the trend slope exceeds the dynamic warning threshold and output the warning record.

[0131] The pairing relationships are sorted in descending order according to the numerical value of the predictive, imperceptible risk intensity index. After sorting, the first 20% of the pairing relationships are selected as the focus of analysis.

[0132] Read the historical values ​​of the predictive, imperceptible risk intensity index of the key pairing relationship in the most recent ten frames from the video frame history cache, and arrange the historical values ​​from the oldest to the newest frame number to form a time series containing ten elements.

[0133] The time series is subjected to the difference operation of adjacent elements. The first difference is obtained by subtracting the value of the first frame from the value of the second frame, the second difference is obtained by subtracting the value of the second frame from the value of the third frame, and so on, to obtain a difference sequence consisting of nine differences.

[0134] Divide each difference in the difference sequence by the reciprocal of the video frame rate, i.e., the single frame time interval, to obtain nine unit time change rate values. Sum these nine unit time change rate values ​​and divide by nine to obtain the slope of the cumulative risk trend of the pairing relationship within a continuous time window.

[0135] Based on the obtained risk accumulation trend slope, the motion speed value of the device body in the current video frame is read, and the motion speed value is divided by the preset speed reference value to obtain the speed adjustment coefficient.

[0136] Read the total number of person detection boxes detected in the current video frame, divide the total number by the preset density baseline value, and obtain the density adjustment coefficient.

[0137] Divide the preset static baseline threshold by the speed adjustment coefficient to obtain the intermediate threshold after speed adjustment. Divide the static baseline threshold by the density adjustment coefficient to obtain the intermediate threshold after density adjustment. Perform a minimum value operation on the two intermediate thresholds and use the smaller intermediate threshold as the dynamic warning threshold.

[0138] If the speed adjustment coefficient is zero, the corresponding intermediate threshold after speed adjustment is set to infinity and is not included in the subsequent minimum value calculation; if the density adjustment coefficient is zero, the corresponding intermediate threshold after density adjustment is set to infinity and is not included in the subsequent minimum value calculation. The speed benchmark value is the average moving speed of equipment during normal operation in the port operation area, and the density benchmark value is the historical average number of personnel detection frames during normal operation periods in the port operation area. The static benchmark threshold is set by the lower quantile of the slope distribution of offline statistical normal risk event samples.

[0139] The value of the risk accumulation trend slope is subtracted from the value of the dynamic early warning threshold. When the result of the subtraction is positive, it means that the slope value is greater than the dynamic early warning threshold value. At this time, the high-risk marker field of the pairing relationship is set to true. Otherwise, the high-risk marker field of the pairing relationship is set to false.

[0140] If the high-risk marker field of the pairing relationship is true, then the predictive imperceptible risk intensity index value and the risk accumulation trend slope value of the pairing relationship are read. The predictive imperceptible risk intensity index value is subtracted from the first indicator boundary value. The positive or negative result of the subtraction operation is used as the first judgment condition. The predictive imperceptible risk intensity index value is subtracted from the second indicator boundary value. The positive or negative result of the subtraction operation is used as the second judgment condition, wherein the first indicator boundary value is greater than the second indicator boundary value.

[0141] The risk accumulation trend slope value is subtracted from the first slope boundary value, and the sign of the subtraction result is used as the third judgment condition. The risk accumulation trend slope value is subtracted from the second slope boundary value, and the sign of the subtraction result is used as the fourth judgment condition, wherein the first slope boundary value is greater than the second slope boundary value.

[0142] If the subtraction result of the first and third judgment conditions is positive, the prediction level field of the event is assigned the integer one.

[0143] If only one of the subtraction operations in the first or third judgment condition is positive and the other is negative, the prediction level field of the event is assigned the integer two.

[0144] If the subtraction results of the first and third judgment conditions are both negative, but the subtraction results of the second and fourth judgment conditions are both positive, then the prediction level field of the event is assigned the integer three.

[0145] If none of the above assignment conditions are met, the prediction level field of the event will be assigned the integer zero, indicating that the current risk level of the pairing relationship has not reached the warning concern standard, and no risk warning record will be generated.

[0146] A risk warning record data structure containing six fields is generated. The unique identifier of the personnel detection box is written into the first field, the unique identifier of the device body is written into the second field, the value of the predictive non-perceptible risk intensity index is written into the third field, the value of the risk accumulation trend slope is written into the fourth field, the integer code of the predicted level is written into the fifth field, and the timestamp of the current video frame is written into the sixth field. The data structure is transmitted to the designated port of the warning display terminal via TCP protocol or written into the warning record table of the risk event database via SQL insert statement.

[0147] In summary, the video-based behavioral feature-based security risk identification and early warning system based on the embodiments of this invention has been clarified. It significantly improves the accuracy of extracting device motion trajectory features by separating the differences between rigid and flexible motion components through a device motion separation module; it introduces human perception capability as an important factor in risk assessment for the first time through quantitative analysis of human orientation vectors and deviation angles via a human perception deviation calculation module; it achieves accurate quantification of multi-dimensional interactive risks by integrating device motion trajectory, human perception deviation angles, and scene mask compensation factors through a non-perceptual risk quantification module; and it effectively solves the problem of delayed response to high-risk interactive events in traditional technologies through dynamic trend analysis and dynamic early warning threshold mechanisms in the risk early warning decision module. This system enables more comprehensive and accurate risk identification and more timely risk early warning in complex dynamic interactive scenarios, providing reliable technical support for reducing the interaction risks between personnel and equipment.

Claims

1. A security risk identification and early warning system based on video behavioral characteristics, characterized in that, include: The device motion separation module is used to extract motion features from each structural region of the device detection box in the video frame sequence, and separate rigid and flexible motion components by the coupling difference of the motion features to obtain the motion trajectory of the device body. The personnel perception deviation calculation module is used to extract the orientation vector of the facial region of the personnel detection box in the video frame sequence, and calculate the deviation angle with the displacement direction of the center point of the personnel detection box as a reference to obtain the personnel perception deviation angle. The non-perceptible risk quantification module is used to calculate and predict the approach rate based on the motion trajectory of the device body and the trajectory of the center point of the personnel detection box, and combine the personnel perception deviation angle and the scene mask compensation factor to obtain a predictive non-perceptible risk intensity index. The risk warning decision module is used to sort the personnel and equipment pairing relationships in descending order according to the predictive, imperceptible risk intensity index, trace the risk accumulation trend of the top-ranked pairing relationships, and mark it as a high-risk interaction event and output a warning record when the trend slope exceeds the dynamic warning threshold.

2. The security risk identification and early warning system based on video behavior features according to claim 1, characterized in that, The device motion separation module includes: The structural area division unit is used to vertically divide the inspection frame of the gantry crane into the gantry area, the boom area, and the lifting equipment area. The coupling relationship verification unit is used to calculate the time-weighted correlation coefficient between each region, and to confirm the rigid coupling relationship and the flexible decoupling relationship based on the correlation coefficient and the standard deviation of the direction angle difference. The motion component separation unit is used to perform a weighted average operation on the motion characteristics of the rigid coupling region to obtain the translational motion vector sequence of the equipment body, and to subtract the translational motion vector sequence from the overall displacement vector sequence of the detection frame to obtain the flexible motion component of the spreader.

3. The security risk identification and early warning system based on video behavior features according to claim 2, characterized in that, In the coupling relationship verification unit, for the two regions to be analyzed and Weighted time series correlation coefficient Defined as: in, For the region With the region Phase delay The time-weighted correlation coefficient under the following conditions For the first Temporal normalization weights of frames, For the region No. The magnitude of the motion vector of the frame. For the region No. ( ) Frame motion vector magnitude, For the set of valid frame indices, For the region The weighted average of the motion vector magnitude over the time window. It is the weighted average of the time-shifted sequence over the valid frames.

4. The security risk identification and early warning system based on video behavior features according to claim 3, characterized in that, The personnel perception bias calculation module includes: The facial landmark localization unit is used to construct a refined facial landmark localization method that integrates sub-pixel-level peak fitting and multi-scale response weighting, and outputs refined localization coordinates. An orientation vector construction unit is used to construct a unit vector from the midpoint of the line connecting the centers of the eyes as the facial reference origin, and to construct a unit vector pointing from the facial reference origin to the tip of the nose as the person's facial orientation vector. The deviation angle calculation unit is used to calculate the difference between the direction angle of the vector pointing to the person's face and the direction angle of the vector of unit displacement to obtain the deviation angle perceived by the person.

5. The security risk identification and early warning system based on video behavior features according to claim 4, characterized in that, In the facial key point localization unit, the first Detailed positioning coordinates of key points The calculation formula is expressed as: in, For the first Detailed positioning coordinates of key points This represents the number of scale levels in the feature pyramid. For the first Response weighting coefficients at each scale level For the first The key point is in the first Integer pixel coordinates corresponding to the peak values ​​in the heatmap at each scale level. For the first Coordinate offset vectors obtained by sub-pixel fitting at each scale level.

6. The security risk identification and early warning system based on video behavior features according to claim 5, characterized in that, The imperceptible risk quantification module includes: The trajectory prediction unit is used to input the motion velocity vector, acceleration vector, and historical position coordinate sequence into a pre-trained LSTM trajectory prediction network to predict the future trajectory. The frame contains the coordinate vectors of the personnel or equipment positions. The approach rate calculation unit is used to calculate the Euclidean distance between the predicted position coordinates of the personnel and the predicted position coordinates of the equipment to obtain the predicted distance. The predicted approach rate is obtained by subtracting the predicted distance from the current distance and dividing by the length of the future time window. The perception attenuation modulation unit is used to calculate the normalized proportion by comparing the absolute value of the perception deviation angle of the person with the half-angle width of the horizontal field of vision of the human eyes. Based on the comparison result between the normalized proportion and the boundary of the field of vision proportion, the perception attenuation coefficient is calculated. The predicted approach rate is multiplied by the perception attenuation coefficient to obtain the predicted risk base value. The scene mask compensation unit is used to calculate the scene mask compensation factor. The scene mask compensation factor is added by one and multiplied with the predicted risk base value to obtain the predictive imperceptible risk intensity index.

7. The security risk identification and early warning system based on video behavior features according to claim 6, characterized in that, In the trajectory prediction unit, the future trajectory prediction is performed. Frame position coordinate vector The calculation formula is expressed as: in, For the predicted future The frame's position coordinate vector; This is the position coordinate vector of the current frame; The velocity vector is obtained by fitting the historical trajectory to the current frame. For the predicted frame number index; This refers to the inter-frame time interval. The time scale parameter is used to control the decay characteristics of the velocity term as the prediction duration; The acceleration vector is obtained by differentiating the historical trajectory in the current frame. The field strength coefficient of the equipment influence field; The device's position coordinate vector; The safe operating radius of the equipment; Output the weight matrix for the neural network; To predict the first The hidden state vector that is recursively updated in frames.

8. The security risk identification and early warning system based on video behavior features according to claim 7, characterized in that, In the scene mask compensation unit, the scene mask compensation factor The calculation formula is expressed as: in, As a scene mask compensation factor, To reveal the lead time, which is the difference between the length of the future time window and the expected arrival time, The length of the future time window. The area influence coefficient of the obstruction. The pixel area of ​​the occluding object's boundary region in the current video frame. The total pixel area of ​​the current video frame. is the base of the natural logarithm. This is the spatial distance sensitivity coefficient. The spatial distance from the current location of the person to the intersection of the boundary of the obstruction. This refers to the safe critical distance between the equipment and the boundary of the obstruction.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the security risk identification and early warning system based on video behavior features as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the security risk identification and early warning system based on video behavior features as described in any one of claims 1 to 8.

Images