A job site safety supervision method based on multi-target tracking

Abstract

This application relates to the field of on-site work supervision technology, and in particular to a method for on-site safety supervision based on multi-target tracking. The method includes: collecting video surveillance data of workers; preprocessing the video surveillance data to obtain image frame data; processing the image frame data using the DeepSORT multi-target tracking algorithm to generate trajectory data for the workers; analyzing the image frame data based on a computer vision model to estimate the workers' attention data; fusing the trajectory data and attention data to construct a behavior vector for the workers; determining the workers' intention state by analyzing the behavior vector; when the intention state is determined to be driven by curiosity, checking the workers' permissions; if they lack permissions, it is confirmed as unauthorized entry, triggering a high-level alarm; if they have permissions, a low-level alert is triggered. This solves the problems of delayed warnings, lack of intent understanding, and limited supervision dimensions in existing technologies.

Description

Technical Field

[0001] This application relates to the field of work site supervision technology, and in particular to a work site safety supervision method based on multi-target tracking. Background Technology

[0002] Industrial production and construction sites are characterized by frequent personnel movement, dense equipment layout, and wide distribution of hazardous areas, making safety accident prevention a core pain point in industry management. Traditional manual inspection and supervision models suffer from inefficiency, limited coverage, and delayed response, making it difficult to achieve real-time, comprehensive safety monitoring of large-scale work sites.

[0003] With the development of computer vision technology, intelligent supervision methods based on video surveillance are gradually being applied. Among them, multi-target tracking technology has become a research hotspot in this field due to its ability to continuously locate and identify multiple workers. Existing supervision schemes based on multi-target tracking mainly follow two technical paths: one is intrusion detection based on electronic fences. This method uses virtual boundaries, i.e., electronic fences, preset in video footage or on a map. When the multi-target tracking system detects personnel entering unauthorized danger zones, it triggers an alarm. However, this method is essentially a passive, reactive response mechanism. It only intervenes after personnel have entered the danger zone and cannot provide early warning before risks occur. The other method is violation detection based on simple behavior recognition: This type of method, based on tracking, attempts to make safety judgments by recognizing specific static behaviors (such as whether a safety helmet is worn or whether smoking) or simple dynamic behaviors (such as running or falling). Although some progress has been made, its supervision dimension is relatively singular, usually limited to detecting obvious, already occurred violations, and unable to deeply understand the behavioral intentions and cognitive states of the workers. Summary of the Invention

[0004] This application provides a work site safety supervision method based on multi-target tracking to solve problems such as delayed early warning, lack of intent understanding, and single supervision dimension in the existing technology.

[0005] The first aspect of this application provides a method for on-site safety supervision based on multi-target tracking, comprising the following steps: deploying high-definition network cameras at the work site to collect video monitoring data of workers in real time; preprocessing the video monitoring data to obtain preprocessed image frame data; processing the preprocessed image frame data using the DeepSORT multi-target tracking algorithm to generate trajectory data for each detected worker, the trajectory data including position, movement speed, and direction of travel; analyzing the preprocessed image frame data using a deep learning-based computer vision model to estimate the visual attention data of the workers, the visual attention data including head orientation and body orientation; synchronizing the trajectory data with the visual attention data in time and space and unifying the coordinate system to construct a real-time behavior vector for each worker; determining the worker's intention state by analyzing the behavior vector, the intention state including goal orientation, observation and learning, and curiosity-driven; when the intention state is determined to be curiosity-driven, checking the worker's permissions, if no permissions are granted, confirming it as an unauthorized entry and triggering a high-level alarm, if permissions are granted, triggering a low-level alert.

[0006] Optionally, the preprocessing of the video surveillance data includes: decoding the video surveillance data into continuous image frame data; for each image frame data: using adaptive Gaussian filtering to remove sensor noise, and combining median filtering to suppress salt-and-pepper noise; separating the illumination component and reflection component of the image using the Retine X algorithm, increasing the brightness of low-light areas, and reducing the contrast of overexposed areas; scaling the image frame data to a fixed size, and normalizing the pixel values.

[0007] Optionally, the step of processing the preprocessed image frame data using the DeepSORT multi-object tracking algorithm to generate trajectory data for each detected worker includes: inputting the preprocessed image frame data into a trained YOLOv8 model to output a list of bounding boxes for the workers, the parameters of which include the coordinates of the bounding box center point, the width and height of the bounding box, the confidence level, and the class; filtering out overlapping and low-confidence bounding boxes using a non-maximum suppression method to obtain high-quality detection results; maintaining a Kalman filter state vector for each existing trajectory; and for each image frame data, predicting the trajectory for each worker using a Kalman filter. The bounding box state of the current image frame is used to obtain the predicted bounding box. For each bounding box in the high-quality detection results, a pre-trained ResNet-50 model is used to extract the appearance feature vector, and the cosine similarity between the appearance feature library of all trajectories and the appearance feature of the current bounding box is calculated. The Mahalanobis distance between the predicted bounding box and all bounding boxes in the current image frame is calculated. The Mahalanobis distance and the cosine similarity are fused by weighting to obtain the association cost matrix. The Hungarian algorithm is used to solve the association cost matrix to obtain the matching result between the bounding box and the existing trajectory. The trajectory is managed according to the matching result, and the trajectory data of each active trajectory is output.

[0008] Optionally, obtaining the matching result between the bounding box and the existing trajectory, and managing the tracking trajectory according to the matching result, includes: if the bounding box does not match the existing trajectory and this occurs for N consecutive frames, then a new trajectory is created and its state vector is initialized; if the bounding box matches the existing trajectory successfully, then the Kalman filter state vector of the corresponding trajectory is updated with the bounding box; if a trajectory is not matched for M consecutive frames, then it is determined that the target has left the field of view and the trajectory is deleted.

[0009] Optionally, the deep learning-based computer vision model analyzes the preprocessed image frame data to estimate the visual attention data of the operator, including: inputting the preprocessed image frame data into a trained pose estimation model to output key points of the human skeleton; estimating the body's orientation vector through the geometric relationship of the key points; locating the key points of the face using a face key point detection model; matching the face key points with a general 3D face model using the PnP algorithm to solve for the rotation and translation vectors of the head, and converting the rotation vectors into yaw, pitch, and roll angles; and concatenating the body's orientation vector with the head's yaw, pitch, and roll angles to obtain the operator's visual attention data.

[0010] Optionally, the behavior vector includes position, velocity scalar, motion direction angle, body orientation vector, head yaw angle, pitch angle, and roll angle.

[0011] Optionally, the step of determining the operator's intention state by analyzing the behavior vector includes: extracting derived features from the behavior vector, the derived features including motion stability, average speed, motion-orientation-attention consistency, and attention concentration; if the motion stability is within the target range, the average speed is within the target speed range, the motion-orientation-attention consistency is greater than the target consistency threshold, the attention concentration is greater than the target concentration threshold, and the duration is greater than the target time, then the operator's intention state is determined to be goal-oriented; if the motion stability is within the target range, the average speed is lower than the target speed range, the attention concentration is greater than the target concentration threshold, and the duration is greater than the target time, then the operator's intention state is determined to be observational learning; if the motion stability is less than the target range, the motion-orientation-attention consistency is less than the target consistency threshold, the attention concentration is less than the target concentration threshold, and the duration is greater than the target time, then the operator's intention state is determined to be curiosity-driven.

[0012] A second aspect of this application provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform a multi-target tracking-based on-site safety monitoring method as described in the above embodiments.

[0013] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement a work site safety supervision method based on multi-target tracking as described in the above embodiments.

[0014] A fourth aspect of this application provides a computer program product storing a computer program that, when executed by a processor, implements a work site safety supervision method based on multi-target tracking as described in the above embodiments.

[0015] The beneficial effects of using the present invention are as follows:

[0016] This application's embodiments construct behavior vectors by fusing trajectory data and visual attention data. These behavior vectors comprehensively depict personnel movement states and attention directions. Combined with intent recognition, they accurately distinguish between three types of intents: goal-oriented, observational learning, and curiosity-driven. This enables the perception and understanding of workers' intent states, allowing for early warnings even before personnel reach hazardous areas, provided their behavioral intent already indicates risk. This allows supervisors to intervene before potential accidents occur, preventing problems before they arise and enhancing proactive on-site safety. A response mechanism linking intent and permissions is constructed. High-level alarms are triggered for unauthorized curiosity-driven behavior, while low-level alerts are given to authorized personnel. This avoids a one-size-fits-all approach to control, ensuring on-site safety without interfering with legitimate and compliant operations and observations. This allows safety managers to focus on real high-risk events, significantly improving supervisory efficiency. Therefore, this addresses the problems of delayed warnings, lack of intent understanding, and limited supervisory dimensions in existing technologies.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein...

[0019] Figure 1 This is a flowchart of a work site safety supervision method based on multi-target tracking, according to an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0022] The following describes a work site safety supervision method based on multi-target tracking, according to an embodiment of this application, with reference to the accompanying drawings. Addressing the issues of delayed early warning, lack of intent understanding, and limited supervisory dimensions mentioned in the background art, this application provides a work site safety supervision method based on multi-target tracking. In this method, a behavior vector is constructed by fusing trajectory data and visual attention data. This behavior vector comprehensively depicts the movement state and attention direction of personnel. Combined with intent recognition, it accurately distinguishes between three types of intents: goal-oriented, observational learning, and curiosity-driven. This enables the perception and understanding of the intent state of workers, allowing for early warnings even before personnel reach the danger zone, but whose behavioral intent already indicates risk. This allows supervisors to intervene before potential accidents occur, preventing problems before they happen and improving the proactiveness of on-site safety. A response mechanism linking intent and permissions is constructed. High-level alarms are triggered for curiosity-driven behaviors without authorization, while only low-level reminders are given to authorized personnel. This avoids a one-size-fits-all approach to control, ensuring work site safety without interfering with legitimate and compliant work and observation behaviors. This allows safety management personnel to focus on handling real high-risk events, thereby significantly improving supervisory efficiency. This solves the problems of delayed early warning, lack of understanding of intent, and single regulatory dimension in existing technologies.

[0023] Specifically, Figure 1 This is a flowchart illustrating a work site safety supervision method based on multi-target tracking, provided in an embodiment of this application.

[0024] like Figure 1 As shown, this method for on-site safety supervision based on multi-target tracking includes the following steps:

[0025] In step S101, high-definition network cameras are deployed at the work site to collect video monitoring data of the workers in real time.

[0026] Specifically, the camera is a 2-megapixel (1920×1080 resolution) high-definition network camera, supporting a frame rate of 25fps to ensure smooth motion capture. The lens focal length is adjustable from 8-12mm to adapt to work areas covering 10-50 meters. It features wide dynamic range, low-light performance, and anti-backlight capabilities, adapting to complex lighting scenarios such as direct sunlight, workshop shadows, and nighttime lighting. It supports IP66 dust and water resistance and can withstand temperatures ranging from -20℃ to 60℃, making it suitable for harsh environments such as outdoor construction and high-temperature workshops.

[0027] Cameras are strategically deployed based on the risk level of the work site. For example, one camera is deployed every 200 square meters in ordinary work areas, focusing on covering personnel operating areas and overlapping work points, with a camera spacing of ≤20 meters. In restricted work areas, one camera is deployed every 100 square meters, covering personnel access routes and the area around equipment, with a camera spacing of ≤15 meters. Additional cameras are installed at personnel entrances and exits, stairwells, corridor corners, and 5 meters outside the boundary of hazardous areas. Cameras are installed at a 45° downward angle to ensure that the workers are fully visible in the frame, and the lenses are kept away from strong light sources (such as windows and spotlights) to reduce backlighting and glare. For areas obstructed by tall equipment, a combination of high-angle and low-angle side shots is used to ensure that there are no blind spots in the obstructed areas.

[0028] The cameras push video surveillance data to a central management server via a standard protocol. The server then uses a streaming media server to receive and manage this video surveillance data.

[0029] It is understood that the embodiments of this application deploy high-definition network cameras at the work site to collect video monitoring data in real time, achieving coverage without blind spots and around the clock, breaking through the time and space limitations of manual inspection, and avoiding missed detection of risks due to insufficient manpower or negligence. Its high-definition image quality and stable frame rate can accurately capture the position, actions, and surrounding environmental details of the workers, providing raw data support for subsequent multi-target tracking and visual attention estimation, ensuring the accuracy of trajectory extraction and intent recognition.

[0030] In step S102, the video surveillance data is preprocessed to obtain preprocessed image frame data.

[0031] It is understood that the embodiments of this application, by preprocessing video surveillance data, can effectively eliminate interference caused by changes in lighting, weather factors, and differences in cameras, providing a stable and consistent input for tracking and pose estimation models, and fundamentally improving the robustness of the algorithm. The preprocessed image frame data can meet the input requirements of deep learning models, significantly reducing unnecessary computational overhead. Furthermore, preprocessing can effectively compensate for insufficient image quality through image enhancement techniques, expanding its applicability in complex on-site environments.

[0032] In this embodiment of the application, the preprocessing of video surveillance data includes:

[0033] Decode video surveillance data into continuous image frame data;

[0034] For each image frame data:

[0035] Adaptive Gaussian filtering is used to remove sensor noise, and median filtering is combined to suppress salt-and-pepper noise.

[0036] The Retinex algorithm is used to separate the illumination and reflection components of an image, thereby increasing the brightness of low-light areas and reducing the contrast of overexposed areas.

[0037] The image frame data is scaled to a fixed size, and the pixel values ​​are normalized.

[0038] Specifically, compressed video surveillance data is read frame by frame from the streaming media server, and the FFmpeg decoding library is used to extract and decode the data frame by frame according to the original frame rate of the camera, outputting a sequence of image frame data arranged in chronological order.

[0039] Using the single-scale Retinex algorithm, with a Gaussian surround kernel radius σ=80, the illumination component L(x,y) and reflection component R(x,y) are separated to satisfy... ,in, This is the original image before separation. For low-brightness areas such as workshop corners and unlit areas at night, the illumination component L(x,y) is multiplied by a gain factor of 1.2~2.0 to improve overall brightness. The CLAHE algorithm is used to enhance local contrast and avoid loss of details in dark areas. For overexposed areas such as those under strong direct sunlight or welding arc light, the illumination component L(x,y) is multiplied by a compression factor of 0.5~0.8 to reduce the brightness of overexposed areas. Then, the reflection component R(x,y) is stretched to map pixel values ​​to the range of [50,200] to restore texture details in overexposed areas. The grayscale mean of the corrected image is controlled between 120 and 150 to ensure uniform brightness and moderate contrast.

[0040] A proportional scaling method is used. First, the image is scaled to the target size range according to the aspect ratio. Then, black pixels are added vertically or horizontally to fill the target size, ensuring no image distortion. The pixel values ​​of the image are normalized from the integer range of [0, 255] to the range of [0, 1]. Image normalization can eliminate the impact of differences in pixel value magnitude on model training and inference, accelerate model convergence, and ensure the consistency of image features under different lighting and contrast conditions.

[0041] It's important to note that adaptive Gaussian filtering calculates the mean value of the grayscale values ​​of neighboring pixels based on their Gaussian distribution weights (the closer to the center, the greater the weight), replacing the center pixel's value. This process strengthens smoothing in flat areas and weakens smoothing in edge areas, thus better preserving edge information while removing noise. Median filtering, on the other hand, is a non-linear filtering technique. It sorts all pixel values ​​within a pixel's neighborhood and takes the median as the new value for that pixel. It effectively suppresses salt-and-pepper noise and removes black-and-white noise.

[0042] It is understood that the embodiments of this application effectively suppress sensor noise and random interference through the collaborative denoising of adaptive Gaussian and median filtering, providing a high-quality data foundation for target localization and attitude estimation. The use of the Retinex algorithm for illumination equalization significantly improves the detail visibility in low-light and overexposed areas, ensuring the robustness of the algorithm under different lighting conditions. Unified size scaling and pixel normalization meet the input specifications of deep learning models, eliminate the influence of device differences, and improve computational efficiency.

[0043] In step S103, the preprocessed image frame data is processed by the DeepSORT multi-target tracking algorithm to generate trajectory data for each detected worker. The trajectory data includes position, movement speed and direction of travel.

[0044] It should be noted that DeepSORT multi-target tracking algorithm is a deep learning-based multi-target tracking algorithm. Its core lies in achieving efficient tracking through the synergy of deep learning feature extraction and the Hungarian matching algorithm. Its main components include a deep neural network feature extractor, a Kalman filter, an appearance model, and the Hungarian algorithm. The deep neural network feature extractor is responsible for extracting appearance features, the Kalman filter predicts the target's current position based on historical motion states, the appearance model integrates and optimizes the extracted features, and the Hungarian algorithm is used to achieve the optimal match between the predicted trajectory and the current detection result.

[0045] It is understood that the embodiments of this application process image data using the DeepSORT multi-target tracking algorithm. Leveraging its advantage of fusing trajectory motion features and appearance features, it can stably track each worker in scenarios with dense crowds and frequent occlusion, effectively reducing ID switching and tracking interruption issues, and generating trajectory data containing position, speed, and direction of travel. This trajectory data can intuitively depict the movement state and trend of personnel, providing core motion feature support for subsequent fusion with visual attention data to construct behavior vectors and accurately identify intent states.

[0046] In this embodiment of the application, the preprocessed image frame data is processed using the DeepSORT multi-target tracking algorithm to generate trajectory data for each detected worker, including:

[0047] The preprocessed image frame data is input into the trained YOLOv8 model, which outputs a list of bounding boxes for the workers. The parameters of the bounding boxes include the coordinates of the center point of the bounding box, the width and height of the bounding box, the confidence score, and the class.

[0048] By using nonmaximum suppression to filter out overlapping, low-confidence bounding boxes, high-quality detection results are obtained.

[0049] Maintain a Kalman filter state vector for each existing trajectory;

[0050] For each image frame data, the bounding box state of each trajectory in the current image frame is predicted by Kalman filtering to obtain the predicted bounding box;

[0051] For each bounding box in the high-quality detection results, the appearance feature vector is extracted using a pre-trained ResNet-50 model, and the cosine similarity between the appearance feature library of all trajectories and the appearance feature of the current bounding box is calculated.

[0052] Calculate the Mahalanobis distance between the predicted bounding box and all bounding boxes in the current image frame;

[0053] The association cost matrix is ​​obtained by weighted fusion of Mahalanobis distance and cosine similarity.

[0054] The Hungarian algorithm is used to solve the association cost matrix to obtain the matching results between the bounding box and the existing trajectory. The trajectory is managed according to the matching results, and the trajectory data of each active trajectory is output.

[0055] Specifically, the non-maximum suppression method eliminates redundant bounding boxes and retains the best detection result by comparing the corner union ratio of two bounding boxes. The specific process is as follows:

[0056] Step 1: Sort all bounding boxes in descending order of confidence;

[0057] Step 2: Select the bounding box with the highest confidence level and add it to the final output list;

[0058] Step 3: Calculate the intersection-union ratio (IUR) of this bounding box with all remaining bounding boxes in the bounding box list. The IUR calculation formula is:

[0059]

[0060] in, The calculated crossover-union ratio; B and C are the two bounding boxes to be compared; The area of ​​the intersection of the two bounding boxes; Let be the area of ​​the union of the two bounding boxes.

[0061] Step 4: Remove bounding boxes from all remaining bounding boxes whose intersection-union ratio exceeds the target threshold;

[0062] Repeat steps 2-4 for the remaining bounding boxes until no boxes remain, to obtain high-quality detection results.

[0063] A Kalman filter state vector is constructed for each existing trajectory to record and update the target's historical motion state, supporting motion prediction in subsequent frames. The Kalman filter state vector uses an 8-dimensional state vector [x,y,a,h,...]. , , , The symbol ] represents the center of the bounding box, x and y represent the aspect ratio of the bounding box, and h represents the height of the bounding box. , , , The motion velocities correspond to the four parameters mentioned above. Each trajectory is identified by a unique ID, and is initialized based on the initial bounding box parameter assignment (the velocity term defaults to 0). The state vector is updated every frame using the matched bounding boxes.

[0064] Assuming the movement speed of personnel is stable, a constant velocity model is used to predict the trajectory. A state transition matrix is ​​constructed to describe the state change from the previous frame to the current frame. The state transition matrix is ​​multiplied by the state vector of the previous frame to obtain the predicted state vector. The bounding box parameters are calculated based on the predicted state vector. Simultaneously, the prediction covariance matrix is ​​updated. The formula is as follows:

[0065]

[0066] in, The covariance matrix is ​​used to predict the uncertainty of the predicted state; F is the state transition matrix. This is the covariance matrix of the previous frame; This is a preset Gaussian noise matrix.

[0067] A feature library is maintained for each trajectory, storing the appearance feature vectors of the most recent k frames. The average of these vectors is used as the baseline feature vector for that trajectory. The cosine similarity is calculated between the current bounding box appearance feature vector (det) and the baseline feature vector for each trajectory, using the following formula:

[0068]

[0069] Where S is the calculated cosine similarity, ranging from 0 to 1, with the closer to 1 indicating greater similarity in appearance. This represents the magnitude of the current bounding box appearance feature vector. The reference feature vector magnitude represents the trajectory.

[0070] The Mahalanobis distance between the predicted bounding box and all bounding boxes in the current image frame is calculated using the following formula:

[0071]

[0072] in, The Mahalanobis distance between the predicted bounding box and the bounding box is used; the smaller the value, the higher the consistency of the motion state. The residual vector is a vector constructed from the difference between the bounding box and the predicted box state vectors.

[0073] Convert the cosine similarity S into cost. To gain an appearance comes at a cost.

[0074] The Mahalanobis distance and appearance cost are weighted and fused to obtain the correlation cost matrix, which has the following form: In this matrix, M represents the number of bounding boxes, N represents the number of existing trajectories, and the matrix element cost[i][j] represents the matching cost between the i-th bounding box and the j-th trajectory. The smaller the value, the higher the matching degree.

[0075] The Hungarian algorithm is used to solve the cost matrix, finding the "boundary box-trajectory" correspondence with the minimum total cost. Matching results between the bounding boxes and existing trajectories are obtained. Trajectories are managed based on these matching results, and trajectory data for each active trajectory is output. Trajectory data includes position, velocity, and direction of travel. Position is the coordinates of the bounding box center point, velocity is the velocity (v_x, v_y) obtained from the Kalman filter state vector, and direction of travel is the direction angle calculated from the velocity vector. The calculation formula is: .

[0076] It is understood that this application's embodiments accurately locate workers using the YOLOv8 model and filter redundant bounding boxes using non-maximum suppression, providing high-quality initial targets for tracking. Kalman filtering is used to predict trajectory motion states, avoiding tracking breaks caused by short-term occlusion. Simultaneously, a pre-trained ResNet-50 model is used to extract appearance features, and cosine similarity is combined to distinguish uniformly dressed personnel, reducing ID switching. A weighted fusion of Mahalanobis distance and cosine similarity is used to construct an association cost matrix, which, combined with the Hungarian algorithm, achieves optimal matching between detection boxes and trajectories. Trajectory management ensures that each worker's trajectory is unique and continuous. The resulting trajectory data, including position, speed, and direction of travel, not only accurately depicts personnel dynamics but also provides reliable motion feature support for subsequent fusion with visual attention data to construct behavior vectors and identify intent states. This significantly improves the stability and accuracy of personnel tracking at the work site, laying a core data foundation for safety supervision.

[0077] In this embodiment of the application, obtaining the matching result between the bounding box and the existing trajectory, and managing the tracking trajectory based on the matching result, includes:

[0078] If the bounding box does not match an existing trajectory and this occurs for N consecutive frames, a new trajectory is created and its state vector is initialized.

[0079] If the bounding box successfully matches an existing trajectory, the Kalman filter state vector of the corresponding trajectory is updated using the bounding box.

[0080] If a trajectory is not matched for M consecutive frames, it is determined that the target has left the field of view and the trajectory is deleted.

[0081] Understandably, this application's embodiments create new trajectories for N consecutive frames of unmatched bounding boxes, enabling rapid capture of newly entering workers and preventing missed detections. Successfully matched trajectories update the Kalman filter state vector with bounding boxes, allowing real-time correction of trajectory prediction biases and ensuring the trajectory accurately follows personnel movement, maintaining tracking continuity and accuracy. Deleting M consecutive frames of unmatched trajectories not only promptly clears trajectories of personnel who have left the field of view, avoiding the consumption of computing resources and interference with new target matching, but also accurately determines the personnel's departure status, ensuring a concise and efficient trajectory library. This provides stable and reliable motion feature support for subsequent behavior vector construction and intent recognition, improving the accuracy and adaptability of multi-target tracking at the work site.

[0082] In step S104, a deep learning-based computer vision model analyzes the preprocessed image frame data to estimate the visual attention data of the workers, including head orientation and body orientation.

[0083] It is understood that the embodiments of this application, by estimating the visual attention data of workers, overcome the limitations of relying solely on trajectory data to judge behavior, effectively reducing misjudgments caused by relying solely on location data, and lowering the false alarm rate of safety supervision. Combined with the dynamic changes in head and body orientation, it can also capture potential risky intentions of personnel in advance (such as curiosity-driven trends of unauthorized approach), providing core feature support for subsequent intention recognition.

[0084] In this embodiment of the application, a deep learning-based computer vision model analyzes the preprocessed image frame data to estimate the visual attention data of the operator, including:

[0085] The preprocessed image frame data is input into the trained pose estimation model, which outputs the key points of the human skeleton.

[0086] Estimate the body's orientation vector by using the geometric relationships of key human points;

[0087] The facial landmark detection model is used to locate the key points of the face;

[0088] Using the PnP algorithm, facial key points are matched with a general 3D face model to solve for the rotation and translation vectors of the head. The rotation vectors are then converted into yaw, pitch, and roll angles.

[0089] By stitching together the body's orientation vector with the head's yaw, pitch, and roll angles, visual attention data of the operator can be obtained.

[0090] Specifically, HRNet is used as the pose estimation model. Its advantage lies in continuously preserving detailed information through high-resolution feature maps, enabling accurate localization of key points even in scenarios with complex worker attire (such as reflective clothing and safety helmets) or partial limb occlusion (such as holding tools). Preprocessed image frame data is input into the HRNet model, which outputs 17 COCO-formatted key points for each human body, including 0 (nose), 1 (left eye), 2 (right eye), 3 (left ear), 4 (right ear), 5 (left shoulder), 6 (right shoulder), 7 (left elbow), 8 (right elbow), 9 (left wrist), 10 (right wrist), 11 (left hip), 12 (right hip), 13 (left knee), 14 (right knee), 15 (left ankle), and 16 (right ankle). Each key point includes its pixel coordinates in the image and its visibility confidence score.

[0091] By analyzing the spatial distribution of key points on the upper body, the body's principal axis direction is calculated to reflect the worker's overall orientation. The midpoints of the shoulder and hip joints are selected, and the unit vector pointing from the midpoint of the hip joint to the midpoint of the shoulder is defined as the body's orientation vector. Its calculation formula is as follows:

[0092]

[0093]

[0094]

[0095] in, Midpoint of the shoulder; , The coordinates of the left shoulder; , The coordinates of the right shoulder; The midpoint of the hip joint; , Left hip coordinates; , The coordinates are for the right hip. This is the difference in x-coordinate between the midpoint of the shoulder and the midpoint of the hip joint; This is the difference in the vertical coordinates between the midpoint of the shoulder and the midpoint of the hip joint; This is the orientation vector of the body.

[0096] A facial landmark detection model is used to first locate the face region in the image frame, and then output 68 landmarks within the face region. The facial landmark detection model can be the RetinaFace model. Twelve core landmarks are selected as the final facial landmarks.

[0097] A predefined 3D face template is used, containing 3D coordinates corresponding to 12 facial key points. The PnP algorithm minimizes the error between the coordinates projected onto the image from the 3D coordinate points and the actual facial key point coordinates, outputting the optimal head rotation and translation vectors. The rotation vector is converted into a rotation matrix using the Rodrigues formula, and then decomposed into three Euler angles: yaw, pitch, and roll. The yaw angle is the rotation angle around the Y-axis, representing left and right head rotation; 0 degrees indicates a frontal view, positive values ​​indicate leftward rotation, and negative values ​​indicate rightward rotation. The pitch angle is the rotation angle around the X-axis, representing head up and down movement; 0 degrees indicates a level gaze, positive values ​​indicate head tilting, and negative values ​​indicate head tilting. The roll angle is the rotation angle around the Z-axis, representing head tilting towards the shoulders; 0 degrees indicates a horizontal tilt, positive values ​​indicate tilting towards the left shoulder, and negative values ​​indicate tilting towards the right shoulder.

[0098] Understandably, the embodiments of this application provide a macroscopic orientation benchmark dominated by the torso based on body orientation estimation of key points of the human skeleton, and accurately capture the microscopic observation and control elements that determine the direction of gaze by solving the head Euler angles through the PnP algorithm. This dual verification mechanism of body orientation combined with head posture not only makes up for the failure risk of single face detection in long-distance and occluded scenarios, but also overcomes the limitation that body orientation alone cannot reflect the actual gaze direction. It constructs robust and hierarchical visual attention data, providing key cognitive state input for subsequent accurate judgment of personnel's work intentions, fundamentally improving the cognitive understanding depth of safety supervision.

[0099] In step S105, the trajectory data and visual attention data are spatiotemporally synchronized and aligned with the coordinate system to construct a real-time behavior vector for each worker.

[0100] It is understood that the embodiments of this application precisely align trajectory data and visual attention data in space and time, forming a complete description of the worker's state. This not only makes up for the limitations of a single data source (such as the inability of a pure trajectory to distinguish between intentional entry and unintentional entry), but also improves the reliability of the data through spatiotemporal consistency verification, so that the generated behavior vector provides the intention recognition model with a joint representation that combines motion features and cognitive features.

[0101] In the embodiments of this application, the behavior vector includes position, velocity scalar, motion direction angle, body orientation vector, head yaw angle, pitch angle, and roll angle.

[0102] Among them, the position and velocity scalars describe the spatial motion state of the operator, the motion direction angle reveals the intention to move, the body orientation vector reflects the macroscopic posture dominated by the torso, and the three Euler angles of the head accurately capture the subtle changes in visual attention.

[0103] It is understood that the present application's embodiments combine macroscopic motion with microscopic posture features, which can analyze behavioral patterns through motion trajectories and determine the focus of attention through the orientation relationship between the body and head. This provides the intention recognition model with a comprehensive input that combines kinematic features and cognitive state, thereby achieving accurate understanding and prediction of the worker's behavioral intentions.

[0104] In step S106, the intention state of the operator is determined by analyzing the behavior vector. The intention state includes goal orientation, observational learning, and curiosity-driven.

[0105] It is understood that the embodiments of this application, by analyzing the intrinsic relationship between motion trajectories and visual attention in behavior vectors, can not only identify goal-oriented behaviors with clear operational objectives, but also distinguish learning states characterized by focused observation, and more acutely capture potential risks caused by the separation of movement direction and visual focus. This intent understanding based on multimodal feature analysis breaks through the limitation of traditional security systems that can only detect physical violations, enabling security management to evolve from passive response to proactive early warning, and providing key technical support for preventing accidents caused by cognitive factors such as distraction and unintentional intrusion.

[0106] In this embodiment of the application, determining the intention state of the operator by analyzing behavior vectors includes:

[0107] Derivative features are extracted from the behavior vectors, including motion stability, average speed, motion-orientation-attention consistency, and attention concentration.

[0108] If the motion stability is within the target range, the average speed is within the target speed range, the motion-orientation-attention consistency is greater than the target consistency threshold, the attention concentration is greater than the target concentration threshold, and the duration is greater than the target time, then the operator's intention state is judged as target-oriented.

[0109] If the motion stability is within the target range, the average speed is lower than the target speed range, the concentration is greater than the target concentration threshold, and the duration is greater than the target time, then the operator's intentional state is judged as observational learning.

[0110] If the motion stability is less than the target range, the motion-orientation-attention consistency is less than the target consistency threshold, the attention concentration is less than the target concentration threshold, and the duration is greater than the target time, then the operator's intention state is judged to be driven by curiosity.

[0111] Among them, motion stability describes the smoothness of the movement speed and direction of a worker's trajectory, reflecting whether their movement has a clear goal. The higher the value, the more regular the trajectory and the smaller the directional fluctuation; the lower the value, the more chaotic the trajectory and the more frequent the directional changes.

[0112] Average speed is the average rate at which a worker moves per unit of time, reflecting the initiative of the movement. A value within the target range indicates active movement towards the target.

[0113] The motion-orientation-attention coherence measure reflects the degree of coordination among "movement direction, body orientation, and head attention direction". The higher the value, the higher the consistency among the three; the lower the value, the more disconnected the three.

[0114] Attention concentration describes the stability of head orientation, reflecting whether a person is focused on a particular target. A higher value indicates less fluctuation in head posture angle; a lower value indicates frequent head turning.

[0115] Specifically, the behavior vector of the most recent R frames is obtained, along with the position, velocity scalar, and motion direction angle. The standard deviations of the velocity scalar and motion direction angle are calculated, and the smoothness of the velocity scalar and motion direction angle is calculated based on a preset maximum fluctuation. The smoothness of both is then weighted and fused to obtain motion stability. For example, the smoothness calculation formula for the motion direction angle is:

[0116]

[0117] in, For the smoothness of the motion direction angle, The standard deviation of the motion direction angle of the most recent R frame. The maximum directional fluctuation is preset.

[0118] Take the behavior vector of the most recent R frames, obtain the velocity scalar, and calculate its arithmetic mean as the average velocity.

[0119] Take the behavior vector from the most recent R frames, obtain the motion direction angle, body orientation vector, and head yaw angle. Convert the motion direction angle and head yaw angle into motion direction vector and head attention direction vector, respectively. For example, its transformed motion direction vector is The cosine similarity of the motion direction vector, the head attention direction vector, and the body orientation vector is calculated separately to obtain motion-orientation consistency and orientation-attention consistency. The motion-orientation-attention consistency response is then obtained by taking the mean of motion-orientation consistency and orientation-attention consistency.

[0120] Take the behavior vector from the most recent R frames to obtain the head's yaw, pitch, and roll angles. Calculate the standard deviation of each of the three angles, and obtain the total fluctuation value through weighted fusion. Calculate the attention concentration based on the preset maximum total fluctuation. The calculation method is similar to that for the smoothness calculation of the motion direction angle, and will not be elaborated here.

[0121] It is understood that the embodiments of this application can accurately distinguish the three core intentions of operators through the quantification and combination of derived features. This data-based intention recognition breaks through the limitations of traditional location judgment, realizing an upgrade from passive monitoring to proactive prediction, and providing accurate decision-making basis for on-site safety management.

[0122] In step S107, when the intention state is determined to be driven by curiosity, the user's permissions are checked. If the user does not have permissions, it is confirmed as an unauthorized entry and a high-level alarm is triggered. If the user has permissions, a low-level alert is triggered.

[0123] Specifically, the personnel access control database stores core authorization information related to work site safety. Each record includes an identity ID, a list of authorized areas, and an access level. When the intent is determined to be driven by curiosity, the personnel's identity ID is obtained. The database is then searched for the authorized area list using this ID. If the work area is listed, the personnel has the necessary permissions; otherwise, they do not. If the personnel's identity ID cannot be obtained, they do not have the necessary permissions.

[0124] When the person does not have the necessary permissions, an alarm message is generated based on their identity ID (or "unknown identity"), work area, current location, and behavior description. This message is then sent to the handheld terminal (such as a walkie-talkie or tablet) of the on-site security personnel, along with an audio-visual alarm and alarm information, along with a screenshot of the real-time monitoring footage.

[0125] When the user has the necessary permissions, a notification is generated and sent to the user's personal device. This information is also recorded and archived in the background for later review by administrators.

[0126] It is understood that this application embodiment matches behavioral intent with identity permissions in real time, and determines the "curiosity-driven" state of unauthorized personnel as high-risk unauthorized entry behavior, triggering a targeted alarm containing identity, location, and behavioral description, making on-site intervention clear and high-priority. At the same time, it provides gentle reminders and records similar behaviors of authorized personnel in the background, which avoids excessive intervention in normal operations and provides data support for analyzing personnel status.

[0127] This application proposes a multi-target tracking-based on-site safety supervision method. By fusing trajectory data and visual attention data to construct a behavior vector, the method comprehensively depicts personnel movement and attention direction. Combined with intent recognition, it accurately distinguishes between three types of intents: goal-oriented, observational learning, and curiosity-driven. This enables the perception and understanding of personnel's intent states, allowing for early warnings even before personnel reach hazardous areas but when their behavioral intent already indicates risk. This allows supervisors to intervene before potential accidents occur, preventing problems before they happen and improving proactive on-site safety. A response mechanism linking intent and permissions is constructed. High-level alarms are triggered for unauthorized curiosity-driven behavior, while low-level alerts are given to authorized personnel. This avoids a one-size-fits-all approach, ensuring on-site safety without interfering with legitimate and compliant operations and observations. This allows safety managers to focus on real high-risk events, significantly improving supervisory efficiency. Therefore, this method solves the problems of delayed warnings, lack of intent understanding, and limited supervisory dimensions in existing technologies.

[0128] The following is a detailed description of a work site safety supervision method based on multi-target tracking, using specific embodiments:

[0129] A large machining workshop adopted this safety monitoring method for real-time control. The workshop deployed 15 high-definition network cameras (1920×1080 resolution, 25fps frame rate, 8-12mm focal length) with 2 megapixels each. In the general work area, one camera was installed every 200 square meters with a spacing of ≤20 meters, focusing on covering machine tool operating areas and material handling points. In the warning area, one camera was installed every 100 square meters with a spacing of ≤15 meters. Additional cameras were installed around high-voltage equipment and in access passages. All cameras were installed at a 45° downward angle. In areas with tall machine tools, a combination of high-level overhead and low-level side shots was used to eliminate blind spots. The cameras have wide dynamic range and low-light capabilities, and can withstand complex environments such as welding arc light and insufficient nighttime lighting in the workshop. Video data is pushed to the central management server in real time via IP protocol.

[0130] Visitor Zhang entered the welding warning area (a non-open area) of the workshop without authorization. The server decodes the video data captured by the camera frame by frame using the FFmpeg decoding library. Each frame is processed by adaptive Gaussian filtering and median filtering to remove sensor noise and salt-and-pepper noise. Then, the Retinex algorithm is used to separate the illumination and reflection components. The illumination component of the overexposed area of ​​the welding arc is multiplied by a compression factor of 0.6 to restore the texture details of the equipment. Finally, it is scaled proportionally to 640×640 and normalized to the range of [0,1].

[0131] The preprocessed image frame data is input into the YOLOv8 model to output Zhang's bounding box (center point coordinates (360, 240), width 60 pixels, height 120 pixels, confidence score 0.92, category "human"). High-quality detection results are obtained by filtering with non-maximum suppression. A new trajectory is created for Zhang, and an 8-dimensional Kalman filter state vector is initialized. The bounding box state of each frame is predicted using a constant velocity model. Cosine similarity is calculated by combining the appearance feature vector extracted by ResNet-50, and a cost matrix is ​​constructed using Mahalanobis distance weighted fusion. Continuous trajectory tracking is achieved through Hungarian algorithm matching. The final output trajectory data is: position (360, 240), velocity scalar 0.4 m / s, and travel direction angle 30°.

[0132] The preprocessed image frame data was input into the HRNet pose estimation model, which output 17 human skeletal key points for Zhang. The vectors of the midpoint of the shoulder (350, 210) and the midpoint of the hip joint (350, 270) were calculated and normalized to obtain the body orientation vector (0.8, 0.2). The RetinaFace model located Zhang's face region and output 68 key points. Twelve core key points were selected and matched with a predefined 3D face template. The head rotation vector was solved using the PnP algorithm, converted into a rotation matrix using the Rodriguez formula, and decomposed to obtain a yaw angle of -15° (right turn), a pitch angle of 5° (head up), and a roll angle of 0°. These were then concatenated with the body orientation vector to obtain visual attention data. The trajectory data and visual attention data were spatiotemporally synchronized according to timestamps to construct Zhang's behavior vector: position (360,240), velocity scalar 0.4m / s, motion direction angle 30°, body orientation vector (0.8,0.2), head yaw angle -15°, pitch angle 5°, and roll angle 0°.

[0133] Based on the behavioral vector, derived features were extracted. Using the most recent 10 frames of data, the motion stability was 0.2 (below the target range of 0.7), the average speed was 0.4 m / s, the motion-orientation-attention consistency was 0.25 (below the threshold of 0.8), and the attention concentration was 0.3 (below the threshold of 0.7). This state lasted for 4 seconds (exceeding the target time by 3 seconds), indicating that Zhang's intention was driven by curiosity. During the permission verification phase, an attempt to obtain Zhang's identity ID via facial recognition failed, indicating he lacked authorization and was confirmed as a mistaken trespasser. An alarm message was immediately generated (identity "unknown," work area "welding warning zone," current location (15.3, 8.7) (world coordinate system), behavior description "chaotic trajectory, frequently turning right to observe welding equipment"), and an audible and visual alarm along with this message and a real-time monitoring screenshot were sent to the tablet of the on-site security personnel. Upon receiving the alarm, the security personnel arrived at the scene within 5 minutes, verified Zhang's identity, guided him out of the workshop, and completed the necessary procedures.

[0134] In summary, this application's embodiments achieve precise personnel tracking using the DeepSORT algorithm. By combining HRNet, RetinaFace, and PnP algorithms, it accurately extracts human posture and head attention features, constructing a behavior vector that comprehensively depicts personnel movement and attention states. Through derived feature analysis, it accurately determines "curiosity-driven" intentions and implements tiered responses based on permission checks. For Zhang, who lacks authorization, a high-level alarm with real-time location and video feed is triggered, allowing for rapid security intervention. This not only achieves timely interception of unauthorized entry but also avoids excessive intervention through accurate intent recognition and permission matching, significantly improving the comprehensiveness, accuracy, and response efficiency of safety supervision in large-scale machining workshops.

[0135] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0136] The memory 201, the processor 202, and the computer program stored on the memory 201 and capable of running on the processor 202.

[0137] When the processor 202 executes the program, it implements the on-site safety supervision method based on multi-target tracking provided in the above embodiments.

[0138] Furthermore, electronic devices also include:

[0139] Communication interface 203 is used for communication between memory 201 and processor 202.

[0140] The memory 201 is used to store computer programs that can run on the processor 202.

[0141] The memory 201 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0142] If the memory 201, processor 202, and communication interface 203 are implemented independently, then the communication interface 203, memory 201, and processor 202 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 2The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0143] Optionally, in a specific implementation, if the memory 201, processor 202, and communication interface 203 are integrated on a single chip, then the memory 201, processor 202, and communication interface 203 can communicate with each other through an internal interface.

[0144] The processor 202 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0145] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for on-site safety supervision based on multi-target tracking.

[0146] This application also provides a computer program product, which stores a computer program that, when executed by a processor, implements the above-described method for on-site safety supervision based on multi-target tracking.

[0147] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0148] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0149] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0150] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0151] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A multi-target tracking-based job site safety supervision method, characterized by, The method comprises the following steps: deploying a high-definition network camera at a work site to collect video monitoring data of workers in real time; preprocessing the video monitoring data to obtain preprocessed image frame data; processing the preprocessed image frame data through a DeepSORT multi-target tracking algorithm to generate trajectory data for each detected worker, the trajectory data including position, movement speed, and direction of travel; analyzing the preprocessed image frame data based on a deep learning computer vision model to estimate visual attention data of the workers, the visual attention data including head orientation and body orientation; spatiotemporally synchronizing and unifying the coordinate systems of the trajectory data and the visual attention data to construct a real-time behavior vector for each worker; judging the intention state of the worker by analyzing the behavior vector, the intention state including target orientation, observation and learning, and curiosity-driven, and extracting derived features from the behavior vector, the derived features including movement stability, average speed, movement-orientation-attention consistency, and attention concentration degree, and if the movement stability is within a target range, the average speed is within a target speed interval, the movement-orientation-attention consistency is greater than a target consistency threshold, the attention concentration degree is greater than a target concentration threshold, and the duration is greater than a target time, the intention state of the worker is judged to be target orientation; if the movement stability is within a target range, the average speed is lower than a target speed interval, the attention concentration degree is greater than a target concentration threshold, and the duration is greater than a target time, the intention state of the worker is judged to be observation and learning; if the movement stability is less than a target range, the movement-orientation-attention consistency is less than a target consistency threshold, the attention concentration degree is less than a target concentration threshold, and the duration is greater than a target time, the intention state of the worker is judged to be curiosity-driven; when the intention state is judged to be curiosity-driven, the authority of the worker is checked, and if there is no authority, it is confirmed that the worker has entered by mistake, triggering a high-level alarm, and if there is authority, a low-level reminder is triggered.

2. The method of claim 1, wherein, The preprocessing of the video monitoring data comprises: decoding the video monitoring data into continuous image frame data; for each image frame data: using adaptive Gaussian filtering to remove sensor noise and combining median filtering to suppress salt and pepper noise; separating the illumination component and the reflection component of the image through the Retinex algorithm to improve brightness in low-illumination areas and reduce contrast in overexposed areas; scaling the image frame data to a fixed size and normalizing the pixel values. 3.The multi-target tracking based job site safety supervision method of claim 1, wherein, The processing of the preprocessed image frame data through the DeepSORT multi-target tracking algorithm to generate trajectory data for each detected worker comprises: inputting the preprocessed image frame data into a trained YOLOv8 model to output a list of bounding boxes of the workers, the parameters of the bounding boxes including the center point coordinates of the bounding boxes, the width and height of the bounding boxes, the confidence, and the class; filtering out overlapping and low-confidence bounding boxes through a non-maximum suppression method to obtain high-quality detection results; maintain a Kalman filter state vector for each existing track; for each image frame data, predict the bounding box state of each track in the current image frame by Kalman filter, and obtain a predicted box; for each bounding box in the high-quality detection result, extract an appearance feature vector using a pre-trained ResNet-50 model, and calculate the cosine similarity between the appearance feature library of all tracks and the appearance feature of the current bounding box; calculate the Mahalanobis distance between the predicted box and all bounding boxes in the current image frame; fuse the Mahalanobis distance and the cosine similarity by weighting to obtain a correlation cost matrix; use the Hungarian algorithm to solve the correlation cost matrix to obtain the matching result of the bounding box and the existing track, manage the tracking track according to the matching result, and output the track data of each active track.

4. The method of claim 3, wherein, The matching result of the bounding box and the existing track is obtained, and the tracking track is managed according to the matching result, including: if the bounding box and the existing track are not matched, and appear continuously for N frames, a new track is created and its state vector is initialized; if the bounding box and the existing track are matched successfully, the Kalman filter state vector of the corresponding track is updated using the bounding box; if a track is not matched for M consecutive frames, it is determined that the target has left the field of view, and the track is deleted.

5. The multi-target tracking based job site safety supervision method of claim 1, wherein, The computer vision model based on deep learning analyzes the preprocessed image frame data to estimate the visual attention data of the worker, including: inputting the preprocessed image frame data into the trained pose estimation model to output the key points of the human body skeleton; estimating the orientation vector of the body through the geometric relationship of the human body key points; locating the key points of the face using a face key point detection model; matching the face key points with a general 3D face model using a PnP algorithm to solve the rotation vector and translation vector of the head, and converting the rotation vector into a yaw angle, a pitch angle, and a roll angle; splicing the orientation vector of the body and the yaw angle, the pitch angle, and the roll angle of the head to obtain the visual attention data of the worker.

6. The multi-target tracking based job site safety supervision method of claim 1, wherein, The behavior vector includes position, velocity scalar, motion direction angle, orientation vector of the body, yaw angle, pitch angle, and roll angle of the head.

7. An electronic device, comprising: including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the method of claim 1-6.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of claim 1-6.

9. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions are executed to implement the method of claim 1-6.

Images