A construction scene intelligent safety monitoring method based on machine vision

Abstract

The application discloses a kind of construction scene wisdom safety monitoring methods based on machine vision, first, the real-time video image of construction scene is collected, the data source of intelligent processing module is obtained;Then the collected video is detected and tracked in real time to construction personnel;Next, the safety state of construction personnel is identified according to the detection result image;Afterwards, the information of detection, identification and tracking is fused and processed;Finally, the result obtained by real-time processing is output to equipment for visual display.The application reduces the labor cost of safety monitoring system while ensuring the effect of safety supervision, and promotes the application of artificial intelligence algorithm in industrial safety actual scene.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pattern recognition, and particularly relates to a construction scene intelligent safety monitoring method based on machine vision. BACKGROUND

[0002] Safety monitoring technology mainly refers to using video technology to detect and monitor a construction scene and display and record images in real time, so as to facilitate monitoring of relevant information of the construction scene, realize real-time monitoring of videos, and at the same time, the recorded information can be called out at any time to achieve traceability. However, most of the safety monitoring technologies of related enterprises at present adopt manual patrol or video supervision, and the video supervision needs manual uninterrupted watching of monitoring videos. The monitoring system only plays the role of an electronic eye. Obviously, workers are easy to slacken in long-term execution of such monotonous and tedious routine monitoring work, thereby affecting the work quality. Therefore, strict compliance with safety technical specifications and safety operation procedures is the primary prerequisite for preventing construction safety accidents, and the relevant departments of enterprises need to pay special attention to safety supervision problems.

[0003] With the continuous development of modern technology, the traditional safety monitoring method has been unable to meet the needs of the construction scene with frequent accidents, and people pay more and more attention to the practical application of artificial intelligence technology in the safety monitoring system. The development trend of safety monitoring technology nowadays can be divided into two categories: a method based on wearable sensors and a method based on machine vision. Although many methods based on wearable sensors use contact sensors to effectively collect data, these sensors are mostly very expensive, which hinders the wide application of this method. On the contrary, the method based on machine vision is a non-contact optical approach, which completes long-term remote monitoring in a very economical way and has robustness. In terms of the method based on machine vision, in the Chinese patent with the publication number CN112911255A, a construction site safety state detection system is disclosed, which provides a solution for the intelligent safety monitoring of a construction site. However, it only detects the wearing of safety helmets in the safety state, and at the same time, there are defects such as insufficient detection precision due to light, obstructions and other reasons, which cannot be applied to actual scenes. SUMMARY

[0004] In order to overcome the deficiencies of the prior art, the present application provides a construction scene intelligent safety monitoring method based on machine vision, which first collects real-time video images of the construction scene to obtain the data source of the intelligent processing module; then detects the collected video in real time and tracks the trajectory of the construction personnel; next, the safety state of the construction personnel is identified according to the detection result image; then the information of detection, identification and tracking is fused; finally, the result obtained by real-time processing is output to the device for visual display. The present application reduces the labor cost of the safety monitoring system while ensuring the effect of safety supervision, and promotes the application of artificial intelligence algorithm in the actual scene of industrial safety.

[0005] The technical solution adopted by the present application to solve its technical problems comprises the following steps:

[0006] Step 1: Constructing a construction scene intelligent safety monitoring system; collecting real-time video images of the construction scene to obtain the data source of the intelligent processing module;

[0007] The construction scene intelligent safety monitoring system comprises a video monitoring camera, a server and a mobile display; the video monitoring camera is connected to the server through a local area network, and the server is connected to the mobile display through a network; the server is equipped with a video image intelligent processing module;

[0008] The monitoring camera sends the collected real-time video images of the construction scene to the server through the RTSP protocol, and the monitoring picture processed by the server is sent to the mobile display through the network for display;

[0009] Step 2: The video image intelligent processing module detects and tracks the trajectory of the construction personnel in real time based on the collected video;

[0010] Step 2-1: In the personnel detection part, the pre-trained target detector on the data set is used to detect the construction personnel in the collected actual construction scene video images, and the detected construction personnel is labeled on the image using an anchor frame;

[0011] Step 2-2: In the trajectory tracking part, two features are tracked: one is the motion feature, which is calculated based on the anchor frame position of the detected construction personnel in the consecutive frames, and the motion feature of the target is obtained through Kalman filtering, and the matching between the adjacent two frames is performed through the Hungarian matching algorithm; the second is the appearance feature, the posture and clothing of each person are different, so the corresponding appearance feature information is also different, the appearance feature information is extracted through a convolutional neural network model, and finally the final tracking result is obtained by combining the appearance feature and the motion feature, and an ID is assigned to realize the counting and tracking of the construction personnel;

[0012] Step 3: Identify the safety status of construction workers based on the detection and trajectory tracking results from Step 2;

[0013] The safe working status of construction workers is identified as a semantic attribute of the image. Based on the anchor boxes of the target detection results, the images of each construction worker are cropped to obtain the specific image of each individual.

[0014] The images of construction workers are fed into a pre-trained attribute recognition classifier. By extracting features from the images, the safe working status of the construction workers is determined, and the identified violations are output. When training the attribute recognition classifier, the input single image of the construction worker is divided into two parts according to the overall vertical proportion, representing the upper body and the lower body respectively. The network is trained by extracting local features from the upper and lower parts respectively and supplementing them with global features.

[0015] Step 4: Fusion processing of detection, identification, and tracking information;

[0016] A detection, recognition, and tracking information fusion processing module is constructed. The detection results and attribute recognition results of steps 2 and 3 are placed on the trajectory of multi-target tracking for supervision. A voting method is used to smooth the tracking display information. That is, if the recognition information of a frame changes abruptly compared with the previous 5 frames, the state of the current frame is judged by the detection state of the previous 30 frames. If the number of frames in the previous 30 frames where no target is detected is more than the number of frames where target is detected, it is determined that the target has disappeared. Otherwise, it is determined that the target exists. The recognition results are corrected according to the multi-target tracking trajectory to complete the smoothing of the result information display.

[0017] Step 5: Output the results of real-time processing to a mobile display for visualization;

[0018] The server and the mobile display are connected via a network. The server processes the received construction scene video images in real time and sends the processed results to the mobile display for display.

[0019] Design a human-computer interaction interface to be displayed on a mobile display, including six functions:

[0020] People count: Displays the total number of people who have appeared in the frame and the number of people currently in the frame;

[0021] Personnel tracking: Tracks and locates people in the video feed in real time;

[0022] Scene switching: Includes two working modes: daily construction and high-altitude operation; the difference between the two modes is that in the daily construction mode, an alarm will be triggered if a worker is not wearing a safety helmet, but will not be triggered if a worker is not wearing a safety belt; in the high-altitude operation mode, an alarm will be triggered if a worker is not wearing a safety helmet or not wearing a safety belt.

[0023] Information display: display the recognized personnel safety operation state, operation picture display and video stream control;

[0024] View button: view the specific construction picture of the specified construction personnel.

[0025] Preferably, the personnel detection of step 2-1 is specifically:

[0026] YOLOv3 network is selected as the detector, head decoupling, data enhancement, Anchor Free and SimOTA sample matching method are introduced to construct an anchor-free end-to-end target detection framework; the video sequence collected by the camera network is sent to the pre-trained YOLOv3 network through the server for construction personnel detection, the detection result includes the position information of the worker, and the detected construction personnel is labeled on the video image using the anchor frame.

[0027] Preferably, the trajectory tracking of step 2-2 is specifically:

[0028] The position information including the worker anchor frame and the feature information including the worker feature map of the adjacent frame are cascaded and matched to realize the effect of multi-target tracking;

[0029] When processing the position information, an 8-dimensional state space describes the state of the target and the motion information in the image coordinate system, and respectively represent the center coordinates of the target detection frame, and respectively represent the width-height ratio and height of the detection frame, represent the relative speed of the first four parameters in the image coordinate; using a standard Kalman filter with a constant speed model and a linear observation model, the detection frame parameters are used as direct observation values of the object state, using the Hungarian algorithm to match the predicted frame and the tracking frame using the motion information;

[0030] For motion information, the Mahalanobis distance is used to describe the correlation degree between the Kalman filter prediction result and the detector result, as shown in the formula:

[0031]

[0032] In the formula, and respectively represent the state vectors of the th detection result and the th prediction result, represent the covariance matrix between the detection result and the average tracking result;

[0033] The cost matrix used by the Hungarian algorithm includes two parts: one part is calculated by a motion model to measure the distance between the trajectory and the bounding box of the observation; the other part is calculated by an appearance model to measure the similarity between the trajectory and the feature of the observation box; finally, the motion model and the appearance model are weighted and summed to obtain the comprehensive matching degree of the trajectory and the observation, and then data association is performed through the comprehensive matching degree.

[0034] Preferably, the step 3 comprises the following steps:

[0035] Step 3-1: According to the results of target detection, the images of each construction worker are recognized and cropped by the anchor frame to obtain the specific image of a single person, and each obtained image is defined as , wherein , respectively represent the height and width of the image;

[0036] Step 3-2: Training the attribute recognition classifier;

[0037] In training the attribute recognition classifier, the input single image of the construction worker is divided into two parts according to the vertical overall proportion, representing the upper body and the lower body, that is, the input vertical of each image is divided into regions of different granularities, which are defined as and , and the network is trained by extracting local features of the upper and lower parts and global features;

[0038] Step 3-3: The processed image is input into the trained model for feature extraction to obtain the output tensor defined as , wherein , , respectively represent the height, length and channel number of the output tensor, and finally the recognition output is obtained through the average pooling layer and the fully connected layer activation , as follows:

[0039]

[0040] , wherein represents the average pooling layer, represents the fully connected layer, represents the sigmoid activation function, and then the safety operation state of the construction worker is judged according to the probability of outputting each safety attribute, and the recognized illegal behavior is output.

[0041] The beneficial effects of the present application are as follows:

[0042] At present, the safety monitoring technology of related enterprises mostly adopts manual patrol or video monitoring, and the video monitoring needs manual uninterrupted watching of monitoring video, and due to the huge number of monitoring videos and long-term watching, visual fatigue is caused, and it is difficult to avoid omissions. The present application aims at the defects of the existing safety monitoring scheme needing a large amount of manpower investment and low efficiency, according to the actual monitoring scene demand, a construction scene intelligent safety monitoring system based on detection, tracking and attribute recognition is proposed, at the same time, an attribute recognition algorithm is introduced to identify the safety operation state of construction personnel, which greatly reduces the detection difficulty problem caused by light, obstacles to a certain extent. The system collects real-time video images of the construction scene through a network camera, sends the video images to a server for back-end AI intelligent analysis and processing, and alarms once illegal behavior is found, realizing remote monitoring of the safety operation state of construction personnel, reducing the labor cost of the safety monitoring system while ensuring the effect of safety supervision, and providing a new scheme for better playing the role of machine vision in industrial safety. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 It is the overall scheme diagram of the intelligent safety monitoring system of the present application.

[0044] Figure 2 It is the worker trajectory tracking algorithm flowchart of the present application.

[0045] Figure 3 It is the safety state recognition algorithm flowchart of the present application.

[0046] Figure 4 It is the information fusion processing module schematic diagram of the present application.

[0047] Figure 5 It is the man-machine interface designed for the building construction scene of the present application. DETAILED DESCRIPTION

[0048] The present application is further illustrated below in combination with the drawings and examples.

[0049] The present application aims at the defects of requiring a large amount of manpower investment and low efficiency of the existing safety monitoring scheme, and proposes a construction scene intelligent safety monitoring system based on detection, tracking and attribute recognition according to the actual monitoring scene demand, which can realize remote supervision of the safety operation state of construction personnel in an electronic mode, and through algorithm recognition and analysis warning for dangerous actions and inadequate protection of personnel in the field operation, the machine vision can better play a role in industrial safety, so as to reduce the labor cost of the safety monitoring system while ensuring the effect of safety supervision. The system can be divided into two modules, a hardware module and a video image intelligent processing module. In the video image intelligent processing module, the position of the construction personnel is first detected by the target detection technology, then the safety operation state is recognized as the attribute of the human body to identify the unsafe behavior of the construction personnel, such as not wearing a safety helmet, not wearing a safety belt during high-altitude operation, etc., and finally the multi-target tracking technology is used to track and alarm the personnel who do not comply with the safety regulations. The overall system scheme design includes the following steps:

[0050] Step 1: Collect real-time video images of the construction scene to obtain the data source of the intelligent processing module;

[0051] Step 2: Real-time detection and trajectory tracking of the collected video for construction personnel;

[0052] Step 3: Safety state recognition of construction personnel according to the detection result image;

[0053] Step 4: Fusion processing of the information of detection, recognition and tracking;

[0054] Step 5: Output the results obtained by real-time processing to the device for visual display.

[0055] Among them, steps 1 and 5 belong to the hardware module, and steps 2, 3 and 4 belong to the video image intelligent processing module.

[0056] The specific process of step 1 is as follows: connect the network of construction site video monitoring cameras to the server through a local area network, so that the monitoring camera and the server are in the same local area network. The monitoring camera sends the collected real-time video images of the construction scene to the server equipped with the video image intelligent processing module through the RTSP protocol, and the server continuously receives the construction scene video images transmitted by the monitoring camera and processes them.

[0057] The specific process of step 2 is as follows:

[0058] Step 2-1: In the personnel detection part, use the pre-trained target detector on the data set to detect the construction personnel in the collected actual construction scene video images, and use anchor boxes (matrix boxes) to label the detected construction personnel on the image.

[0059] Step 2-2: In the trajectory tracking part, the application mainly tracks based on two features, one is the motion feature, according to the anchor frame position of the detected construction personnel in the continuous frames, the Markov distance is calculated, and the motion feature of the target is obtained through Kalman filtering, and the adjacent two frames are matched through the Hungarian matching algorithm; the second is the appearance feature, because the posture, clothing, etc. of each person are different, so the corresponding appearance feature information is also different, the application extracts the appearance feature information through the convolutional neural network model, and finally combines the matching results of the appearance feature and the motion feature to obtain the final tracking result, and assigns an ID, so as to realize the counting and trajectory tracking of the construction personnel.

[0060] Step 3: In order to further reduce the influence caused by light, shielding and other reasons, and improve the recognition accuracy, the application identifies the safety operation state of the construction personnel as a kind of semantic attribute of the image. First, according to the anchor frame of the target detection result, the images of each construction personnel are cropped to obtain the specific image of a single person. The application identifies the specific image of a single person, which can exclude the interference caused by the safety attribute factors in some scenes, so that the system pays more attention to the safety operation state of the construction personnel itself, which meets the needs of the actual construction scene. Then, the image of the construction personnel is sent to the trained attribute recognition classifier, the safety operation state of the construction personnel is judged by extracting the features of the image, and the identified illegal behavior is output, such as not wearing work clothes according to regulations, not wearing a safety helmet, not wearing a safety belt during high-altitude operation, smoking and other illegal behaviors. At present, most of the attribute recognition algorithms are identified by extracting the global features of the image, which cannot solve the problem of non-standard protection of the construction personnel, for example, the safety belt appears in the image of the construction personnel, but it is not correctly worn on the chest. In order to solve this problem and obtain more accurate recognition effect, the application divides the input single image of the construction personnel into two parts according to the vertical overall proportion during the training of the attribute recognition classifier, representing the upper body and the lower body respectively, and trains the network by extracting the local features of the upper and lower parts respectively and supplementing the global features. In this way, on the one hand, the identification result of the safety operation state is more accurate by extracting more fine-grained local information, and on the other hand, the problem caused by non-standard protection is avoided to some extent.

[0061] Step 4: In order to solve the problem of sudden alarm in the display interface caused by the misjudgment and missed detection in the identification process, the application constructs a detection, identification and tracking information fusion processing module, and places the detection result and the attribute recognition result on the trajectory of multi-target tracking for supervision. The application uses a voting method to smooth the displayed information, that is, if the identification information of this frame has a sudden change compared with the previous several frames, the state of this frame is judged through the detection state of the previous several frames, and the smoothing of the result information display is successfully realized.

[0062] The specific process of step 5 is that the server is connected with the mobile display, the server processes the received construction scene video image in real time, and the processed result is transmitted to the mobile display through a TCP protocol, and the mobile display displays. In order to better and more intuitively visualize the recognition result, the application designs a man-machine interaction interface. The designed interface has six functions, which are personnel counting, displaying the total number of people appearing in the picture and the number of people in the current picture; personnel tracking, tracking the personnel in the picture in real time and positioning; scene switching, including two working modes: daily construction and high-altitude operation, the difference between the two modes is that when daily construction, detecting that the worker does not wear a safety helmet will cause an alarm, and not wearing a safety belt will not alarm; when high-altitude operation, detecting that the worker does not wear a safety helmet or safety belt will alarm; information display, displaying the safety operation state of the recognized personnel; and operation picture display and video stream control. At the same time, the view button is added, which can view the specific construction picture of a construction personnel through the button. Embodiments

[0064] Reference Figure 1 , the application proposes a construction scene intelligent safety monitoring system based on detection, tracking and attribute recognition, which can be divided into two modules, a hardware module and a video image intelligent processing module, and the system can realize remote supervision of the safety operation state of construction personnel in an electronic mode.

[0065] 1. Collecting real-time video images of construction scenes to obtain data sources of the intelligent processing module

[0066] In this embodiment, the construction site video monitoring camera network is connected with the server through a local area network, so that the monitoring camera and the server are in the same local area network. The monitoring camera sends the collected real-time video images of the construction scene to the server carrying the video image intelligent processing module through the RTSP protocol. This protocol supports playing, pausing, rewinding, advancing and other operations on streaming media. The server continuously receives the construction scene video image information transmitted by the monitoring camera and processes it.

[0067] 2. Real-time detection and trajectory tracking of construction personnel on the collected video images

[0068] In terms of real-time detection of personnel, the embodiment selects YOLOX network as the detector, which is an anchor-free end-to-end target detection framework constructed on the basis of YOLOv3 by introducing head decoupling, data enhancement, Anchor Free and SimOTA sample matching methods. The video sequence collected by the camera network is sent to the pre-trained YOLOX detector through the server to detect the construction personnel, and the detection result mainly includes the position information of the worker, and the detected construction personnel is labeled on the video image using the anchor box (matrix box).

[0069] In the personnel trajectory tracking part, referring to Figure 2 , the embodiment realizes the effect of multi-target tracking by cascading matching of the construction personnel detection results of adjacent frames, including position information (worker anchor box) and feature information (worker feature map). When processing the position information, an 8-dimensional state space is used to describe the state of the target and the motion information in the image coordinate system. and represent the center coordinates of the target detection box, and represent the width-height ratio and height of the detection box respectively, represent the relative speed of the first four parameters in the image coordinates. The algorithm uses a standard Kalman filter with a constant speed model and a linear observation model, and takes the detection box parameters as the direct observation value of the object state. The algorithm uses the Hungarian algorithm to match the predicted box and the tracking box using the motion information. For motion information, the algorithm uses Mahalanobis distance to describe the correlation between the Kalman filter prediction result and the detector result, as shown in the formula:

[0070]

[0071] In the formula, and represent the state vectors of the th detection result and the th prediction result respectively, The covariance matrix between the detection result and the average tracking result is represented. The Mahalanobis distance takes into account the uncertainty of state estimation by measuring the standard deviation of the detection result from the average tracking result, which can exclude the possibility of low correlation. However, matching only by measuring the distance similarity can lead to a serious identity switch. In order to reduce the identity transformation frequency, in addition to measuring the distance similarity, the feature similarity also needs to be measured, so the cost matrix used by the Hungarian matching algorithm mainly includes two parts: one part calculates the distance between the bounding box of the trajectory and the observation result through the motion model; the other part measures the similarity of the features of the trajectory and the observation box through the appearance model. Finally, the motion model and the appearance model are weighted and summed to obtain the comprehensive matching degree of the trajectory and the observation, and then the data association is performed through the comprehensive matching degree.

[0072] 3. Recognizing the safety state of the construction personnel according to the detection result image

[0073] Referring to Figure 3 , the safety operation state of the construction personnel is recognized as a semantic attribute of an image, which further reduces the influence caused by factors such as light and shielding, and improves the recognition accuracy. The specific process is as follows:

[0074] Step 3-1: Process state recognition data input. The current safety monitoring system rarely considers the interference of scene factors, for example, the placement of a safety helmet appears in the construction site scene, but it is not worn on the human body. In order to solve this problem, the embodiment first anchors the image of each construction personnel according to the detection result to obtain the specific image of a single person, and the specific image of a single person is recognized to exclude the interference of safety attribute factors in the scene, so that the system pays more attention to the safety operation state of the construction personnel itself, which meets the needs of the actual construction scene, and finally defines each obtained image as ;

[0075] Step 3-2: Train the attribute recognition classifier. Most of the current attribute recognition algorithms are recognized by extracting global features of the image, which cannot solve the problem of non-standard protection of the construction personnel, for example, a safety belt appears in the image of the construction personnel, but it is not correctly worn on the chest, or a safety helmet appears, but it is not correctly worn on the head. In order to solve this problem and obtain more accurate recognition results, the embodiment divides the input single image of the construction personnel into two parts according to the vertical overall proportion when training the attribute recognition classifier, representing the upper body and the lower body, i.e. the input vertical direction of each image is divided into regions of different granularities, defined as and , the network is trained by extracting local features of the upper and lower parts and supplementing global features, which on the one hand makes the recognition result of the safe operation state more accurate by extracting more fine-grained local information, and on the other hand avoids the problem caused by non-standard protection to a certain extent;

[0076] Step 3-3: The processed image is input into the model trained in the embodiment for feature extraction, and the output tensor defined as , wherein , , respectively represent the height, length and channel number of the output tensor, and finally the recognition output , as shown below:

[0077]

[0078] wherein, represents the average pooling layer, represents the full connection layer, represents the sigmoid activation function, and finally the possible probability of each safety attribute output is used to judge the safety operation state of the construction personnel and output the recognized violation behaviors, such as not wearing work clothes, not wearing safety helmets, not wearing safety belts, not wearing insulating shoes and smoking and other violation behaviors.

[0079] Step 4: Fusion processing of detection, recognition and tracking information

[0080] After obtaining the detection, recognition and tracking results, in order to solve the problem of sudden alarm in a frame in the display interface caused by false detection and missed detection in the recognition process, the application constructs a detection, recognition and tracking information fusion processing module, as shown in Figure 4 , the detection result and the attribute recognition result are placed on the multi-target tracking track for supervision. In the embodiment, a voting method is used to smooth the displayed information, that is, if the recognition information of the frame is changed compared with the previous 5 frames, for example, the target is not detected, then the detection state of the previous thirty frames is used to judge the state of the frame, if the number of undetected frames is more than the number of detected frames in the previous thirty frames, it is determined that the target has disappeared, otherwise it is determined that the target exists, the recognition result is corrected according to the multi-target tracking track, and the smooth display of the result information is realized.

[0081] Step 5: Output the result obtained by real-time processing to the device for visual display

[0082] The embodiment connects the server with the mobile display in the control room, the server processes the received construction scene video image in real time, and transmits the processed result to the mobile display through the TCP protocol, the mobile display displays the result, and the manager views the result, when the result shows that the worker violates the rules, the operation system controls the hardware equipment in the construction site to give an audible and visual alarm and sends a short message to the safety supervisor, the server labels the processed detection result on the video image, and then transmits the real-time scene image to the mobile display terminal such as a mobile phone and a notebook computer, so that the safety supervisor can view the real-time situation of the construction site.

[0083] Referring to Figure 5 In order to better and more intuitively display the recognition result, the embodiment designs a man-machine interaction interface. The designed interface has six functions, which are personnel counting, personnel tracking, scene switching, information display, picture display and video stream control, and the specific functions are introduced as follows. Personnel counting displays the total number of people appearing in the picture and the number of people in the current picture; personnel tracking tracks the personnel in the picture in real time and locates them; scene switching includes two working modes: daily construction and aerial work, the difference between the two modes is that when the daily construction is detected, the worker without a safety helmet will cause an alarm, and the worker without a safety belt will not cause an alarm; when the aerial work is detected, the worker without a safety helmet or the worker without a safety belt will cause an alarm; information display displays the safety operation state of the recognized personnel; and there are operation picture display and video stream control. Meanwhile, the embodiment adds a view button, which can be used to view the specific construction picture of a construction worker.

Claims

1. A smart safety monitoring method for construction scenarios based on machine vision, characterized in that, Includes the following steps: Step 1: Construct a smart safety monitoring system for construction scenarios; collect real-time video images of construction scenarios and obtain data sources for the intelligent processing module; The intelligent safety monitoring system for the construction scene includes video surveillance cameras, a server, and a mobile display; the video surveillance cameras are connected to the server via a local area network, and the server is connected to the mobile display via a network; the server is equipped with a video image intelligent processing module. The surveillance camera transmits real-time video images of the construction scene to the server via the RTSP protocol. The processed surveillance images are then transmitted to a mobile display via the network. Step 2: The video image intelligent processing module performs real-time detection and trajectory tracking of construction personnel in the acquired video; Step 2-1: In the personnel detection part, the target detector pre-trained on the dataset is used to detect construction personnel in the collected actual construction scene video images, and the detected construction personnel are marked on the images using anchor boxes. The personnel testing is specifically as follows: We selected the YOLOv3 network as the detector and introduced head decoupling, data augmentation, anchor-free, and SimOTA sample matching methods to construct an anchor-free end-to-end target detection framework. The video sequences acquired by the camera network were fed into the pre-trained YOLOv3 network via a server to detect construction workers. The detection results included the workers' location information, and the detected construction workers were labeled with anchor boxes on the video images. Step 2-2: In the trajectory tracking part, tracking is based on two features: first, motion features. Based on the anchor frame positions of the detected construction workers in consecutive frames, the Markov distance is calculated, and the motion features of the target are obtained through Kalman filtering. The Hungarian matching algorithm is used to match adjacent frames. Second, appearance features. Each person's posture and clothing are different, so they also have different appearance feature information. The appearance feature information is extracted through a convolutional neural network model. Finally, the final tracking result is obtained by combining the cascaded matching results of appearance features and motion features, and an ID is assigned to realize the counting of construction workers and the tracking of their trajectories. Trajectory tracking specifically refers to: By cascading and matching the detection results of construction workers in adjacent frames, including location information (worker anchor frames) and feature information (worker feature maps), the effect of multi-target tracking can be achieved. An 8-dimensional state space is used when processing location information. Describe the target's state and motion information in the image coordinate system. and These represent the center coordinates of the target detection box, and These represent the aspect ratio and height of the detection frame, respectively. This represents the relative velocity of the first four parameters in the image coordinates; a standard Kalman filter with a constant velocity model and a linear observation model is used to filter the detection box parameters. As a direct observation of the object's state, motion information is used to match the predicted bounding box and the tracking box using the Hungarian algorithm; For motion information, Mahalanobis distance is used to describe the correlation between the Kalman filter prediction results and the detector results, as shown in the formula: In the formula, and They represent the first The first test result and the first The state vector of the prediction result This represents the covariance matrix between the detection results and the average tracking results; The cost matrix used in the Hungarian algorithm consists of two parts: one part calculates the distance between the bounding boxes of the trajectory and the observation results through the motion model; the other part measures the similarity of the features of the trajectory and the observation box through the appearance model; finally, the motion model and the appearance model are weighted and summed to obtain the comprehensive matching degree of the trajectory and the observation, and then the data is associated based on the comprehensive matching degree. Step 3: Identify the safety status of construction workers based on the detection and trajectory tracking results from Step 2; The safe working status of construction workers is identified as a semantic attribute of the image. Based on the anchor boxes of the target detection results, the images of each construction worker are cropped to obtain the specific image of each individual. The images of construction workers are fed into a pre-trained attribute recognition classifier. By extracting features from the images, the safe working status of the construction workers is determined, and the identified violations are output. When training the attribute recognition classifier, the input single image of the construction worker is divided into two parts according to the overall vertical proportion, representing the upper body and the lower body respectively. The network is trained by extracting local features from the upper and lower parts respectively and supplementing them with global features. Step 4: Fusion processing of detection, identification, and tracking information; A detection, recognition, and tracking information fusion processing module is constructed. The detection results and attribute recognition results of steps 2 and 3 are placed on the trajectory of multi-target tracking for supervision. A voting method is used to smooth the tracking display information. That is, if the recognition information of a frame changes abruptly compared with the previous 5 frames, the state of the current frame is judged by the detection state of the previous 30 frames. If the number of frames in the previous 30 frames where no target is detected is more than the number of frames where target is detected, it is determined that the target has disappeared. Otherwise, it is determined that the target exists. The recognition results are corrected according to the multi-target tracking trajectory to complete the smoothing of the result information display. Step 5: Output the results of real-time processing to a mobile display for visualization; The server and the mobile display are connected via a network. The server processes the received construction scene video images in real time and sends the processed results to the mobile display for display. Design a human-computer interaction interface to be displayed on a mobile display, including six functions: People count: Displays the total number of people who have appeared in the frame and the number of people currently in the frame; Personnel tracking: Tracks and locates people in the video feed in real time; Scene switching: Includes two working modes: daily construction and high-altitude operation; the difference between the two modes is that in the daily construction mode, an alarm will be triggered if a worker is not wearing a safety helmet, but will not be triggered if a worker is not wearing a safety belt; in the high-altitude operation mode, an alarm will be triggered if a worker is not wearing a safety helmet or not wearing a safety belt. Information display: Displays the identified personnel's safe working status, the work screen display, and video stream control; View button: View the specific construction footage of the designated construction worker.

2. The intelligent safety monitoring method for construction scenarios based on machine vision according to claim 1, characterized in that, Step 3 includes the following steps: Step 3-1: Based on the target detection results, identify and crop the images of each construction worker to obtain a specific image of each individual worker. Define each obtained image as... ,in , These represent the height and width of the image, respectively. Step 3-2: Train the attribute recognition classifier; When training the attribute recognition classifier, the input single image of a construction worker is divided into two parts according to the overall vertical proportion, representing the upper body and lower body respectively. That is, the input vertical division of each image is divided into regions of different granularities, defined as follows: and The network is trained by extracting local features from the upper and lower parts and supplementing them with global features; Step 3-3: Input the processed image into the trained model for feature extraction, and the output tensor is defined as follows: ,in , , These represent the height, length, and number of channels of the output tensor, respectively. The final output is obtained after activation by an average pooling layer and a fully connected layer. As shown below: in, Represents the average pooling layer. Represents a fully connected layer. This represents the sigmoid activation function. Based on the probability of various safety attributes, the safe working status of construction workers is determined, and the identified violations are output.

Images