Large-scale construction machinery protection detection method and system based on computer vision

Abstract

The invention discloses a large-scale construction machinery protection detection method and system based on computer vision. The system comprises a hardware layer and a software layer. The hardware layer collects video streams through an industrial-grade camera, and a server runs a target detection and tracking algorithm; and the software layer realizes mechanical state judgment, dynamic region generation and a multi-level alarm mechanism. The system utilizes YOLOv8 and DeepSORT algorithms to monitor the positions of construction machinery and protection personnel in real time, dynamically delimit a safety area, trigger an alarm and record violation events to optimize safety management when no person is supervised. The problems that manual inspection efficiency is low, positioning equipment is prone to being interfered, and on-duty performance is not achieved can be solved, the safety management level of a construction scene is remarkably improved, and an innovative solution is provided for the high-strength and high-timeliness safety protection requirement along a railway.

Description

Technical Field

[0001] The present invention relates to the field of intelligent management of the construction industry and construction safety near railway operating lines, and in particular to a computer vision-based large-scale construction machinery protection detection method and system. Background Art

[0002] Safety protection is crucial in the intelligent management of the construction industry and in the safety supervision of large-scale construction machinery operations near operating lines, particularly along railway lines. The "one machine, one dedicated guard" system for operations along railway lines was developed based on lessons learned from numerous historical accidents, such as the 2008 Jiaoji Railway train collision and the 2020 incident where an excavator struck the overhead wire on a high-speed railway line. These incidents directly prompted the strengthening of the protection system. my country's "Regulations on the Safety Management of Railway Operating Line Construction" clearly stipulate that when large-scale machinery operates within the railway line's safety protection zone, a "one machine, one person" dedicated guard system must be implemented. Guards must be trained and certified. Current management solutions fall into two main categories: one manually registers the association between guards and construction machinery and conducts regular spot checks; the other relies on guards carrying positioning systems that monitor their specific location and are associated with the construction machinery. However, existing technologies have significant drawbacks. Manual registration relies on paper or spreadsheet records, which are prone to human errors and data tampering. It also fails to monitor the actual presence and performance of guards in real time, making it difficult to detect violations such as absenteeism and substitute work. Monitoring equipment that relies on GPS Beidou or base station positioning is susceptible to signal interference in complex environments such as tunnels and high-density urban areas, and its positioning accuracy is insufficient, with errors often exceeding 10 meters. It also faces problems such as short battery life, equipment damage, or human signal shielding. In addition, traditional technologies can only track location information and cannot verify whether the guards are focused on performing their duties, such as lookout or early warning. More importantly, the data isolation of traditional technologies makes it difficult to achieve linkage between mechanical operations, train scheduling, and personnel supervision, and it is impossible to build a dynamic risk warning closed loop, making it difficult to meet the high-intensity and high-efficiency safety protection needs along the railway. Therefore, there is an urgent need for an innovative solution that can achieve real-time monitoring, accurate identification, and dynamic early warning to overcome the shortcomings of existing technologies and improve the safety management level of construction machinery operations. Summary of the Invention

[0003] The purpose of the invention is to provide a large-scale construction machinery protection detection method and system based on computer vision, which solves the problems mentioned in the background technology.

[0004] The present invention is implemented as follows: a computer vision-based method and system for large-scale construction machinery protection detection. The system utilizes intelligent cameras and advanced target detection algorithms to achieve real-time monitoring and dynamic management of construction machinery and its protective personnel. The method includes the following steps: S1. Real-time detection of construction machinery and tracking of its motion; S2. Accurate identification of protective personnel positions; S3. Dynamic determination of whether the safety zone surrounding the machinery is supervised; S4. Triggering an alarm and recording violations if no supervision is available; and S5. Compilation of protective personnel performance data to optimize safety management.

[0005] Furthermore, the system is divided into a hardware layer and a software layer. The hardware layer is responsible for data acquisition, video streaming, and software deployment, while the software layer is responsible for algorithm reasoning, data push, and front-end and back-end statistical presentation. In the hardware layer, an industrial-grade dome camera that supports 4K resolution and infrared night vision is set up to collect on-site video streams and is deployed at high points of the machinery or fixed poles around it. The server is equipped with an Ubuntu operating system, running a stream acquisition module, a YOLOv8 detection model, a DeepSORT tracking algorithm, a motion state determination algorithm, etc., to process video streams and perform target detection and tracking. In addition, it also includes a 4G sound column for event alarms and the playback of intelligent voice reminders; a Windows desktop computer as a front-end presentation device to display real-time monitoring images and alarm information; and an LCD screen that displays a smart large-screen interface, providing comprehensive monitoring and management information.

[0006] Specifically, the software layer includes a stream acquisition module that uses an interface protocol to read the video stream from the front-end camera for subsequent algorithm inference. The machinery detection and tracking module, based on the YOLOv8 detection model and the DeepSORT tracking algorithm, enables real-time detection and tracking of construction machinery. The guard detection module uses customized YOLOv8 training to enhance the characteristics of reflective vests of specific colors, enabling accurate identification of guards. The dynamic safety zone generation module delineates dynamic safety zones based on the excavator's motion state. The alarm and statistics module implements a multi-level alarm mechanism, including audible alarms, SMS notifications, archiving of violation images, and event statistics.

[0007] Furthermore, the specific steps for implementing the method are as follows: S1. Data collection and model training, collect more than 10,000 images of on-site construction scenes, annotate categories such as "protection officer" (reflective vest of a specific color), "excavator", and use YOLOv8s (lightweight version) to fine-tune the COCO pre-trained model to improve detection accuracy. S2. Model deployment, deploy the trained model to the server, obtain the on-site video stream in real time, and detect the positions of the protection officer and the excavator. S3. Construction machinery tracking, use the DeepSORT algorithm, use ReID features + Kalman filtering to delineate the mechanical ID, and predict its motion trajectory to achieve real-time tracking of the excavator. S4. Mechanical construction status determination, based on the construction status determination of the spatiotemporal logical chain, obtain the target frame of the excavator through the YOLOv8 detection model, calculate its center point, and determine that its center point has a continuous change of a certain pixel threshold for several consecutive frames (assuming 15 frames), then it is determined to be under construction, otherwise it is in a stationary state. S5. Dynamic area demarcation and safety officer presence determination: In real time, a circular area is demarcated with a certain radius using the center point of the excavator's detection frame as the dot. If the machine is in operation and there is no safety officer in its dynamic area for more than 30 seconds, a level 1 alarm is triggered. If there is no safety officer on the same machine for 5 consecutive minutes, it is upgraded to a level 2 alarm (notified by SMS to the project manager). S6. Alarm and data management: Level 1 alarms use on-site sound alarms (4G sound bars) to remind safety officers to return to their positions. Level 2 alarms push SMS messages to administrators, including the location and violation screenshots. Violation clips (1 minute before and after the trigger) are uploaded to the backend, marked with time, location, and machine type. The backend generates daily / weekly reports and calculates the lack of supervision of each machine.

[0008] In particular, the beneficial effects of the system are reflected in multiple aspects. First, it has strong real-time performance. Computer vision technology is used to achieve real-time monitoring, without relying on manual inspections or positioning equipment, thus avoiding human errors and equipment signal interference. Secondly, dynamic area management dynamically defines safety areas based on the movement status of the excavator to ensure that the guards are always within the effective supervision range. Furthermore, the multi-level alarm mechanism realizes the full process of automated response from risk warning to active intervention through sound alarms, SMS notifications, etc. Finally, data tracing and statistics are carried out, and violations are automatically recorded and archived, which facilitates subsequent tracing and safety management optimization.

[0009] Furthermore, the system uses the YOLOv8 deep learning model to accurately identify reflective vests of specific colors. Specifically, the YOLOv8 model has undergone customized training to enhance its ability to detect the characteristics of reflective vests of specific colors, thereby accurately distinguishing between protective personnel and other construction site personnel. During the object detection process, each frame of the video stream is first extracted. Then, the YOLOv8 model performs forward inference, outputting the category and confidence score of each target box. Finally, the target boxes with confidence scores above the set threshold are selected as the detection results.

[0010] Furthermore, for tracking construction machinery, the DeepSORT algorithm is combined with ReID features and a Kalman filter. ReID features are used to extract the target's appearance, while the Kalman filter is used to predict its trajectory. The DeepSORT algorithm determines the target's identity by matching the target box in the current frame with its historical trajectory and updates its motion state. If the center point of the target box changes significantly over several consecutive frames, the target is considered to be in an active state; otherwise, it is considered to be stationary.

[0011] Furthermore, in the dynamic safety zone generation module, a circular area with a set radius, centered around the excavator's detection frame, is defined as the safety monitoring area. While the excavator is operating, the system continuously checks for the presence of a security guard within this area. If no security guard is detected within a set time, an alarm of the corresponding level is triggered.

[0012] Furthermore, the alarm and statistics module implements risk warnings and proactive intervention through a multi-level alarm mechanism. Level 1 alarms use a 4G speaker to play intelligent voice reminders, prompting guards to return to their positions promptly. Level 2 alarms notify administrators via text message, along with a screenshot of the violation and detailed location information, facilitating rapid response and resolution. All violations are recorded and archived, including a one-minute video clip before and after the trigger, tagged with information such as time, location, and machine type. Daily and weekly reports are generated regularly in the backend, analyzing the rate of under-regulation for each machine and providing data support for safety management.

[0013] In summary, the present invention uses computer vision technology to achieve intelligent supervision of large-scale construction machinery and its protective personnel, solving the problems of low efficiency, susceptibility to interference, and "on-the-job failure" in the existing technology, significantly improving the safety management level of construction scenes, and providing an innovative solution to the high-intensity and high-efficiency safety protection needs along the railway. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of the present invention, showing the complete steps from data collection and model training to multi-level alarm and data tracing;

[0015] Figure 2 This is a detection diagram of the present invention, showing multiple excavators and a security guard wearing a red reflective vest. The target detection module marks the excavators with a red frame and the security guard with a green frame.

[0016] Figure 3 This is a schematic diagram of the dynamic area generation of the present invention. The upper figure shows the excavator in the actual working scene and the predicted working area represented by the blue dotted circle. The lower figure is a schematic diagram of the dynamic working area division, including the predicted working area (blue dotted line), the actual working area (red solid line), and the maximum working area (red dotted line);

[0017] Figure 4 This is the hardware architecture diagram of the present invention, showing the deployment and functions of the camera, server, 4G sound bar, Windows desktop and LCD screen.

[0018] The accompanying drawings are numbered as follows:

[0019] 1. Camera; 2. Server; 3. 4G speaker; 4. Windows desktop; 5. LCD screen; 6. Excavator; 7. Security guard; 8. Predicted working area; 9. Actual working area; 10. Maximum working area. DETAILED DESCRIPTION

[0020] The present invention provides a large-scale construction machinery protection detection method and system based on computer vision, which realizes real-time monitoring and dynamic management of construction machinery and its protection personnel by deploying intelligent cameras and advanced target detection algorithms. Figure 1 To the attached Figure 4 , the specific implementation methods of the present invention are described in detail below.

[0021] In practice, the system hardware layer consists of camera 1, server 2, 4G speaker 3, Windows desktop 4, and LCD screen 5. Camera 1 is an industrial-grade dome camera with 4K resolution and infrared night vision. It is mounted at a high point or on a fixed pole at the construction site to collect video stream data. Server 2 runs the Ubuntu operating system and is equipped with a stream acquisition module, a YOLOv8 detection model, a DeepSORT tracking algorithm, and other related software modules. It processes the video stream and performs target detection and tracking. 4G speaker 3 plays alarm notifications. Windows desktop 4 serves as the front-end display device, displaying the monitoring images. LCD screen 5 displays a large, intelligent interface, allowing managers to fully understand the on-site situation.

[0022] S1 data collection and model training phase is the starting point of the entire system. Figure 1As shown in the figure, at this stage, at least 10,000 construction site images must be collected. These images contain two types of objects: an excavator 6 and a security guard 7 wearing a reflective vest of a specific color. Manual annotation is used to identify the location and category of each object to form a training dataset. Subsequently, a lightweight version of the YOLOv8s model is selected and fine-tuned using COCO pre-trained weights. During training, customized enhancements are performed to the specific color of the security guard 7's reflective vest to ensure the model can accurately identify the security guard. After multiple rounds of iterative optimization, a target detection model suitable for construction sites is obtained.

[0023] During the S2 model deployment phase, the trained YOLOv8s model is loaded into Server 2. Server 2 uses the stream acquisition module to read the real-time video stream transmitted by Camera 1 and extract image data frame by frame. After each frame is input into the YOLOv8s model, the model outputs the detection box locations, category labels, and corresponding confidence scores for excavator 6 and security guard 7. To improve detection accuracy, a confidence threshold of 0.7 is set to select high-confidence target boxes as the final detection results. At this point, the system is able to detect the location information of excavator 6 and security guard 7 in real time.

[0024] During the S3 construction machinery tracking phase, the DeepSORT algorithm is used to track Excavator 6 in real time. The DeepSORT algorithm combines ReID features with Kalman filtering technology to first extract the appearance features of Excavator 6's target frame and then predict its motion trajectory. In each frame, the system determines Excavator 6's identity ID by matching the current target frame with its historical trajectory and updates its motion state. Furthermore, the system determines Excavator 6's construction status based on a spatiotemporal logic chain. For example, if the displacement of the center point of Excavator 6's target frame exceeds a set pixel threshold within 15 consecutive frames, it is determined to be in the construction state; otherwise, it is determined to be stationary.

[0025] The core of the S4 dynamic area demarcation and guard on-duty determination stage is to generate a dynamic safety area for the excavator 6. Figure 3 As shown, the system defines a circular area with a set radius, centered around the center of the detection frame for excavator 6, as the dynamic safety monitoring range. This radius can be adjusted based on actual needs; for example, the default value is 10 meters. When excavator 6 is working, the system continuously checks for the presence of a safety officer 7 within this area. If no safety officer 7 is detected within 30 seconds, a level 1 alarm is triggered. If the same machine is absent from a safety officer 7 for five consecutive minutes, a level 2 alarm is triggered.

[0026] The S5 alarm and data management phase utilizes a multi-tiered alarm mechanism to enable risk early warning and proactive intervention. Level 1 alarms are activated by a 4G sound column 3, which plays an intelligent voice reminder, prompting the guard 7 to return to their post promptly. Level 2 alarms notify administrators via text message, including the specific location and screenshots of the violation. The system automatically records and archives the violation, storing a one-minute video clip before and after the trigger, tagged with key information such as time, location, and machine type. The backend regularly generates daily and weekly reports, analyzing the rate of missed inspections for each machine, providing data support for subsequent safety management optimization.

[0027] The implementation process of the above steps fully demonstrates the technical advantages of the present invention. For example, through customized training of the YOLOv8s model, the system significantly improves the recognition ability of reflective vests of specific colors, thereby avoiding the error and omission problems in the traditional manual registration method. In addition, the DeepSORT algorithm combined with the Kalman filter achieves accurate tracking of the excavator 6, effectively solving the problem that GPS positioning equipment is susceptible to signal interference. Figure 2 A schematic diagram of the detection results is shown, in which the red box marks the excavator 6 and the green box marks the guard 7, which intuitively reflects the detection effect of the system in complex scenarios.

[0028] The design of dynamic safety zone generation module further enhances the practicality of the system. Figure 3 As shown, the blue dashed line represents the predicted working area 8, the red solid line represents the actual working area 9, and the red dashed line represents the maximum working area 10. The system dynamically adjusts the safety zone based on the motion of the excavator 6, ensuring that the safety officer 7 remains within the effective monitoring range. This design not only improves detection accuracy but also significantly reduces false alarm rates.

[0029] The alarm and statistics modules function throughout the entire system. Level 1 alarms provide instant voice alerts via the 4G speaker 3, quickly guiding the security guard 7 back to their assigned position. Level 2 alarms notify administrators via text message, ensuring a rapid response to violations. All violations are fully recorded and uploaded to the backend database for easy tracing and analysis. Daily and weekly reports generated by the system provide management with detailed data support, facilitating the development of more effective safety management strategies.

[0030] In summary, the present invention uses computer vision technology to achieve intelligent supervision of large-scale construction machinery and its protective personnel, solving the problems of low efficiency, susceptibility to interference, and "on-the-job failure" in existing technologies. From data collection and model training to multi-level alarms and data tracing, each link has been carefully designed to ensure that the system can operate stably in complex construction scenarios. Figure 1 To the attached Figure 4It clearly demonstrates the specific details of the system architecture, detection principles, dynamic region generation, and hardware deployment, providing a complete implementation reference for technicians.

[0031] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large-scale construction machinery protection detection method based on computer vision, characterized in that: The large-scale construction machinery protection detection method based on computer vision includes the following steps: S1: Real-time detection of construction machinery and tracking of its motion status; S2: Accurately identify the location of protective personnel; S3: Dynamically determine whether there is anyone supervising the safety area around the machine; S4: Trigger an alarm and record violations when there is no supervision; S5: Collect data on the performance of protective personnel to optimize safety management.

2. A large-scale construction machinery protection detection method based on computer vision as claimed in claim 1, characterized in that: The real-time detection of construction machinery and tracking of its motion state are achieved through the YOLOv8 detection model and the DeepSORT tracking algorithm, wherein the DeepSORT algorithm combines ReID features with Kalman filtering technology to perform identity matching and trajectory prediction on construction machinery.

3. A large-scale construction machinery protection detection method based on computer vision as claimed in claim 2, characterized in that: The continuous frame number threshold is further set to 15 frames. When the displacement change of the center point of the target frame exceeds the set pixel value, it is determined that the construction machinery is in the construction state.

4. A large-scale construction machinery protection detection method based on computer vision as claimed in claim 4, characterized in that: The dynamic determination of whether there is anyone supervising the safety area around the machine is performed by demarcating a circular area with the center point of the construction machinery detection frame as the center, and detecting the presence status of the protective personnel in the area.

5. A large-scale construction machinery protection detection method based on computer vision as claimed in claim 4, characterized in that: The radius of the circular area is further set to 10 meters, and a level one alarm is triggered when the construction machinery is in operation and no protective personnel are detected in the area for 30 seconds.

6. A large-scale construction machinery protection detection method based on computer vision as claimed in claim 5, characterized in that: The trigger alarm includes a first-level alarm that plays a voice reminder through the 4G speaker, and a second-level alarm that notifies the administrator via text message with a screenshot of the violation.

7. A large-scale construction machinery protection detection method based on computer vision as claimed in claim 6, characterized in that: The statistical data on the performance of duties of protection personnel include recording the time, location, machine type and 1 minute video clips before and after the triggering of the violation, and generating daily and weekly reports.

8. A large-scale construction machinery protection detection system based on computer vision, characterized in that: The computer vision-based large-scale construction machinery protection and detection system includes: a hardware layer and a software layer. The hardware layer is responsible for data acquisition, video streaming, and software deployment, while the software layer is responsible for algorithm reasoning, data push, and front-end and back-end statistical presentation.

Images