Construction safety supervision method, system and equipment based on artificial intelligence

Abstract

The invention discloses a construction safety supervision method, supervision system and equipment based on artificial intelligence, the method is executed in the supervision system, the supervision system comprises a plurality of cameras and a server, and a visible light camera and an infrared camera are adopted to collect images of a construction engineering construction site and transmit the images to the server; visible light and infrared images are fused, then YOLOv9 is used for training and model establishment, whether a worker wears a safety helmet and a reflective vest or not is trained and recognized in real time, the recognition result is transmitted back to a database after the images are recognized, and finally a supervision system positions the situation which does not meet safety regulations and gives an alarm. According to the method, an artificial intelligence identification mode is used for replacing a traditional manual monitoring mode, accidents are reduced, and meanwhile the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to an artificial intelligence-based construction safety supervision method, supervision system and equipment. Background Art

[0002] From a global perspective, the incidence of safety accidents in engineering construction has always been the highest among all industries. This has a significant impact on the lives of workers and the productivity of the industry. Among them, unsafe behavior of construction workers is the main factor in the occurrence of safety accidents. Even if cameras are installed on the construction site or supervisors are sent to monitor whether the workers are working in accordance with safety management regulations, this traditional method not only consumes manpower costs but also easily distracts managers and creates dangers.

[0003] Construction projects are closely related to people's daily lives. Whether it is the construction of road pipelines or subway lines, workers need to wear safety helmets and reflective vests during construction. The purpose is not only to protect the workers themselves, but also to remind drivers and passers-by in the evening or at night. At present, many domestic construction sites have set up cameras to monitor the safety of workers, but due to the high cost of manpower and time, the effect of preventing danger is not good.

[0004] Therefore, how to effectively monitor the construction site and control labor costs has become one of the urgent problems to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a construction safety supervision method, supervision system and equipment based on artificial intelligence. The method uses artificial intelligence automatic recognition to replace traditional monitoring methods, so as to improve the quality of labor and reduce accidents, while reducing labor costs.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A construction safety supervision method based on artificial intelligence is implemented in a supervision system, the supervision system includes a plurality of cameras and a server, the cameras include a visible light camera and an infrared camera, and the method includes the following steps: S1: image acquisition step, using a camera to collect images of the construction site; S2: Image fusion step, using YOLOv9 multimodality to achieve visible light and infrared image fusion; S3: Classification step, classify and label the database according to safety management regulations; S4: Model training step, the labeled data is trained using YOLOv9, and the trained model is used to test the data of the test set; S5: Violation alarm step, using the model to perform real-time monitoring. When a violator appears in the picture within a period of time, the person will be recorded. When the number of records reaches a predetermined number, an alarm message will be issued.

[0007] Preferably, the step S1 specifically includes: S11: collect visible light images and infrared images simultaneously; S12: Perform registration based on the scale-invariant feature transformation algorithm and unify the image size.

[0008] Preferably, the step S2 specifically includes: The multimodal function of YOLOv9 is adopted, visible light images and infrared images are input simultaneously, and image fusion is realized with the help of the front-end fusion network.

[0009] Preferably, the step S3 specifically includes: S31: According to the safety management regulations, it is divided into four categories: wearing a helmet, not wearing a helmet, wearing a reflective vest and not wearing a reflective vest; S32: Use the AI ​​model target detection labeling tool Makesense.ai to label the data.

[0010] Preferably, the step S4 specifically includes: YOLOv9 combines programmable gradient information PGI and generalized efficient layer aggregation network GELAN to reduce incorrect predictions caused by information loss and improve the accuracy and efficiency of real-time target detection.

[0011] Preferably, the step S4 further comprises: The PGI includes auxiliary reversible branches and multi-level auxiliary information, including auxiliary reversible branches and multi-level auxiliary information, the auxiliary reversible branches are used to generate reliable gradients, and the multi-level auxiliary information is used to integrate the gradient information of different prediction heads. PGI provides reliable gradients through auxiliary reversible branches and multi-level auxiliary information, and guides the main branch learning to form a complementary and enhanced network structure.

[0012] Preferably, the step S4 further comprises: The GELAN reduces redundant computation by splitting and merging feature maps and uses layer aggregation to enhance the representation capability of features. GELAN allows the use of various types of computational blocks within its framework to optimize model performance and efficiency.

[0013] Preferably, the step S5 specifically includes: When the camera footage at the construction site shows a person violating the rules within a certain period of time, a record will be taken and the number of records within the preset time will be recorded. When the number of records reaches the preset number and there is no violation record after the preset time interval, the calculation will be reset to zero to avoid false alarms caused by accumulated records caused by passers-by or background objects. If a violation record appears again, an alarm message will be issued using LINE Notify.

[0014] On the other hand, the present invention also discloses a construction safety supervision system based on artificial intelligence, which executes the above safety supervision method and specifically includes the following modules: An image acquisition module, used to acquire images of a construction site using a camera; Image fusion module, used to achieve visible light and infrared image fusion using YOLOv9 multimodality; The labeling category module is used to classify and label the database according to safety management regulations; The model training module is used to train the labeled data using YOLOv9 and use the trained model to test the data of the test set; The violation alarm module is used to use the model to perform real-time monitoring. When a violator appears in the picture within a period of time, it will be recorded, and an alarm message will be issued after the record reaches a predetermined number of times.

[0015] On the other hand, the present invention also discloses an artificial intelligence-based construction safety supervision device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned safety supervision method when executing the computer program.

[0016] The present invention uses a safety supervision system to conduct safety supervision on the construction site of a construction project. Based on image recognition YOLOv9, the present invention trains relevant data to automatically and in real time identify whether workers on site are wearing safety helmets and reflective vests, thereby reducing traditional manpower costs, issuing notification reminders to reduce the occurrence of accidents, and reducing safety risks.

[0017] The safety supervision system of the present invention realizes automation and intelligence of engineering supervision, which not only solves the complicated manual supervision process but also reduces the workload of manual supervision.

[0018] Compared with the prior art, the present invention provides a safety supervision method, safety supervision system and device for a construction site based on artificial intelligence, which has the following beneficial effects: 1. By installing visible light and infrared cameras, the advantages of infrared images and visible light images are combined to improve recognition accuracy; 2. YOLOv9 combines programmable gradient information PGI and generalized efficient layer aggregation network GELAN to reduce false predictions caused by information loss and improve the accuracy and efficiency of real-time target detection; 3. Data labeling is achieved by using AI model target detection and labeling tools, and intelligent means are used to achieve automatic alarm reminders; 4. By adopting edge computing devices, the limitation that the training model can only be used on the server side is solved, the pressure on cloud resources and server load is reduced, and energy and costs are saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a safety supervision method for a construction site of a building project disclosed in the present invention Figure 2 A flowchart of the YOLOv9 model training in the security monitoring system disclosed in the present invention Figure 3 A schematic diagram of the marking categories in the safety supervision system disclosed in the present invention Figure 4 Example of a rectangular box labeled for the AI ​​model target detection labeling tool disclosed in the present invention Figure 5 This is the mAP result diagram after model training disclosed by the present invention Figure 6 The system architecture diagram of the edge computing device Jetson Nano disclosed in the present invention DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Example 1

[0021] A construction safety supervision method based on artificial intelligence is implemented in a supervision system, the supervision system includes a plurality of cameras and a server, the cameras include a visible light camera and an infrared camera, and the method includes the following steps: S1: image acquisition step, using a camera to collect images of the construction site; S2: Image fusion step, using YOLOv9 multimodality to achieve visible light and infrared image fusion; S3: Classification step, classify and label the database according to safety management regulations; S4: Model training step, the labeled data is trained using YOLOv9, and the trained model is used to test the data of the test set; S5: Violation alarm step, using the model to perform real-time monitoring. When a violator appears in the picture within a period of time, the person will be recorded. When the number of records reaches a predetermined number, an alarm message will be issued.

[0022] Most current target detection algorithms are mainly based on visible light images. In the case of sufficient light, visible light cameras can effectively capture information such as the color and texture of the target. However, during the construction process at a construction site, due to interference from various environmental factors, such as occlusion, fog, and uneven lighting, visible light cameras often find it difficult to obtain complete target information; while infrared cameras are less affected by light and can provide clear contour information under insufficient light conditions. In order to improve the accuracy of target recognition, the present invention uses visible light cameras and infrared cameras to collect images.

[0023] Preferably, the step S1 specifically includes: S11: collect visible light images and infrared images simultaneously; S12: Perform registration based on the scale-invariant feature transformation algorithm and unify the image size.

[0024] The collected visible light images and infrared images are registered based on the scale-invariant feature transformation algorithm, which includes two main steps: feature matching and affine transformation. After the registration, the resolutions of the visible light and infrared images are set to the same.

[0025] The present invention adopts the YOLOv9 target recognition model, which is more outstanding in accuracy, efficiency and applicability compared with previous versions. YOLOv9 introduces two key technologies: a new model architecture called General Efficient Layer Aggregation Network GELAN maximizes accuracy while minimizing parameters and number of failures. A training technology called Programmable Gradient Information PGI provides more reliable learning gradients, especially for smaller models.

[0026] By combining the architectural advances of GELAN with the training improvements of PGI, YOLOv9 achieves unprecedented efficiency and performance: compared to previous YOLO versions, YOLOv9 achieves higher accuracy, 10%-15% fewer parameters, and 25% less computation. This brings significant improvements in speed and functionality across model sizes. YOLOv9 surpasses other real-time detectors such as YOLO-MS and RT-DETR in terms of parameter efficiency and FLOPs. It requires fewer resources to achieve a given level of performance, and the smaller YOLOv9 model even beats larger pre-trained models such as RT-DETR-X. Despite using 36% fewer parameters, YOLOv9-E achieves better results.

[0027] Preferably, the step S2 specifically includes: The multimodal function of YOLOv9 is adopted, visible light images and infrared images are input simultaneously, and image fusion is realized with the help of the front-end fusion network.

[0028] The specific steps to implement multimodal fusion in YOLOv9 include: Feature extraction: feature extraction is performed on visible light images and infrared images respectively. Commonly used feature extraction methods include convolutional neural networks; Feature fusion, the features of two modalities are combined to obtain a comprehensive feature representation. Common fusion methods include weighted fusion and cascade fusion.

[0029] Preferably, the step S3 specifically includes: S31: According to the safety management regulations, it is divided into four categories: wearing a helmet, not wearing a helmet, wearing a reflective vest and not wearing a reflective vest. For details, please refer to the attached Figure 3 ; The present invention follows the key points of supervision and inspection in the safety management regulations. In order to prevent accidents, workers must wear safety helmets and reflective vests with bright colors during road operations and road construction. The selection of categories is divided into four categories according to the above, including wearing safety helmets, not wearing safety helmets, wearing reflective vests and not wearing reflective vests.

[0030] (1) Wearing a safety helmet: To prevent collisions, road construction workers are required to wear a safety helmet that meets safety standards. Even if an accident unfortunately occurs, it can reduce the damage caused by the accident.

[0031] (2) Not wearing a helmet: When an accident occurs, if you do not wear a helmet, your head may be completely unprotected, resulting in serious permanent injuries.

[0032] (3) Wear reflective vests: reflective vests can remind car and motorcycle drivers and pedestrians to notice workers. In addition to roadblocks, reflective vests are very useful in the evening and at night, and can protect workers in conditions of poor visibility at night.

[0033] (4) Not wearing a reflective vest: The purpose of wearing a reflective vest is to alert pedestrians and vehicles, while not wearing a reflective vest cannot preventively alert pedestrians and vehicles, and the incidence of accidents will increase.

[0034] S32: Use the AI ​​model target detection labeling tool Makesense.ai to label the data.

[0035] The AI ​​model target detection labeling tool Makesense.ai of the present invention performs data labeling. In addition to labeling points and lines, it can also label rectangular boxes. For details, see the attached Figure 4 , and the rectangular frame is the marking method used by the present invention, and after the marking is completed, it can be output into txt and xml files.

[0036] Preferably, the step S4 specifically includes: YOLOv9 combines programmable gradient information PGI and generalized efficient layer aggregation network GELAN to reduce incorrect predictions caused by information loss and improve the accuracy and efficiency of real-time target detection.

[0037] Preferably, the step S4 further comprises: The PGI includes auxiliary reversible branches and multi-level auxiliary information, including auxiliary reversible branches and multi-level auxiliary information, the auxiliary reversible branches are used to generate reliable gradients, and the multi-level auxiliary information is used to integrate the gradient information of different prediction heads. PGI provides reliable gradients through auxiliary reversible branches and multi-level auxiliary information, and guides the main branch learning to form a complementary and enhanced network structure.

[0038] Preferably, the step S4 further comprises: The GELAN reduces redundant computation by splitting and merging feature maps and uses layer aggregation to enhance the representation capability of features. GELAN allows the use of various types of computational blocks within its framework to optimize model performance and efficiency.

[0039] Preferably, the step S5 specifically includes: When the camera footage at the construction site shows a person violating the rules within a certain period of time, a record will be taken and the number of records within the preset time will be recorded. When the number of records reaches the preset number and there is no violation record after the preset time interval, the calculation will be reset to zero to avoid false alarms caused by accumulated records caused by passers-by or background objects. If a violation record appears again, an alarm message will be issued using LINE Notify.

[0040] The present invention uses ROC curve to evaluate performance. The main concept of ROC curve (receiver operating characteristic curve) is a binary classification model, and its output results will only have two categories, such as correct / incorrect, compliant / non-compliant, target / non-target, etc. Therefore, ROC curve is more commonly used in fields such as machine learning and target detection that require model evaluation. Some commonly used evaluation indicators, such as accuracy, precision rate (TPR), recall rate (Recall), false positive rate (FPR) and other indicator values ​​are all generated based on the confusion matrix. Accuracy is the most basic indicator for measuring model performance, that is, the percentage of correctly predicted numbers of all categories in all predicted samples. Accuracy represents the proportion of correctly predicted samples in the predicted true samples, and recall rate is the proportion of correctly predicted values ​​in the test samples that are actually true. The indicator value calculation method of each classification model is as follows: Accuracy = (TP+TN) / (TP+TN+FP+FN) TPR = TP / (TP+FP) Recall = TP / (TP+FN) False Positive Rate (FPR) = TN / (TN+FP) in: TP (True positive): The actual value is True and the prediction is True, which means the prediction is correct; FN (False negative): The actual value is True, but the prediction is False, which is an incorrect prediction; FP (False positive): The actual value is False, but the prediction is True, which is a wrong prediction; TN (True negative): The actual value is False, and the prediction is False, which is correct.

[0041] AP (Average Precision) represents the average accuracy between accuracy (vertical axis) and recall (horizontal axis). In the relationship curve between the two axes, the area under the curve represents the average accuracy of single category recognition.

[0042] mAP is the average of APs of multiple categories and is an indicator often used to judge the quality of a model’s target recognition ability. Figure 5The AP and mAP (i.e., the average of the AP of the four categories) of each category (including pwh: wearing a helmet, pwv: wearing a reflective vest, pnh: not wearing a helmet, and pnv: not wearing a reflective vest) are displayed. Figure 5 The obtained result is that mAP is as high as 0.994, which is an excellent result, indicating that both the precision and recall rates are close to the ideal value of 1. Example 2

[0043] A construction safety supervision system based on artificial intelligence, which implements the above safety supervision method, specifically includes the following modules: An image acquisition module, used to acquire images of a construction site using a camera; Image fusion module, used to achieve visible light and infrared image fusion using YOLOv9 multimodality; The labeling category module is used to classify and label the database according to safety management regulations; The model training module is used to train the labeled data using YOLOv9 and use the trained model to test the data of the test set; The violation alarm module is used to use the model to perform real-time monitoring. When a violator appears in the picture within a period of time, it will be recorded, and an alarm message will be issued after the record reaches a predetermined number of times.

[0044] If the model trained by YOLOv9 can only be used on the server side, it will be subject to more restrictions in use. Edge computing uses the computing power of edge nodes to move some computing and processing functions from the cloud to the terminal device, reducing the pressure on cloud resources and server load, saving energy and cost. The present invention selects Jetson Nano v3 as the edge computing device. The main reason for using Jetson Nano is that when the number of photographic devices increases, the server will not be able to bear the computing load.

[0045] The environment configuration of the Jetson Nano v3 edge computing device is divided into the following four steps: (1) Basic environment configuration First, the Jetson Nano v3 development kit needs to install the official image file, then connect HDMI to the monitor to complete the subsequent settings. After the settings are completed, use SSH to remotely connect to the Jetson Nano v3 to reduce the equipment and cumbersome steps required during development.

[0046] (2) Power control Jetson Nano v3 is a high-power device that requires a stable power input, so this development kit is provided with an uninterruptible power supply system (UPS), which can provide the required voltage and charge the lithium battery when connected to an external power source, and can be powered by the lithium battery when the power is cut off. In addition, a monitoring program is also required on the monitoring end to detect whether the battery power is too low, and if the power is too low, a message will be sent to the user.

[0047] (3) Camera selection There are two common interfaces for commonly used cameras: MIPI CSI-2 and USB 3.0. The present invention uses a camera with a MIPI CSI-2 interface. Although USB supports Plug-and-Play (PnP), its power consumption is quite high and its price is relatively expensive. In addition to being able to directly transfer raw data through DMA (Direct Memory Access), MIPI CSI-2 also has a faster overall startup speed. Except for the need to install the driver, it has other advantages over USB 3.0.

[0048] (4) Network configuration When performing calculations, it is necessary to transmit data to the database through the 4G network, and it is also necessary to use the LINE Notify function through the API officially released by LINE to send instant messages to notify managers. The present invention selects the IntelAC8265 M.2 wireless network card. The M.2 interface can provide high-speed transmission efficiency and there will be no delay when transmitting data. This wireless network card uses the IEEE 802.11ac (Wi-Fi 5) standard wireless network, and is equipped with Bluetooth 4.2 wireless communication technology, supports 5GHz bandwidth, and has a faster signal transmission speed than 2.4GHz in short distances, and is not easily interfered by other signals.

[0049] The safety monitoring system of the present invention first uses a camera to collect real-time images, and simultaneously transmits 5 screenshots collected by the camera every second to the YOLOv9 model. The YOLOv9 model determines whether a person is wearing a safety helmet or a reflective vest. If a person who violates the rules is detected, the time and location of the violation will be recorded and the status will be reported through LINE Notify, and an alarm message will be issued. For details, see the attached Figure 6 . Example 3

[0050] A construction safety supervision device based on artificial intelligence includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned safety supervision method when executing the computer program.

[0051] The present invention uses the safety supervision system to realize the automation and intelligence of engineering supervision, conducts safety supervision on the construction site of the construction project, and uses the YOLOv9 model as the basis to train real-time recognition of whether the workers on the construction site are wearing safety helmets and reflective vests, thereby reducing traditional manpower costs, issuing notification reminders to reduce the occurrence of accidents, and reducing safety risks.

[0052] Those skilled in the art may appreciate that the units of the various examples described in conjunction with the embodiments disclosed herein may be implemented in the form of electronic hardware, or a combination of computer software and electronic hardware.

[0053] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0054] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0055] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A construction safety supervision method based on artificial intelligence, the method is executed in a supervision system, the supervision system includes several cameras, a server, the cameras include visible light cameras and infrared cameras, the method includes the following steps: S1: image acquisition step, using a camera to collect images of the construction site; S2: Image fusion step, using YOLOv9 multimodality to achieve visible light and infrared image fusion; S3: Classification step, classify and label the database according to safety management regulations; S4: Model training step, the labeled data is trained using YOLOv9, and the trained model is used to test the data of the test set; S5: Violation alarm step, using the model to perform real-time monitoring. When a violator appears in the picture within a period of time, the person will be recorded. When the number of records reaches a predetermined number, an alarm message will be issued.

2. The safety supervision method according to claim 1, characterized in that: The step S1 specifically includes: S11: collect visible light images and infrared images simultaneously; S12: perform registration based on a scale-invariant feature transformation algorithm and unify the image sizes.

3. The safety supervision method according to claim 1, characterized in that: The step S2 specifically includes: The multimodal function of YOLOv9 is adopted, visible light images and infrared images are input simultaneously, and image fusion is realized with the help of the front-end fusion network.

4. The safety supervision method according to claim 1, characterized in that: The step S3 specifically includes: S31: According to the safety management regulations, it is divided into four categories: wearing a helmet, not wearing a helmet, wearing a reflective vest and not wearing a reflective vest; S32: Use the AI ​​model target detection labeling tool Makesense.ai to label the data.

5. The safety supervision method according to claim 1, characterized in that: The step S4 specifically includes: YOLOv9 combines programmable gradient information PGI and generalized efficient layer aggregation network GELAN to reduce incorrect predictions caused by information loss and improve the accuracy and efficiency of real-time target detection.

6. The safety supervision method according to claim 5, characterized in that: The step S4 further comprises: The PGI includes auxiliary reversible branches and multi-level auxiliary information, including auxiliary reversible branches and multi-level auxiliary information, the auxiliary reversible branches are used to generate reliable gradients, and the multi-level auxiliary information is used to integrate the gradient information of different prediction heads; PGI provides reliable gradients through auxiliary reversible branches and multi-level auxiliary information, and guides the learning of the main branch to form a complementary and enhanced network structure.

7. The safety supervision method according to claim 5, characterized in that: The step S4 further comprises: The GELAN reduces redundant computation by splitting and merging feature maps and uses layer aggregation to enhance the representation capability of features. GELAN allows the use of various types of computational blocks within its framework to optimize model performance and efficiency.

8. The safety supervision method according to claim 1, characterized in that: The step S5 specifically includes: When the camera footage at the construction site shows a person violating the rules within a certain period of time, a record will be taken and the number of records within the preset time will be recorded. When the number of records reaches the preset number and there is no violation record after the preset time interval, the calculation will be reset to zero to avoid false alarms caused by accumulated records caused by passers-by or background objects. If a violation record appears again, an alarm message will be issued using LINE Notify.

9. A construction safety supervision system based on artificial intelligence, the system is used to execute the safety supervision method according to any one of claims 1 to 8, and specifically comprises the following modules: An image acquisition module for using a camera to capture images of a construction site; Image fusion module, used to achieve visible light and infrared image fusion using YOLOv9 multimodality; The labeling category module is used to classify and label the database according to safety management regulations; The model training module is used to train the labeled data using YOLOv9 and use the trained model to test the data of the test set; The violation alarm module is used to use the model to perform real-time monitoring. When a violator appears in the picture within a period of time, it will be recorded, and an alarm message will be issued after the record reaches a predetermined number of times.

10. A construction safety monitoring device based on artificial intelligence, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the security supervision method according to any one of claims 1 to 8 is implemented.

Images