A safety monitoring method and system for a hoisting device construction site

Abstract

This application relates to the safety monitoring of a hoisting device construction site. Among them, the method includes: obtaining a historical image set of the hoisting device construction site and annotating the target objects therein; constructing a detection network and a classification network, and respectively training the detection network and the classification network based on the annotated historical image set to obtain a detection model and a classification model respectively; collecting the current image of the construction site, detecting the human body object and the hook in the current image through the detection model, and identifying the specific human body object in the human body object through the classification model; determining the target area in the current image according to the hook and the specific human body object in the current image; real-time tracking the specific human body object, obtaining its coordinates corresponding in the current image, and judging whether the coordinates are in the target area within a preset time period. If not, indicating to send a safety alarm. Through this application, the safety monitoring ability of the tower crane construction site is improved, and the safety of construction personnel is further guaranteed.

Description

Technical Field

[0001] This application relates to the field of security, and particularly to a safety monitoring method, system, computer device, and computer-readable storage medium for a hoisting device construction site. Background Art

[0002] With the continuous development of modern construction, tower cranes have become essential common mechanical equipment on construction sites. Due to the high distance from the ground and the large mass of the hoisted objects, safety early warning and alarm during the hoisting process are crucial.

[0003] In the related art, Chinese Patent CN109019335A provides a hoisting safety distance detection method based on deep learning, which describes the detection of the hook and pedestrians under the hook, and calculates the actual distance through the detection results to determine whether to alarm. Chinese Patent CN111062373A provides a hoisting process hazard identification method and system based on deep learning. By monitoring the hook, workers wearing safety helmets correctly, and workers not wearing safety helmets correctly, it is determined whether the workers' behaviors meet the safety operation requirements of the hoisting construction site. Further, it can also be determined whether the workers are within the predicted path range of the hook, thereby improving the safety of the hoisting construction site.

[0004] However, during the process of hoisting heavy objects, to prevent the occurrence of safety accidents, relying solely on machines for detection is not enough. A signalman still needs to be on-site for supervision all the time. Even if an accident occurs suddenly, the signalman can respond in a timely manner. If the signalman leaves the supervision post, it will lead to great safety risks. Summary of the Invention

[0005] The embodiments of this application provide a safety monitoring method, system, computer device, and computer-readable storage medium for a hoisting device construction site, so as to at least solve the problem of poor safety guarantee in the hoisting device monitoring method in the related art.

[0006] In a first aspect, the embodiments of this application provide a safety monitoring method for a hoisting device construction site, and the method includes:

[0007] Obtain a historical image set of a hoisting device construction site and label the target objects therein, where the target objects include a hook, specific human objects, and non-specific human objects;

[0008] Construct a detection network and a classification network, and based on the historical image set after annotation, train the detection network and the classification network respectively to obtain a detection model and a classification model;

[0009] Collect the current image of the construction site, detect the human objects and hooks in the current image through the detection model, and identify the specific human object among the human objects through the classification model;

[0010] Determine the target area in the current image according to the hook in the current image;

[0011] Track the specific human object in real time, obtain its coordinates in the current image, and determine whether the coordinates are within the target area within a preset time period. If not, instruct to send a safety alarm.

[0012] In some embodiments, the specific human object is a signalman wearing a red safety helmet, and the non-specific human is other personnel except the signalman.

[0013] In some embodiments, detecting the human objects and hooks in the current image through the detection model includes:

[0014] Extract the features in the current image through the backbone network, where the backbone network is a Ghostnet structure,

[0015] Fuse the features through the neck layer to obtain a feature map, where the neck layer includes a top-down sampling layer and a bottom-up sampling layer. The top-down sampling layer obtains semantic information, and the bottom-up sampling layer obtains localization information. Fuse the features based on the semantic information and the localization information;

[0016] Detect the feature map through the detection head to obtain the human objects and hooks in the current image.

[0017] In some embodiments, the method further includes:

[0018] During the training process of the detection model, expand the historical data set through mixup data augmentation,

[0019] Adopt BCEWithLogitsLoss as the classification loss function and IoU loss as the regression loss function.

[0020] In some embodiments, the classification model is a mobilenetv3 model. During the training process of the classification model, balance the positive and negative samples through the focalloss loss function.

[0021] In some embodiments, determining the target area in the current image according to the hook in the current image includes:

[0022] Obtain the pixel length of the hook corresponding in the current image, and obtain the actual length of the hook;

[0023] Determine the conversion ratio between the actual length and the pixel length, and the preset activity radius value of the signalman in the real space;

[0024] According to the conversion ratio, convert the preset activity radius value into the pixel radius value in the current image;

[0025] Taking the center of the hook as the center of the circle, determine the target area in the current image according to the pixel radius value.

[0026] In some embodiments, before obtaining the historical image set of the construction site of the lifting device, the method further includes:

[0027] Collect the video of the construction site of the lifting device through a camera device, divide the video into multiple groups of single-frame images, and form the historical image set from the multiple groups of single-frame images, wherein the camera device is installed on the boom of the lifting device, and the camera angle is a vertical depression angle.

[0028] In a second aspect, an embodiment of the present application provides a safety monitoring system for the construction site of a lifting device, the system includes: a preprocessing module, a training module, and a detection module, wherein;

[0029] The preprocessing module is used to obtain the historical image set of the construction site of the lifting device and label the target objects therein, wherein the target objects include hooks, specific human objects, and non-specific human objects;

[0030] The training module is used to construct a detection network and a classification network, and based on the labeled historical image set, train the detection network and the classification network respectively to obtain a detection model and a classification model;

[0031] The detection module is used to collect the current image of the construction site, detect the human objects and hooks in the current image through the detection model, identify the specific human objects in the human objects through the classification model, and

[0032] Determine the target area in the current image according to the hook in the current image, and

[0033] Real-time track the specific human object, obtain its coordinates corresponding in the current image, and judge whether the coordinates are within the target area within a preset time period. If not, instruct to send a safety alarm.

[0034] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method described in the first aspect above is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect above is implemented.

[0036] Compared with the related art, a safety monitoring method for a construction site provided by an embodiment of the present application includes obtaining multiple sets of historical images of the construction site and annotating the target objects therein; training a detection model and a classification model based on the multiple sets of historical images after annotation. Further, in the actual detection process, the trained detection model and classification model are used to detect the human objects and hooks in the current image, and identify specific human objects among the human objects. Finally, the specific human object is tracked in real time, and it is determined whether the specific human object is in the target area within a preset time period. If not, it is indicated that the specific human has exceeded the time limit for leaving the work post, and other devices are instructed to send a safety alarm. This solves the problem of poor safety guarantee in the hoisting device monitoring method in the related art, realizes the detection and tracking of the signalman responsible for supervision, and outputs an alarm signal after the safety officer has been outside the preset activity range below the hook for a certain period of time, thereby improving the safety monitoring ability of the tower crane construction site and further ensuring the safety of construction workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0038] Figure 1 is a schematic diagram of the application environment of a safety monitoring method for a hoisting device construction site according to an embodiment of the present application;

[0039] Figure 2 is a flowchart of a safety monitoring method for a hoisting device construction site according to an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of feature fusion in the neck layer according to an embodiment of the present application;

[0041] Figure 4 is a structural block diagram of a safety monitoring system for a hoisting device construction site according to an embodiment of the present application;

[0042] Figure 5 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application;

[0043] Figure 6 It is a schematic internal structure diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0044] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided in the present application without creative efforts fall within the scope of protection of the present application.

[0045] Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present application. For those of ordinary skill in the art, without creative efforts, the present application can also be applied to other similar scenarios based on these drawings. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood as the content disclosed in the present application being insufficient.

[0046] Referring to "embodiment" in the present application means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those of ordinary skill in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the ordinary meanings understood by those with ordinary skills in the technical field to which this application belongs. The words such as "a", "an", "one kind", "the" and the like involved in this application do not indicate a quantity limitation and may represent a singular or plural number. The terms "include", "comprise", "have" and any variations thereof involved in this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may further include steps or units not listed, or may further include other steps or units inherent to these processes, methods, products or devices. The words such as "connect", "be connected", "couple" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the front and rear associated objects. The terms "first", "second", "third" and the like involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0048] A safety monitoring method for a hoisting device construction site provided by this application can be applied in an application environment such as Figure 1 shown. Figure 1 FIG. is a schematic diagram of an application environment of a safety monitoring method for a hoisting device construction site according to an embodiment of this application. As Figure 1 shown, an imaging device 10 installed on the tower crane jib is used to collect images of the construction site in real time. Further, a server 11 set in a monitoring room or in the cloud receives the images, and identifies human objects in the images through a built-in detection model, and identifies specific human objects (such as a signalman wearing a red safety helmet) from all humans through a classification model. Finally, the specific human object is tracked in real time, and by determining whether the specific human object is within a preset activity range under the hook, it is identified whether the signalman responsible for safety supervision is on duty, thereby optimizing the construction security monitoring ability and ensuring the safety of construction personnel.

[0049] Figure 2 FIG. is a flowchart of a safety monitoring method for a hoisting device construction site according to an embodiment of this application. As Figure 2 shown, the process includes the following steps:

[0050] S201. Collect the video of the construction site of the lifting device through a camera device, divide the video into multiple groups of single-frame images, and form a historical image set from the multiple groups of single-frame images. Among them, the camera device is installed on the boom of the lifting device, and the camera angle is a vertical depression angle;

[0051] In this embodiment, the camera device can be any one of a conventional camera, an IPC camera, a depth camera, and an infrared camera.

[0052] Optionally, the camera device can be fixedly or slidably installed on the boom of the lifting device. Specifically: when installed slidably, a trolley capable of sliding back and forth is arranged on the boom of the lifting device, and then the camera device is installed on the trolley, so as to adjust the shooting position according to the actual situation on site.

[0053] Furthermore, the camera angle of the camera device is a vertical depression angle, which is used to capture on-site images with less distortion and closer to the real scene.

[0054] It should be noted that in the historical image set obtained in this step, the number of images should meet a certain intensity to meet the minimum requirements for model training.

[0055] S202. Obtain the historical image set of the construction site of the lifting device and label the target objects therein. Among them, the target objects include a hook, a specific human object, and a non-specific human object;

[0056] Among them, the historical image set is composed of multiple single-frame images with different contents. Each single-frame image can be similar or completely different; for example, the human body contour is clear or blurred, the human body contour is large or small, the human body is blocked or unblocked, and the human body has overlap or no overlap.

[0057] It should be noted that in the above historical image set, the types of single-frame images must be rich to ensure that the subsequent obtained detection model can obtain an identification effect that meets the requirements.

[0058] Furthermore, the number of labeled images should also reach a certain order of magnitude. Among them, for a deep network model, under the same conditions, if the amount of data used in the training process is larger and the data types are more diverse, the obtained model will have better effects. However, correspondingly, the increase in the amount of data will also lead to an increase in the amount of computation, and ordinary devices may not have the required computing power. In this example, according to the actual situation, 10,000 images of specific human objects and 10,000 images of non-specific human objects are labeled respectively.

[0059] In this embodiment, the annotation process includes the annotation process of the human body bounding box and the classification annotation process. Among them, the human body bounding box annotation process is to frame the area covered by the human body in the original image, where the range of the frame should include the entire part of the human body. Further, the classification annotation process is to cut out the above-mentioned area of the human body bounding box diagram to obtain the human body bounding box diagram, and then classify the human body bounding box diagram according to whether it is a specific human object.

[0060] It should be noted that in this embodiment, the specific human object is a signalman responsible for safety supervision. According to the current construction industry specification requirements, the significant identification features of the signalman usually manifest as wearing a red safety helmet. Of course, in some special scenarios, the signalman can also have other identification features, such as wearing a blue safety helmet, wearing a red coat, etc.

[0061] It should be understood that in the case where the signalman has a unified identifier and the identifier can be recognized and processed by the algorithm model, the technical solution of this application can utilize the unified identifier to achieve the expected technical effect. Therefore, in the embodiments of this application, no specific limitation is imposed on the identification features adopted by the specific human object.

[0062] S203, construct a detection network and a classification network, and based on the annotated historical image set, train the detection network and the classification network respectively to obtain a detection model and a classification model respectively;

[0063] Among them, the above detection network is obtained by further improvement on the basis of the yolov5 algorithm, combined with the characteristics of the hoisting device construction site. Specifically:

[0064] Replace the backbone in the original yolov5 algorithm with the Ghostnet structure. Since Ghostnet has a more lightweight network structure compared to the backbone, it can generate more features with fewer parameters, thereby improving the overall detection speed;

[0065] And in the feature fusion layer of the model, a layer of PAN structure is added. On the basis of the original bottom-up feature fusion, top-down feature fusion is further performed to obtain better feature maps;

[0066] Optionally, mixup can be used to enhance the data during training. The main classification loss function can use BCEWithLogitsLoss, and the regression loss function can use IoU loss. Finally, the output head still remains as the output heads yolo1, yolo2, yolo3 of yolov5. Optionally, train for 200 epochs.

[0067] Further, any common binary classification recognition algorithm can be used for the above classification network. Optionally, the mobilenetv3 algorithm model can be adopted. This model is small and precise, and its input resolution is 224*224. In this embodiment, the classification model is trained and iterated for 100 epochs.

[0068] S204. Collect the current image at the construction site, detect the human object and the hook in the current image through the detection model, and identify the specific human object in the human object through the classification model.

[0069] After the above steps S201 to S203, the trained detection model and classification model have been obtained. In this step, the model can be deployed at the construction site for monitoring. The detection model is used to identify and obtain the hook and human objects in the real-time image on site, and then through the classification model, the specific human object is identified from these human objects.

[0070] It should be noted that since the camera device takes pictures at a vertical depression angle and the shooting distance is relatively high, in the image, the identifying features of the specific human object (such as a red safety helmet) will only occupy a very small area of the entire image. Therefore, if the method of directly applying the detection model to detect the specific human object in the entire picture in the related technology is adopted, there will be a great risk of false detection. For example, a non-specific human is identified as a specific human, and a non-human object is identified as the above specific human object.

[0071] In the embodiment of the present application, compared with the related technology, the specific human object is not directly detected from the image. Instead, first, through the detection model, all human objects (including specific humans and non-specific humans) are obtained from the real-time image to form a human block diagram. Further, all human object block diagrams are input into the classification model, and through this classification model, binary classification is performed within the relatively small range of all human object block diagrams to obtain the specific human object. Thus, the risk of false detection is greatly avoided and the detection accuracy is improved.

[0072] S205. Determine the target area in the current image according to the hook in the current image, track the specific human object in real time, obtain its coordinates corresponding in the current image, and determine whether the coordinates are within the target area within a preset time period. If not, instruct to send a safety alarm.

[0073] It should be noted that the process of tracking the specific human object can be implemented by combining the Kalman filter and the Hungarian algorithm.

[0074] Optionally, the above preset time period can be 10 minutes. When it is recognized that the specific human object is not within the target area under the hook in the image within the preset time period, it means that the signalman has been away from the supervision post for more than the required time.

[0075] In this case, the server instructs the alarm device to send a safety alarm to remind the signalman to quickly return to the supervision post or notify the management personnel for further decision-making. Among them, the alarm device can be a voice alarm device, such as a construction site broadcast, a portable instrument communication device, etc.

[0076] Through the above steps S201 to S205, compared with the safety detection method of the hoisting device construction site in the related art, in this application, all human objects are obtained through the detection model. Further, the signalman wearing a red safety helmet responsible for construction supervision is identified from all human objects through the classification model. Thus, after it is identified that the signalman has exceeded the activity range below the hook for a certain period of time, an alarm signal is sent to remind him to quickly return to the supervision post. The embodiment of this application realizes the identification of specific groups, improves the safety monitoring ability of the tower crane construction site, and further ensures the safety of construction personnel.

[0077] In some of the embodiments, the process of the detection model detecting the target object includes: extracting features in the historical image or the current image through the backbone network, where the backbone network is a Ghostnet structure, fusing the features through the neck layer to obtain a feature map, where the neck layer is a PAN structure, and detecting the feature map through the detection head to obtain the target object.

[0078] Among them, the Ghost module can be used as a plug-and-play component to upgrade the existing convolutional neural network, and the lightweight Ghostnet network structure is established based on the stacking of Ghost modules. Specifically:

[0079] In the Ghostnet network structure, an ordinary convolutional layer is divided into two parts. The first part involves ordinary convolutions, but the total number of them will be strictly controlled; further, given the internal feature map of the first part, simple linear operations are applied to generate more feature maps. Thus, without changing the size of the output feature map, compared with the traditional Bckbone convolutional neural network, the total number of parameters and the computational complexity required by the Ghostnet network structure are reduced.

[0080] Further, Figure 3 is a schematic diagram of feature fusion in the neck layer according to an embodiment of this application, as Figure 3 shown,

[0081] The traditional FPN structure only performs top-down sampling, and fuses the high-level features through upsampling and the low-level features to obtain a feature map.

[0082] In this embodiment, on the basis of the FPN layer, a bottom-up feature pyramid structure (PAN structure) is further added. Through the combined operation of the two sampling layers, the FPN layer conveys strong semantic features from top to bottom, and the pyramid structure conveys localization features from bottom to top. By combining the semantic features and the localization features, parameter aggregation is performed on the features obtained from different detection layers of different backbone layers, so as to obtain a better feature map.

[0083] Figure 4 is a structural diagram of a detection model according to an embodiment of the present application. As Figure 4 shown, the construction site image is input into the model, and after feature extraction by Ghostnet and PAN feature fusion, a feature map is obtained, and then it is processed by the detection head to obtain the outputs yolo1, yolo2, and yolo3.

[0084] In some of these embodiments, during the training process of the detection model, the historical dataset is augmented by mixup data augmentation. Further, BCEWithLogitsLoss is used as the classification loss function, and IoU loss is used as the regression loss function.

[0085] It should be noted that since the images corresponding to the actual construction process have various different types of situations, specifically, including images with human body occlusion or human body overlap, images with a long or short distance of the human body, images with a large or small human body contour, etc. And because the number of collected and labeled images is limited, the historical dataset used for training cannot cover all situations of the construction site. Therefore, through a data augmentation mechanism, existing images are fused and superimposed to generate image data that is similar but not exactly the same as the original images, thereby filling the gaps between the existing images, enriching the diversity of the data, and also improving the generalization ability of the model.

[0086] In the embodiment of the present application, by using the characteristics of mixup data augmentation, while enriching the data diversity, when performing BN (Batch Normalization) operation, the mean and variance can be better statistically calculated.

[0087] In some of these embodiments, according to the hook in the current image, a target area is determined in the current image, including: obtaining the pixel length of the hook corresponding in the current image, and obtaining the actual length of the hook; determining the conversion ratio between the actual length and the pixel length, and the preset activity radius value of the signalman in the real space; according to the conversion ratio, converting the preset activity radius value into the pixel radius value in the current image; with the center of the hook as the center, determining the target area in the current image according to the pixel radius value.

[0088] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0089] This embodiment also provides a safety monitoring system for the construction site of a lifting device. This system is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated here. As used hereinafter, terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0090] Figure 5 is a structural block diagram of the safety monitoring system for the construction site of a lifting device according to an embodiment of the present application. As Figure 5 shown, the system includes: a preprocessing module 50, a training module 51, and a detection module 52, wherein;

[0091] The preprocessing module 50 is used to obtain a historical image set of the construction site of the lifting device and label the target objects therein, where the target objects include a hook, a specific human object, and a non-specific human object;

[0092] The training module 51 is used to construct a detection network and a classification network, and based on the labeled historical image set, train the detection network and the classification network to obtain a detection model and a classification model respectively;

[0093] The detection module 52 is used to collect the current image of the construction site, detect the human object and the hook in the current image through the detection model, identify the specific human object in the human object through the classification model, and determine the target area in the current image according to the hook and the specific human object in the current image, and real-time track the specific human object, obtain its coordinates corresponding in the current image, and judge whether the coordinates are in the target area within a preset time period. If not, it is instructed to send a safety alarm.

[0094] In one embodiment, a computer device is provided, and the computer device may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a safety monitoring method for a hoisting device construction site. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0095] In one embodiment, Figure 6 is a schematic internal structure diagram of an electronic device according to an embodiment of the present application, as Figure 6 shown, an electronic device is provided, and the electronic device may be a server, and its internal structure diagram may be as Figure 6 shown. The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory connected through an internal bus. Among them, the non-volatile memory stores an operating system, a computer program, and a database. The processor is used to provide computing and control capabilities. The network interface is used to communicate with an external terminal through a network connection. The internal memory is used to provide an environment for the operation of the operating system and the computer program. When the computer program is executed by the processor, it implements a safety monitoring method for a hoisting device construction site. The database is used to store data.

[0096] Those skilled in the art can understand that Figure 6 the structure shown in

[0097] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0098] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A safety monitoring method for a hoisting device at a construction site, characterized in that, The method includes: Obtaining a historical image set of the construction site of the lifting device and annotating the target objects therein, where the target objects include a hook, a specific human object, and a non-specific human object; Constructing a detection network and a classification network, and respectively training the detection network and the classification network based on the annotated historical image set to obtain a detection model and a classification model respectively. The classification model is a mobilenetv3 model. During the training process of the classification model, the positive and negative samples are balanced through the focal loss function; Collecting the current image of the construction site, detecting the human object and the hook in the current image through the detection model, and identifying the specific human object in the human object through the classification model. Wherein, detecting the human object and the hook in the current image through the detection model includes: Extracting features in the current image through the backbone network, where the backbone network is a Ghostnet structure; Fusing the features through the neck layer to obtain a feature map, where the neck layer includes a top-down sampling layer and a bottom-up sampling layer. The top-down sampling layer obtains semantic information, and the bottom-up sampling layer obtains localization information, and fuses the features based on the semantic information and the localization information; Detecting the feature map through the detection head to obtain the human object and the hook in the current image; Determining a target area in the current image according to the hook in the current image; Real-time tracking the specific human object, obtaining its coordinates corresponding in the current image, and judging whether the coordinates are within the target area within a preset time period. If not, instruct to send a safety alarm.

2. The method according to claim 1, wherein The specific human object is a signalman wearing a red safety helmet, and the non-specific human is other personnel except the signalman.

3. The method according to claim 1, characterized in that, The method further includes: During the training process of the detection model, expanding the historical data set through mixup data augmentation; Using BCEWithLogitsLoss as the classification loss function and IoU loss as the regression loss function.

4. The method according to claim 2, wherein Determining a target area in the current image according to the hook in the current image includes: Obtaining the pixel length of the hook corresponding in the current image, and obtaining the actual length of the hook; Determining the conversion ratio between the actual length and the pixel length, and the preset activity radius value of the signalman in the real space; According to the conversion ratio, converting the preset activity radius value into the pixel radius value in the current image; Taking the center of the hook as the center of the circle, and determining the target area in the current image according to the pixel radius value.

5. The method according to claim 1, wherein Before obtaining the historical image set of the construction site of the lifting device, the method further includes: Collecting the video of the construction site of the lifting device through a camera device, dividing the video into multiple groups of single-frame images, and forming the historical image set by the multiple groups of single-frame images. The camera device is installed on the boom of the lifting device, and the camera angle is a vertical depression angle.

6. A safety monitoring system for a hoisting device construction site, characterized in that, The system includes: a preprocessing module, a training module, and a detection module, where; The preprocessing module is used to obtain the historical image set of the construction site of the lifting device and label the target objects therein, where the target objects include a hook, a specific human object, and a non-specific human object; The training module is used to construct a detection network and a classification network, and based on the labeled historical image set, train the detection network and the classification network respectively to obtain a detection model and a classification model respectively. Among them, the classification model is a mobilenetv3 model. During the training process of the classification model, the focal loss function is used to balance positive and negative samples; The detection module is used to collect the current image of the construction site, detect the human object and the hook in the current image through the detection model, and identify the specific human object in the human object through the classification model, and determine the target area in the current image according to the hook and the specific human object in the current image, and real-time track the specific human object, obtain its coordinates corresponding to the current image, and judge whether the coordinates are within the target area within a preset time period. If not, it is indicated to send a safety alarm. Among them, detecting the human object and the hook in the current image through the detection model includes: extracting the features in the current image through the backbone network, where the backbone network is a Ghostnet structure, fusing the features through the neck layer to obtain a feature map, where the neck layer includes a top-down sampling layer and a bottom-up sampling layer. The top-down sampling layer obtains semantic information, and the bottom-up sampling layer obtains localization information. The features are fused based on the semantic information and the localization information; detecting the feature map through the detection head to obtain the human object and the hook in the current image.

7. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 5.

8. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1 to 5.

Images