Construction site safety management method based on cloud-side cooperation

Abstract

The invention provides a construction site safety management method based on cloud edge collaboration, which is applied to a cloud and comprises the following steps: pre-training to obtain a first dangerous behavior recognition model; configuring an edge device and a camera; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for the edge device so as to deploy the second dangerous behavior recognition model to the edge device; the first dangerous behavior recognition model and the second dangerous behavior recognition model are used for outputting a dangerous behavior recognition result through the input camera data. The invention also provides a construction site safety management method based on cloud-side cooperation, which is applied to edge equipment and comprises the following steps: receiving a second dangerous behavior recognition model sent by a cloud; recognizing the camera data by using a second danger recognition model to obtain a dangerous behavior recognition result; and the front end displays the dangerous behavior recognition result. The invention further provides a construction site safety management system based on cloud edge collaboration, electronic equipment and a medium.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer vision and edge computing, and specifically to a construction site safety management method based on cloud-edge collaboration. Background Art

[0002] With the rapid development of information technology and artificial intelligence, computer vision and edge computing have shown significant potential in many fields. In terms of construction site safety management, traditional monitoring methods and manual inspections often fail to meet the requirements of real-time and accuracy. To this end, the construction of smart construction sites is introducing advanced technical means to improve the efficiency and effectiveness of safety management. Summary of the invention

[0003] In view of the above problems, the present disclosure provides a construction site safety management method based on cloud-edge collaboration, which is applied to the cloud, including: pre-training to obtain a first dangerous behavior recognition model; configuring edge devices and cameras; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for edge devices for deploying the second dangerous behavior recognition model to the edge devices; the first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

[0004] According to an embodiment of the present disclosure, configuring edge devices and cameras includes: configuring edge device information; the edge device information includes at least one of the edge device IP, edge device port, edge device name and edge device attributes; configuring camera information; the camera information includes at least one of the camera IP address, camera port number, camera user name and password, camera data sending interval, camera area fixed point and camera group information; for each camera, selecting an edge device to bind to the camera so that the edge device can process camera data.

[0005] According to an embodiment of the present disclosure, converting a first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for an edge device includes: using the first dangerous behavior recognition model to generate a BModel in FP32 floating point format as the second dangerous behavior recognition model; and / or, using the first dangerous behavior recognition model to generate a BModel in FP32 floating point format; establishing a representative input data set; using an automatic quantization tool to convert the BModel into a quantized BModel as the second dangerous behavior recognition model based on the representative input data set; and / or, using the first dangerous behavior recognition model to generate an intermediate UModel in FP32 floating point format; establishing a representative input data set; processing the representative input data set into an LMDB format; using the representative input data set to manually quantize the UModel to obtain the quantized UModel as the second dangerous behavior recognition model; wherein the representative input data set includes typical input samples of the dangerous behavior recognition model; the typical input samples include at least camera data of cameras when a helmet is not worn, a reflective vest is not worn, and both a helmet and a reflective vest are worn.

[0006] The second aspect of the present disclosure provides a construction site safety management method based on cloud-edge collaboration, which is applied to edge devices, including: receiving a second dangerous behavior recognition model sent by the cloud; using the second dangerous behavior recognition model to recognize camera data to obtain dangerous behavior recognition results; the second dangerous behavior recognition model is used to output dangerous behavior recognition results through input camera data; and the front end displays the dangerous behavior recognition results.

[0007] According to an embodiment of the present disclosure, a second hazard identification model is used to identify camera data to obtain dangerous behavior identification results, including: extracting video frames in a video queue using a multi-process mode based on the IP address of the camera; identifying video frames using the second hazard identification model to obtain dangerous behavior identification results; the dangerous behavior identification results include an identification box, category, area image of the dangerous behavior, and video frames containing the dangerous behavior.

[0008] According to an embodiment of the present disclosure, after obtaining the dangerous behavior identification result, it also includes: storing the dangerous behavior identification result locally; uploading the dangerous behavior identification result, the IP address of the camera, the video timestamp and the location to the cloud for the cloud to optimize the second dangerous behavior identification model and record the dangerous behavior.

[0009] According to an embodiment of the present disclosure, the method also includes: receiving camera binding information sent from the cloud; the camera binding information includes the binding relationship between the edge device and the camera; configuring the camera according to the camera binding information to achieve local management of the camera; wherein, local management of the camera includes at least one of turning on the camera, turning off the camera and changing the camera information in the cloud online.

[0010] The third aspect of the present disclosure provides a construction site safety management system based on cloud-edge collaboration, including: a cloud, used to configure edge devices and cameras; pre-training to obtain a first dangerous behavior recognition model; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for edge devices; deploying the second dangerous behavior recognition model to edge devices; edge devices, used to receive the second dangerous behavior recognition model sent by the cloud; using the second dangerous recognition model to recognize camera data to obtain dangerous behavior recognition results; the front end displays the dangerous behavior recognition results; a camera, used to obtain camera data and send it to the cloud and edge devices; wherein the first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

[0011] The fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned construction site safety management method based on cloud-edge collaboration.

[0012] The fifth aspect of the present disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to execute the above-mentioned construction site safety management method based on cloud-edge collaboration.

[0013] According to the construction site safety management method based on cloud-edge collaboration provided by the present disclosure, by migrating data processing to edge nodes closer to the source, latency is significantly reduced, bandwidth usage is reduced, and the efficiency of the safety management system is improved. Since the low-precision model is deployed on the edge device, the technical problem of data transmission affecting recognition efficiency and occupying bandwidth resources is at least partially solved, and the technical effect of cloud-edge collaboration for construction site safety management is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of a construction site safety management method based on cloud-edge collaboration applied to the cloud according to an embodiment of the present disclosure is schematically shown;

[0015] Figure 2 A functional module diagram of a construction site safety management method based on cloud-edge collaboration according to an embodiment of the present disclosure is schematically shown;

[0016] Figure 3 A flowchart of model conversion according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 4 The structural block diagram of a construction site safety management system based on cloud-edge collaboration according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 5A block diagram of an electronic device suitable for implementing a construction site safety management method based on cloud-edge collaboration according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0020] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0021] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0022] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0023] First, the technical terms involved in this disclosure are explained as follows:

[0024] BModel: BModel is a deep neural network model file format for the Compute Energy TPU processor. This format is generated by a model compiler tool (such as bmnetc / bmnett, etc.) and contains parameter information of one or more networks, such as input and output. At runtime, this model file can be loaded and used to support various applications of deep neural networks.

[0025] UModel: It is an intermediate model Umodel generated in the process of converting the traditional deep learning model .pt model to .onnx model and then to the Suanneng Technology exclusive model .bmodel in the Suanneng Technology model conversion tool chain, that is, .pt to .onnx to .umodel to .bmodel. UMODEL is an intermediate representation that facilitates further optimization and adjustment of the model on different hardware platforms or software tools.

[0026] LMDB format: An efficient memory-mapped database format suitable for storing large-scale data.

[0027] Edge device: A device located at the edge of the network that performs a certain degree of computing and data processing and interacts with the cloud and other terminals. By migrating data processing to edge nodes closer to the source, latency is significantly reduced, bandwidth usage is reduced, and system efficiency and user experience are improved. Moreover, based on the self-developed domestically produced edge devices, the software and hardware are independently controllable to store historical data locally, reducing the risk of data leakage and improving data security.

[0028] Figure 1 The flowchart of the construction site safety management method based on cloud-edge collaboration applied to the cloud according to an embodiment of the present disclosure is schematically shown. Figure 1 As shown, an embodiment of the present disclosure provides a construction site safety management method based on cloud-edge collaboration, which is applied to the cloud, including: pre-training to obtain a first dangerous behavior recognition model; configuring edge devices and cameras; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for edge devices for deploying the second dangerous behavior recognition model to the edge devices; the first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

[0029] In this embodiment, deployment of edge devices is performed by copying.

[0030] Through the embodiments of the present disclosure, by deploying the model on the edge device and performing preliminary data processing on the edge device, the delay of data transmission can be greatly reduced, and the dependence on cloud computing resources can be reduced while ensuring real-time performance.

[0031] On the basis of the above embodiment, edge devices and cameras are configured, including: configuring edge device information; the edge device information includes at least one of the edge device IP, edge device port, edge device name and edge device attributes; configuring camera information; the camera information includes at least one of the camera IP address, camera port number, camera user name and password, camera data sending interval, camera area fixed point and camera group information; for each camera, an edge device is selected to bind to the camera so that the edge device can process camera data.

[0032] In this embodiment, configuration information is added, deleted, and modified online through the PC dynamic service module in the cloud, mainly including edge device IP, port, edge device name, edge device attributes (large, small) and other related information; camera IP, camera port number, account, password, collection interval (i.e., camera violation (dangerous behavior) data sending interval), area fixed-point related information and other information; related information of the edge device and camera binding.

[0033] Optionally, the camera information also includes camera group information and information related to the cameras in the group, where the encoding format should be H264. The camera group information is the comprehensive information of all cameras in a certain area, and the group information is the individual information of each camera in the area. The camera information also includes monitoring area information. That is, the area information corresponding to the camera IP address, mainly including four coordinates starting from the upper left corner, in the order of upper left, upper right, lower right and lower left corner coordinates.

[0034] Through the embodiments of the present disclosure, management is performed by configuring the information of edge devices and cameras; by binding the edge devices and cameras, it is ensured that the edge devices can efficiently process the data of specific cameras, thereby improving the pertinence and efficiency of data processing.

[0035] Based on the above embodiment, the first dangerous behavior recognition model is converted to obtain a second dangerous behavior recognition model suitable for edge devices, including: using the first dangerous behavior recognition model to generate a BModel in FP32 floating point format as the second dangerous behavior recognition model; and / or, using the first dangerous behavior recognition model to generate a BModel in FP32 floating point format; establishing a representative input data set; using an automatic quantization tool to convert the BModel into a quantized BModel as the second dangerous behavior recognition model according to the representative input data set; and / or, using the first dangerous behavior recognition model to generate an intermediate UModel in FP32 floating point format; establishing a representative input data set; processing the representative input data set into an LMDB format; using the representative input data set to manually quantize the UModel to obtain a quantized UModel as the second dangerous behavior recognition model; wherein the representative input data set includes typical input samples of the dangerous behavior recognition model; the typical input samples at least include camera data of cameras when a helmet is not worn, a reflective vest is not worn, and both a helmet and a reflective vest are worn.

[0036] Among them, a representative input dataset is used to evaluate the performance of the model during the quantization process.

[0037] It should be noted that the first dangerous behavior recognition model needs to be prepared. The model is usually expressed in FP32 floating point format and contains the network architecture and its corresponding weight parameters. The preparation work includes verifying the correctness and integrity of the model and ensuring its compatibility with the target hardware platform. This stage lays the foundation for subsequent compilation and optimization.

[0038] Figure 3 The flowchart of model conversion according to the embodiment of the present disclosure is schematically shown; the first line indicates that the BModel in fp32 format can be directly compiled; the second line indicates the preparation of a quantized data set (i.e., a representative input data set), and then the automatic quantization compilation is performed to obtain a low-precision Bmodel in INT8 format; the third line indicates the preparation of a quantized data set, which will be processed and converted to LMDB format for manual quantization step by step, from Umodel in fp32 format to Umodel in INT8 format, and finally obtains Bmodel in INT8 format. The purpose of generating LMDB from bmodel is to provide efficient access data support in the subsequent model optimization and reasoning process, and improve the overall processing performance.

[0039] Specifically, the process of the first line is as follows: After obtaining the original model, the model needs to be compiled to generate BMODEL in FP32 format. BMODEL is a specific format that is optimized for specific hardware accelerators or inference engines. The purpose of compilation is to ensure that the model can run efficiently on the target device while retaining the computing power of FP32 floating point precision.

[0040] The process in the second row is as follows: In order to perform the quantization process, a representative dataset for calibration needs to be prepared. This dataset should cover the typical inputs of the model in actual application scenarios to ensure that the accuracy loss of the model is minimal during the quantization process. The selection of the dataset is crucial to the performance of the final quantized model, so it must be strictly screened and processed. Using the automatic quantization tool, the FP32 model is converted to a low-precision (usually INT8 format) BMODEL. The quantization process relies on the previously prepared calibration dataset. By adjusting the model parameters, a quantized BMODEL with higher computational efficiency and smaller memory usage is generated, which is suitable for resource-constrained hardware environments.

[0041] The process in the third line is as follows: The calibration dataset is processed and converted to LMDB format. LMDB is an efficient memory-mapped database format suitable for storing large-scale data. The purpose of generating LMDB is to provide efficient data access support in the subsequent model optimization and inference process, and improve the overall processing performance. In addition to generating a quantized model, it is also necessary to export UMODEL in FP32 format. UMODEL is an intermediate representation that facilitates further optimization and adjustment of the model on different hardware platforms or software tools. This step ensures the flexibility and scalability of the model and provides a basis for subsequent refined quantization tuning. Based on FP32UMODEL, the optimized quantized UMODEL is generated through manual or automated quantization parameter adjustment. The purpose of this step is to further reduce the computational complexity and memory usage while ensuring the performance of the model, so that it can adapt to the inference requirements of specific hardware. The last step is to compile the tuned quantized UMODEL to generate the final quantized BMODEL. After multiple optimizations, the model has higher inference efficiency while retaining the inference accuracy of the model to the greatest extent on the target hardware platform. This quantized BMODEL is the final deployable inference model suitable for actual production environments.

[0042] Through the embodiments of the present disclosure, the model is carefully trained in the cloud. The model trained in the cloud cannot be directly run on the edge side due to file format issues, so it is necessary to develop the algorithm on the edge side to meet the compatible operation of the algorithm on the edge side. Through professional model conversion technology, the algorithm logic is reconstructed according to the specific situation on the edge side and optimized into a version suitable for edge devices.

[0043] The present disclosure also provides a construction site safety management method based on cloud-edge collaboration, which is applied to edge devices, including: receiving a second dangerous behavior recognition model sent by the cloud; using the second dangerous behavior recognition model to recognize camera data to obtain a dangerous behavior recognition result; the second dangerous behavior recognition model is used to output the dangerous behavior recognition result through the input camera data; and the front end displays the dangerous behavior recognition result.

[0044] In this embodiment, the principle of Http communication is adopted to send the dangerous behavior identification results to the front end, and the dangerous behavior identification results are displayed through the front end, specifically including the real-time rotation of the violation information (not wearing a safety helmet, reflective clothing), violation time, screenshot information, handling suggestions, etc. in the platform-related algorithm through the front-end visualization module. Through the front-end visualization module, it is possible to quickly grasp the safety wearing problems of workers in various key areas, locate the camera location in time, and take screenshots of violations for preservation.

[0045] In this embodiment, the hardware part of the edge device includes CPU, GPU, memory, network interface, input and output, cooling system, power management module, etc., and domestic hardware or self-developed hardware is used to achieve hardware autonomy and control. The software part of the edge device includes operating system, driver, application, middleware, framework and library, etc., and domestic software or self-developed software is used to achieve software autonomy and control.

[0046] Through the embodiments of the present disclosure, the model generated by PC cloud training is first converted and migrated and deployed on the edge side, and the locally collected data is quickly processed and analyzed. Then, real-time data analysis is performed on the workers under the camera to promptly identify whether the workers are wearing the necessary safety equipment, and to realize real-time screenshot reporting of safety wearing issues of safety helmets and reflective clothing. By performing data analysis on the video stream in the construction site scene, the detection and data storage of the required targets in the video are realized, which greatly improves the safety service level of the construction site and better adapts to the current application scenario requirements of safety construction; by performing in-depth data analysis on the video stream of the smart construction site, the system realizes comprehensive safety management.

[0047] On the basis of the above embodiment, the second hazard identification model is used to identify the camera data to obtain the dangerous behavior identification result, including: extracting the video frames in the video queue using the multi-process mode according to the IP address of the camera; using the second hazard identification model to identify the video frames to obtain the dangerous behavior identification result; the dangerous behavior identification result includes the identification box, category, area image of the dangerous behavior and the video frame containing the dangerous behavior.

[0048] In this embodiment, the video frames in the video queue are extracted in a multi-process mode according to the IP address of the camera; for the pulled video frame images, they are searched to determine whether they contain the required target (for example, a safety helmet, a reflective vest), and if so, the identification frame and category of the target are returned, and the regional image is extracted. Optionally, after identifying the violation, further target tracking is performed to obtain the identity ID information of the violating target, and the violation behavior (dangerous behavior) is judged by algorithm logic, and the identification result is intelligently sent to the database and displayed on the front end.

[0049] In this embodiment, the real-time video stream is obtained through the IP address of the camera, and each frame of video is transmitted as input data to the target detection module to start the subsequent processing flow. A model composed of backbone network architectures such as SPP, FPN+PAN, combined with the CSPDarknet53 backbone network, Mish activation function and Dropblock regularization technology, extracts key feature information in each frame of the image to provide a basis for target recognition. Through the adaptive anchor frame mechanism and the dynamic adjustment of the image size, in conjunction with the CIOU_Loss and DIOU_NMS algorithms, the specific position of the target is predicted from the extracted image features, and it is determined whether the prior frame contains the target object, and the position of the prediction frame is finally decoded. The non-maximum suppression (NMS) algorithm is used to remove redundant prediction frames with high overlap to ensure that only the most accurate recognition result frame is retained for each target object. During the processing of the video stream, real-time target detection and analysis are performed to identify all possible violations or dangerous actions, and dynamic monitoring is performed. The identified targets and behaviors are intelligently analyzed, and the identification information is sent to the database according to the preset time interval for real-time display, and the relevant warning information is displayed on the front end.

[0050] Through the embodiments of the present disclosure, real-time video streams are monitored and processed, and potential safety hazards are automatically detected. Through continuous observation and data mining of on-site activities, the system can promptly identify abnormal situations, provide early warnings, and support decision-making, thereby significantly improving the overall safety level and management efficiency of the construction site.

[0051] Based on the above embodiment, after obtaining the dangerous behavior identification result, it also includes: storing the dangerous behavior identification result locally; uploading the dangerous behavior identification result, the IP address of the camera, the video timestamp and the location to the cloud for the cloud to optimize the second dangerous behavior identification model and record the dangerous behavior.

[0052] In this embodiment, the data storage mode can be configured, and cloud storage, local storage, and dual-end storage are optional. In order to ensure data security, the default mode is dual-end storage, and data from different ports are stored through the set service interface. The edge device receives and feedbacks in real time according to the cloud configuration of the storage mode.

[0053] Through the embodiments of the present disclosure, the relevant identification information, identification results and data transmission status are stored in the database on both ends. When the cloud communication is disconnected and the signal is weak, the data is synchronized and saved locally, so that the historical records can be checked and complete and provide data support for intelligent decision-making, ensuring that subsequent data can be traced and management has a basis to follow. At the same time, the algorithm is continuously optimized and upgraded through the feedback of field data (video frame-level screenshots of the detection results and field data collection), the environmental pertinence of the algorithm is improved, and the self-increment training of the algorithm is realized.

[0054] At the same time, the cloud computing platform is responsible for in-depth analysis and storage of massive data, providing high-level intelligent analysis and decision support. Through the collaborative work of the cloud and the edge, the system can achieve more comprehensive and refined construction site safety management, timely warning and handling of safety hazards. This technology not only improves the level of intelligent construction site safety management, but also reduces labor costs and safety risks to a certain extent.

[0055] Based on the above embodiments, the method also includes: receiving camera binding information sent from the cloud; the camera binding information includes the binding relationship between the edge device and the camera; configuring the camera according to the camera binding information to achieve local management of the camera; wherein, local management of the camera includes at least one of turning on the camera, turning off the camera, and changing the camera information in the cloud online.

[0056] In this embodiment, the edge device can also add, delete, modify edge device related information, camera visualization configuration information, modify camera configuration information (for example, data volume sending interval information), and dynamically maintain it to the running system.

[0057] Through the embodiments of the present disclosure, after configuring an algorithm for a certain area (a certain camera), an edge device is specified to perform service operation in a selected manner, thereby achieving flexible configuration, unified shutdown, and online change of configuration information of the PC dynamic service module. The PC dynamic service module changes the configuration information of the data configuration module online based on Http communication, and the configuration can be updated without restarting the system to meet daily maintenance needs.

[0058] The second aspect of the present disclosure provides a construction site safety management system based on cloud-edge collaboration, including: a cloud, used to configure edge devices and cameras; pre-training to obtain a first dangerous behavior recognition model; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for edge devices; deploying the second dangerous behavior recognition model to edge devices; edge devices, used to receive the second dangerous behavior recognition model sent by the cloud; using the second dangerous recognition model to recognize camera data to obtain dangerous behavior recognition results; the front end displays the dangerous behavior recognition results; a camera, used to obtain camera data and send it to the cloud and edge devices; wherein the first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

[0059] Based on the above method, the present disclosure also provides a construction site safety management system based on cloud-edge collaboration. Figure 4 The device is described in detail.

[0060] like Figure 4 As shown, the construction site safety management system based on cloud-edge collaboration of this embodiment includes a cloud, used to configure edge devices and cameras; pre-training to obtain a first dangerous behavior recognition model; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for edge devices; deploying the second dangerous behavior recognition model to the edge device; an edge device, used to receive the second dangerous behavior recognition model sent by the cloud; using the second dangerous recognition model to recognize camera data to obtain dangerous behavior recognition results; the front end displays the dangerous behavior recognition results; a camera, used to obtain camera data and send it to the cloud and edge devices; wherein the first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

[0061] It should be noted that the cloud can be used to execute the above-mentioned construction site safety management method based on cloud-edge collaboration applied to the cloud, and the edge device can be used to execute the above-mentioned construction site safety management method based on cloud-edge collaboration applied to the edge device.

[0062] Figure 2The functional module diagram of the construction site safety management method based on cloud-edge collaboration according to the embodiment of the present disclosure is schematically shown; a smart construction site safety wear intelligent identification system based on cloud-edge-end collaboration, the method and system include a PC dynamic service configuration module, a database management module, a front-end visualization module, and an edge-side high-precision algorithm recognition module (deployed on a fully autonomous and controllable domestically produced edge device), wherein the database management module is used to store relevant identification information, identification results and data sending status in the database on both ends; the front-end visualization module is used to perform real-time rotation of violation information (not wearing a safety helmet, reflective clothing), violation time, screenshot information, processing suggestions, etc. in the platform-related algorithms; the PC dynamic service configuration module is used to add, delete, and modify configuration information online, mainly including edge device IP, port and other information, camera IP, account, password, collection interval and other information, as well as related information of the edge device and the camera; the edge-side high-precision algorithm recognition module is used to deploy a hazard recognition behavior model on the edge side to perform real-time data analysis on the workers under the camera, and continuously optimize and upgrade the algorithm through on-site data feedback to improve the environmental pertinence of the algorithm.

[0063] Figure 5 A block diagram of an electronic device suitable for implementing a construction site safety management method based on cloud-edge collaboration according to an embodiment of the present disclosure is schematically shown.

[0064] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0065] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0066] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0067] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0068] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0069] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0070] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0071] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0072] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A construction site safety management method based on cloud-edge collaboration, characterized in that: Applied in the cloud, including: Pre-training to obtain the first dangerous behavior recognition model; Configure edge devices and cameras; The first dangerous behavior recognition model is converted to obtain a second dangerous behavior recognition model suitable for the edge device so as to deploy the second dangerous behavior recognition model to the edge device; the first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

2. The method according to claim 1, wherein: The configuration of the edge device and the camera includes: Configure edge device information; the edge device information includes at least one of edge device IP, edge device port, edge device name and edge device attribute; Configure camera information; the camera information includes at least one of the camera IP address, camera port number, camera user name and password, camera data transmission interval, camera area fixed point and camera group information; For each camera, an edge device is selected to be bound to the camera so that the edge device can process camera data.

3. The method according to claim 2, wherein: The converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for the edge device includes: Using the first dangerous behavior recognition model, generate a BModel in FP32 floating point format as a second dangerous behavior recognition model; And / or, using the first dangerous behavior recognition model, generating a BModel in FP32 floating point format; establishing a representative input data set; using an automatic quantization tool, converting the BModel into a quantized BModel as a second dangerous behavior recognition model according to the representative input data set; And / or, using the first dangerous behavior recognition model, generating an intermediate UModel in FP32 floating point format; establishing a representative input data set; processing the representative input data set into LMDB format; using the representative input data set to manually quantize the UModel, and obtaining the quantized UModel as the second dangerous behavior recognition model; Among them, the representative input data set includes typical input samples of the dangerous behavior recognition model; the typical input samples at least include camera data of cameras when a safety helmet is not worn, a reflective vest is not worn, and a safety helmet and reflective vest are both worn.

4. A construction site safety management method based on cloud-edge collaboration, characterized in that: Applied to edge devices, including: receiving a second dangerous behavior recognition model sent by the cloud; The second danger identification model is used to identify the camera data to obtain a dangerous behavior identification result; the second dangerous behavior identification model is used to output the dangerous behavior identification result through the input camera data; Call the front end to display the dangerous behavior identification results.

5. The method according to claim 4, wherein: The step of using the second danger identification model to identify camera data to obtain a dangerous behavior identification result includes: Extract video frames from the video queue using multi-process mode based on the IP address of the camera; The video frame is identified by using the second danger identification model to obtain a dangerous behavior identification result; the dangerous behavior identification result includes an identification box, a category, a region image and a video frame containing the dangerous behavior.

6. The method according to claim 5, wherein: After obtaining the dangerous behavior identification result, the following steps are further included: Storing the dangerous behavior identification result locally; The dangerous behavior identification results, the IP address of the camera, the video timestamp and the location are uploaded to the cloud for the cloud to optimize the second dangerous behavior identification model and record the dangerous behavior.

7. The method according to claim 1, wherein: The method further comprises: Receive camera binding information sent by the cloud; the camera binding information includes a binding relationship between the edge device and the camera; According to the camera binding information, configure the camera to achieve local camera management; Wherein, the local camera management includes at least one of turning on the camera, turning off the camera and changing the camera information in the cloud online.

8. A construction site safety management system based on cloud-edge collaboration, characterized in that: include: The cloud, used to configure edge devices and cameras; Pre-training to obtain a first dangerous behavior recognition model; converting the first dangerous behavior recognition model to obtain a second dangerous behavior recognition model suitable for an edge device; and deploying the second dangerous behavior recognition model to the edge device; The edge device is used to receive the second dangerous behavior recognition model sent by the cloud; use the second dangerous behavior recognition model to recognize the camera data to obtain a dangerous behavior recognition result; and call the front end to display the dangerous behavior recognition result; A camera, used to obtain camera data and send it to the cloud and the edge device; The first dangerous behavior recognition model and the second dangerous behavior recognition model are used to output dangerous behavior recognition results through input camera data.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images