Operation safety monitoring method and device based on artificial intelligence model collaboration

Abstract

The invention relates to an operation safety monitoring method and device based on artificial intelligence model collaboration, and relates to the technical field of operation safety monitoring. The method comprises the following steps: acquiring a monitoring video stream in a construction site, and determining a to-be-monitored risk operation event corresponding to the monitoring video stream in a plurality of preset risk operation events; based on the to-be-monitored risk operation event, determining a corresponding target operation safety monitoring model from a plurality of pre-trained operation safety monitoring models; each operation safety monitoring model at least comprises a first operation safety monitoring model and a second operation safety monitoring model, and the number of model parameters of the first operation safety monitoring model is smaller than the number of model parameters of the second operation safety monitoring model; and according to the target operation safety monitoring model and the monitoring video stream, determining an operation safety monitoring result of the construction site under the to-be-monitored risk operation event. By adopting the method, the operation risk in the construction site can be reduced.

Description

Technical Field

[0001] This application relates to the field of job safety monitoring technology, and in particular to a job safety monitoring method, device, computer equipment, computer-readable storage medium, and computer program product based on artificial intelligence model collaboration. Background Technology

[0002] During the construction phase, infrastructure projects typically present complex and dangerous environments, large numbers of personnel, and high mobility, making it difficult to guarantee the safety of operations. To ensure safe operations during the construction phase, it is usually necessary to monitor the safety of the construction site.

[0003] In related technologies, the safety of construction sites is usually monitored manually based on surveillance videos. However, the construction work is complex and there is a lot of personnel movement at the construction site, making it difficult to achieve real-time and comprehensive safety monitoring manually, which results in a relatively high risk of operation at the construction site. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for work safety monitoring based on artificial intelligence model collaboration, which can reduce the work risks in construction sites, in order to address the technical problem that the work risks in construction sites are still relatively high.

[0005] Firstly, this application provides a work safety monitoring method based on collaborative artificial intelligence models, including:

[0006] Acquire surveillance video streams within the construction site, and determine the risk operation event to be monitored corresponding to the surveillance video streams from a set of preset risk operation events;

[0007] Based on the risky work event to be monitored, a corresponding target work safety monitoring model is determined from multiple pre-trained work safety monitoring models; the multiple work safety monitoring models include at least a first work safety monitoring model and a second work safety monitoring model, wherein the number of model parameters of the first work safety monitoring model is less than the number of model parameters of the second work safety monitoring model;

[0008] Based on the target operation safety monitoring model and the monitoring video stream, determine the operation safety monitoring results of the construction site under the monitored risk operation event.

[0009] In one embodiment, determining the risk operation event to be monitored corresponding to the monitoring video stream from a preset plurality of risk operation events includes:

[0010] Determine the target object present in the monitoring screen corresponding to the monitoring video stream;

[0011] Based on the target object, the risk operation event to be monitored is determined from among the multiple risk operation events.

[0012] In one embodiment, the risky operation event includes a first risky operation event and a second risky operation event. The difficulty of judging the first risky operation event is less than the difficulty of judging the second risky operation event. The difficulty of judging the event is used to characterize the difficulty of judging whether the event exists.

[0013] The step of determining the corresponding target operation safety monitoring model from multiple pre-trained operation safety monitoring models based on the risk operation event to be monitored includes:

[0014] If the risky work event to be monitored is the first risky work event, the first work safety monitoring model is determined as the target work safety monitoring model;

[0015] If the risky work event to be monitored is the second risky work event, the second work safety monitoring model is determined as the target work safety monitoring model;

[0016] If the risk operation event to be monitored is either the first risk operation event or the second risk operation event, then both the first operation safety monitoring model and the second operation safety monitoring model are determined as the target operation safety monitoring model.

[0017] In one embodiment, determining the work safety monitoring results of the construction site under the monitored risk event based on the target work safety monitoring model and the monitoring video stream includes:

[0018] When the target operation safety monitoring model is the first operation safety monitoring model, based on the first operation safety monitoring model, the first target detection result corresponding to the first risk operation event in the monitoring video stream is determined, and the operation safety monitoring result of the construction site under the first risk operation event is obtained.

[0019] When the target operation safety monitoring model is the second operation safety monitoring model, based on the second operation safety monitoring model, the second target detection result corresponding to the second risk operation event in the monitoring video stream is determined, and the operation safety monitoring result of the construction site under the second risk operation event is obtained.

[0020] When the target operation safety monitoring model is the first operation safety monitoring model and the second operation safety monitoring model, based on the first operation safety monitoring model, the first target detection result corresponding to the first risk operation event is determined by the monitoring video stream, and based on the second operation safety monitoring model, the second target detection result corresponding to the second risk operation event is determined by the monitoring video stream, thereby obtaining the operation safety monitoring results of the construction site under the first risk operation event and the second risk operation event.

[0021] In one embodiment, the method further includes:

[0022] When the verification condition of the first target detection result is triggered, the first target detection result determined by the first operation safety monitoring model is verified according to the second operation safety monitoring model and the monitoring video stream to obtain the verification target detection result of the first risk operation event;

[0023] Based on the detection results of the verified target, the operational safety monitoring results of the construction site under the first risky operation event are obtained.

[0024] In one embodiment, the method is applied to any edge node in a collaborative network;

[0025] The step of determining the corresponding target operation safety monitoring model from multiple pre-trained operation safety monitoring models includes:

[0026] The target operation safety monitoring model is determined from the operation safety monitoring models deployed on the edge nodes;

[0027] After determining the operational safety monitoring results of the construction site under the monitored risk event, the process also includes:

[0028] The operation safety monitoring results are reported to the central terminal in the collaborative network.

[0029] Secondly, this application also provides a work safety monitoring device based on artificial intelligence model collaboration, comprising:

[0030] The monitoring event determination module is used to acquire the monitoring video stream within the construction site and determine the risk operation event to be monitored corresponding to the monitoring video stream from a set of preset risk operation events.

[0031] The monitoring model determination module is used to determine the corresponding target operation safety monitoring model from a plurality of pre-trained operation safety monitoring models based on the risk operation event to be monitored; the plurality of operation safety monitoring models include at least a first operation safety monitoring model and a second operation safety monitoring model, wherein the number of model parameters of the first operation safety monitoring model is less than the number of model parameters of the second operation safety monitoring model;

[0032] The operation safety monitoring module is used to determine the operation safety monitoring results of the construction site under the monitored risk operation event based on the target operation safety monitoring model and the monitoring video stream.

[0033] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0034] Acquire surveillance video streams within the construction site, and determine the risk operation event to be monitored corresponding to the surveillance video streams from a set of preset risk operation events;

[0035] Based on the risky work event to be monitored, a corresponding target work safety monitoring model is determined from multiple pre-trained work safety monitoring models; the multiple work safety monitoring models include at least a first work safety monitoring model and a second work safety monitoring model, wherein the number of model parameters of the first work safety monitoring model is less than the number of model parameters of the second work safety monitoring model;

[0036] Based on the target operation safety monitoring model and the monitoring video stream, determine the operation safety monitoring results of the construction site under the monitored risk operation event.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0038] Acquire surveillance video streams within the construction site, and determine the risk operation event to be monitored corresponding to the surveillance video streams from a set of preset risk operation events;

[0039] Based on the risky work event to be monitored, a corresponding target work safety monitoring model is determined from multiple pre-trained work safety monitoring models; the multiple work safety monitoring models include at least a first work safety monitoring model and a second work safety monitoring model, wherein the number of model parameters of the first work safety monitoring model is less than the number of model parameters of the second work safety monitoring model;

[0040] Based on the target operation safety monitoring model and the monitoring video stream, determine the operation safety monitoring results of the construction site under the monitored risk operation event.

[0041] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0042] Acquire surveillance video streams within the construction site, and determine the risk operation event to be monitored corresponding to the surveillance video streams from a set of preset risk operation events;

[0043] Based on the risky work event to be monitored, a corresponding target work safety monitoring model is determined from multiple pre-trained work safety monitoring models; the multiple work safety monitoring models include at least a first work safety monitoring model and a second work safety monitoring model, wherein the number of model parameters of the first work safety monitoring model is less than the number of model parameters of the second work safety monitoring model;

[0044] Based on the target operation safety monitoring model and the monitoring video stream, determine the operation safety monitoring results of the construction site under the monitored risk operation event.

[0045] The aforementioned work safety monitoring method, device, computer equipment, computer-readable storage medium, and computer program product based on artificial intelligence model collaboration, based on monitoring video streams, can identify the current risky work event to be monitored from multiple preset risky work events. Based on the current risky work event to be monitored, it can determine a suitable target work safety monitoring model from a first work safety monitoring model with fewer model parameters and a second work safety monitoring model with more model parameters. Through the target work safety monitoring model and the monitoring video stream, it can perform work safety monitoring on the risky work event to be monitored and obtain the work safety monitoring results of the construction site under the risky work event to be monitored. Based on the above process, the work safety monitoring method based on artificial intelligence model collaboration no longer relies on manual monitoring and can realize real-time and comprehensive work safety monitoring of the construction site, thus reducing the work risks in the construction site. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1This is an application environment diagram of an artificial intelligence model-based collaborative operation safety monitoring method in one embodiment;

[0048] Figure 2 This is a flowchart illustrating a work safety monitoring method based on artificial intelligence model collaboration in one embodiment;

[0049] Figure 3 This is a flowchart illustrating a method for monitoring operational safety during the construction phase of a pumped storage power station based on artificial intelligence model collaboration, as described in another embodiment.

[0050] Figure 4 This is a structural block diagram of a work safety monitoring device based on an artificial intelligence model collaboration in one embodiment;

[0051] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0054] The work safety monitoring method based on artificial intelligence model collaboration provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the application environment is a collaborative network, which includes a central terminal 102 and multiple edge nodes 104. The central terminal 102 communicates with the edge nodes 104 through the network. The central terminal 102 and the edge nodes 104 can be servers, terminals, or systems that include both servers and terminals. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services. Terminals can be, but are not limited to, various video capture devices, personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, etc.

[0055] In practical applications, each construction site corresponds to at least one edge node 104. Edge node 104 first acquires the monitoring video stream within the construction site and, among a set of pre-defined risky operation events, identifies the risky operation event to be monitored corresponding to the monitoring video stream. Then, based on the risky operation event to be monitored, it determines the corresponding target operation safety monitoring model from a set of pre-trained operation safety monitoring models. Each operation safety monitoring model includes at least a first operation safety monitoring model and a second operation safety monitoring model, with the number of model parameters in the first operation safety monitoring model being less than the number of model parameters in the second operation safety monitoring model. Next, based on the target operation safety monitoring model and the monitoring video stream, it determines the operation safety monitoring results of the construction site under the risky operation event to be monitored. Finally, it reports the operation safety monitoring results to the central terminal 102.

[0056] In one exemplary embodiment, such as Figure 2 As shown, a work safety monitoring method based on artificial intelligence model collaboration is provided, which is then applied to... Figure 1 Taking edge nodes in the example, the explanation includes the following steps:

[0057] Step S202: Obtain the monitoring video stream within the construction site, and determine the risk operation event to be monitored corresponding to the monitoring video stream from among a set of preset risk operation events.

[0058] The construction site is equipped with multiple video acquisition devices, and the monitoring video stream includes the monitoring video streams collected by each video acquisition device.

[0059] Among them, a work event refers to an event that includes work behavior; a risky work event refers to a work event that may lead to safety risks; the difficulty of judging different risky work events is different, and the difficulty of judging events is used to characterize the difficulty of judging whether an event exists.

[0060] Specifically, the edge node acquires the monitoring video streams collected by each video acquisition device in the construction site, and for each monitoring video stream, based on the monitoring screen corresponding to the monitoring video stream, determines the risk operation event to be monitored among multiple risk operation events.

[0061] Among them, the monitoring screen corresponding to the monitoring video stream is the screen captured by that monitoring video stream.

[0062] Step S204: Based on the risky operation event to be monitored, determine the corresponding target operation safety monitoring model from multiple pre-trained operation safety monitoring models.

[0063] Each operation safety monitoring model includes at least a first operation safety monitoring model and a second operation safety monitoring model. The first operation safety monitoring model has fewer model parameters than the second operation safety monitoring model; that is, the first operation safety monitoring model is a small model, and the second operation safety monitoring model is a large model. In practical applications, the first operation safety monitoring model has between several million and tens of millions of model parameters, while the second operation safety monitoring model has hundreds of millions or even hundreds of millions of model parameters. The first operation safety monitoring model has relatively low computational resource requirements, faster inference and training speeds, and can handle simple tasks. The second operation safety monitoring model has relatively high computational resource requirements, slower inference and training speeds, but higher inference accuracy for complex tasks. In practical applications, the first operation safety monitoring model uses the CNN (Convolutional Neural Network) framework, while the second operation safety monitoring model uses the Transformer framework.

[0064] Specifically, for each monitoring video stream, the edge node selects a suitable model from the first and second operation safety monitoring models deployed on the edge node based on the difficulty of event judgment of the risk operation event to be monitored corresponding to the monitoring video stream, and uses it as the target operation safety monitoring model corresponding to that monitoring video stream.

[0065] In practical applications, if the difficulty of event judgment is relatively low, the edge node can choose the first operation safety monitoring model as the target operation safety monitoring model to save computing resources and improve monitoring speed; if the difficulty of event judgment is relatively high, the edge node can choose the second operation safety monitoring model as the target operation safety monitoring model to improve monitoring accuracy.

[0066] Step S206: Based on the target operation safety monitoring model and the monitoring video stream, determine the operation safety monitoring results of the construction site under the risk operation event to be monitored.

[0067] Among them, the work safety monitoring results corresponding to the risky work events to be monitored are used to characterize whether the risky work events to be monitored exist.

[0068] Specifically, for each monitoring video stream, the edge node calls the target operation safety monitoring model corresponding to that monitoring video stream to perform target detection on the monitoring video stream for the risk operation event to be monitored, determines whether the risk operation event to be monitored exists, and obtains the operation safety monitoring result of the construction site under that monitoring video stream for the risk operation event to be monitored in the corresponding monitoring video stream.

[0069] In the aforementioned AI-based collaborative operation safety monitoring method, edge nodes, based on monitoring video streams, can identify the current risky operation event to be monitored from a set of preset risky operation events. Based on this event, a suitable target operation safety monitoring model can be selected from a first operation safety monitoring model with fewer parameters and a second model with more parameters. Through the target model and the monitoring video stream, operation safety monitoring can be performed on the risky operation event, yielding the operation safety monitoring results for the construction site under the event. This AI-based collaborative operation safety monitoring method, based on the above process, no longer relies on manual monitoring and can achieve real-time and comprehensive operation safety monitoring of the construction site, thus reducing operational risks within the construction site.

[0070] In an exemplary embodiment, step S102 above, which involves determining the risk operation event to be monitored corresponding to the monitoring video stream among a plurality of preset risk operation events, specifically includes the following steps: determining the target object present in the monitoring screen corresponding to the monitoring video stream; and determining the risk operation event to be monitored among the plurality of risk operation events based on the target object.

[0071] The target object is at least one of an event, behavior, object, and humanoid object.

[0072] Specifically, the edge node has multiple target objects preset; for each monitoring video stream, the edge node determines the target objects existing in the monitoring screen corresponding to the monitoring video stream, and based on the existing target objects, determines the risk operation event to be monitored among multiple risk operation events.

[0073] For example, if the target object is at least one of high-altitude work events, high-altitude work behaviors, or high-altitude work personnel, then the edge node will take "high-altitude work without safety rope" as the risky work event to be monitored among multiple risky work events.

[0074] For example, if the target objects are high-risk areas and humanoid objects near high-risk areas, then the edge node will take "unauthorized entry into dangerous areas" from multiple risk operation events as the risk event to be monitored.

[0075] For example, if the target object is large construction equipment, then the edge node will take "abnormal equipment operation" from multiple risk operation events as the risk event to be monitored.

[0076] For example, if the target object is at least one of the following: construction equipment that is prone to fire, construction activities that are prone to fire, or construction sites that are prone to fire, then the edge node will take at least one of "fire occurred" and "fire hazard exists" from the multiple risk operation events as the risk events to be monitored.

[0077] In this embodiment, the edge node can monitor the target objects in the video stream and determine the risk operation events that need to be monitored in the current video stream, so as to facilitate the subsequent reasonable selection of a suitable target operation safety monitoring model for operation safety monitoring.

[0078] In an exemplary embodiment, the risky operation events include a first risky operation event and a second risky operation event. The difficulty of judging the first risky operation event is lower than that of judging the second risky operation event. The difficulty of judging the event is used to characterize the difficulty of determining whether the event exists. In specific applications, the edge node also presets a corresponding difficulty of judging the event for each risky operation event. Furthermore, the edge node can also update the corresponding difficulty of judging the event based on the accuracy of judging the event for each risky operation event, wherein the accuracy of judging the event exists is used to characterize the accuracy of determining whether the event exists.

[0079] Step S104 above, based on the risky work event to be monitored, determines the corresponding target work safety monitoring model from multiple pre-trained work safety monitoring models, specifically including the following steps: if the risky work event to be monitored is a first risky work event, the first work safety monitoring model is determined as the target work safety monitoring model; if the risky work event to be monitored is a second risky work event, the second work safety monitoring model is determined as the target work safety monitoring model; if the risky work event to be monitored is both a first risky work event and a second risky work event, both the first work safety monitoring model and the second work safety monitoring model are determined as target work safety monitoring models.

[0080] Specifically, if the risk operation events to be monitored only include the first risk operation events with lower difficulty in event judgment, then the edge node will determine the first operation safety monitoring model with fewer model parameters, lower requirements for computing resources, and faster inference speed as the target operation safety monitoring model, so as to save computing resources and improve monitoring speed.

[0081] If the risk events to be monitored only include the second risk operation events that are more difficult to judge, then the edge node will determine the second operation safety monitoring model with more model parameters and higher inference accuracy for complex tasks as the target operation safety monitoring model, so as to improve the monitoring accuracy.

[0082] If the risk event to be monitored includes both the first risk operation event and the second risk operation event, the edge node will select both the first operation safety monitoring model and the second operation safety monitoring model as the target operation safety monitoring model to balance the consumption of computing resources, inference speed and inference accuracy.

[0083] In this embodiment, for simple tasks, the edge nodes select a small model as the target job safety monitoring model, while for complex tasks, the edge nodes select a large model as the target job safety monitoring model, thereby balancing the consumption of computing resources, inference speed, and inference accuracy.

[0084] In an exemplary embodiment, step S106 above, determining the operation safety monitoring results of the construction site under the monitored risk operation event based on the target operation safety monitoring model and the monitoring video stream, specifically includes the following steps: If the target operation safety monitoring model is a first operation safety monitoring model, based on the first operation safety monitoring model, determine the first target detection result of the monitoring video stream corresponding to the first risk operation event, thus obtaining the operation safety monitoring results of the construction site under the first risk operation event; if the target operation safety monitoring model is a second operation safety monitoring model, based on the second operation safety monitoring model, determine the second target detection result of the monitoring video stream corresponding to the second risk operation event, thus obtaining the operation safety monitoring results of the construction site under the second risk operation event; if the target operation safety monitoring model is both a first operation safety monitoring model and a second operation safety monitoring model, based on the first operation safety monitoring model, determine the first target detection result of the monitoring video stream corresponding to the first risk operation event, and based on the second operation safety monitoring model, determine the second target detection result of the monitoring video stream corresponding to the second risk operation event, thus determining the operation safety monitoring results of the construction site under both the first and second risk operation events.

[0085] Specifically, when the risk operation events to be monitored only include the first risk operation events with low difficulty in event judgment, the edge node selects the first operation safety monitoring model as the target operation safety monitoring model. Therefore, the edge node calls the first operation safety monitoring model to perform target detection processing on the monitoring video stream corresponding to the first risk operation event, obtains the first target detection result that characterizes whether the first risk operation event exists, and uses it as the operation safety monitoring result of the construction site under the monitoring video stream corresponding to the first risk operation event.

[0086] When the risk operation events to be monitored only include the second risk operation events, which are more difficult to judge, the edge node selects the second operation safety monitoring model as the target operation safety monitoring model. Therefore, the edge node calls the second operation safety monitoring model to perform target detection processing on the monitoring video stream corresponding to the second risk operation event, obtains the second target detection result that represents whether the second risk operation event exists, and uses it as the operation safety monitoring result of the construction site in the monitoring video stream corresponding to the second risk operation event.

[0087] When the monitored risk operation event includes both a first risk operation event and a second risk operation event, the edge node simultaneously selects the first operation safety monitoring model and the second operation safety monitoring model as the target operation safety monitoring model. Therefore, the edge node calls the first operation safety monitoring model and the second operation safety monitoring model respectively. The first operation safety monitoring model is used to perform target detection processing on the monitoring video stream corresponding to the first risk operation event to obtain a first target detection result that characterizes whether the first risk operation event exists. This first target detection result is then used as the operation safety monitoring result of the construction site under the monitoring video stream corresponding to the first risk operation event. The second operation safety monitoring model is used to perform target detection processing on the monitoring video stream corresponding to the second risk operation event to obtain a second target detection result that characterizes whether the second risk operation event exists. This second target detection result is then used as the operation safety monitoring result of the construction site under the monitoring video stream corresponding to the second risk operation event.

[0088] In this embodiment, for simple tasks, the edge node selects a small model as the target job safety monitoring model for inference, while for complex tasks, the edge node selects a large model as the target job safety monitoring model for inference, thereby balancing the consumption of computing resources, inference speed, and inference accuracy.

[0089] In an exemplary embodiment, the method further includes the following steps for reviewing the first target detection result determined by the first operation safety monitoring model: when the review conditions of the first target detection result are triggered, the first target detection result determined by the first operation safety monitoring model is reviewed according to the second operation safety monitoring model and the monitoring video stream to obtain the review target detection result of the first risk operation event; based on the review target detection result, the operation safety monitoring result of the construction site under the first risk operation event is obtained.

[0090] Specifically, in response to the result verification instruction triggered by the supervisor for the first target detection result, the edge node determines the verification condition trigger for the first target detection result, and then calls the second operation safety monitoring model to perform target detection processing on the monitoring video stream corresponding to the first risk operation event, so as to verify the first target detection result determined by the first operation safety monitoring model, obtain the verification target detection result that characterizes whether the first risk operation event exists, and use it as the operation safety monitoring result of the construction site under the monitoring video stream corresponding to the first risk operation event.

[0091] In this embodiment, for simple tasks, in addition to reasoning based on a small model, edge nodes can also be verified based on a large model to achieve accurate reasoning, thereby ensuring monitoring accuracy.

[0092] In an exemplary embodiment, step S104 above, which determines the corresponding target job safety monitoring model from a plurality of pre-trained job safety monitoring models, specifically includes the following steps: determining the target job safety monitoring model from each job safety monitoring model deployed on the edge node.

[0093] Specifically, the first and second operation safety monitoring models are deployed on edge nodes, and the edge nodes determine the appropriate target operation safety monitoring model from the first and second operation safety monitoring models they have deployed.

[0094] In this embodiment, after determining the operation safety monitoring results of the construction site under the risk operation event to be monitored in step S106 above, the following steps are also included: reporting the operation safety monitoring results to the central end in the collaborative network.

[0095] Specifically, edge nodes report the operation safety monitoring results to the central end; the central end is used to store the operation safety monitoring results and generate operation safety reports for one or more construction sites based on the operation safety monitoring results reported by different edge nodes.

[0096] In this embodiment, based on the collaborative network, the local deployment of the model and the aggregation of operation safety monitoring results can be realized; the central end can perform overall analysis of operation safety reports of one or more construction sites based on the aggregation of operation safety monitoring results reported by different edge nodes.

[0097] In an exemplary embodiment, after determining the work safety monitoring results of the construction site under the monitored risk operation event in step S106 above, the step further includes the following steps: in response to the work safety monitoring results, controlling the equipment in the construction site to alarm and stop the existing risk operation event.

[0098] Specifically, if the existence of a risky work event to be monitored is determined based on the results of work safety monitoring, the edge node controls the audible and visual alarm devices in the construction site to issue an alarm for the existing risky work event, or controls the equipment in the construction site related to the existing risky work event to stop operating, so as to prevent the further development of the existing risky work event.

[0099] To more clearly illustrate the work safety monitoring method based on artificial intelligence model collaboration provided in this application, a specific embodiment is given below for detailed description. However, it should be understood that the embodiments of this application are not limited thereto. Figure 3 As shown, in one exemplary embodiment, this application also provides a method for monitoring the operational safety of a pumped storage power station during the infrastructure construction phase based on an artificial intelligence model collaboration technology, specifically including the following steps:

[0100] Step S302: The edge node acquires the monitoring video stream within the construction site of the pumped storage power station.

[0101] Step S304: The edge node determines the risk operation event to be monitored from among multiple risk operation events based on the target object present in the monitoring screen corresponding to the monitoring video stream.

[0102] In step S306, when the risk operation event to be monitored is a simple first risk operation event, the edge node calls the first operation safety monitoring model deployed on the edge node and trained with a small model framework to determine the first target detection result corresponding to the first risk operation event in the monitoring video stream, and obtains the operation safety monitoring result of the construction site under the first risk operation event.

[0103] In step S308, when the risk operation event to be monitored is a complex second risk operation event, the edge node calls the second operation safety monitoring model deployed on the edge node and trained with a large model framework to determine the second target detection result corresponding to the second risk operation event in the monitoring video stream, and obtains the operation safety monitoring result of the construction site under the second risk operation event.

[0104] In step S310, when the risk operation event to be monitored is the first risk operation event and the second risk operation event, the edge node calls the first operation safety monitoring model trained with a small model framework and the second operation safety monitoring model trained with a large model framework deployed on the edge node, respectively, to determine the first target detection result corresponding to the first risk operation event and the second target detection result corresponding to the second risk operation event in the monitoring video stream, and obtains the operation safety monitoring results of the construction site under the first risk operation event and the second risk operation event.

[0105] In step S312, the edge node reports the operation safety monitoring results to the central terminal.

[0106] In this embodiment, the collaborative use of large and small models enables operational safety management during the construction phase of a pumped storage power station, overcoming the limitations of traditional manual inspections and single-modal monitoring, and improving risk identification efficiency and emergency response capabilities.

[0107] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0108] Based on the same inventive concept, this application also provides an artificial intelligence model-based collaborative work safety monitoring device for implementing the aforementioned work safety monitoring method based on artificial intelligence model collaboration. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the work safety monitoring device based on artificial intelligence model collaboration provided below can be found in the limitations of the work safety monitoring method based on artificial intelligence model collaboration described above, and will not be repeated here.

[0109] In one exemplary embodiment, such as Figure 4 As shown, a work safety monitoring device based on artificial intelligence model collaboration is provided, including: a monitoring event determination module 402, a monitoring model determination module 404, and a work safety monitoring module 406, wherein:

[0110] The monitoring event determination module 402 is used to acquire the monitoring video stream in the construction site and determine the risk operation event to be monitored corresponding to the monitoring video stream among a number of preset risk operation events.

[0111] The monitoring model determination module 404 is used to determine the corresponding target operation safety monitoring model from a number of pre-trained operation safety monitoring models based on the risk operation event to be monitored. Each operation safety monitoring model includes at least a first operation safety monitoring model and a second operation safety monitoring model, and the number of model parameters of the first operation safety monitoring model is less than the number of model parameters of the second operation safety monitoring model.

[0112] The operation safety monitoring module 406 is used to determine the operation safety monitoring results of the construction site under the risk operation event to be monitored, based on the target operation safety monitoring model and the monitoring video stream.

[0113] In an exemplary embodiment, the monitoring event determination module 402 is further configured to determine the target object present in the monitoring screen corresponding to the monitoring video stream; and based on the target object, determine the risk operation event to be monitored among multiple risk operation events.

[0114] In an exemplary embodiment, the risky operation event includes a first risky operation event and a second risky operation event. The difficulty of judging the first risky operation event is less than the difficulty of judging the second risky operation event. The difficulty of judging the event is used to characterize the difficulty of judging whether the event exists.

[0115] Based on the risky work event to be monitored, the monitoring model determination module 404 is further configured to determine the first work safety monitoring model as the target work safety monitoring model when the risky work event to be monitored is a first risky work event; determine the second work safety monitoring model as the target work safety monitoring model when the risky work event to be monitored is a second risky work event; and determine both the first work safety monitoring model and the second work safety monitoring model as target work safety monitoring models when the risky work event to be monitored is both a first risky work event and a second risky work event.

[0116] In an exemplary embodiment, the operation safety monitoring module 406 is further configured to: when the target operation safety monitoring model is a first operation safety monitoring model, determine the first target detection result of the monitoring video stream corresponding to the first risk operation event based on the first operation safety monitoring model, and obtain the operation safety monitoring result of the construction site under the first risk operation event; when the target operation safety monitoring model is a second operation safety monitoring model, determine the second target detection result of the monitoring video stream corresponding to the second risk operation event based on the second operation safety monitoring model, and obtain the operation safety monitoring result of the construction site under the second risk operation event; when the target operation safety monitoring model is both a first operation safety monitoring model and a second operation safety monitoring model, determine the first target detection result of the monitoring video stream corresponding to the first risk operation event based on the first operation safety monitoring model, and determine the second target detection result of the monitoring video stream corresponding to the second risk operation event based on the second operation safety monitoring model, and obtain the operation safety monitoring result of the construction site under both the first and second risk operation events.

[0117] In an exemplary embodiment, the operation safety monitoring module 406 is further configured to, when the verification condition of the first target detection result is triggered, verify the first target detection result determined by the first operation safety monitoring model according to the second operation safety monitoring model and the monitoring video stream, to obtain the verification target detection result of the first risk operation event; and obtain the operation safety monitoring result of the construction site under the first risk operation event according to the verification target detection result.

[0118] In an exemplary embodiment, the monitoring model determination module 404 is further configured to determine the target job safety monitoring model from among the various job safety monitoring models deployed on the edge node.

[0119] The work safety monitoring device based on artificial intelligence model collaboration also includes a monitoring result reporting module, which is used to report the work safety monitoring results to the central end in the collaborative network.

[0120] The modules in the aforementioned work safety monitoring device based on artificial intelligence model collaboration can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0121] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data such as the results of work safety monitoring at the construction site. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a work safety monitoring method based on an artificial intelligence model.

[0122] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0124] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0125] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0127] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0128] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for monitoring operational safety based on collaborative artificial intelligence models, characterized in that, The method includes: Acquire surveillance video streams within the construction site, and determine the risk operation event to be monitored corresponding to the surveillance video streams from a set of preset risk operation events; Based on the risky work event to be monitored, a corresponding target work safety monitoring model is determined from multiple pre-trained work safety monitoring models; the multiple work safety monitoring models include at least a first work safety monitoring model and a second work safety monitoring model, wherein the number of model parameters of the first work safety monitoring model is less than the number of model parameters of the second work safety monitoring model; Based on the target operation safety monitoring model and the monitoring video stream, determine the operation safety monitoring results of the construction site under the monitored risk operation event.

2. The method according to claim 1, characterized in that, The step of determining the risk operation event to be monitored corresponding to the monitoring video stream from a set of preset risk operation events includes: Determine the target object present in the monitoring screen corresponding to the monitoring video stream; Based on the target object, the risk operation event to be monitored is determined from among the multiple risk operation events.

3. The method according to claim 1, characterized in that, The risky operation event includes a first risky operation event and a second risky operation event. The difficulty of judging the first risky operation event is less than the difficulty of judging the second risky operation event. The difficulty of judging the event is used to characterize the difficulty of judging whether the event exists. The step of determining the corresponding target operation safety monitoring model from multiple pre-trained operation safety monitoring models based on the risk operation event to be monitored includes: If the risky work event to be monitored is the first risky work event, the first work safety monitoring model is determined as the target work safety monitoring model; If the risky work event to be monitored is the second risky work event, the second work safety monitoring model is determined as the target work safety monitoring model; If the risk operation event to be monitored is either the first risk operation event or the second risk operation event, then both the first operation safety monitoring model and the second operation safety monitoring model are determined as the target operation safety monitoring model.

4. The method according to claim 3, characterized in that, The step of determining the operational safety monitoring results of the construction site under the monitored risk event based on the target operational safety monitoring model and the monitoring video stream includes: When the target operation safety monitoring model is the first operation safety monitoring model, based on the first operation safety monitoring model, the first target detection result corresponding to the first risk operation event in the monitoring video stream is determined, and the operation safety monitoring result of the construction site under the first risk operation event is obtained. When the target operation safety monitoring model is the second operation safety monitoring model, based on the second operation safety monitoring model, the second target detection result corresponding to the second risk operation event in the monitoring video stream is determined, and the operation safety monitoring result of the construction site under the second risk operation event is obtained. When the target operation safety monitoring model is the first operation safety monitoring model and the second operation safety monitoring model, based on the first operation safety monitoring model, the first target detection result corresponding to the first risk operation event is determined by the monitoring video stream, and based on the second operation safety monitoring model, the second target detection result corresponding to the second risk operation event is determined by the monitoring video stream, thereby obtaining the operation safety monitoring results of the construction site under the first risk operation event and the second risk operation event.

5. The method according to claim 4, characterized in that, The method further includes: When the verification condition of the first target detection result is triggered, the first target detection result determined by the first operation safety monitoring model is verified according to the second operation safety monitoring model and the monitoring video stream to obtain the verification target detection result of the first risk operation event; Based on the detection results of the verified target, the operational safety monitoring results of the construction site under the first risky operation event are obtained.

6. The method according to any one of claims 1 to 5, characterized in that, The method is applied to any edge node in a collaborative network; The step of determining the corresponding target operation safety monitoring model from multiple pre-trained operation safety monitoring models includes: The target operation safety monitoring model is determined from the operation safety monitoring models deployed on the edge nodes; After determining the operational safety monitoring results of the construction site under the monitored risk event, the process also includes: The operation safety monitoring results are reported to the central terminal in the collaborative network.

7. A work safety monitoring device based on artificial intelligence model collaboration, characterized in that, The device includes: The monitoring event determination module is used to acquire the monitoring video stream within the construction site and determine the risk operation event to be monitored corresponding to the monitoring video stream from a set of preset risk operation events. The monitoring model determination module is used to determine the corresponding target operation safety monitoring model from a plurality of pre-trained operation safety monitoring models based on the risk operation event to be monitored; the plurality of operation safety monitoring models include at least a first operation safety monitoring model and a second operation safety monitoring model, wherein the number of model parameters of the first operation safety monitoring model is less than the number of model parameters of the second operation safety monitoring model; The operation safety monitoring module is used to determine the operation safety monitoring results of the construction site under the monitored risk operation event based on the target operation safety monitoring model and the monitoring video stream.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images