Dangerous operation automatic alarm method and device, dangerous operation management equipment and medium

Abstract

The invention provides an automatic dangerous operation alarm method and device, dangerous operation management equipment and a medium, and belongs to the technical field of dangerous operation process monitoring, and the method comprises the steps: determining a plurality of operation process nodes of dangerous operation, and determining a plurality of node key elements of the operation process nodes based on operation process specifications; the node key elements comprise personnel elements, equipment elements, field measure elements and field environment elements; determining an image keyword based on the node key elements, and performing region extraction on the field image of the dangerous work based on the image keyword to obtain an interested image corresponding to the node key elements; inputting the interested image into the hidden danger prediction model to obtain a predicted hidden danger value; and when the predicted hidden danger value is greater than the hidden danger safety threshold, generating an alarm signal. According to the invention, hidden danger value prediction is carried out based on four factors of personnel factors, equipment factors, field measure factors and field environment factors, and the timeliness and accuracy of automatic alarm of dangerous operation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of dangerous operation process monitoring, and in particular to a dangerous operation automatic alarm method, device, dangerous operation management equipment and medium. Background Art

[0002] Site management is the focus of enterprise safety management, and dangerous operation management is the top priority. In order to solve the problem of "dangerous operation personnel perform on-site operations based on experience and fill in paper operation records by hand, on-site supervisors fill in paper operation supervision records, safety management personnel conduct on-site spot checks, and leadership conducts spot checks on operation records and operation supervision records", this traditional "human defense" management method has problems such as incomplete and inaccurate hidden danger inspection, inadequate monitoring of dangerous operation processes, and failure to timely discover risks and hidden dangers.

[0003] In order to solve the above technical problems, the prior art has proposed an AI-based automatic alarm method for dangerous operations to identify safety hazards in dangerous operations; however, the current automatic alarm method for dangerous operations can only identify violations such as dangerous operation personnel not wearing safety helmets or work clothes. The algorithm is single and cannot meet the comprehensive identification of risk hazards of all factors involved in the dangerous operation process, such as personnel, equipment, on-site measures, and on-site environment.

[0004] Therefore, there is an urgent need to provide an automatic alarm method, device, dangerous operation management equipment and medium for dangerous operations to improve the identification of safety hazards in the entire operation process, to ensure that the operation process and operation behavior meet the requirements, to achieve automatic alarm, thereby reducing accidents caused by risk hazards during dangerous operations and improving the safety of dangerous operations. Summary of the invention

[0005] In view of this, it is necessary to provide an automatic alarm method, device, dangerous operation management equipment and medium for dangerous operations, so as to solve the technical problem that the identification algorithm in the prior art can only identify a single factor, resulting in incomplete identification of risks and hidden dangers, and thus causing low safety of dangerous operations.

[0006] On the one hand, in order to solve the above technical problems, the present invention provides a dangerous operation automatic alarm method, comprising: Determine multiple operation process nodes of dangerous operations, and determine multiple node key elements of the operation process nodes based on the operation process specifications; the node key elements include personnel elements, equipment elements, on-site measures elements and on-site environment elements; Determine image keywords based on the node key elements, and perform region extraction on the on-site image of the dangerous operation based on the image keywords to obtain multiple images of interest corresponding to the multiple node key elements; Inputting the plurality of images of interest into a hidden danger prediction model to obtain a predicted hidden danger value; When the predicted hidden danger value is greater than the hidden danger safety threshold, an alarm signal is generated.

[0007] In a possible implementation, the determining of multiple operation flow nodes of dangerous operations includes: Acquire the type of dangerous operation, and determine the dangerous operation process based on the dangerous operation type; the dangerous operation process includes a dangerous operation sub-process before the operation, a dangerous operation sub-process during the operation, and a dangerous operation sub-process after the operation; Determine a plurality of dangerous operation sub-steps in the dangerous operation process, and use the dangerous operation sub-steps as the operation process nodes.

[0008] In a possible implementation, the personnel factors include the behavior of the operators, including whether the operators have work permits, whether the operators are not in the process of applying for work, whether they wear protective equipment as required, whether the safety belts are hung high and used low, whether there is someone to supervise during high-altitude operations, and whether the tools are used in a standardized manner; the equipment factors include the equipment status; the on-site measures factors include whether fencing measures are set up, whether warning signs or warning tapes are set up, whether there are shading measures at the work site where arc laying is carried out, whether there are anti-slip measures for the high-altitude equipment, whether the fire sources at the site are cleared, and whether the flammable materials at the site are cleared; the on-site environmental factors include the on-site temperature, on-site noise, on-site wind speed, and whether there is water accumulation.

[0009] In a possible implementation, the hidden danger prediction model includes a deformable convolution initial recognition module and a recognition reasoning module; then the step of inputting the multiple images of interest into the hidden danger prediction model to obtain predicted hidden danger values ​​includes: Based on the deformable convolution initial recognition module, the multiple region of interest images are recognized and classified to obtain multiple initial classification features; Based on the recognition and reasoning module, a plurality of surrounding features of the image of the region of interest are acquired; and relational reasoning and recognition are performed on the plurality of initial classification features and the plurality of surrounding features to obtain the predicted hidden danger value.

[0010] In a possible implementation, the deformable convolution initial recognition module includes a deformable convolution unit, a deformable component model unit and an initial recognition unit; The deformable convolution unit is used to perform irregular sampling on the region of interest to obtain an adaptive feature map; The deformable component model unit is used to determine at least one candidate region in the adaptive feature map; The initial recognition unit is used to perform weighted fusion on the at least one candidate region to obtain a weighted region, and perform deformable pooling and full connection processing on the weighted region to obtain the initial classification feature.

[0011] In a possible implementation, the recognition and reasoning module includes a surrounding information recognition unit and a recognition and reasoning unit; The surrounding information recognition unit is used to obtain a plurality of surrounding features of the image of the region of interest; The recognition and reasoning unit is used to fuse the initial classification features and the surrounding features to obtain the predicted hidden danger value.

[0012] In a possible implementation, the surrounding information recognition unit is Faster RCNN, and the recognition and reasoning unit is GRU.

[0013] On the other hand, the present invention also provides an automatic warning device for dangerous operations, comprising: A node key element determination unit is used to determine a plurality of operation process nodes of a dangerous operation, and determine a plurality of node key elements of the operation process nodes based on the operation process specification; the node key elements include personnel elements, equipment elements, on-site measure elements and on-site environment elements; An image of interest determining unit, configured to determine image keywords based on the node key elements, and perform region extraction on the on-site image of the dangerous operation based on the image keywords to obtain a plurality of images of interest corresponding to the plurality of node key elements; A hidden danger prediction unit, used for inputting the plurality of images of interest into a hidden danger prediction model to obtain a predicted hidden danger value; The hidden danger alarm unit is used to generate an alarm signal when the predicted hidden danger value is greater than the hidden danger safety threshold.

[0014] On the other hand, the present invention also provides a dangerous operation management device, including a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the dangerous operation automatic alarm method described in any one of the possible implementation methods mentioned above.

[0015] On the other hand, the present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the dangerous operation automatic alarm method described in any of the above possible implementation methods.

[0016] The beneficial effects of the present invention are as follows: the dangerous operation automatic alarm method provided by the present invention determines the node key elements of the operation process nodes based on the operation process specifications, and then performs regional extraction on the on-site images of the dangerous operation based on the image keywords determined by the node key elements, to obtain multiple images of interest corresponding to multiple node key elements, and determines the predicted hidden danger value based on the multiple images of interest, so that in the process of determining the predicted hidden danger value, the four dimensions of personnel factors, equipment factors, on-site measure factors and on-site environmental factors are taken into consideration, that is, the personnel factors, equipment factors, on-site measure factors and on-site environmental factors jointly determine the predicted hidden danger value. Compared with the method in the prior art that predicts hidden dangers only through the single factor of personnel, the accuracy of the predicted hidden danger value can be improved, thereby improving the timeliness and accuracy of the automatic alarm of dangerous operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic flow chart of an embodiment of the method for automatically warning of dangerous operations provided by the present invention; Figure 2 For the present invention Figure 1 A flowchart of an embodiment of determining multiple operation process nodes of a dangerous operation in step S101; Figure 3 A schematic diagram of the structure of an embodiment of the hidden danger prediction model provided by the present invention; Figure 4 For the present invention Figure 1 A schematic flow chart of an embodiment of step S103; Figure 5 A schematic diagram of the structure of an embodiment of the automatic warning device for dangerous operations provided by the present invention; Figure 6 A schematic structural diagram of an embodiment of the hazardous operations management equipment provided by the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] It should be understood that the schematic drawings are not drawn to scale. The flowchart used in the present invention shows the operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present invention, and can also remove one or more operations from the flowchart. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0021] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] The present invention provides a dangerous operation automatic alarm method, device, dangerous operation management equipment and medium, which are described below respectively.

[0023] Figure 1 A flow chart of an embodiment of the method for automatically warning of dangerous operations provided by the present invention is shown in FIG. Figure 1 As shown, the dangerous operation automatic alarm method includes: S101. Determine multiple operation process nodes of dangerous operations, and determine multiple node key elements of the operation process nodes based on the operation process specification; the node key elements include personnel elements, equipment elements, on-site measure elements and on-site environment elements; S102, determining image keywords based on the key elements of the nodes, and performing region extraction on the on-site images of the dangerous operation based on the image keywords to obtain multiple images of interest corresponding to multiple key elements of the nodes; S103, inputting a plurality of images of interest into a hidden danger prediction model to obtain predicted hidden danger values; S104: When the predicted hidden danger value is greater than the hidden danger safety threshold, an alarm signal is generated.

[0024] Among them, the operating process specifications are determined jointly by the regulatory standards and operating specifications of the operating process.

[0025] It should be understood that the on-site image of the dangerous operation in step S102 may be directly acquired based on on-site acquisition equipment, or may be retrieved from a storage medium storing the on-site image.

[0026] Among them, on-site collection equipment includes but is not limited to fixed cameras and mobile control balls.

[0027] It should be noted that the warning signal in step S104 may be a sound and / or a light signal.

[0028] Compared with the prior art, the method for automatic warning of dangerous operations provided by the embodiment of the present invention determines the node key elements of the operation process nodes based on the operation process specifications, and then performs regional extraction on the on-site images of the dangerous operations based on the image keywords determined by the node key elements, to obtain multiple images of interest corresponding to multiple node key elements, and determines the predicted hidden danger value based on the multiple images of interest, so that in the process of determining the predicted hidden danger value, the four dimensions of personnel factors, equipment factors, on-site measure factors and on-site environmental factors are taken into consideration, that is, the personnel factors, equipment factors, on-site measure factors and on-site environmental factors jointly determine the predicted hidden danger value. Compared with the method in the prior art that predicts hidden dangers only through the single factor of personnel, the accuracy of the predicted hidden danger value can be improved, thereby improving the timeliness and accuracy of the automatic warning of dangerous operations.

[0029] In order to avoid the technical problem of inaccurate warning of hidden dangers of dangerous operations due to omission of operation process nodes, in some embodiments of the present invention, such as Figure 2 As shown, the multiple operation process nodes of the dangerous operation determined in step S101 include: S201, obtaining the type of dangerous operation, and determining the dangerous operation process based on the dangerous operation type; the dangerous operation process includes a dangerous operation sub-process before the operation, a dangerous operation sub-process during the operation, and a dangerous operation sub-process after the operation; S202. Determine multiple dangerous operation sub-steps in the dangerous operation process, and use the dangerous operation sub-steps as operation process nodes.

[0030] The embodiments of the present invention achieve diversity in the types of dangerous operation processes by determining dangerous operation processes based on the types of dangerous operations, making them adaptable to various real dangerous operation scenarios, and determining dangerous operation sub-processes in the three links of before, during and after the operation, thereby achieving full-cycle acquisition of dangerous operation processes, thereby improving the comprehensiveness and accuracy of the determined operation process nodes, avoiding the omission of operation process nodes, and further improving the alarm accuracy of the automatic alarm method for dangerous operations.

[0031] It should be understood that dangerous work types include but are not limited to: hot work, work at height, earth-moving work, lifting work, temporary power work, circuit breaking work, and confined space work.

[0032] In a specific embodiment of the present invention, personnel factors include operator behavior, which includes whether the operator has a work permit, whether the operator is not a person applying for an operation, whether protective equipment is worn as required, whether the safety belt is hung high and used low, whether there is someone to supervise during high-altitude operations, and whether the tools are used in a standardized manner; equipment factors include equipment status; on-site measures factors include whether fencing measures are set up, whether warning signs or warning tapes are set up, whether there are shading measures at the work site where electric arcs are laid, whether there are anti-slip measures for high-altitude equipment, whether the fire source at the site is cleared, and whether flammable materials at the site are cleared; on-site environmental factors include on-site temperature, on-site noise, on-site wind speed, and whether there is water accumulation.

[0033] In a specific embodiment of the present invention, the key elements of the pre-operation process nodes include: personnel work permit, personnel on-site protection (protective railings, warning tape, warning signs), operation site cleaning (fire source, flammable materials, etc.), etc. The key elements of the process nodes during the operation include: personnel behavior (), on-site equipment status (fire equipment, climbing equipment), on-site environment (temperature, noise, wind force, etc.), on-site measures (enclosure measures, warning signs, warning tapes, etc.).

[0034] In a specific embodiment of the present invention, the hidden danger identification contents at different stages are specifically as follows: Hidden dangers before operations include: Behavior of operators: operators do not have work permits, operators are not the ones included in the operation application, etc. Operation measures: failure to process the operation application, fire sources on site not being cleared, flammable materials on site not being cleared, protective railings not being set up on site, failure to conduct pre-operation publicity and vote counting, etc. Operation environment: conducting hot work and high-altitude operations when wind speed exceeds level 5; conducting hot work when there is water on site, etc.

[0035] Hidden dangers during operation include: Operator behavior: failure to wear protective equipment as required, safety belts hung high and used low, no one to supervise height work, improper use of tools, etc. Operation measures: no shading measures at the operation site with arc radiation; no anti-slip measures for height equipment, etc. Operation environment: no shielding measures during windy weather, use of live equipment in places with accumulated water, etc.

[0036] In some embodiments of the present invention, Figure 3 As shown in Figure 2, the hidden danger prediction model includes a deformable convolution initial recognition module and a recognition reasoning module; Figure 4 As shown, step S103 includes: S401, identifying and classifying a plurality of region-of-interest images based on a deformable convolution initial recognition module to obtain a plurality of initial classification features; S402, acquiring multiple surrounding features of the image of the region of interest based on the recognition and reasoning module; and performing relational reasoning and recognition on multiple initial classification features and multiple surrounding features to obtain a predicted hidden danger value.

[0037] Among them, the principle of the deformable convolution initial recognition module is deformable convolution. Deformable convolution is guided by the offset output, and the original fixed sampling points are adjusted in the horizontal and vertical directions to obtain the final sampling point position for convolution operation. Through the deformable convolution layer, the change of convolution operation from fixed sampling to irregular sampling is realized, and the offset learned from the target task does not require additional supervision information, which is convenient for end-to-end training through back propagation. Deformable convolution enhances the network's spatial modeling capabilities and can adaptively extract discriminative features based on the content characteristics of the region of interest.

[0038] The embodiment of the present invention can improve the accuracy of feature extraction and recognition classification of multiple region of interest images by identifying and classifying multiple region of interest images based on a deformable convolution initial module, that is, improve the expressiveness and accuracy of the initial classification features.

[0039] In addition, the recognition and reasoning module of the embodiment of the present invention obtains multiple surrounding features, performs relational reasoning and identification based on multiple initial classification features and multiple surrounding features, and obtains a predicted hidden danger value. In the process of determining the predicted hidden danger value, the influence of the surrounding features on the recognition result is taken into account, thereby further improving the accuracy of the predicted hidden danger value.

[0040] In a specific embodiment of the present invention, Figure 3 As shown, the deformable convolution initial recognition module includes a deformable convolution unit, a deformable component model unit and an initial recognition unit; The deformable convolution unit is used to perform irregular sampling of the region of interest to obtain an adaptive feature map; The deformable component model unit is used to determine at least one candidate region in the adaptive feature map; The initial recognition unit is used to perform weighted fusion on at least one candidate region to obtain a weighted region, and perform deformable pooling and full connection processing on the weighted region to obtain an initial classification feature.

[0041] In layman's terms, deformable convolution changes the position of the convolution kernel corresponding to the receptive field, thereby achieving the purpose of changing the output feature map.

[0042] It should be noted that deformable pooling is similar to deformable convolution. Deformable pooling adds a bias to the position of each pooling box. The difference is that the displacement of deformable pooling is obtained by adding a fully connected layer after the pooling layer.

[0043] It should be further explained that the deformable component model unit has a root filter with relatively coarse accuracy and multiple component filters with relatively fine accuracy. The root filter is intended to approximately cover the entire image of the region of interest, and the component filter is intended to finely cover the different parts of the image of the region of interest. Therefore, in the process of determining the candidate region, including a large candidate region and multiple refined small candidate regions, the image of the region of interest can be described from multiple scales, thereby further improving the accuracy of the initial classification features.

[0044] In a specific embodiment of the present invention, Figure 3 As shown, the recognition and reasoning module includes a surrounding information recognition unit and a recognition and reasoning unit; The surrounding information recognition unit is used to obtain multiple surrounding features of the image of the region of interest; The recognition and reasoning unit is used to fuse the initial classification features and surrounding features to obtain the predicted hidden danger value.

[0045] In a specific embodiment of the present invention, the surrounding information recognition unit is Faster RCNN, and the recognition reasoning unit is GRU.

[0046] In order to better implement the dangerous operation automatic alarm method in the embodiment of the present invention, based on the dangerous operation automatic alarm method, the embodiment of the present invention also provides a dangerous operation automatic alarm device, such as Figure 5 As shown, the dangerous operation automatic warning device 500 includes: The node key element determination unit 501 is used to determine multiple operation process nodes of the dangerous operation, and determine multiple node key elements of the operation process nodes based on the operation process specification; the node key elements include personnel elements, equipment elements, on-site measure elements and on-site environment elements; An image of interest determining unit 502 is used to determine image keywords based on the node key elements, and perform region extraction on the on-site image of the dangerous operation based on the image keywords to obtain multiple images of interest corresponding to multiple node key elements; The hidden danger prediction unit 503 is used to input multiple images of interest into the hidden danger prediction model to obtain predicted hidden danger values; The hidden danger alarm unit 504 is used to generate an alarm signal when the predicted hidden danger value is greater than the hidden danger safety threshold.

[0047] The dangerous operation automatic alarm device 500 provided in the above embodiment can implement the technical solution described in the above dangerous operation automatic alarm method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above dangerous operation automatic alarm method embodiment, which will not be repeated here.

[0048] like Figure 6As shown, the present invention also provides a dangerous operation management device 600. The dangerous operation management device 600 includes a processor 601, a memory 602 and a display 603. Figure 6 Only some of the components of the hazardous work management device 600 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0049] In some embodiments, the processor 601 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 602, such as the automatic alarm method for dangerous operations in the present invention.

[0050] In some embodiments of the present invention, the processor 601 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 601 may be local or remote. In some embodiments, the processor 601 may be implemented in a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0051] In some embodiments, the memory 602 may be an internal storage unit of the hazardous operation management device 600, such as a hard disk or memory of the hazardous operation management device 600. In other embodiments, the memory 602 may also be an external storage device of the hazardous operation management device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the hazardous operation management device 600.

[0052] Furthermore, the memory 602 may include both an internal storage unit of the hazardous operation management device 600 and an external storage device. The memory 602 is used to store application software installed in the hazardous operation management device 600 and various data.

[0053] In some embodiments, the display 603 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 603 is used to display information of the hazardous operation management device 600 and to display a visual user interface. The components 601-603 of the hazardous operation management device 600 communicate with each other via a system bus.

[0054] In some embodiments of the present invention, when the processor 601 executes the dangerous operation automatic warning program in the memory 602, the following steps may be implemented: Determine multiple operation process nodes of dangerous operations, and determine multiple key elements of the operation process nodes based on the operation process specifications; the key elements of the nodes include personnel elements, equipment elements, on-site measures elements and on-site environment elements; Determine image keywords based on key elements of nodes, and perform region extraction on site images of dangerous operations based on image keywords to obtain multiple images of interest corresponding to multiple key elements of nodes; Inputting multiple images of interest into a hidden danger prediction model to obtain predicted hidden danger values; When the predicted hidden danger value is greater than the hidden danger safety threshold, an alarm signal is generated.

[0055] It should be understood that: when the processor 601 executes the dangerous operation automatic warning program in the memory 602, in addition to the above functions, other functions can also be realized. For details, please refer to the description of the corresponding method embodiment above.

[0056] Furthermore, the embodiment of the present invention does not specifically limit the type of the hazardous operation management device 600 mentioned. The hazardous operation management device 600 may be a portable hazardous operation management device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable hazardous operation management devices include but are not limited to portable hazardous operation management devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable hazardous operation management device may also be other portable hazardous operation management devices. It should also be understood that in some other embodiments of the present invention, the hazardous operation management device 600 may not be a portable hazardous operation management device, but a desktop computer with a touch-sensitive surface (such as a touch panel).

[0057] Correspondingly, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the steps or functions of the dangerous operation automatic alarm method provided by the above-mentioned method embodiments.

[0058] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0059] The above is a detailed introduction to a dangerous operation automatic alarm method, device, dangerous operation management equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A dangerous operation automatic warning method, characterized in that: include: Determine multiple operation process nodes of the dangerous operation, and determine multiple node key elements of the operation process nodes based on the operation process specification; The key elements of the node include personnel elements, equipment elements, on-site measures elements and on-site environment elements; Determine image keywords based on the node key elements, and perform region extraction on the on-site image of the dangerous operation based on the image keywords to obtain multiple images of interest corresponding to the multiple node key elements; Inputting the plurality of images of interest into a hidden danger prediction model to obtain a predicted hidden danger value; When the predicted hidden danger value is greater than the hidden danger safety threshold, an alarm signal is generated.

2. The method for automatically warning of dangerous operations according to claim 1, characterized in that: The multiple operation process nodes of the dangerous operation are determined, including: Acquire the type of dangerous operation, and determine the dangerous operation process based on the dangerous operation type; the dangerous operation process includes a dangerous operation sub-process before the operation, a dangerous operation sub-process during the operation, and a dangerous operation sub-process after the operation; Determine a plurality of dangerous operation sub-steps in the dangerous operation process, and use the dangerous operation sub-steps as the operation process nodes.

3. The method for automatically warning of dangerous operations according to claim 1, characterized in that: The personnel factors include the behavior of operators, including whether the operators have work permits, whether the operators are not in the process of applying for work, whether they wear protective equipment as required, whether the safety belts are hung high and used low, whether there is someone to supervise during high-altitude operations, and whether the tools are used in a standardized manner; the equipment factors include the status of the equipment; the on-site measures factors include whether fencing measures are set up, whether warning signs or warning tapes are set up, whether there are shading measures at the work site where electric arcs are laid, whether there are anti-slip measures for high-altitude equipment, whether the fire sources on site are cleared, and whether flammable materials on site are cleared; the on-site environmental factors include the on-site temperature, on-site noise, on-site wind speed, and whether there is water accumulation.

4. The method for automatically warning of dangerous operations according to claim 1, characterized in that: The hidden danger prediction model includes a deformable convolution initial recognition module and a recognition reasoning module; Then, inputting the plurality of images of interest into the hidden danger prediction model to obtain the predicted hidden danger value comprises: Based on the deformable convolution initial recognition module, the multiple region of interest images are recognized and classified to obtain multiple initial classification features; Based on the recognition and reasoning module, a plurality of surrounding features of the image of the region of interest are acquired; and relational reasoning and recognition are performed on the plurality of initial classification features and the plurality of surrounding features to obtain the predicted hidden danger value.

5. The method for automatically warning of dangerous operations according to claim 4 is characterized in that: The deformable convolution initial recognition module includes a deformable convolution unit, a deformable component model unit and an initial recognition unit; The deformable convolution unit is used to perform irregular sampling on the region of interest to obtain an adaptive feature map; The deformable component model unit is used to determine at least one candidate region in the adaptive feature map; The initial recognition unit is used to perform weighted fusion on the at least one candidate region to obtain a weighted region, and perform deformable pooling and full connection processing on the weighted region to obtain the initial classification feature.

6. The method for automatically warning of dangerous operations according to claim 4 is characterized in that: The recognition and reasoning module includes a surrounding information recognition unit and a recognition and reasoning unit; The surrounding information recognition unit is used to obtain a plurality of surrounding features of the image of the region of interest; The recognition and reasoning unit is used to fuse the initial classification features and the surrounding features to obtain the predicted hidden danger value.

7. The method for automatically warning of dangerous operations according to claim 6, characterized in that: The surrounding information recognition unit is Faster RCNN, and the recognition reasoning unit is GRU.

8. An automatic warning device for dangerous operations, characterized in that: include: A node key element determination unit, used to determine a plurality of operation process nodes of a dangerous operation, and determine a plurality of node key elements of the operation process nodes based on an operation process specification; The key elements of the node include personnel elements, equipment elements, on-site measures elements and on-site environment elements; An image of interest determining unit, configured to determine image keywords based on the node key elements, and perform region extraction on the on-site image of the dangerous operation based on the image keywords to obtain a plurality of images of interest corresponding to the plurality of node key elements; A hidden danger prediction unit, used for inputting the plurality of images of interest into a hidden danger prediction model to obtain a predicted hidden danger value; The hidden danger alarm unit is used to generate an alarm signal when the predicted hidden danger value is greater than the hidden danger safety threshold.

9. A dangerous operation management device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the dangerous operation automatic alarm method described in any one of claims 1 to 7 above.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the dangerous operation automatic alarm method described in any one of claims 1 to 7 above.

Images