Safety confirmation system

Abstract

Even when numerous pieces of equipment are located on a vast site, safety during equipment startup is ensured. [Solution] The safety management computer 3 includes an image analysis means for analyzing images, and an audio analysis means for identifying the equipment to be activated by analyzing audio paging broadcast when the equipment is activated, which includes identification information of the equipment to be activated. The drone 2 flies to the equipment identified by the audio analysis means and takes pictures of the area around the equipment with the camera 21. The image analysis means analyzes the images of the area around the equipment taken by the camera 21 and determines and outputs whether or not there are people around the equipment.

Description

Technical Field

[0001] The present invention relates to a safety confirmation system for safely starting machines and equipment.

Background Art

[0002] For example, when a machine or equipment in a power plant compound is put through a test run after inspection, people must be prevented from being involved in the machine or equipment and getting injured. For this reason, conventionally, during a test run or the like, voice paging (in-house broadcasting) was used to alert people not to approach.

[0003] On the other hand, a peripheral monitoring device for a working machine that can prompt an operator to confirm that a person around the working machine, such as an excavator, has withdrawn is known (see, for example, Patent Document 1). This device determines the presence or absence of people around the working machine, outputs an alarm when it is determined that there are people, and continues to output the alarm until an operator input for stopping the alarm is received.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, simply alerting with voice paging may cause people concentrating on their work to remain near the machine or equipment to be started. In addition, in a power plant, a large variety of machines and equipment are arranged in a vast site, and it is difficult to arrange and operate the monitoring devices described in Patent Document 1 for all machines and equipment.

[0006] Therefore, the present invention aims to provide a safety confirmation system that can ensure safety when equipment is started up, even when many pieces of equipment are located on a large site. [Means for solving the problem]

[0007] To solve the above problems, the invention of claim 1 is a safety confirmation system comprising: a layout diagram storage means for storing a layout diagram showing the arrangement of multiple pieces of equipment within a power plant site; an unmanned aerial vehicle that flies within the site and is equipped with a camera; an image analysis means for analyzing images captured by the camera; and an audio analysis means for identifying the equipment to be activated by analyzing audio paging broadcast when the equipment is activated, which includes identification information of the equipment to be activated. The unmanned aerial vehicle flies to the equipment identified by the audio analysis means based on the layout diagram, and captures images of the area around the equipment with the camera; and the image analysis means analyzes the images of the area around the equipment captured by the camera, and determines and outputs whether or not there are people around the equipment.

[0008] The invention of claim 2 is characterized in that, in the safety confirmation system described in claim 1, the unmanned aerial vehicle is equipped with an audio output means that outputs an audio message to alert, and when the image analysis means determines that there is a person in the vicinity of the equipment, the audio output means outputs the audio message.

[0009] The invention of claim 3 is a safety confirmation system according to claim 1, comprising confirmation area storage means for storing the area where safety should be confirmed when the equipment is activated as the area around the equipment. The unmanned aerial vehicle is characterized by using the photographing means to photograph the area surrounding the equipment stored in the confirmation area storage means.

[0010] The invention of claim 4 is characterized in that, in the safety confirmation system described in claim 1, the system includes an authorization action storage means that stores equipment on which a person is to perform a predetermined action and the location where the person should be at startup, and the image analysis means, when it determines that a person is in the vicinity of the equipment, determines and outputs whether the equipment is equipment stored in the authorization action storage means and whether the person is in a location stored in the authorization action storage means.

[0011] The invention of claim 5 is characterized in that, in the safety confirmation system described in claim 1, the image analysis means uses an image analysis learning model that has been machine-trained based on past performance data, such that when an image of the area around the equipment is input, it outputs whether or not a person is present.

[0012] The invention of claim 6 is characterized in that, in the safety confirmation system described in claim 1, the voice analysis means uses a voice analysis learning model that has been machine-trained based on past performance data, such that when voice paging is input, the equipment to be activated is output.

[0013] The invention of claim 7 is characterized in that, in the safety confirmation system described in claim 1, the site is the site of a power plant, and the image analysis means outputs the determination result of the presence or absence of the person to a central control device that controls the plurality of facilities. [Effects of the Invention]

[0014] According to the invention described in claim 1, prior to the activation of a certain piece of equipment, when an audio paging message containing identification information of the equipment is broadcast, the audio analysis means identifies the equipment to be activated from the audio paging message. Then, an unmanned aerial vehicle flies to this equipment and takes pictures of the area around the equipment with a photography means. The captured images are analyzed by an image analysis means to determine whether or not there are people around the equipment and output the result. In this way, simply by performing audio paging as in the conventional method, the unmanned aerial vehicle flies to the equipment and determines and outputs whether or not there are people. Therefore, even when many pieces of equipment are arranged on a large site, it is possible to ensure safety when activating equipment.

[0015] According to the invention described in claim 2, if there are people around the equipment, an audio message alerting them is output from the unmanned aerial vehicle, making it possible to evacuate people from the equipment and ensure their safety.

[0016] According to the invention described in claim 3, when the equipment is activated, the area where safety needs to be checked is photographed by an unmanned aerial vehicle, and it is determined whether or not there are people in that area, so that safety can be reliably ensured in the area where safety needs to be checked.

[0017] Furthermore, there are cases where a person should perform a specific action in a specific location when the equipment is started up. According to the invention described in claim 4, it is determined and output whether or not a person is present in the equipment or location where they should be at startup. Therefore, if a person should be present at startup, the equipment will start up as is, and if a person should not be present, it is possible to wait for startup to be completed to ensure safety. In this way, by determining whether or not it is permissible for a person to be present, it is possible to ensure safety appropriately.

[0018] According to the invention described in claim 5, a machine learning-based image analysis model is used to output whether or not there are people around the equipment, making it possible to more accurately determine the presence or absence of people. As a result, it becomes possible to more accurately ensure safety when the equipment is started up.

[0019] According to the invention described in claim 6, using a machine learning model for speech analysis, Since the equipment to be started is output from the paging system, it becomes possible to more accurately identify the equipment to be started. As a result, safety during equipment startup can be more properly ensured.

[0020] According to the invention described in claim 7, a central control unit that controls multiple pieces of equipment outputs whether or not there are people around the equipment being started. Therefore, even if many pieces of equipment are located on the vast site of a power plant, it is possible to centrally and comprehensively ensure safety during equipment startup with a single central control unit.

Brief Description of the Drawings

[0021] [Figure 1] This is a schematic configuration diagram showing a safety confirmation system according to an embodiment of the present invention. [Figure 2] This is a schematic configuration block diagram showing the safety management computer of the safety confirmation system in FIG. 1. [Figure 3] This is a functional block diagram showing the schematic configuration of the speech analysis learning model of the safety management computer in FIG. 2. [Figure 4] This is a functional block diagram showing the schematic configuration of the image analysis learning model of the safety management computer in FIG. 2.

Embodiments for Carrying Out the Invention

[0022] Hereinafter, the present invention will be described based on the illustrated embodiments.

[0023] FIG. 1 is a schematic configuration diagram showing a safety confirmation system 1 according to an embodiment of the present invention. This safety confirmation system 1 is a system for safely starting machines and equipment. In this embodiment, a case where a large number of equipment are arranged within the vast site of a power plant G will be described, but other cases where equipment is arranged within a site may also be applicable.

[0024] The safety confirmation system 1 mainly includes a drone (unmanned aerial vehicle) 2, a safety management computer 3, and a central control device 4. The safety management computer 3 is communicably connected to the drone 2 and the central control device 4. Also, in this embodiment, a case where one drone 2 is provided will be described, but a plurality of drones 2 may be made to operate as described later respectively.

[0025] Drone 2 is an unmanned aerial vehicle equipped with a camera (photography device) 21 and a speaker (audio output device) 22 that flies within the grounds of power plant G. When Drone 2 receives a flight command from the safety management computer 3, as described later, it flies toward the latitude and longitude of the target equipment included in the flight command. In other words, it is equipped with GPS (Global Positioning System) and flies autonomously toward the target equipment while confirming its own position with GPS. At this time, it is equipped with a function to avoid obstacles to flight depending on the flight environment.

[0026] Upon arriving at the equipment to be activated, the camera 21 takes pictures of the area surrounding the equipment and transmits the captured images, along with the location information (latitude and longitude) of the shooting location, to the safety management computer 3. Here, the area surrounding the equipment is the area where safety must be confirmed, as will be described later, and is determined by the safety management computer 3 and included in the flight command. Furthermore, the camera 21 continues to take pictures and transmit images until a stop command is received from the safety management computer 3.

[0027] Furthermore, when drone 2 receives an audio output command from safety management computer 3, as described later, it outputs a warning voice message from speaker 22. That is, it is equipped with a sound source and outputs a voice message from speaker 22 that includes identification information of the equipment to be activated, as included in the audio output command. For example, if the identification information of the equipment to be activated is a conveyor belt, it will output a voice message such as, "We are activating the conveyor belt, so please stay at least 3 meters away." The system outputs a sage message. Subsequently, upon receiving a stop command from the safety management computer 3, it stops outputting voice messages and stopping the camera 21 from taking pictures.

[0028] The safety management computer 3 is a computer for ensuring safety when the equipment is started up, and as shown in Figure 2, it mainly comprises an input unit 31, a display unit 32, a communication unit 33, a storage unit 34, a voice analysis task (voice analysis means) 35, an image analysis task (image analysis means) 36, a learning task 37, and a central processing unit 38 that controls these.

[0029] The input unit 31 is an interface for inputting various information and commands, specifically for inputting equipment information into the equipment information database 342, which will be described later. The display unit 32 is a display for displaying various data and information, specifically for displaying images received from the drone 2. The communication unit 33 is an interface for communicating with the outside world, specifically for sending commands to the drone 2, receiving images from the drone 2, and sending analysis and judgment results from the image analysis task 36 to the central control unit 4.

[0030] The memory unit 34 mainly comprises a layout diagram database (layout diagram storage means) 341, an equipment information database (confirmation area storage means, permission action storage means) 342, a learning model 343, and a performance database 344. Databases 341 and 342 will be described here, and the learning model 343 and the performance database 344 will be described later. Furthermore, as will be described later, the learning model 343 includes a learning model 343A for speech analysis and a learning model 343B for image analysis, and the performance database 344 includes a performance database 344A for speech analysis and a performance database 344B for image analysis.

[0031] The layout diagram database 341 is a database that stores layout diagrams showing the arrangement of multiple facilities within the site of power plant G. In other words, it stores layout diagrams showing what facilities are located where on the site, and stores facility identification information and location information (latitude and longitude) in association with each other.

[0032] The equipment information database 342 is a database that stores information about each piece of equipment, and primarily stores safety area information and authorized action information for each piece of equipment's identification information. Safety area information refers to information about the area where safety should be confirmed when the equipment is activated. In other words, it is information that defines the area around the equipment that should be photographed by drone 2 when the equipment is activated, and the range of the area to be photographed by drone 2 (the range to be captured in the image) is stored. When a flight command is transmitted to drone 2, the area where safety should be confirmed for the equipment to be activated is obtained from the safety area information and included in the flight command.

[0033] Permitted action information refers to information about actions in which a person is permitted to be present around the equipment when the equipment is started. In other words, it is information indicating whether a person should perform a prescribed action (such as measurement or visual inspection) when the equipment is started, and, if so, where the person should be. Specifically, for equipment where a person is required to perform a prescribed action, an image of the area where the person should be (near instruments, etc.) is stored.

[0034] The voice analysis task 35 is a task program that identifies the equipment to be started by analyzing the voice paging broadcast when the equipment is started, which includes the identification information of the equipment to be started. In other words, voice paging (public address announcement) is made to warn users before the equipment is started, and this voice paging includes the identification information of the equipment to be started. For example, voice paging such as "We will start up equipment number ×, △△, for trial operation." is made. In this case, "equipment number ×, △△" is the identification information of the equipment to be started.

[0035] In this embodiment, voice paging can be performed from multiple pre-configured locations within the power plant G. The voice paging is collected by microphones connected to the safety management computer 3, and the voice analysis task 35 is activated. The voice analysis task 35 then analyzes the voice paging to identify the equipment to be activated. Specifically, it converts the voice data (voice paging) into text using speech recognition technology, and analyzes the text data to identify which equipment will be activated.

[0036] This type of speech analysis task 35 uses a speech analysis learning model 343A that has been trained on past performance data so that when speech paging is input, the equipment to be activated is output.

[0037] In other words, the learning task 37 creates a learning model 343A for speech analysis using a known machine learning algorithm such as a neural network, based on past performance data recorded and stored in the performance database 344A for speech analysis. This performance database 344A for speech analysis is a database in which performance data is recorded and stored, including equipment to be activated, which has been identified by experts in speech analysis and skilled personnel in speech paging at power plants G, based on speech paging as input information. The past performance data includes data created based on actual speech paging and equipment to be activated, which has been identified by experts and skilled personnel, as well as data created through pre-training, etc.

[0038] As shown in Figure 3, this learning task 37 uses machine learning and deep learning with a neural network to create a neural network based on the performance data recorded in the performance database 344A for speech analysis. For example, it uses speech paging as the input layer, the equipment to be activated as the output layer, and the analysis processing from the input layer to the output layer as the hidden layer. Then, learning task 37 uses the performance data of the learning model 343A for speech analysis as training data to learn various parameters in the hidden layer. In other words, learning task 37 learns various parameters in the hidden layer so that the equipment to be activated is output appropriately based on speech paging.

[0039] The image analysis task 36 is a task program that analyzes images captured by the camera 21 of the drone 2 to determine whether or not there are people around the equipment to be activated (whether or not the equipment can be activated) and outputs the result. In other words, it is activated when it receives an image from the drone 2, and first analyzes the image of the area around the equipment to be activated captured by the camera 21 as described above to determine whether or not there are people within a predetermined distance (for example, less than 3m) from the equipment to be activated. If no people are found as a result, it transmits activation permission information, including identification information of the equipment to be activated, to the central control unit 4.

[0040] On the other hand, if there are people present, the system sends activation denial information, including identification information of the equipment to be activated, to the central control unit 4, and simultaneously sends an audio output command, also including identification information of the equipment to be activated, to the drone 2. This process is repeated until the people have evacuated, at which point activation permission information is sent to the central control unit 4.

[0041] Even if it is determined that there is a person in the vicinity of the equipment, if that person is in the location where the equipment is to perform a prescribed action as stored in the authorized action information of the equipment information database 342, the activation permission information is transmitted to the central control unit 4. In other words, if there is a person in the prescribed location of the equipment where measurements or visual checks should be performed upon activation, it is determined that there is no problem in activating the equipment (activation permitted). Here, whether or not there is a person in the prescribed location is determined based on the image of the location where the person should be, which is stored as authorized action information.

[0042] This image analysis task 36, when an image of the area surrounding the equipment is input, determines whether or not the equipment can be started. A machine learning model 343B for image analysis, which has been trained based on past performance data, is used to output information such as the presence or absence of people.

[0043] In other words, the learning task 37 creates an image analysis learning model 343B using a known machine learning algorithm such as a neural network, based on past performance data recorded and stored in the image analysis performance database 344B. This image analysis performance database 344B is a database in which performance data, including whether or not the equipment can be started (such as the presence or absence of people), is determined by experts in image analysis and skilled personnel in equipment startup at the power plant G, based on images of the surrounding area of ​​the equipment as input information. The past performance data includes actual images, data created based on the determination of whether or not the equipment can be started by experts and skilled personnel, and data created through pre-training, etc.

[0044] As shown in Figure 4, this learning task 37 uses machine learning and deep learning with a neural network to create a neural network based on actual data recorded in the image analysis database 344B. For example, it uses images as the input layer, whether the equipment can be started as the output layer, and the analysis processing from the input layer to the output layer as the hidden layer. Then, learning task 37 uses the actual data of the image analysis learning model 343B as training data to learn various parameters in the hidden layer. In other words, learning task 37 learns various parameters in the hidden layer so that it can output whether the equipment can be started as appropriate based on images of the surrounding area of ​​the equipment.

[0045] The central control unit 4 is a device that manages and controls all equipment located in the power plant G, and has a configuration that is basically the same as off-the-shelf or existing central control units. Furthermore, upon receiving activation permission information from the safety management computer 3, it enables the activation operation of the relevant equipment, and in this embodiment, it also pages and broadcasts information throughout the plant indicating that no one is present. Alternatively, it may be configured to notify the operator's mobile terminal.

[0046] On the other hand, if the safety management computer 3 receives information that the equipment cannot be started, it will disable the startup operation of the equipment in question (it will not start even if an operator tries to start it). Also, if it receives information that the equipment can be started or not, it will display whether or not a person is present on the display.

[0047] Next, we will explain the operation and function of the safety confirmation system 1 with this configuration, as well as the method for starting up equipment using the safety confirmation system 1.

[0048] First, prior to the activation of a certain piece of equipment, an audio paging message containing the identification information of that equipment is broadcast. The audio analysis task 35 identifies the equipment to be activated from the audio paging message, and the location information (latitude and longitude) of this equipment is obtained from the layout map database 341. Next, a flight command containing the location information of the equipment to be activated is transmitted from the safety management computer 3 to the drone 2. The drone 2 flies to this equipment and takes photographs of the area around the equipment with the camera 21. At this time, the photographs are taken of areas that need to be checked for safety as included in the flight command.

[0049] Next, the captured image is transmitted to the safety management computer 3 along with location information of the shooting location. The image analysis task 36 analyzes the image to determine whether or not the equipment can be activated, such as whether or not there are people around the equipment. If there are people around the equipment, an audio output command including identification information of the equipment to be activated is transmitted to the drone 2. In response, the speaker 22 of the drone 2 outputs an audio message to warn the person. In this case, the safety management computer 3 transmits activation denial information to the central control unit 4, making it impossible to activate the equipment, and the presence of people around the equipment is displayed on the screen.

[0050] Furthermore, people must evacuate from the vicinity of the equipment, and those who should be performing measurements or other tasks during startup must leave the equipment unattended. If a person is in the designated location, activation permission information is transmitted from the safety management computer 3 to the central control unit 4. This allows the equipment to be activated, and a paging and public address system broadcasts announcing that no one is present, and the display shows that no one is around the equipment.

[0051] As described above, with this safety confirmation system 1, simply by using voice paging as in the conventional method, the drone 2 flies to the equipment to be activated, determines whether or not there are people present, and outputs the result. Therefore, even when many pieces of equipment are located on a large site, it is possible to ensure safety when activating the equipment.

[0052] Furthermore, if there are people around the equipment, Drone 2 will emit a voice message to warn them, making it possible to evacuate people from the equipment and ensure their safety.

[0053] Furthermore, when the equipment is activated, the area that needs to be checked for safety is photographed by Drone 2, and it is determined whether or not there are people in that area, making it possible to reliably ensure safety in the area that needs to be checked.

[0054] Furthermore, there are cases where a person should perform measurements or visual checks at a designated location when the equipment is started up. The system determines and outputs whether or not a person is present at the equipment or location where they should be at startup. Therefore, if a person should be present when the equipment is started up, it will start up as is; if a person should not be present, it will wait to start up to ensure safety. In this way, by determining whether or not it is acceptable for a person to be present, it is possible to ensure safety appropriately.

[0055] Furthermore, since the equipment to be activated is output from the voice paging using the machine learning-trained speech analysis model 343A, it becomes possible to more accurately identify the equipment to be activated. As a result, it becomes possible to more accurately ensure safety when activating equipment.

[0056] Similarly, using the machine learning-trained image analysis model 343B, information such as whether or not there are people around the equipment can be output, making it possible to more accurately determine the presence or absence of people (and whether or not the equipment can be started). As a result, safety during equipment startup can be more accurately ensured.

[0057] Furthermore, the central control unit 4, which controls all the equipment, outputs information such as whether or not there are people around the equipment to be started (whether or not the equipment can be started). Therefore, even if many pieces of equipment are located on the vast site of the power plant G, it is possible to centrally and comprehensively ensure safety during equipment startup with a single central control unit 4.

[0058] Although embodiments of this invention have been described in detail above, the specific configuration is not limited to these embodiments, and any design changes, etc., that do not depart from the gist of this invention are also included. For example, in the above embodiment, the image analysis task 36 is provided on the safety management computer 3, but it may also be provided on the drone 2, and the analysis results from the image analysis task 36 may be transmitted from the drone 2 to the safety management computer 3. [Explanation of Symbols]

[0059] 1. Safety Confirmation System 2. Drones (unmanned aerial vehicles) 21. Camera (Method of taking pictures) 22. Speaker (Audio output means) 3 Operator 342 Equipment Information Database (Verification Area Storage Means, Permission Action Storage Means) 343A Learning model for speech analysis 343B Image Analysis Learning Model 35. Speech analysis task (speech analysis means) 36 Image Analysis Task (Image Analysis Means) 4. Central Control System G Power Plant

Claims

1. A layout diagram storage means that stores a layout diagram showing the arrangement of multiple facilities on the site, An unmanned aerial vehicle equipped with a camera and flying within the aforementioned site, Image analysis means for analyzing images captured by the aforementioned shooting means, The system includes an audio analysis means that analyzes audio paging broadcast when the equipment is started, which includes identification information of the equipment to be started, in order to identify the equipment to be started. The unmanned aerial vehicle flies to the equipment identified by the sound analysis means based on the layout diagram, and photographs the area around the equipment with the photography means. The image analysis means analyzes the image of the surrounding area of ​​the equipment captured by the shooting means, determines whether or not there are people in the surrounding area of ​​the equipment, and outputs the result. A safety verification system characterized by the following features.

2. The aforementioned unmanned aerial vehicle is equipped with an audio output means that outputs an audio message to draw attention, If the image analysis means determines that there is a person in the vicinity of the equipment, the voice output means outputs the voice message. The safety confirmation system described in claim 1.

3. The equipment is equipped with a confirmation area storage means that stores the area around the equipment as the area where safety should be confirmed when the equipment is started. The unmanned aircraft uses the photographing means to photograph the area surrounding the equipment stored in the confirmation area storage means. The safety confirmation system described in claim 1.

4. It is equipped with an authorization action memory means that stores the equipment on which a person is supposed to perform a predetermined action and the location where that person should be, When the image analysis means determines that there is a person in the vicinity of the equipment, it determines and outputs whether the equipment is equipment stored in the authorization action storage means and whether the person is in a location stored in the authorization action storage means. The safety confirmation system described in claim 1.

5. The safety confirmation system according to claim 1, characterized in that the image analysis means uses an image analysis learning model that has been machine-trained based on past performance data, such that when an image of the area around the equipment is input, it outputs whether or not a person is present.

6. The voice analysis means uses a voice analysis learning model that has been trained on past performance data so that when voice paging is input, the equipment to be activated is output. The safety confirmation system described in claim 1.

7. The aforementioned site is the site of a power plant, The image analysis means outputs the result of determining the presence or absence of the person to a central control device that controls the plurality of equipment. The safety confirmation system described in claim 1.

Images