Danger target warning device, danger target alarm method, and program

Abstract

To provide a dangerous object warning device, a dangerous object alarm method and a program for calling attention so that the position of a dangerous object can be grasped at appropriate timing.SOLUTION: A dangerous target warning device according to an embodiment acquires viewpoint information from a viewpoint measurement device, acquires an image in which a plurality of targets are recognized from an image recognition device, and causes an alarm output device to output an alarm sound, and includes a detection unit that detects a dangerous target from among the plurality of targets in the image according to a viewpoint movement range included in the viewpoint information, a position measurement unit that measures a distance and a direction to the dangerous target with the image recognition device as a reference point, and an alarm unit that causes the alarm output device to output an alarm sound by transmitting an alarm output signal to the alarm output device according to a positional relationship between the image recognition device and the dangerous target.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a hazard warning device, a hazard warning method, and a program. [Background technology]

[0002] In places where safety is not ensured, such as construction sites, the most common accidents are falls, and there are many cases where people are so focused on the work at hand that they lose attention to what is behind them, leading to falls. For example, a conventional technology to solve such falls at construction sites is to alert the wearer whose line of sight is not directed in the direction of a hazard, and to direct the wearer to the hazard of falling. There are helmets that can recognize this in advance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-002514 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the wearer may be alerted even when not approaching a hazard, which may result in frequent alerts at construction sites where the helmet is expected to be worn. As a result, the wearer may become accustomed to the alerts and may not be able to correctly recognize hazards. In addition, the wearer needs to accurately determine the location of hazards in order to avoid danger.

[0005] The embodiments of the present invention have been made in consideration of the above circumstances, and provide a hazard warning device, a hazard warning method, and a program that alert a user so that the location of a hazard can be grasped at an appropriate time. [Means for solving the problem]

[0006] A hazardous object warning device according to one embodiment acquires viewpoint information from a viewpoint measurement device, acquires an image recognizing multiple objects from an image recognition device, and causes an alarm output device to output an alarm sound. The hazardous object warning device includes a detection unit that detects a hazardous object from among the multiple objects in the image according to the viewpoint movement range included in the viewpoint information, a position measurement unit that measures the distance and direction to the hazardous object using the image recognition device as a reference point, the image recognition device, and an alarm unit that outputs the alarm sound by sending an alarm output signal to the alarm output device according to the positional relationship with the hazardous object. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a danger object warning system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a danger object warning system including a danger object warning device according to an embodiment. [Figure 3] FIG. 3 is a flowchart showing an example of a hazard object warning process of the hazard object warning device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] The hazard warning device, hazard warning method, and program will be described in detail below with reference to the accompanying drawings. In the following description of each embodiment and modification, parts with the same reference numerals have substantially the same functions, and descriptions of overlapping parts will be omitted as appropriate.

[0009] 1 is a schematic diagram showing an example of the configuration of a hazard warning system according to an embodiment. The hazard warning system 10 includes a helmet 11, a viewpoint measurement device 12, an image recognition device 13, a warning output device 14, and a hazard warning device 15.

[0010] The viewpoint measuring device 12, the image recognition device 13, the alarm output device 14, and the hazard object warning device 15 are connected by a bus communication line or the like so as to be able to send and receive data to and from each other. Note that the viewpoint measuring device 12, the image recognition device 13, the alarm output device 14, and the hazard object warning device 15 may also be connected by wireless communication or the like so as to be able to send and receive data to and from each other.

[0011] The helmet 11 may be a helmet that is used, for example, at a construction site or the like, to protect the head of a person wearing the helmet 11.

[0012] The viewpoint measuring device 12 is a device that tracks the viewpoint of the wearer of the helmet 11, and is attached, for example, to the front of the helmet 11. The viewpoint measuring device 12 irradiates the pupil of the wearer of the helmet 11 with infrared light, and measures the field of view and movement from the positional relationship between the light reflected from the wearer's cornea and the pupil. The method by which the viewpoint measuring device 12 measures the viewpoint of the wearer is not limited to this, and various existing methods can be used to measure the viewpoint of the wearer.

[0013] The viewpoint measurement device 12 accumulates viewpoint information including the viewpoint movement range of the wearer of the helmet 11, and transmits the accumulated viewpoint information in response to a request from the hazard warning device 15. Note that the viewpoint measurement device 12 may transmit the viewpoint information to the hazard warning device 15 periodically or in real time.

[0014] The image recognition device 13 is a monocular camera that recognizes objects that exist beyond the field of view of the wearer of the helmet 11, and is attached, for example, to the rear of the helmet 11. The image recognition device 13 recognizes objects by extracting target regions of the objects by learning images containing multiple objects using AI. The method of object recognition by the image recognition device 13 is not limited to this, and multiple objects can be recognized by various existing methods.

[0015] The image recognition device 13 transmits image information including information on the multiple recognized objects in response to a request from the hazard object warning device 15. Note that the image recognition device 13 may transmit the image information to the hazard object warning device 15 periodically or in real time.

[0016] In addition, the image recognition device 13 may be configured to measure the distance to a recognized object in real time by learning the distance to the object using aberration, a phenomenon in which a bundle of light rays does not completely converge to a single point when light from an object is converged to a single point using a lens.

[0017] In this case, the image recognition device 13 may transmit to the hazardous object warning device 15 image information including distance information and direction information associated with information on the recognized plurality of objects.

[0018] The alarm output device 14 is a speaker that notifies the wearer of the helmet 11 of the location of a dangerous object, and is attached, for example, to both the left and right ends of the helmet 11. The alarm output device 14 is configured to output an alarm from any point, and has the function of setting the point at which the alarm is output according to the positional relationship of the dangerous object recognized by the image recognition device 13, and adjusting the volume etc. according to the distance.

[0019] The alarm output device 14 receives an alarm output signal including the distance and direction of the hazardous object from the hazardous object warning device 15, and outputs an alarm while adjusting the volume according to the distance so that the sound can be heard from the direction of the hazardous object based on the viewpoint of the wearer of the helmet 11.

[0020] For example, if there is a dangerous object directly behind the wearer of the helmet 11, the alarm output device 14 issues an alarm so that the alarm can be heard from directly behind the wearer of the helmet 11.

[0021] In addition, when there are dangerous objects at positions 5 m and 10 m behind the wearer of the helmet 11, the alarm output device 14 outputs an alarm for the dangerous object at a position 5 m at a louder volume than an alarm for the dangerous object at a position 10 m.

[0022] 2 is a block diagram showing an example of the configuration of a hazard warning system including a hazard warning device according to an embodiment. The hazard warning device 15 includes a control unit 16, a storage unit 17, and a communication unit 18, and is attached to a helmet 11 at an arbitrary position.

[0023] The control unit 16 is a processor, typically a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), but may also be a microcomputer, an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or the like.

[0024] The control unit 16 can realize various functions of the hazard object warning device 15 by executing programs such as system software, application software, or firmware stored in the storage unit 17.

[0025] The hazard object warning device 15 may be configured to combine two or more control units 16 and execute various processes through cooperation of these two or more processors.

[0026] The control unit 16 includes a detection unit 161, a position measurement unit 162, and an alarm unit 163. The control unit 16 is configured to transmit and receive data to and from the viewpoint measurement device 12, the image recognition device 13, and the alarm output device 14.

[0027] By executing the above program, the control unit 16 has the function of detecting dangerous objects from images obtained from the image recognition device according to the viewpoint movement range contained in the viewpoint information obtained from the viewpoint measurement device 12, the function of measuring the distance and direction to the detected dangerous object, and the function of causing the alarm output device to output an alarm.

[0028] The detection unit 161 extracts the viewpoint movement range included in the viewpoint information transmitted from the viewpoint measurement device 12. For example, the detection unit 161 extracts the movement range of the pupils of the person wearing the helmet 11 as the viewpoint movement range, and determines whether the movement range is equal to or smaller than a preset movement range (first threshold value).

[0029] Furthermore, the detection unit 161 detects a dangerous object from among a plurality of objects included in the image information transmitted from the image recognition device 13, according to the viewpoint movement range. When detecting the dangerous object, the detection unit 161 uses a dangerous object list acquired from the storage unit 17. The dangerous object list will be described later.

[0030] The position measurement unit 162 measures the distance and direction of a dangerous object detected by the detection unit 161. For example, the position measurement unit 162 has a function of measuring the distance and direction of multiple objects by learning blur information caused by aberration included in image information.

[0031] The position measurement unit 162 measures the distance and direction based on the point where the image was captured. That is, the image recognition device 13 is used as the reference point to measure the distance and direction of the dangerous object.

[0032] The alarm unit 163 acquires information regarding the distance and direction of the hazardous object measured by the position measurement unit 162, and outputs an alarm output signal to the alarm output device 14 if the position of the hazardous object is within a predetermined range (second threshold value).

[0033] The storage unit 17 includes, for example, a main storage unit and an auxiliary storage unit. The main storage unit may include, for example, a ROM (read-only memory) and a RAM (random-access memory). The ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values ​​used by the control unit 16 when performing various processes. The RAM may be used as a so-called work area for temporarily storing data when the control unit 16 performs various processes. The main storage unit in this embodiment is, for example, a RAM, and is used as a memory.

[0034] The auxiliary storage unit is a non-transitory computer-readable storage medium of a computer centered around the control unit 16. The auxiliary storage unit is, for example, an EEPROM (registered trademark) (electric erasable programmable read-only memory), an HDD (hard disk drive), or an SSD (solid state drive).

[0035] The auxiliary storage unit can store data used by the control unit 16 when it performs various processes, data generated by the processes in the control unit 16, various setting values, etc. The auxiliary storage unit stores, for example, viewpoint information, image information, a list of dangerous objects, etc.

[0036] The communication unit 18 can transmit information received from outside to the components of the danger target warning device 15, and can also transmit information received from the components within the danger target warning device 15 to the outside. The communication unit 18 is communicably connected to a terminal 19 and a cloud 20 via a network such as the Internet.

[0037] The terminal 19 is, for example, equipped with at least one processor and a memory in which a program executed by the processor is recorded, and is configured to be able to realize various functions by software or a combination of software and hardware.

[0038] The terminal 19 may be an electronic device such as a computer, and may be, for example, a television receiver (including an Internet television), a PC (Personal Computer), a mobile terminal (for example, a tablet, a smartphone, a laptop, a feature phone, a digital music player, an e-book reader, a smartwatch, etc.), a game device (a home game console, a portable game console), a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, etc. The number of terminals 19 connected to the danger object warning device 15 via the network may be zero, or may increase or decrease at any time.

[0039] In this embodiment, the terminal 19 is, for example, a smartphone, and real-time images from the image recognition device 13 are displayed on the screen of the terminal 19 via a dedicated app or the like. This allows the wearer of the helmet 11 to recognize dangerous objects that are outside the wearer's field of vision.

[0040] The cloud 20 stores a list of dangerous objects in cloud storage. For example, the cloud 20 acquires images from the image recognition device 13 attached to the helmet 11 via the network, performs learning to classify dangerous objects from non-dangerous objects, and generates a learning model.

[0041] Furthermore, the cloud 20 performs learning to classify dangerous objects, such as obstacles, and dangerous objects, such as holes that could lead to a fall, and generates a learning model. The cloud 20 classifies dangerous objects based on the learning results and creates a dangerous object list.

[0042] The cloud 20 also classifies objects acquired from the multiple image recognition devices 13 using a learning model and updates the dangerous object list. The cloud 20 transmits the dangerous object list held in the cloud storage in response to a request from the dangerous object warning device 15.

[0043] Next, a hazard object warning process executed by the hazard object warning device 15 in this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the hazard object warning process of the hazard object warning device according to this embodiment.

[0044] The hazard object warning process of this embodiment is started at any timing. The hazard object warning device 15 transmits a transmission request signal for viewpoint information to the viewpoint measurement device 12. The viewpoint measurement device 12 transmits viewpoint information of the wearer of the helmet 11 to the hazard object warning device 15 (step S1).

[0045] The detection unit 161 extracts the viewpoint movement range included in the viewpoint information transmitted from the viewpoint measurement device 12, and determines whether the viewpoint movement range is equal to or smaller than a first threshold value (step S2).

[0046] When the detection unit 161 determines that the viewpoint movement range is equal to or less than the first threshold value (step S2, YES), the hazard object warning device 15 transmits a signal requesting transmission of image information to the image recognition device 13. The image recognition device 13 transmits image information including information on the recognized plurality of objects to the hazard object warning device 15 (step S3).

[0047] If the detection unit 161 determines that the viewpoint movement range is greater than the first threshold value (step S2, NO), the process returns to step S1.

[0048] The detection unit 161 determines whether a dangerous object has been detected from among the objects using multiple object information contained in the image information transmitted from the image recognition device 13 and the dangerous object list transmitted from the cloud 20 (step S4).

[0049] When the detection unit 161 determines that it has detected at least one dangerous object from among the objects based on the dangerous object list (step S4, YES), the position measurement unit 162 measures the distance and direction of the dangerous object detected by the detection unit 161 (step S5).

[0050] When the detection unit 161 determines that a dangerous object cannot be detected from among the objects based on the dangerous object list (step S4, NO), the process returns to step S1.

[0051] The warning unit 163 determines whether the distance (straight-line distance) between the hazardous object and the reference point, which is included in the position information of the hazardous object measured by the position measurement unit 162, is equal to or less than a second threshold value (step S6).

[0052] When the alarm unit 163 determines that the distance between the hazardous object and the reference point is less than or equal to the second threshold value (step S6, YES), it transmits an alarm output signal to the alarm output device 14, causes the alarm output device 14 to output an alarm sound (step S7), and terminates the processing.

[0053] According to the hazard warning device 15 of this embodiment, when the wearer of the helmet 11 focuses his or her attention on one point and the field of vision becomes narrow, it is possible to alert the wearer to holes, obstacles, etc. that exist in the wearer's blind spot. Also, by measuring the distance to hazards such as holes or obstacles from an image, an alarm is not sounded for hazards that are greater than a certain standard, so an alarm is issued only for necessary hazards, and it is possible to reduce the wearer's habituation to hazards compared to when hazard warnings are always issued.

[0054] In addition, by sending images from the image recognition device 13 to the cloud 20 and generating a learning model using the images, it becomes possible to constantly update the dangerous object list to the latest state.

[0055] In other words, the hazard warning device, hazard warning method, and program of this embodiment make it possible to issue a warning so that the location of the hazard can be grasped at the appropriate time.

[0056] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0057] 10...Danger warning system 11. Helmet 12...Point of view measurement device 13...Image recognition device 14…Alarm output device 15...Hazard warning device 16...Control unit 161...Detection unit 162...Position measurement section 163...Alarm section 17...Storage section 18…Communications Department 19...Terminal 20. Cloud

Claims

1. A hazard object warning device that acquires viewpoint information from a viewpoint measurement device, acquires an image in which a plurality of objects are recognized from an image recognition device, and causes an alarm output device to output an alarm sound, a detection unit that detects a hazardous object from among the plurality of objects in the image according to a viewpoint movement range included in the viewpoint information; a position measurement unit that measures the distance and direction to the dangerous object using the image recognition device as a reference point; an alarm unit that outputs the alarm sound by transmitting an alarm output signal to the alarm output device according to a positional relationship between the image recognition device and the hazardous object; A hazard warning device comprising:

2. The danger object warning device is Further, a storage unit is provided for storing a list of dangerous objects including the plurality of dangerous objects transmitted from the cloud, The detection unit 2. The hazard warning device according to claim 1, wherein when it is determined that the viewpoint movement range is equal to or smaller than a first threshold, the hazard is detected from the image based on the hazard list.

3. The alarm unit The hazard warning device according to claim 2 , wherein when it is determined that the distance between the image recognition device and the hazardous object is equal to or less than a second threshold, the device causes the alarm output device to output the alarm sound.

4. A hazard object warning method that acquires viewpoint information from a viewpoint measurement device, acquires an image in which a plurality of objects are recognized from an image recognition device, and causes a warning output device to output a warning sound, detecting a dangerous object from among the plurality of objects in the image according to a viewpoint movement range included in the viewpoint information; Measure the distance and direction to the dangerous object using the image recognition device as a reference point; A hazard warning method that outputs the alarm sound by transmitting an alarm output signal to the alarm output device according to a positional relationship between the image recognition device and the hazard.

5. A program executed by a danger object warning device, which acquires viewpoint information from a viewpoint measurement device, acquires an image in which a plurality of objects are recognized from an image recognition device, and causes an alarm output device to output an alarm sound, The danger object warning device, a detection means for detecting a dangerous object from among the plurality of objects in the image in accordance with a viewpoint movement range included in the viewpoint information; a position measuring means for measuring the distance and direction to the dangerous object using the image recognition device as a reference point; an alarm means for outputting the alarm sound by transmitting an alarm output signal to the alarm output device according to a positional relationship between the image recognition device and the hazardous object; A program to function as a

Images