Warning device and warning method
The alarm device enhances safety and efficiency by determining worker awareness of heavy machinery, using imaging and detection units to output tailored alarms, ensuring smooth operations.
Patent Information
- Application Number
- JP2023086969
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-12-06
AI Technical Summary
Conventional systems that stop heavy machinery when a worker is near it can reduce work efficiency and hinder smooth operation, even if the worker is aware of the machinery and can take preventive action.
An alarm device that uses imaging and detection units to determine if a person is near heavy machinery and recognizes it, outputting alarms based on the person's awareness to ensure safety without halting operations.
Allows work to proceed smoothly while ensuring safety by differentiating between workers who recognize and do not recognize the machinery, using colored warnings or alarms to manage risks effectively.
Smart Images

Figure 2024170041000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an alarm device and an alarm method. [Background technology]
[0002] At construction sites, heavy machinery is used in the work area, and workers may work nearby. In such cases, measures are taken to prevent contact between the heavy machinery and the workers. For example, a device that notifies the operator when a worker approaches the heavy machinery within a certain distance has been disclosed (for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2007-120110 A Summary of the Invention [Problem to be solved by the invention]
[0004] In the above-mentioned conventional system, if a worker is near the heavy machinery, a warning is issued and the operation of the heavy machinery is stopped, but if the worker is aware of the presence and movement of the heavy machinery, the worker can take action to move away from the heavy machinery even if no warning is issued. If the heavy machinery is stopped when the heavy machinery and the worker come close to each other, a certain level of safety can be ensured, but the heavy machinery is stopped, which reduces work efficiency and makes it difficult to carry out work smoothly.
[0005] In view of the above-mentioned problems, an object of the present invention is to provide an alarm device and an alarm method that can notify a worker of a danger and enable the worker to proceed with work smoothly. [Means for solving the problem]
[0006] An alarm device according to one embodiment of the present invention includes an imaging unit that acquires an image of the area around a heavy machine, a person detection unit that detects an image of a person included in the image, a distance determination unit that determines whether the person is within a predetermined distance from the heavy machine based on the image of the detected person, a recognition determination unit that determines whether the detected person is aware of the heavy machine, and a warning unit that outputs an alarm depending on the determination result of the distance determination unit and the determination result of the recognition determination unit.
[0007] An alarm method according to one aspect of the present invention is an alarm method that causes a computer to execute the steps of acquiring an image of the area around heavy machinery, detecting an image of a person contained in the acquired image, determining whether the person is within a predetermined distance from the heavy machinery based on the image of the detected person, determining whether the detected person is aware of the heavy machinery, and outputting an alarm in response to the determination result of whether the person is within the predetermined distance and the determination result of whether the person is aware of the heavy machinery. Effect of the Invention
[0008] According to the present invention, even if a person is near the heavy machinery, if the person is aware of the heavy machinery, the work may be able to proceed. This allows the work to proceed smoothly while ensuring safety. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram showing an overview of a danger warning system according to an embodiment of the present invention. [Diagram 2] 5 is an explanatory diagram of the operation of the danger warning device according to the embodiment of the present invention. FIG. [Diagram 3] 5 is an explanatory diagram of the operation of the danger warning device according to the embodiment of the present invention. FIG. [Figure 4] 5 is an explanatory diagram of the operation of the danger warning device according to the embodiment of the present invention. FIG. [Diagram 5] 1 is a block diagram showing an overview of a danger warning device according to an embodiment of the present invention. [Figure 6]4 is a flowchart showing a process of the danger warning device according to the embodiment of the present invention. [Figure 7] 13 is a flowchart showing details of a recognition determination process. [Figure 8] FIG. 11 is an explanatory diagram of a face direction vector. [Figure 9] FIG. 1 is an explanatory diagram of a person's field of view. [Figure 10] 13 is a flowchart showing details of a risk assessment process. [Figure 11] FIG. 11 is a block diagram of another aspect of the danger warning device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing an overview of a danger warning system 1 according to an embodiment of the present invention.
[0011] In FIG. 1, heavy equipment 10 is a machine used in civil engineering and construction sites, such as a backhoe, a shovel, a loader, a crane, etc. Heavy equipment 10 is equipped with a danger alarm device 20. The danger alarm device 20 detects whether or not a person is near heavy equipment 10 from an image captured by a camera 21. If a person is near heavy equipment 10, the danger alarm device 20 judges whether or not the person recognizes heavy equipment 10, evaluates the risk of contact between the surrounding people and the heavy equipment, and turns on an alarm lamp 22 according to the evaluation result. Regarding whether or not a person recognizes heavy equipment 10, for example, an effective visual field is estimated, and if the heavy equipment is in the effective visual field, it can be estimated that the person recognizes the heavy equipment. The effective visual field is the area in which human visual recognition is performed, and is said to be about 30° to 40° in the horizontal direction and about 20° in the vertical direction. The operator of the heavy equipment 10 can determine the danger around the heavy equipment 10 based on the lighting state of this warning lamp 22 (whether it is lit or not, its color, etc.).
[0012] Figures 2 to 4 are explanatory diagrams of the operation of the danger warning device 20 according to an embodiment of the present invention. Figure 2 shows a case where there is no person near the heavy equipment 10 (for example, within a distance L1 based on the heavy equipment 10). Figures 3 and 4 show a case where there is a person near the heavy equipment 10 (for example, within a distance L1 based on the heavy equipment 10).
[0013] 2, when there is no person near the heavy equipment 10, there is a low risk of contact between the heavy equipment 10 and people in the vicinity. In such a case, the warning lamp 22 is lit in green. In this case, the warning lamp may be in an unlit state.
[0014] As shown in Fig. 3 and Fig. 4, when a person 2a or 2b is present near the heavy machinery 10, it is considered that there is a risk of contact between the heavy machinery 10 and the surrounding person 2a or 2b. However, the risk of contact between the heavy machinery 10 and the surrounding person 2a or 2b differs between a case where the person 2a visually recognizes the heavy machinery 10 as shown in Fig. 3 and a case where the person 2a does not visually recognize the heavy machinery 10 as shown in Fig. 4. As shown in Fig. 3, when the person 2a visually recognizes the heavy machinery 10 (when the line of sight of the person 2a is directed toward the heavy machinery 10), the person 2a can recognize the presence (position and behavior) of the heavy machinery 10, and therefore the person 2a can take action to move away from the heavy machinery 10 as necessary, and the risk of contact between the heavy machinery 10 and the person 2a is low. On the other hand, as shown in Fig. 4, when the person 2b is not looking at the heavy machinery 10 (when the person 2a's line of sight is not directed toward the heavy machinery 10), the person 2b does not visually recognize the heavy machinery 10 and therefore cannot visually recognize the position or behavior of the heavy machinery 10, and therefore has fewer clues to determine whether or not it is better to move away from the heavy machinery 10. Therefore, it becomes difficult for the person 2b to determine whether or not it is better to move away from the heavy machinery 10, and the risk of contact between the heavy machinery 10 and the person 2b increases.
[0015] For this reason, in this embodiment, as shown in FIG. 3, if a person 2a is present near the heavy equipment 10, but the person 2a recognizes the heavy equipment 10, the warning lamp 22 is turned on in yellow to warn the operator 3 of the heavy equipment. In this case, the operator 3 of the heavy equipment can continue working while being aware of the presence of the person 2a. Furthermore, as shown in FIG. 4, if a person 2b is present near the heavy equipment 10 and the person 2b does not recognize the heavy equipment 10, the warning lamp 22 is turned on in red to warn the operator 3 of the heavy equipment 10 of danger. Here, turning on the warning lamp 22 in red indicates a higher degree of danger than turning on the warning lamp 22 in yellow. When the warning lamp 22 is turned on in red, the operator 3 of the heavy equipment can avoid contact with the person by stopping work.
[0016] FIG. 5 is a block diagram showing an outline of a danger warning device 20 according to an embodiment of the present invention. As shown in FIG. 5, the danger warning device 20 includes a camera 21, a warning lamp 22, and a control unit 25.
[0017] The camera 21 functions as an imaging unit that acquires images of the surroundings of the heavy equipment 10. The camera 21 may be installed so as to image an area that is a blind spot for the operator 3 of the heavy equipment. In addition, as the camera 21, a plurality of cameras 21 may be arranged so as to image different imaging areas. In addition, the camera 21 may be capable of changing the imaging direction according to the moving direction of the heavy equipment 10, etc.
[0018] The warning lamp 22 functions as a warning unit that warns the heavy equipment operator 3 by lighting up in different colors according to the danger. The warning lamp 22 is, for example, a rotating light of three colors, red, yellow, and green, and warns the operator 3 by lighting up one of the lamps of each color according to the danger analysis result.
[0019] The distance between the person and the heavy equipment 10 can be estimated from an image acquired from the camera 21 (for example, Japanese Patent Application No. 2019-190536, Japanese Patent Application Laid-Open No. 2021-67469). The distance between the person and the heavy equipment 10 may also be measured using LiDAR or radar.
[0020] The control unit 25 includes a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), etc., and performs various processes based on a program. In this embodiment, the control unit 25 realizes the functions of a person detection function unit 251, a distance determination function unit 252, a recognition determination function unit 253, and a risk assessment function unit 254.
[0021] The human detection function unit 251 acquires an image captured by the camera 21, and detects whether or not an image of a human has been detected in the captured image.
[0022] The distance determination function unit 252 determines whether the distance between the heavy equipment 10 and the person is within a predetermined distance based on the measurement results of the camera image or LiDAR, etc. The predetermined distance may be determined in advance based on the range to which attention should be paid.
[0023] The recognition determination function unit 253 estimates from the image captured by the camera 21 whether the heavy equipment 10 is within the person's field of view, thereby determining whether the person recognizes the heavy equipment 10.
[0024] The risk assessment function unit 254 assesses the risk of contact between the person and the heavy machinery according to the person detection result, the distance between the person and the heavy machinery, and the person's awareness of the heavy machinery (whether or not the person is aware of it), and turns on the warning lamp 22 in various colors according to the risk assessment result. In this embodiment, the risk assessment function unit 254 determines the level of risk of contact between the person and the heavy machinery according to the judgment result of the distance judgment function unit 252 and the judgment result of the recognition judgment function unit 253. There may be multiple risk levels, for example, three levels. The risk assessment function unit 254 turns on the warning lamp 22 as follows according to the judged risk level.
[0025] (1) Red light (Level 3): There is a person near the heavy equipment 10, but the person does not see the heavy equipment 10. In this case, the warning lamp 22 is lit in red to warn the surrounding people that there is a high risk of contact with the heavy equipment 10. (2) Yellow light (Level 2): There is a person near the heavy equipment 10, but that person has visual contact with the heavy equipment 10. In this case, there is a person nearby, but because that person has visual contact with the heavy equipment 10, there is a risk that contact can be avoided. In this case, compared to the above-mentioned case (1), there is a lower risk of nearby people coming into contact with the heavy equipment 10. Therefore, the warning lamp 22 is lit yellow to warn nearby people that there is a risk of contact with the heavy equipment 10. (3) Green light (Level 1): There is no person near the heavy equipment 10. In this case, the risk of contact between nearby people and the heavy equipment is lower than in (2) above. In this case, the warning lamp 22 is lit in green to notify nearby people that there is a low risk of contact with the heavy equipment 10. Note that in this case, the warning lamp 22 does not have to be lit.
[0026] FIG. 6 is a flowchart showing the process of the danger warning device 20 according to the embodiment of the present invention.
[0027] (Step S1) The control unit 25 acquires an image captured by the camera 21 and detects whether or not a person is detected in the captured image. If a person is detected, the control unit 25 advances the process to step S2, and if a person is not detected, the control unit 25 executes the process of step S1 again after a certain waiting time has elapsed.
[0028] (Step S2) The control unit 25 measures the distance between the heavy equipment 10 and the detected person from a camera image or a detection signal from LiDAR or the like, and determines whether the distance between the heavy equipment 10 and the person is within a predetermined distance. If the control unit 25 determines that the distance between the heavy equipment 10 and the person is within the predetermined distance (Step S2: Yes), the control unit 25 advances the process to Step S3.
[0029] (Step S3) The control unit 25 determines whether or not the detected person recognizes the heavy machinery 10, and proceeds to the process of step S4.
[0030] (Step S4) The control unit 25 determines the level of risk of contact between nearby people and the heavy machinery based on the person detection result, the person's distance, and the person's awareness of the heavy machinery, and proceeds to step S5.
[0031] (Step S5) The control unit 25 lights up the alarm lamp 22 in one of the colors according to the determined level.
[0032] FIG. 7 is a flowchart showing details of the recognition determination process in step S3 of FIG.
[0033] (Step S101) Control unit 25 performs a skeletal analysis based on AI (Artificial Intelligence) technology from the image captured by camera 21, detects each part of the person, including both eyes, both ears, and the nose, and proceeds to the process of step S102.
[0034] (Step S102) The control unit 25 determines whether each face part has been acquired. If each face part has been acquired (Step S102: Yes), the control unit 25 advances the process to Step S103, and if the face part cannot be acquired because the user is wearing a helmet or the like, or the user is not facing the heavy equipment 10 and therefore the face part cannot be acquired (Step S102: No), the control unit 25 advances the process to Step S106.
[0035] (Step S103) Control unit 25 estimates a face direction vector based on the positional relationship of the acquired five face features, namely, both eyes, both ears, and nose, and advances the process to step S104.
[0036] (Step S104) The control unit 25 judges whether or not the heavy machine 10 is present in the field of view of the person based on the face direction vector of the person. If the control unit 25 judges based on this judgment result that the heavy machine 10 is present in the field of view of the person, it judges that the person is able to recognize the heavy machine 10, and if it judges that the heavy machine 10 is not present in the field of view of the person, it judges that the person is not able to recognize the heavy machine 10.
[0037] Here, FIG. 8 is an explanatory diagram of the face direction vector. Also, FIG. 9 is an explanatory diagram of a person's field of view. As shown in FIG. 8, a face direction vector indicating the direction of the face direction of the person can be estimated from the positional relationship of five points, the eyes, the ears, and the nose. Since a person looks (visually recognizes) the direction in which the face is facing, the face direction vector indicates the direction of the person's line of sight. Therefore, as shown in FIG. 9, the range of the field of view that the person is looking at is the range of the field of view P1 centered on the face direction vector V1. If heavy equipment 10 is present within the range of this field of view P1, it can be estimated that the person recognizes heavy equipment 10. Therefore, if heavy equipment 10 is present within the field of view of the person (step S104: Yes), the control unit 25 determines that the person recognizes heavy equipment 10 and proceeds to the process at step S105, and if not (step S104: No), determines that the person does not recognize heavy equipment 10 and proceeds to the process at step S106. Also, if no facial parts can be acquired in step S102 due to a helmet or the like (step S102: No), for safety reasons, it is determined that the person does not recognize the heavy equipment 10, and the process proceeds to step S106.
[0038] (Step S105) The control unit 25 outputs a determination result that the person recognizes the heavy machine 10, and returns to the main processing.
[0039] (Step S106) The control unit 25 outputs a determination result that the person cannot recognize the heavy machinery 10, and returns to the main processing.
[0040] FIG. 10 is a flowchart showing the details of the risk assessment process in step S4 of FIG.
[0041] (Step S201): The control unit 25 determines whether or not a person is detected within a predetermined distance from the heavy equipment 10 based on the processing results of steps S1 and S2 in Fig. 6. If a person is not detected within the predetermined distance from the heavy equipment 10 (step S201: No), the control unit 25 advances the process to step S203, and if a person is detected within the predetermined distance from the heavy equipment 10 (step S201: Yes), the control unit 25 advances the process to step S202.
[0042] (Step S202) The control unit 25 determines whether or not the detected person recognizes the heavy machine 10 from the processing result of step S3 in Fig. 6. Then, if the control unit 25 determines that the detected person recognizes the heavy machine 10 (step S202: Yes), the control unit 25 advances the processing to step S204, and if the control unit 25 determines that the detected person does not recognize the heavy machine 10 (step S202: No), the control unit 25 advances the processing to step S205.
[0043] (Step S203) The control unit 25 turns on the alarm lamp 22 in green, and returns to the main processing.
[0044] (Step S204) The control unit 25 turns on the alarm lamp 22 in yellow, and returns to the main processing.
[0045] (Step S9) The control unit 25 turns on the alarm lamp 22 in red, and returns to the main processing.
[0046] As described above, in the danger warning device 20 according to the embodiment of the present invention, if a person is present near the heavy equipment 10 and the person recognizes the heavy equipment 10, the warning lamp 22 is turned on in yellow to warn the operator 3 of the heavy equipment, and if the person does not recognize the heavy equipment 10, the warning lamp 22 is turned on in red to warn the operator 3 of the heavy equipment 10 of the danger. In this embodiment, when the warning lamp 22 is turned on in yellow, the operator can continue working while paying attention to the danger. Also, when the warning lamp 22 is turned on in red, the operator can avoid the danger by stopping the work. Thus, in this embodiment, even if a person is present near the heavy equipment 10, the operator can proceed with the work as long as the person recognizes the heavy equipment. This allows the work to proceed smoothly while ensuring safety.
[0047] In the above embodiment, the warning lamp 22 that lights up in three colors is used as a warning unit that notifies the operator 3 of the heavy equipment of a danger depending on the danger, but the warning is not limited to this. For example, a warning function that issues a warning not only by lighting a lamp but also by using different volumes or types of sounds depending on the level may be used.
[0048] Furthermore, the warning may be transmitted not only to the operator 3 of the heavy machine 10 but also to workers (persons 2a, 2b) working around the heavy machine 10. For example, the worker may bring a mobile terminal or an IC tag, and the warning may be transmitted to the mobile terminal or the IC tag. In addition, various types of warning units that notify of danger depending on the danger are envisioned.
[0049] Furthermore, the operation of the heavy equipment 10 may be controlled according to the degree of danger determined by the danger warning device 20. For example, if a person is present near the heavy equipment 10 and is not aware of the heavy equipment 10, the warning lamp 22 may be illuminated in red and the operation of the heavy equipment 10 may be stopped.
[0050] FIG. 11 shows a block diagram of another aspect of the danger warning device. As shown in FIG. 11, the danger warning device 520 includes an imaging unit 521, a person detection unit 751, a distance determination unit 752, a recognition determination unit 753, a risk assessment unit 754, and a warning unit 522. The imaging unit 521 generates image data by capturing an image of the periphery of the heavy machine. The person detection unit 751 detects an image of a person included in the image data of the periphery of the heavy machine. The distance determination unit 752 determines whether the person is within a predetermined distance based on the image of the detected person. The recognition determination unit 753 determines whether the detected person recognizes the heavy machine. The risk assessment unit 754 evaluates the degree of risk according to the person detection result and the person recognition result. The warning unit 522 notifies the danger according to the degree of risk. According to this embodiment, even if the LiDAR 23 is not used, it is possible to issue an alarm.
[0051] The danger warning device 20 or the danger warning device 520 in the above-mentioned embodiment may be realized in whole or in part by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA.
[0052] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]
[0053] 10... heavy machinery, 11... camera 11, 20... danger warning device, 21... camera, 22... warning lamp, 25... control unit, 251... person detection function unit, 252... distance judgment function unit, 253... recognition judgment function unit, 254... danger evaluation function unit, 520... danger warning device, 521... imaging unit, 522... warning unit, 751... person detection unit, 752... distance judgment unit, 753... recognition judgment unit, 754... danger evaluation unit
Claims
1. An imaging unit that captures an image of the surroundings of the heavy machinery; a person detection unit that detects an image of a person included in the image; a distance determination unit that determines whether or not the detected person is within a predetermined distance from the heavy machinery based on an image of the detected person; a recognition determination unit that determines whether the detected person recognizes the heavy machinery; a warning unit that outputs a warning in accordance with a determination result of the distance determination unit and a determination result of the recognition determination unit; An alarm device comprising:
2. The warning device according to claim 1 , wherein the recognition determination unit determines whether or not the heavy machinery is recognized based on whether or not the heavy machinery is present within a person's field of vision.
3. The warning device according to claim 2 , wherein the recognition determination unit estimates whether the heavy machinery is present within the field of view of the person from a face direction vector calculated based on the positional relationship of facial features and the field of view of the person.
4. a risk assessment unit that assesses a level of risk to the person based on a result of the assessment by the distance assessment unit and a result of the assessment by the recognition assessment unit; The warning unit outputs a warning in accordance with a result of the determination by the risk assessment unit. An alarm device according to any one of claims 1 to 3.
5. 5. The alarm device according to claim 4, wherein the alarm unit is an alarm lamp that lights up in a different color depending on the result of the determination.
6. On the computer, acquiring an image of the surroundings of the heavy machine; detecting an image of a person included in the captured image; determining whether or not the detected person is within a predetermined distance from the heavy machinery based on an image of the detected person; determining whether the detected person recognizes the heavy machinery; outputting an alarm according to a result of the determination as to whether or not the object is within the predetermined distance and a result of the determination as to whether or not the object is recognized; An alarm method for executing the above.
Citation Information
Patent Citations
Alarm about person getting close to heavy equipment
JP2007120110A
Cited By
System, method and program
JP7815533B1
