Work monitoring support device, work monitoring support system, work monitoring support terminal, and work monitoring support method

Abstract

The work monitoring support device according to the present disclosure includes an image acquisition unit (11) that acquires video of a work site, a motion area extraction unit (12) that extracts the area of ​​the worker's motion contained in the video, a hook area extraction unit (13) that extracts an area including the hook connected to the safety belt worn by the worker as the hook area, a hook usage status analysis unit (14) that analyzes the motion information of the worker contained in the motion area and the movement information of the hook contained in the hook area and analyzes the relationship between the motion information and the movement information as a usage status, a hook usage status determination unit (15) that determines, from the analyzed usage status, at least one of the usage statuses of the hook, whether or not the hook is hung on the hanging point and whether or not the hook is detached from the hanging point, and a determination result output unit (16) that outputs the determination result of the hook usage status determination unit.

Description

[Technical Field]

[0001] The present disclosure relates to a work monitoring support device, a work monitoring support system, a work monitoring support terminal, and a work monitoring support method. [Background technology]

[0002] At work sites such as construction sites, workers are sometimes required to wear safety harnesses. In particular, when working on high floors or at high altitudes, it is necessary to monitor whether the worker has fastened the hook of the safety harness to a lifeline or the like in order to prevent falls. Patent Document 1 discloses a technology that acquires a site image of a site where a worker wearing a safety harness is working, identifies from the site image the position of the hook and the position of the lifeline to which the hook is to be fastened, and determines whether the hook is fastened to the lifeline based on the positional relationship between the hook position and the lifeline. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-137361 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technology determines whether or not a hook has been attached to a main rope based on the positional relationship, which makes it difficult for workers to reliably confirm whether or not the hook has been attached to the target point.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a work monitoring support device that can determine whether a worker or the like has reliably hooked a hook at a target position. It also aims to provide a work monitoring support system, a work monitoring support terminal, and a work monitoring support method. [Means for solving the problem]

[0006] The work monitoring support device according to the present disclosure includes a video acquisition unit that acquires a video of a work site, and a video acquisition unit that acquires a video of a work site and, from frames that make up the video, determines the positions of workers included in the frames. finger joints A motion area extraction unit extracts a motion range as a motion area, a hook area extraction unit extracts a range including a hook connected to a safety belt worn by a worker included in a frame from a frame constituting the video as a hook area, and a frame extracting unit extracts a motion area from the frame from which the motion area and the hook area have been extracted and extracts a motion area from the motion area. Regarding the position of the finger joints a hook usage status analysis unit that analyzes the motion information and the movement information acquired from the hook area as a usage status of the hook; It is determined that the worker has grasped the hook based on the positional relationship between the positions of the finger joints, and The hook use state determining unit determines at least one of the use states of the hook, whether or not the hook is hung on the hanging point and whether or not the hook is detached from the hanging point, based on the analyzed use state, and the determination result output unit outputs the determination result of the hook use state determining unit. In addition, the work monitoring support device according to the present disclosure includes an image acquisition unit that acquires video of a work site; a motion area extraction unit that extracts, from the frames that make up the video, a range in which at least a part of the worker's body included in the frames moves as a motion area; a hook area extraction unit that extracts, from the frames that make up the video, a range that includes a hook connected to a safety belt worn by the worker included in the frames as a hook area; a hook usage status analysis unit that receives the frames from which the motion area and hook area have been extracted and analyzes the motion information acquired from the motion area and the movement information acquired from the hook area as the usage status of the hook; a hook usage status determination unit that determines, based on the analysis results of the motion information and movement information, at least one of whether the worker has performed a hooking action in which the hook is attached to a hooking point and whether the worker has performed an unhooking action in which the hook is removed from the hooking point; and a determination result output unit that outputs the determination result of the hook usage status determination unit.

[0007] The work monitoring support system according to the present disclosure includes an imaging unit that captures images of a work site, a work monitoring support device according to the present disclosure, and an alarm unit that issues at least one of a warning popup, light, sound, and vibration based on a determination result of the work monitoring support device. do. Furthermore, the work monitoring support system according to the present disclosure includes: In order to capture images of the bodies of workers working at the work site,a hook area extraction unit that extracts, from the frames that make up the image, an area in which at least a part of the worker's body moves, as an area of ​​movement; a hook usage status analysis unit that receives the frames from which the area of ​​movement and the area of ​​the hook have been extracted, and analyzes the action information obtained from the area of ​​movement and the movement information obtained from the area of ​​the hook as a usage status of the hook; a hook usage status determination unit that determines, based on the analyzed usage status, at least one of the usage statuses of the hook, whether the hook is attached to the hanging point or whether the hook has been detached from the hanging point; a determination result output unit that outputs the determination result of the hook usage status determination unit; and an alarm unit that emits at least one of a warning pop-up, light, sound, and vibration, based on the determination result. The multiple cameras include a right camera that images the worker's right arm, and a left camera that images the worker's left arm.

[0008] The work monitoring support terminal according to the present disclosure includes a video acquisition unit that acquires a video of a work site, and a video acquisition unit that acquires a video of a worker included in the video. finger joints A motion area extraction unit extracts a motion range as a motion area, a hook area extraction unit extracts a hook area connected to a safety belt worn by a worker included in the video as a hook area, and a motion area extraction unit extracts a motion area of ​​the worker included in the motion area extracted by the motion area extraction unit. Regarding the position of the finger joints a hook usage status analysis unit that analyzes the motion information and movement information including at least one of the position and movement direction of the hook included in the hook area extracted by the hook area extraction unit, and analyzes the relationship between the motion information and the movement information as a usage status; It is determined that the worker has grasped the hook based on the positional relationship between the positions of the finger joints, and The hook use state determination unit determines the use state of the hook based on the analyzed use state, at least one of whether the hook is hung on the hanging point or whether the hook is detached from the hanging point, and the determination result output unit outputs the determination result of the hook use state determination unit.

[0009] In addition, work related to this disclosure monitoring support The method includes steps of acquiring a video of a work site, and finger joints A step of extracting a range of movement as a movement area, and a step of extracting a range of a hook connected to a safety belt worn by a worker included in the video as a hook area; Regarding the position of the worker's finger joints included in the area of ​​motion Operational information, and The hook area includes at least one of the position and the direction of movement of the hook. analyzing the movement information and analyzing the relationship between the operation information and the movement information as a usage situation; It is determined that the worker has grasped the hook based on the positional relationship between the positions of the finger joints, and The method includes a step of determining at least one of the usage states of the hook, whether the hook is hung on the hanging point or whether the hook is removed from the hanging point, based on the analyzed usage state, and a step of outputting the determination result of the usage state of the hook. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to determine whether the hook has been securely attached to the target point and whether the hook has been removed from the target point based on the worker's motion information and the hook movement information, and the determination result can be notified to the worker himself, the work supervisor, etc. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a configuration of a work monitoring support system according to a first embodiment; [Figure 2] FIG. 2 is an explanatory diagram illustrating an example of a flow of the work monitoring support system according to the first embodiment. [Figure 3] FIG. 2 is an explanatory diagram illustrating an example of a flow of the work monitoring support system according to the first embodiment. [Figure 4] 4 is an explanatory diagram illustrating each operation when determining whether or not a hooking operation has been performed using the work support device according to the first embodiment. FIG. [Figure 5] 4 is an explanatory diagram illustrating each operation when determining whether or not a hooking operation has been performed using the work support device according to the first embodiment. FIG. [Figure 6]4 is an explanatory diagram illustrating each operation when determining whether or not a hook-unhooking operation has been performed using the work support device according to the first embodiment. FIG. [Figure 7] 3 is a schematic diagram showing an example of a warning pop-up displayed by a reporting unit of the work support system according to the first embodiment. FIG. [Figure 8] 1 is a block diagram showing an example of the configuration of a work monitoring support system according to a first embodiment. [Figure 9] 10 is a flowchart showing a processing routine executed by a work monitoring support device according to a second embodiment. [Figure 10] FIG. 10 is a schematic block diagram showing an example of a processing circuit that realizes each function of the work monitoring support device according to the second embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of a hook use state determination unit according to the third embodiment. [Figure 12] FIG. 11 is a schematic diagram showing a three-layer neural network model according to a third embodiment. [Figure 13] 11 is a flowchart showing the flow of learning of the hook use state determination unit according to the third embodiment. [Figure 14] FIG. 11 is a block diagram showing the configuration of a hook use state determination unit according to the third embodiment. [Figure 15] 11 is a flowchart showing the flow of determination by the hook use state determination unit according to the third embodiment. [Figure 16] FIG. 10 is a block diagram showing an example of the configuration of a work monitoring support device according to a fourth embodiment. [Figure 17] FIG. 10 is an explanatory diagram illustrating an example of the flow of the work monitoring support system according to the fourth embodiment. [Figure 18] FIG. 10 is a block diagram showing an example of the configuration of a work monitoring support device according to a fifth embodiment. [Figure 19] FIG. 13 is an explanatory diagram illustrating an example of the flow of the work monitoring support system according to the fifth embodiment. [Figure 20] FIG. 20 is an explanatory diagram illustrating an example of the flow of the work monitoring support system according to the sixth embodiment. [Figure 21]FIG. 13 is a block diagram showing an example of the configuration of a work monitoring support device according to a sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that identical or corresponding parts in each drawing are designated by the same reference numerals. In the description of the embodiments, the description of identical or corresponding parts will be omitted or simplified as appropriate.

[0013] Embodiment 1 A work monitoring support system 1000 according to a first embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of the work monitoring support system 1000. The work monitoring support system 1000 includes an imaging unit 31 that captures images of a work site, a work monitoring support device 100 that determines whether a hook 42 connected to a safety belt 41 worn by a worker 43 is attached to a destination 48, and an alarm unit 32 that issues a warning pop-up, light, sound, vibration, or the like based on the determination result of the work monitoring support device 100. The hook 42 connected to the safety belt 41 is, for example, a so-called loop hook, which has a spring that closes with one touch and makes it difficult to come off. The destination 48 is, for example, a lifeline, handrail, single pipe, or a screw hole in a steel frame at the work site.

[0014] The work monitoring support device 100 includes an image acquisition unit 11 that acquires an image 49 of a work site; a motion area extraction unit 12 that extracts, from the frames that make up the image 49, a range in which at least a part of the body of a worker 43 included in the frame moves as an action area 46; a hook area extraction unit 13 that extracts, from the frames that make up the image 49, a range that includes a hook 42 connected to a safety belt 41 worn by the worker 43 included in the frame as a hook area 47; a hook usage status analysis unit 14 that receives the frames from which the motion area 46 and the hook area 47 have been extracted and analyzes the motion information acquired from the motion area 46 and the movement information acquired from the hook area as the usage status of the hook 42; a hook usage status determination unit 15 that determines, based on the analyzed usage status, at least one of the usage statuses of the hook 42, i.e., whether or not the hook 42 is hung on the hanging point 48 and whether or not the hook 42 has been detached from the hanging point 48; and a determination result output unit 16 that outputs the determination result of the hook usage status determination unit 15. Then, from an image 49 of the work site captured, the motion area extraction unit 12 extracts a part of the body of the worker 43, such as a hand, and the hook area extraction unit 13 extracts an area 47 of the hook connected to the safety belt 41 worn by the worker 43. The use status of the hook 42 by the worker 43, such as hand motion information and movement information of the hook 42, is analyzed to determine the use status of the hook 42, such as whether the hook 42 has been hung on the hanging point 48 by the hand or whether the hook 42 has been removed from the hanging point 48. The determination result is output from the determination result output unit 16.

[0015] The processing flow of the work monitoring support device 100 when, for example, a fixed camera fixed to a work site is used as the imaging unit 31 will be described. As shown in FIG. 2, the video acquisition unit 11 acquires video 49 captured by the fixed camera. If the acquired video 49 includes a worker 43, the motion area extraction unit 12 detects the range of motion of the worker 43's hands and other objects from the frames constituting the video 49 and extracts a bounding box of the worker 43 that includes and surrounds the detected hands and other objects (hereinafter referred to as a motion area 46). If the acquired video 49 includes a hook 42, the hook area extraction unit 13 extracts a bounding box that includes and surrounds the hook 42 from the frames constituting the video 49 (hereinafter referred to as a hook area 47). The hook usage status analysis unit 14 analyzes, for example, the center coordinates of the motion area 46 and the center coordinates of the hook area 47 as the usage status of the hook 42. The hook use state determination unit 15 determines whether the worker 43 has hung the hook 42 on the target 48 based on the positional relationship between the center coordinates of the analyzed action area 46 and the center coordinates of the hook area 47. The determination result is notified to the worker terminal 201 carried by the worker 43 and including the alarm unit 32, the management terminal 202 carried by the work supervisor, and an alarm unit connected to a work management device 301 installed in a work management unit that manages the work of the worker 43. For example, if the worker 43 has not hung the hook 42, the alarm unit 32 provided in each of them issues a warning pop-up, light, sound, vibration, or the like to warn the worker.

[0016] The imaging unit 31 may be, for example, a wearable camera attached to the helmet of the worker 43. For example, as shown in Fig. 3, the image acquisition unit 11 acquires an image 49 by combining images captured by two imaging units 31 attached to the helmet of the worker 43. Here, if the imaging unit 31 attached to the right half of the helmet of the worker 43 captures an image of the front left side of the worker 43, and the imaging unit 31 attached to the left half captures an image of the front left side of the worker 43, one arm and one hook 42 are captured in one image 49, and therefore, detection errors such as confusing multiple arms are less likely to occur when extracting a movement area 46 and a hook area 47, which will be described later. If the right hand of the worker 43 is included in the video 49 acquired by the imaging unit 31 on the right side of the body, the motion area extraction unit 12 detects, for example, the right hand of the worker 43 from the frames constituting the video 49, and extracts a motion area 46 of the worker 43 that includes and surrounds the detected right hand, etc. Furthermore, if the left hand of the worker is included in the video 49 acquired by the imaging unit 31 on the left side of the body, the motion area extraction unit 12 detects, for example, the left hand of the worker 43 from the frames constituting the video 49, and extracts a motion area 46 of the worker 43 that includes and surrounds the detected left hand, etc. When video 49 acquired by imaging unit 31 on the right side of the body includes hook 42 grasped with right hand 44, hook region extraction unit 13 detects right hook 42 from the frames constituting video 49 and extracts right hook region 47. When video 49 acquired by imaging unit 31 on the left side of the body includes hook 42 grasped with the left hand, hook region extraction unit 13 detects left hook 42 from the frames constituting video 49 and extracts left hook region 47. The hook usage status analysis unit 14 analyzes the center coordinates of the action area 46 and the center coordinates of the hook area 47 as the usage status of the hook 42. The hook use state determination unit 15 determines whether the worker 43 has hung the hook 42 on the hooking point 48 based on the positional relationship between the center coordinates of the action area 46 and the center coordinates of the hook area 47.

[0017] If the imaging unit 31 is a fixed camera installed at the work site or a wearable camera attached to the helmet of the worker 43, the camera or the like captures an image 49 including the body of the worker 43 and the hook. The imaging unit 31 uses, for example, a CCD (Charge Coupled Device) camera that converts the image into an electrical signal, extracts it, and forms pixels. It may also be a video camera that arranges the electrical signals in time series to generate a video signal. It may be a monocular camera with a single lens, a stereo camera, or a camera system combining multiple cameras. It may also be a network camera or IP (Internet Protocol) camera that communicates over the Internet.

[0018] The video acquisition unit 11 acquires video 49 captured by the imaging unit 31 and transmits it to the motion area extraction unit 12 and the hook area extraction unit 13. The video 49 is acquired at intervals of, for example, several fps to approximately 30 fps and transmitted to the video acquisition unit 11 of the work monitoring support device 100 via wireless communication. The video may also be transmitted via wired communication using a USB (Universal Area Network) cable or the like.

[0019] The motion region extraction unit 12 uses an object detection model such as YOLO (You Only Look Once) or R-CNN (Region Based Convolutional Neural Networks) to detect the region of the hand of the worker 43 included in the frames constituting the video 49, and acquires a motion region 46 including the detected hand. Here, an object detection model such as OpenPose that extracts key points of fingers (hereinafter referred to as specific positions) may be used to detect the joints of the fingers as specific positions, and acquire a motion region 46 including the detected specific positions.

[0020] The hook region extraction unit 13 uses an object detection model such as YOLO or R-CNN to detect the hook 42 included in the frames constituting the video 49, and acquires the hook region 47 including the hook 42. Here, the outline of the hook or the like may be detected as a specific position, and the hook region 47 including the detected specific position may be acquired.

[0021] The hook usage status analyzer 14 analyzes the center coordinates of the movement area 46 as movement information of the worker 43 from frames in which a movement area 46 including at least a portion of the body of the worker 43, such as a hand, is extracted. Because the center coordinate position of the movement area 46 can be obtained by analyzing one frame, a movement trajectory may be created by connecting the center coordinate positions of the movement area 46 accumulated by analyzing multiple frames. Furthermore, as movement information, a body movement vector of the worker 43 may be obtained, which is the direction of change over time when moving from the center coordinate position of the movement area 46 at time t to the center coordinate position of the movement area 46 at time t+1. Here, the coordinates of specific positions such as finger joints, the movement trajectory, and the body movement vector may be analyzed as movement information of the worker 43.

[0022] Furthermore, the hook usage status analysis unit 14 analyzes the center coordinates of the hook region 47 as movement information of the hook 42 from the frames in which the hook region 47 is extracted. Because the center coordinates of the hook region 47 can be obtained by analyzing one frame, a movement trajectory may be created by connecting the positions of the center coordinates of the hook region 47 accumulated by analyzing multiple frames. Furthermore, as movement information, a hook movement vector may be obtained, which is the direction of change over time when moving from the position of the center coordinates of the hook region 47 at time t to the position of the center coordinates of the hook region 47 at time t+1. Here, the movement information of the hook 42 may be analyzed as the coordinates of a specific position, such as the outline of the hook 42, the movement trajectory, or the hook movement vector.

[0023] FIG. 4 is a diagram illustrating an example of the flow of a hooking operation performed by the worker 43 when the imaging unit 31 captures an image of the body of the worker 43 and the hook 42. The hook use state determination unit 15 determines whether a hook preparation action has been performed in which the worker 43 grasps the hook 42, as shown in FIG. 4A, for example, based on the analyzed usage status. The hook use state determination unit 15 determines that a hook preparation action has been performed when, for example, the distance between the motion area 46 and the hook area 47 is less than a set grippable distance at which the worker 43 can grasp the hook 42. For example, the distance between the motion area 46 and the hook area 47 is calculated as the distance between the center coordinates of the motion area 46 and the center coordinates of the hook area 47. When a hook preparation action has been performed, the distance between the hand and the hook 42 is small, and the hand and the hook 42 are located in a predetermined position, such as the chest. By determining the hook preparation action in this manner, it can be determined that the distance between the hand and the hook 42 is small. Here, a small distance between the hand and the hook 42 means, for example, that the distance between the center coordinates of the motion area 46 and the center coordinates of the hook area 47 is less than the grippable distance. The grippable distance is, for example, 50 mm.

[0024] Next, as shown in FIG. 4B, it is determined whether or not the worker 43 has performed a hooking action to hook the hook 42 onto the hooking destination 48. If the hooking action has been performed, the distance between the worker 43's hand and the hook 42 is small, and the body movement vector and the hook movement vector are similar. Here, the small distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinate position of the movement area 46 and the center coordinate position of the hook area 47 is less than the graspable distance. The graspable distance is, for example, 50 mm. The similarity between the body movement vector and the hook movement vector means that the cosine value of the angle between the body movement vector and the hook movement vector (hereinafter referred to as cosine similarity) is, for example, 0.9 or more.

[0025] Next, as shown in FIG. 4C , it is determined whether the worker 43 has performed a hook confirmation action to confirm the state of the hook 42. If the hook confirmation action has been performed, the distance between the worker 43's hand and the hook 42 is small, and the change in the position of the worker 43's hand and the position of the hook 42 is small over a predetermined period of time. Here, a small distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinate of the motion area 46 and the center coordinate of the hook area 47 is less than the graspable distance. Here, a small change in the position of the worker 43's hand and the hook 42 means, for example, that the position of the center coordinate of the motion area 46 and the position of the center coordinate of the hook area 47 are within a confirmable distance area in which the worker 43 can confirm that the hook 42 is engaged at the destination 48 by touching the hook with his / her hand. The confirmable distance is, for example, 50 mm.

[0026] Next, as shown in 4D of Fig. 4, it is determined whether or not the worker 43 has performed a hook release action, in which part of his body is removed from the hook 42. If the hook release action has been performed, the hook 42 is in an area where it can be determined that it is engaged with the hook end 48, and the distance between the worker 43's hand and the hook 42 is large. Here, the long distance between the hand and the hook 42 means, for example, that the distance between the position of the center coordinates of the action area 46 and the position of the center coordinates of the hook area 47 is equal to or greater than the grippable distance. The grippable distance is, for example, 50 mm.

[0027] For example, when all of the hooking action, hook confirmation action, and hook release action have been performed, the hook use state determination unit 15 determines that the worker 43 has hooked the hook 42 to the destination 48. In cases such as when the worker 43 re-hooks the hook 42 that was already hooked to the destination 48 to a different destination 48, the determination of the hook preparation action may be omitted. In this way, by analyzing the situation in which the worker 43 hooks the hook 42 based on the motion information such as the position of the motion area 46 and the body motion vector, and the movement information such as the position of the hook area 47 and the hook movement vector, it can be reliably determined that the worker 43 has hooked the hook 42 to the destination 48.

[0028] FIG. 5 is a diagram illustrating an example of the flow of a hooking operation performed by a worker 43 when the imaging unit 31 captures an image of the hook 42 and the hand including the fingers of the worker 43. The hook usage state determination unit 15 determines, for example, based on the analyzed usage status, whether or not a hook preparation action has been performed in which the hook 42 is grasped by the worker 43, as shown in 5A of FIG. 5. The hook usage state determination unit 15 first determines whether or not the distance between the action area 46 and the hook area 47 is less than a set graspable distance at which the hook 42 can be grasped by the worker 43. The graspable distance is, for example, 50 mm. As the distance between the action area 46 and the hook area 47, for example, the distance between the center coordinates of the action area 46 and the center coordinates of the hook area 47 is calculated. After determining the distance between the movement region and the hook region, if the shape of the distribution of specific positions is the shape of a hand joint gripping an object, based on the positional relationship between a plurality of specific positions such as finger joints, it is determined that the worker 43 has gripped the hook 42. The positional relationship of the finger joints is detected using an object detection model such as MediaPipe Hands. The determination of the shape of the distribution of specific positions and the determination of the distance between the movement region and the clothing region may be performed simultaneously, or the distance between the movement region and the hook region may be determined after determining the shape of the distribution of specific positions. In this way, if it is determined that the distance between the hand of the worker 43 and the hook 42 is less than the grippable distance and it is determined that the hook is being gripped by the hand of the worker 43, a hook preparation operation is determined.

[0029] Next, as shown in FIG. 5B, it is determined whether or not the worker 43 has performed a hooking action to hook the hook 42 onto the hooking destination 48. When the hooking action is performed, the distance between the worker 43's hand and the hook 42 is small, and the body movement vector and the hook movement vector are similar. Here, the small distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinate position of the movement area 46 and the center coordinate position of the hook area 47 is less than the graspable distance. The graspable distance is, for example, 50 mm. The similarity between the body movement vector and the hook movement vector means, for example, that the cosine similarity between the body movement vector and the hook movement vector is 0.9 or more.

[0030] Next, as shown in FIG. 5C , it is determined whether the worker 43 has performed a hook confirmation action to confirm the state of the hook 42. If the hook confirmation action is performed, the distance between the worker 43's hand and the hook 42 is small, and the change in the position of the worker 43's hand and the hook 42 is small over a predetermined period of time. Here, a small distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinate of the motion area 46 and the center coordinate of the hook area 47 is less than the graspable distance. Here, a small change in the position of the worker 43's hand and the hook 42 means, for example, that the position of the center coordinate of the motion area 46 and the position of the center coordinate of the hook area 47 are within a confirmable distance area in which the worker 43 can confirm that the hook 42 is engaged with the destination 48 by touching the hook with his / her hand. The confirmable distance is, for example, 50 mm.

[0031] Next, as shown in 5D of Fig. 5, it is determined whether or not the worker 43 has performed a hook release action, in which part of his body is removed from the hook 42. If a hook release action is performed, the hook 42 is within a hook engagement area where it can be determined that the hook 42 is engaged with the hooking point 48, and the distance between the worker 43's hand and the hook 42 is large. Here, the distance between the worker 43's hand and the hook 42 is large means, for example, that the distance between the center coordinates of the movement area 46 and the center coordinates of the hook area 47 is less than the grippable distance. The grippable distance is, for example, 50 mm. Then, for example, when all of the hook preparation action, hooking action, hook confirmation action, and hook release action have been performed, the hook use state determination unit 15 determines that the worker 43 has hooked the hook 42 to the destination 48. In this way, by analyzing the manner in which the worker 43 grasps and hooks the hook 42 with his / her hand based on motion information such as the position of the motion area 46, specific positions of the fingers, and body motion vector, and movement information such as the position of the hook area 47 and the hook movement vector, it can be reliably determined that the hook 42 has been hooked to the destination 48.

[0032] FIG. 6 is a diagram illustrating an example of the flow of the hook removal work performed by the worker 43 when the imaging unit 31 captures an image of the body of the worker 43 and the hook 42. The hook usage state determination unit 15 determines, for example, based on the analyzed usage status, whether or not a hook preparation action has been performed in which the hook 42 is grasped by the worker 43, as shown in 6A of Fig. 6. When determining whether or not a hook preparation action has been performed, the hook usage state determination unit 15 determines that a hook preparation action has been performed if the distance between a part of the body area of ​​the worker 43 and an area 47 of the hook is less than the graspable distance at which the set hook 42 can be grasped by the worker 43. The graspable distance is, for example, 50 mm.

[0033] Next, as shown in FIG. 6B , it is determined whether or not the worker 43 has performed a hook 42 release action to release the hook 42 from the hooking point 48. When the hook 42 release action is performed, the distance between the worker 43's hand and the hook 42 is small, and the body movement vector and the hook movement vector are similar. Here, a small distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinate position of the movement area 46 and the center coordinate position of the hook area 47 is less than the grippable distance. The grippable distance is, for example, 50 mm. A similarity between the body movement vector and the hook movement vector means, for example, that the cosine similarity between the body movement vector and the hook movement vector is 0.9 or more.

[0034] Next, as shown in FIG. 6C , it is determined whether the worker 43 has performed a hook return motion to return the hook 42 to a predetermined position on his body. If the hook return motion is performed, the distance between the worker 43's hand and the hook 42 is small, and the change in the position of the hand and the hook 42 is small for a predetermined time. Here, a small distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinate of the movement area 46 and the center coordinate of the hook area 47 is less than the graspable distance. A small change in the position of the worker 43's hand and the hook 42 means, for example, that the position of the center coordinate of the movement area 46 and the position of the center coordinate of the hook area 47 are within a confirmable distance area in which the worker 43 can confirm that the hook 42 is attached to the string of the safety belt 41 by touching the hook with his hand. The confirmable distance is, for example, 50 mm.

[0035] Next, as shown in 6D of Fig. 6, it is determined whether or not the worker 43 has performed a hook 42 release action to release his / her hand from the hook 42. When the hook release action is performed, the hook 42 is in an area where it can be determined that the hook 42 is fastened to the safety belt 41 or the like worn by the worker 43, and the distance between the worker 43's hand and the hook 42 is large. Here, a large distance between the worker 43's hand and the hook 42 means, for example, that the distance between the center coordinates of the action area 46 and the center coordinates of the hook area 47 is equal to or greater than the grippable distance. The grippable distance is, for example, 50 mm.

[0036] For example, when all of the hook preparation operation, hook release operation, hook return operation, and hook release operation have been performed, the hook use state determination unit 15 determines that the hook 42 has been released from the destination 48 by the worker 43. In cases such as when the worker 43 re-attaches the hook 42 that was already attached to the destination 48 to another destination 48, the determination of the hook return operation and the hook release operation may be omitted. In this way, by analyzing the manner in which the worker 43 sets the hook 42 based on motion information such as the center coordinates of the region 46 of the worker 43's motion and the body motion vector, and movement information such as the center coordinates of the region 47 of the hook and the hook movement vector, it can be reliably determined that the worker 43 has set the hook 42 to the destination 48.

[0037] Next, an example of determining whether a hooking or unhooking action has been performed by the worker 43 will be described. The hook use state determination unit 15 determines, for example, whether the distance between the center coordinate of the area 46 of the worker's 43's action and the center coordinate of the hook area 47 is less than the set grippable distance at which the hook 42 can be gripped by the worker 43. The grippable distance may be stored in the hook use state determination unit 15 as, for example, 50 mm, or may be a value input by, for example, a work supervisor from the input unit of the work monitoring support device 100 or from outside the work monitoring support device 100. Here, an example has been described in which the center coordinate of the hook area and the center coordinate of the hook area are used, but the coordinate of a specific position, such as a finger joint, or the coordinate of a specific position, such as the outline of the hook 42, may also be used. In addition, the hook 42 usage status determination unit determines whether the similarity between the body movement vector analyzed by the hook usage status analysis unit 14 and the hook movement vector is equal to or greater than a set similarity in order to check whether the positions of the hand and hook 42 have changed in the same way. The similarity is a value indicating the degree of similarity between the directions of the body motion vector and the hook movement vector. The hook usage status analysis unit 14 analyzes the body motion vector and the hook movement vector using, for example, the Pearson correlation coefficient or cosine similarity, and the hook usage status determination unit 15 receives the analyzed similarity from the hook usage status analysis unit 14. If the hook usage status determination unit 15 determines that the distance between the center coordinate of the motion area 46 of the worker 43 and the center coordinate of the hook area 47 is less than the graspable distance and determines that the similarity is equal to or greater than a preset similarity, it can be assumed that the hook and the hand are close and moving in the same direction, and that a hooking operation has been performed by the hand gripping the hook. Here, an example has been described in which a body motion vector based on the movement of the center coordinate of the motion area 46 and a hook movement vector based on the movement of the center coordinate of the hook area 47 have been used. However, a body motion vector based on the movement of the coordinates of specific positions, such as finger joints, and a hook movement vector based on the movement of the coordinates of specific positions, such as the outline of the hook 42, may also be used. In addition, the hook usage status analysis unit may analyze the similarity between the worker's movement trajectory and the hook's movement trajectory using a method such as DTW (Dynamic Time Warping), and use the similarity based on the movement trajectory and movement trajectory to determine whether the hooking operation has been performed. The grippable distance may be, for example, 50 mm, the set similarity when determining using cosine similarity may be, for example, 0.9, and the set similarity when using DTW may be, for example, 0.01, and these may be stored in the hook usage status determination unit 15, or may be values ​​input from the input unit of the work monitoring support device 100 or from outside the work monitoring support device 100, for example, by a work supervisor. In this way, the body motion vector and the hook movement vector can be used to quantitatively determine the degree of similarity between the hand motion information of the worker 43 and the movement information of the hook 42. The unhooking motion may also be determined using the same determination method as for the hooking motion.

[0038] An example of determining whether or not the worker 43 has performed a hook confirmation action will be described. The hook use state determination unit 15 determines that the hook confirmation action has been performed, for example, if the distance between the center coordinate position of the region 46 of the worker's movement and the center coordinate position of the region 47 of the hook remains within a grippable distance for the worker 43 to grasp the set hook 42 during a confirmation time during which it can be confirmed that the set hook 42 is hooked on the hook end 48. By using the center coordinates of the region 46 of the worker's movement and the center coordinates of the region 47 of the hook, it is possible to quantitatively determine the action of the worker 43 to confirm whether or not the hook 42 is hooked on the hook end 48. Here, an example has been described in which the center coordinates of the region of the hook and the center coordinates of the region of the hook are used. However, coordinates of specific positions, such as the joints of the fingers, and coordinates of specific positions, such as the outline of the hook 42, may also be used. The grippable distance is, for example, 50 mm.

[0039] An example of determining whether or not the worker 43 has performed a hook release action will be described. The hook use state determination unit 15 determines that the hook release action has been performed, for example, when the position of the hook region 47 is within a hook engagement area where the hook 42 can be engaged, and the distance between the center coordinate of the region 46 of the worker 43's movement and the center coordinate of the hook region 47 is equal to or greater than the grippable distance at which the worker 43 can grip the set hook 42. By using the center coordinate of the region of the worker 43's movement and the center coordinate of the hook region 47, it is possible to quantitatively determine whether or not the worker 43's hand has released the hook 42. Here, an example has been described in which the center coordinate of the hook region and the center coordinate of the hook region are used. However, coordinates of specific positions, such as the joints of the fingers, and coordinates of specific positions, such as the outline of the hook 42, may also be used. The grippable distance is, for example, 50 mm.

[0040] The determination result output unit 16 receives the result determined by the hook use state determination unit 15 of the work monitoring support device 100, and outputs to the alarm unit 32 the determination result that the hook 42 has not been hung by the worker 43. Alternatively, the determination result output unit 16 may output the determination result that the worker 43 has hung the hook 42 on the target 48.

[0041] When the alarm issuing unit 32 receives a determination result from the determination result output unit 16, it is provided in an operator terminal 201 carried by the operator 43, or in a management terminal 202 carried by a work supervisor or installed in a work management unit that manages the work of the operator 43. The determination result is transmitted to the alarm issuing unit 32 of the work monitoring support device 100 by wireless communication. It may also be transmitted by wired communication using a USB (Universal Area Network) cable or the like. The alarm unit 32 displays the worker identification ID corresponding to the worker 43, the position of the worker 43, the date and time of the determination, the determination result, and a recorded video showing the state of forgetting to hook in a pop-up screen such as that shown in Fig. 7. The alarm unit 32 may issue the worker identification ID and the determination result by automatic voice or the like. Furthermore, the alarm unit 32 may notify that there is a worker 43 who has forgotten to hook by flashing an LED lamp or the like.

[0042] As described above, the work monitoring support device 100 includes an image acquisition unit 11 that acquires an image 49 of a work site, a motion area extraction unit 12 that extracts, from the frames that make up the image 49, a range in which at least a part of the body of the worker 43 included in the frame moves as a motion area 46, a hook area extraction unit 13 that extracts, from the frames that make up the image 49, a range that includes the hook 42 connected to the safety belt 41 worn by the worker 43 included in the frame as a hook area 47, and a control unit 14 that receives the frames from which the motion area 46 and the hook area 47 have been extracted, and calculates the motion information acquired from the motion area 46 and the movement information acquired from the hook area as the usage status of the hook 42. the hook usage status determination unit 15 for determining the usage status of the hook 42, whether or not the hook 42 is securely attached to the destination 48, or whether or not the hook 42 has been removed from the destination 48, based on the analyzed usage status; and the determination result output unit 16 for outputting the determination result of the hook usage status determination unit 15. Therefore, regardless of the type of destination 48 of the hook 42, it is possible to determine whether or not the hook 42 has been securely attached to the destination 48, or whether or not the hook 42 has been removed from the destination 48, based on the operation information of the worker 43 and the movement information of the hook 42, and notify the determination result to the worker 43 himself, a work supervisor, etc.

[0043] The work monitoring support system 1000 is equipped with an imaging unit 31 that images the work site, a work monitoring support device 100 according to the present disclosure, and an alarm unit 32 that issues a warning pop-up, light, sound, vibration, etc. based on the determination result of the work monitoring support device 100. Therefore, when the hook 42 is not hung on the hanging point 48, an alarm can be issued on a worker terminal 201 that is carried by the worker 43 and includes the alarm unit 32, or on a management terminal 202 that is carried by a work supervisor or installed in a work management unit that manages the work of the worker 43.

[0044] The work monitoring support device 100 may be located in the worker terminal 201, the management terminal 202, or the work management device 301. When the work management device 301 is equipped with the work monitoring support device 100, the work management device 301 can manage the worker terminals 201 carried by multiple workers 43, as shown in FIG. 8, for example. The work management device 301 may be located on the cloud. Furthermore, video 49 may be acquired from a wearable camera worn by the worker 43, and the work monitoring support device 100 may be provided on the worker terminal 201 of each worker 43.

[0045] Furthermore, although an example has been described in which video 49 showing one worker 43 is used to analyze the motion information of the worker 43 and the movement information of the hook 42, video 49 may include multiple workers 43. When multiple workers 43 are included, the motion information for each worker 43 and the movement information of the hook 42 used by each worker 43 are analyzed to determine whether the hook 42 has been set, thereby making it possible to efficiently analyze the usage status of the hook 42 by multiple workers 43.

[0046] In addition, although an example in which the worker 43 himself sets the hook 42 has been described, the action of setting the hook 42 by a work assistant assisting the worker 43, a supervisor supervising the work site, or the like may be analyzed and determined. The object detection model may be used to recognize that the hook 42 is being set or removed by someone other than the worker 43.

[0047] Embodiment 2 The following describes the operation of the work monitoring support device 100. Fig. 9 is a flowchart showing a processing routine executed by the work monitoring support device 100 according to the second embodiment. The work monitoring support device 100 first acquires an image 49 including a part of the body of the worker 43 from the imaging unit 31 (step S101). Then, from the acquired image 49, an area 46 of the detected movement of the worker 43 is extracted (step S102), and an area 47 of the detected hook is extracted (step S103). Steps S102 and S103 may be performed simultaneously. Frames of the image 49 from which the area 46 of the movement of the worker 43 and the area 47 of the hook have been extracted are accumulated until the number of frames of the image 49 reaches a predetermined number N of frames, and the frames are temporarily stored in the storage device 51 provided in the work monitoring support device 100 (S104). If the predetermined number of frames have been accumulated (YES in S104), the center coordinates of the area of ​​the worker's 43's movement, the coordinates of a specific position, the movement trajectory of the center coordinate position, the movement trajectory of the specific position, the body movement vector, etc. are calculated as the movement information of the worker 43 (step S105). Simultaneously with S105, the center coordinates of the hook area 47, the coordinates of a specific position, the movement trajectory of the center coordinate, the movement trajectory of the specific position, the hook movement vector, etc. are calculated as the movement information of the hook 42 (step S106). Next, the correlation between the movement of the worker 43 and the movement of the hook 42 is analyzed from the calculated movement information and movement information (step S107). After S107, it is determined whether the hook 42 has been hooked to the target 48 based on the analyzed correlation between the movement of the worker 43 and the movement of the hook 42 (step S108). If the hook 42 has not been hooked to the target 48 (NO in S108), the determination result is output to the alarm unit 32.

[0048] Here, each function of the work monitoring support device 100 is realized by a processing circuit. Fig. 10 is a schematic configuration diagram showing an example of a processing circuit that realizes each function of the work monitoring support device 100. The work monitoring support device 100 has a processor 50, a storage device 51, a communication I / F (interface) 52, etc. For example, a CPU (Central Processing Unit) is used as the processor 50. The storage device 51 transmits and receives data to and from the processor 50 and stores the data. The image 49 captured by the imaging unit 31 is acquired by the image acquisition unit 11 via the communication I / F 52. The calculations and judgments of the image acquisition unit 11, the motion area extraction unit 12, the hook area extraction unit 13, the hook usage situation analysis unit 14, the hook usage state determination unit 15, and the determination result output unit 16 are executed by the processor 50. The parameters, calculation formulas, etc. used for the judgments are stored in the storage device 51.

[0049] The processor 50 and the storage device 51 may be one shared device or multiple devices may be used. The processor 50 may also be equipped with, for example, a logic circuit using an ASIC (Application Specific Integrated Circuit), an IC (Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or various signal processing circuits. Multiple processors 50 of the same type or different types may be provided, so that each process may be shared and executed by multiple arithmetic processing devices.

[0050] The multiple storage devices 51 may include, for example, a RAM (Random Access Memory) configured to allow data to be read and written from the processor 50, a ROM (Read Only Memory) configured to allow data to be read from the processor 50, a hard disk (HDD), etc.

[0051] Each function of the work monitoring support device 100 is realized by the processor 50 executing software or a program 54 stored in the storage device 51 and working in cooperation with the hardware. The setting data to be set in the work monitoring support device 100 may be stored in the storage device 51 as part of the software or program 54, or may be input by the user. A non-transitory recording medium 53 on which the program 54 for the work monitoring support device is recorded may be distributed and installed in the storage device 51 of the work monitoring support device 100.

[0052] In this way, an image 49 including a part of the body of the worker 43 is acquired from the imaging unit 31, and the area of ​​the detected body of the worker 43 is extracted from the acquired image 49, and the area 47 of the detected hook is extracted. Coordinates of specific positions in the area 46 of the movement, a movement trajectory, a body movement vector, etc. are calculated as movement information of the worker 43, and coordinates of specific positions in the area 47 of the hook, a movement trajectory, a body movement vector, etc. are calculated as movement information of the hook 42, and the relationship between the movement information of the worker 43 and the movement information of the hook 42 is analyzed. By outputting a determination result when it is determined that the hook 42 is not hung on the destination 48 based on the analyzed relationship between the movement information and the movement information, it is possible to determine whether the worker 43 has reliably hung the hook 42 on the destination 48, and to prevent forgetting to hang the hook 42.

[0053] Embodiment 3 A work monitoring support device 100 according to a third embodiment will be described with reference to the drawings. In the first embodiment, for example, an example was described in which it was determined whether the hook 42 was hung on the target 48 based on the relationship between the motion information and the movement information. However, in the second embodiment, the use state of the hook 42 is determined based on the motion information, the movement information, and the correct label for the use state of the hook 42. The following description will focus on the differences from the first embodiment, and a description of the same or corresponding parts will be omitted as appropriate.

[0054] The work monitoring support device 100 performs supervised learning to learn a discrimination criterion for determining whether or not the hook 42 has been set, using, for example, the motion trajectory of the worker 43 and the movement trajectory of the hook 42, and a correct answer label of the usage state, which is work content information such as the hooking work actually performed by the worker 43. Then, the work monitoring support device 100 determines whether or not the hook 42 has been set by the worker 43 based on, for example, the movement information, the action information, and the learned discrimination criterion.

[0055] We will now explain the generation of a reference trained model that has learned the criteria for determining whether or not the hook 42 is hooked on the hook destination 48. Figure 11 is a functional block diagram showing the configuration of the hook usage state determination unit 15. The hook usage state determination unit 15 includes a hook state learning unit 151 that has a data acquisition unit that acquires the analyzed usage situation and the correct answer for the usage state of the hook 42 and a model generation unit 153 that generates a reference trained model, and a reference trained model storage unit 154 that stores the reference trained model.

[0056] The learning data acquisition unit 152 acquires the analyzed operation information, movement information, and correct labels of the usage states of the hook 42 as learning data. Here, the learning data acquisition unit 152 acquires the operation information and movement information from the hook usage state analysis unit 14. The correct labels of the usage states of the hook 42 are input by, for example, a correct label input unit 20 provided outside the hook usage state determination unit 15.

[0057] The model generation unit 153 learns the discrimination criteria for determining whether the hook 42 is hung or not, based on the learning data created based on a combination of the action information, movement information, and correct label. That is, it generates a trained model that infers the optimal discrimination criteria from the action information, movement information, and correct label. Here, the learning data is data that associates the action information, movement information, and correct label of the use state of the hook 42 with each other.

[0058] The learning algorithm used by the model generation unit 153 can be a known algorithm such as supervised learning, unsupervised learning, reinforcement learning, etc. As an example, a case where a neural network is applied will be described.

[0059] The model generation unit 153 learns the discrimination criteria by so-called supervised learning, for example, according to a neural network model. Here, supervised learning refers to a method of providing pairs of input and result (label) data to the hook usage state determination unit 15, learning the features of the learning data, and inferring the result from the input.

[0060] A neural network consists of an input layer consisting of multiple neurons, an intermediate layer (hidden layer) consisting of multiple neurons, and an output layer consisting of multiple neurons. The intermediate layer may be one layer, or two or more layers.

[0061] For example, in a three-layer neural network as shown in Figure 12, when multiple inputs are input to the input layer (X1-X3), the values ​​are multiplied by weight W1 (w11-w16) and input to the middle layer (Y1-Y2), and the result is further multiplied by weight W2 (w21-w26) and output from the output layer (Z1-Z3). This output result changes depending on the values ​​of weights W1 and W2.

[0062] The neural network learns the discrimination criteria by so-called supervised learning in accordance with learning data created based on a combination of the motion information, movement information, and the correct label of the usage state of the hook 42.

[0063] In other words, the neural network learns by inputting motion information and movement information into the input layer and adjusting the weights W1 and W2 so that the results output from the output layer approach the correct label.

[0064] The model generation unit 153 generates and outputs a trained model by performing the above-described learning.

[0065] The reference trained model storage unit 154 stores the reference trained model output from the model generation unit 153.

[0066] Next, the learning process of the hook usage state determination unit 15 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the flow of learning by the hook usage state determination unit 15.

[0067] In step S201, the learning data acquisition unit 152 acquires the operation information, movement information, and the correct label for the use state of the hook 42. Although the operation information, movement information, and the correct label for the use state of the hook 42 are acquired simultaneously, it is sufficient if the operation information, movement information, and the correct label for the use state of the hook 42 are input in association with each other, and the operation information, movement information, and the correct label for the use state of the hook 42 may be acquired at different times.

[0068] In step S202, the model generation unit 153 learns the judgment criteria by so-called supervised learning in accordance with the learning data created based on a combination of the action information and movement information acquired by the learning data acquisition unit 152 and the correct label of the usage state of the hook 42, and generates a criterion-trained model.

[0069] In step S203, the reference learned model storage unit 154 stores the reference learned model generated by the model generation unit 153, which is the reference for determining whether the hook 42 is hooked on the hook destination 48 or not.

[0070] Next, we will explain the use of a trained model in this embodiment for determining whether or not the hook 42 has been hung on the hanging destination 48. As shown in Fig. 14, the work monitoring support device 100 includes an image acquisition unit 11 that acquires image 49 of the work site, a motion area extraction unit 12 that extracts an image of the motion from the acquired image 49, a hook area extraction unit 13 that extracts an image of the hook 47 from the acquired image 49, a hook usage status analysis unit 14 that analyzes the motion information of the motion area 46 and the movement information of the hook area 47, a hook usage status determination unit 15 that determines whether or not the hook 42 has been hung on the hanging destination 48, and a determination result output unit 16 that outputs the determination result to the outside of the work monitoring support device 100. The hook usage status determination unit 15 includes a reference learned model memory unit 154 that stores a reference learned model for determining whether the hook 42 is hung on the hanging destination 48, an analysis data acquisition unit 155 that acquires the analyzed usage status from the hook usage status analysis unit 14, and a hook hanging determination unit 156 that determines whether the hook 42 is hung on the hanging destination 48 based on the data acquired by the analysis data acquisition unit 155.

[0071] The analysis data acquisition unit 155 acquires the operation information and movement information from the hook usage status analysis unit 14 .

[0072] The hooking determination unit 156 determines whether or not the hook 42 is hooked on the destination 48 based on the data received from the analysis data acquisition unit 155 and the reference learned model stored in the reference learned model storage unit 154, and outputs the determination result to the determination result output unit 16. By inputting the operation information and movement information acquired by the analysis data acquisition unit 155 into this reference learned model, it is possible to output whether or not the hook 42, determined based on the operation information and movement information, is hooked on the destination 48.

[0073] Next, with reference to FIG. 15, a process for obtaining a determination result as to whether or not the hook 42 is hooked on the hooking destination 48 using the hook use state determination unit 15 will be described.

[0074] In step S301, the analysis data acquisition unit 155 acquires the motion information and movement information.

[0075] In step S302, the hook usage status determination unit 15 acquires a reference learned model from the reference learned model memory unit 154, inputs the data acquired by the analysis data acquisition unit 155 into the acquired reference learned model, and obtains a determination result as to whether the hook 42 has been hooked on the hook destination 48.

[0076] In step S303, the hooking determination unit 156 outputs the determination result, obtained from the reference trained model, as to whether or not the hook 42 has been hooked on the hooking destination 48 to the determination result output unit 16. This makes it possible to accurately determine whether or not the hook 42 has been hooked on the hooking destination 48 based on the motion information and movement information.

[0077] Although the above description has been given of a case where supervised learning is applied to the learning algorithm used by the model generation unit 153, the present invention is not limited to this. As for the learning algorithm, reinforcement learning, semi-supervised learning, etc. can also be applied in addition to supervised learning.

[0078] In this way, in the work monitoring support device 100 according to embodiment 3, by determining whether the hook 42 is hooked on the target 48 based on the operation information and movement information, it is possible to reliably and accurately determine whether the hook 42 is hooked on the target 48 even in various operation and movement states.

[0079] In this embodiment, the reference trained model stored in the reference trained model memory unit 154 provided in the work monitoring support device 100 is used to output the determination result of whether the hook 42 has been engaged, but it is also possible to obtain a reference trained model from an external source, such as another work monitoring support device 100, and output the determination result of whether the hook 42 has been engaged on the target 48 based on this reference trained model. By obtaining a reference trained model from an external source, the work monitoring support system 1000 can be updated to an appropriate reference trained model even after it has started operating at the work site, making it possible to more reliably determine whether the hook 42 has been engaged.

[0080] Embodiment 4 A work monitoring support device 100 according to a fourth embodiment will be described with reference to the drawings. In the first embodiment, for example, an example was described in which it was determined whether the hook 42 was hung on the target 48 based on the correlation between the motion information and the movement information, but in the fourth embodiment, the use state of the hook 42 is identified based on the state of the hook 42. The following description will focus on the differences from the first embodiment, and descriptions of the same or corresponding parts will be omitted as appropriate.

[0081] The work monitoring support device 100 according to the fourth embodiment has the configuration of the work monitoring support device 100 described in the first embodiment, and a hook state identification unit 17 that identifies whether the hook 42 is hung on the hooking point 48 based on the state of the hook 42. As shown in Fig. 16 , the hook state identification unit 17 has a hook area acquisition unit 171 that acquires the hook area 47 from the hook area extraction unit 13, and an identification processing unit 172 that identifies the open / closed state or the hung state of the hook 42 from the video 49 including the hook area 47 using a state-learned model for identifying the open / closed state or the hung state of the hook 42 from the hook area 47. Then, in the work monitoring support device 100, the work area extraction unit and hook area extraction unit 13 extract an action area 46 and a hook area 47, and then the hook usage status analysis unit 14 analyzes the usage status of the hook 42, and the hook state identification unit 17 identifies whether the hook 42 is in a state where it is hung on the destination 48 based on the open / closed state of the hook 42, etc. The hook usage status determination unit 15 performs the determination described in the first embodiment to determine whether the hook 42 is in a state where it is hung on the destination 48. Here, if the result of the identification by the hook state identification unit 17 and the result of the determination by the hook usage status determination unit 15 indicate that the hook 42 is in a state where it is hung on the destination 48, the determination result output unit 16 outputs a determination result that the hook 42 is not in a state where it is hung on the destination 48.

[0082] The following describes the processing flow of the work monitoring support device 100 when, for example, two wearable cameras attached to the helmet of the worker 43 are used as the imaging unit 31. As shown in FIG. 17 , the video acquisition unit 11 acquires a video 49 capturing a right hand 44 and a video 49 capturing a left hand 45 captured by the two wearable cameras attached to the helmet of the worker 43. The motion area extraction unit 12 detects the joints of the fingers of the worker 43 as specific positions and extracts a motion area 46 including the extracted specific positions. If the acquired video 49 includes a hook 42 grasped with the right hand 44 and a hook 42 grasped with the left hand 45, the hook area extraction unit 13 separately detects the hook 42 on the right hand 44 side and the hook 42 on the left hand 45 side, and extracts a region 47 of the hook on the right hand 44 side and a region 47 of the hook on the left hand 45 side. The hook use status analysis unit 14 analyzes the positions of the hand joints included in the action area 46 and the position of the center coordinates of the hook area 47 as the use status of the hook 42. The hook use state determination unit 15 determines whether or not the hook 42 has been hooked on the destination 48 by the worker 43, based on the positional relationship between the position of the hand joint included in the motion area 46 and the position of the center coordinate of the hook area 47. The hook state identification unit 17 then identifies the state of the hook 42 extracted by the hook area extraction unit 13, and determines whether or not the hook 42 is hooked on the destination 48. When the hook use state determination unit 15 determines that the hook 42 has been hooked on the destination 48, and the hook state identification unit 17 identifies that the hook 42 is in a state of being hooked on the destination 48, it is determined that the hook 42 is securely hooked on the destination 48.

[0083] The hook region acquisition unit 171 acquires the hook region 47 including the hook 42 from the hook region extraction unit 13 .

[0084] The identification processing unit 172 identifies whether the hook 42 is in a state where it is hung on the destination 48, based on the state of the hook 42 included in the hook region 47 received from the hook region acquisition unit 171 and the state learned model stored in the state learned model storage unit 173. The identification result is transmitted to the hook use state determination unit 15. By inputting the state of the hook 42 acquired by the hook region acquisition unit 171 into this state learned model, it is possible to identify whether the hook 42 identified based on the state of the hook 42 is in a state where it is hung on the destination 48. The state-trained model is, for example, an object detection model such as YOLO (You Only Look Once) or R-CNN (Region Based Convolutional Neural Networks).

[0085] In this way, the hook usage status determination unit 15 determines whether the hook 42 is hung on the destination 48 based on the operation information and movement information, and the hook status identification unit 17 determines whether the hook 42 is hung on the destination 48 based on the status of the hook 42, thereby making it possible to more reliably determine whether the hook 42 is hung on the destination 48.

[0086] While the hook state identification unit 17 has been described as identifying whether the hook 42 is in a state where it is hung on the destination 48 based on the state of the hook 42, the hook 42 use state identification unit may determine the use state of the hook 42 by associating the identified result of the hook 42 state with the action to be determined by the hook 42 use state determination unit. For example, the hook 42 state is preset for each of the four actions of the hooking work shown in FIG. 4, such as 4A being the closed state, 4B being the open state, 4C being the closed state, and 4D being the closed state, and it is determined whether the state transition of the hook 42 is as set. Here, the state in which the movable part of the hook 42 is open is defined as the open state, and the state in which the movable part of the hook 42 is closed is defined as the closed state. While determining whether the hand movement information and the movement information of the hook 42 meet the predetermined judgment conditions for each action shown in FIG. 4, it is also determined whether the state transition of the hook 42 meets the predetermined settings, and it is determined whether the hook 42 has been left hanging based on the judgment results based on the movement information and movement information and the identification result of the hook 42 state. By determining whether the hook 42 has been hooked onto the destination 48 based on the judgment results based on the operation information and movement information and the identification results of the state of the hook 42, it is possible to more reliably determine whether the hooking operation has been performed.

[0087] Embodiment 5. A work monitoring support device 100 according to embodiment 5 will be described with reference to the drawings. In embodiment 1, an example was described in which the use state of the hook 42 was determined regardless of whether the image 49 acquired by the image acquisition unit 11 was captured in a location where use of the hook 42 is mandatory. However, embodiment 5 differs in that when it is necessary to determine the use state of the hook 42, it is determined whether the hook 42 is hung on the hanging destination 48. The following description will focus on the differences from embodiment 1, and descriptions of identical or corresponding parts will be omitted as appropriate.

[0088] The worker 43 moves to various heights, from ground level to floor level on the second floor or higher. When determining whether the hook is engaged using a camera attached to the helmet of the worker 43, for example, it is not necessary to determine the use status of the hook 42 when the worker 43 is at ground level. Therefore, an example will be described in which the surrounding situation is read from the video 49 captured by the camera attached to the helmet, and it is determined whether or not the use status of the hook 42 needs to be determined. As shown in Figure 18, the work monitoring support device 100 includes an image acquisition unit 11 that acquires image 49 of the work site, a judgment necessity determination unit 18 that determines whether it is necessary to determine whether or not the hook 42 has been hung on the hanging destination 48 based on the judgment necessity information contained in the image 49, a motion area extraction unit 12 that extracts an action area 46 from the acquired image 49, a hook area extraction unit 13 that extracts a hook area 47 from the acquired image 49, a hook usage status analysis unit 14 that analyzes the motion information of the action area 46 and the movement information of the hook area 47, a hook usage status determination unit 15 that determines whether or not the hook 42 has been hung on the hanging destination 48, and a determination result output unit 16 that outputs the determination result to an outside of the work monitoring support device 100. The judgment necessity determination unit 18 has a judgment necessity information acquisition unit 181 that acquires information necessary for determining whether a judgment is necessary, a hook use necessity information storage unit 183 that stores hook use necessity information in which the judgment necessity information is associated with whether or not to judge the usage status of the hook 42, and a decision processing unit 182 that determines whether or not it is necessary to judge the usage status of the hook 42 based on the judgment necessity information and the hook use necessity information.

[0089] The determination necessity information acquisition unit 181 receives determination necessity information including the video 49 acquired by the video acquisition unit 11, the video 49 including an identification code 55 such as a QR code (a registered trademark of DENSO WAVE Corporation) posted at the work site, the video 49 capturing the image of the worker 43 ascending, and the ascent detection data measured by the ascent detection measurement unit 56 held by the worker 43. As shown in FIG. 19 , the identification code 55 is associated with information such as the height of the location where it is posted and is posted, for example, at the exit of a staircase leading from the first floor to the second floor of the work site. In this case, when the imaging unit 31 captures the identification code 55, it can be determined that the worker 43 is on the second floor. Furthermore, when the imaging unit 31 reads the surrounding video 49 and captures the image of a higher floor of another building, it can be determined that the worker 43 is on a higher floor of the work site. The ascent detection measurement unit 56 is, for example, an acceleration sensor that measures the ascent of the worker 43, a barometric pressure sensor that can calculate the altitude, etc.

[0090] The hook use necessity information storage unit 183 stores hook use necessity information that associates judgment necessity information required to determine whether judgment is necessary with hook use necessity information that indicates whether it is necessary to judge the use state of the hook 42. Specifically, the hook use necessity information is information that includes, for example, "judgment not required at ground level," "judgment required on the second floor or higher," and "judgment required above the legal height from ground level where the use of the safety belt 41 is required by law."

[0091] The determination processing unit 182 determines whether it is necessary to determine the usage state of the hook 42 based on the determination necessity information and the hook use necessity information corresponding to the determination necessity information. If it is determined that it is necessary to determine the usage state of the hook 42, it transmits a determination necessity flag to the hook usage state determination unit 15. Then, after receiving the determination necessity flag, the hook usage state determination unit 15 performs processing to determine the usage state of the hook 42.

[0092] In this way, based on the judgment necessity information on whether it is necessary to judge the usage status of the hook 42 and the hook use necessity information corresponding to the judgment necessity information, the work monitoring support device 100 can determine that the worker 43 is in a location where the use of the hook 42 is mandatory, and by configuring the device to analyze the usage status of the hook 42 when use of the hook 42 is required, it is possible to prevent the detection process from operating in a location where hooking judgment is not required and causing a false alarm, and it can be reliably determined whether the hook 42 has been hung on the hanging destination 48 only in locations where use of the hook 42 is required.

[0093] While the description has been given of transmitting whether or not it is necessary to determine the use status of the hook 42 using the hook use necessity information stored in the hook use necessity information storage unit 183 provided in the work monitoring support device 100, it is also possible to obtain hook use necessity information from an external source, such as another work monitoring support device 100, and transmit whether or not it is necessary to determine the use status of the hook 42 based on this hook use necessity information. By obtaining hook use necessity information from an external source, it is possible to accurately determine whether or not a location requires determining the use status of the hook 42 using hook use necessity information appropriate for each work site, and therefore it is possible to reliably determine whether or not the hook 42 has been hung on the destination 48 only in locations where use of the hook 42 is required.

[0094] Also, while an example has been shown in which a wearable camera is used as imaging unit 31 to capture images of identification code 55 and the surroundings of worker 43, a fixed camera fixed to a structure or the like at the work site may be used as imaging unit 31, and it may be determined that a determination of the use state of hook 42 is necessary if the determination necessity information is included within the range of the fixed camera's imaging field of view. By including determination necessity information such as identification code 55 in the fixed camera's field of view, determination processing is not performed on image 49 from a fixed camera installed in a location where determination of the use state of hook 42 is not necessary, thereby preventing false alarms from occurring when detection processing is performed in a location where hooking determination is not necessary, and making it possible to reliably determine whether hook 42 is hung on destination 48 only in locations where use of hook 42 is required.

[0095] Embodiment 6 A work monitoring support device 100 according to a sixth embodiment will be described with reference to the drawings. In the first embodiment, an example was described in which an action area 46 and a hook area 47 were extracted from an acquired image 49 without performing preprocessing on the image 49 acquired by the image acquisition unit 11. However, in the sixth embodiment, when an image 49 is acquired from an imaging unit 31 carried by a worker 43, preprocessing is performed to correct image blur of the worker 43. The following description will focus on the differences from the first embodiment, and descriptions of identical or corresponding parts will be omitted as appropriate.

[0096] As shown in Figure 20, when an image 49 is captured by the imaging unit 31 carried by a worker 43, if the imaging unit 31 moves with the movement of the worker's head, the worker's right hand 44 and left hand 45 holding the hook 42 appear to be moving in the image 49, even though they are not moving. Therefore, as shown in FIG. 21, the work monitoring support device 100 according to the sixth embodiment: The work monitoring support device 100 according to the fifth embodiment includes an image acquisition unit 11 that acquires an image 49 of the work site, an image blur correction unit 19 that corrects image blur in the acquired image 49, a motion area extraction unit 12 that extracts an action area 46 from the acquired image 49, a hook area extraction unit 13 that extracts a hook area 47 from the acquired image 49, a hook usage status analysis unit 14 that analyzes the motion information of the action area 46 and the movement information of the hook area 47, a hook usage status determination unit 15 that determines whether the hook 42 has been hooked on the hook destination 48, and a determination result output unit 16 that outputs the determination result to an outside of the work monitoring support device 100. After correcting the image blur, the motion area 46 is extracted, the hook area 47 is extracted, and the hooking is determined.

[0097] When the image acquisition unit 11 acquires an image 49 from the imaging unit 31 carried by the worker 43, the image blur correction unit 19 corrects image blur caused by the movement of the worker 43 and transmits the corrected image to the motion area extraction unit 12 and the hook area extraction unit 13.

[0098] For example, the following steps (1) to (3) show the process of image stabilization. (1) Blur extraction: Image blur is extracted from two frames using image processing techniques such as feature point matching, which matches the features extracted from each frame, and optical flow, which compares the two frames to detect object movement and display the speed as a vector. (2) Motion correction: The shift amount is calculated based on the extracted image blur, and the pixels are shifted by the shift amount. (3) Frame reconstruction: The gaps caused by the shift are filled in using pixel interpolation techniques such as bilinear interpolation. By performing shake correction, even if the worker 43 moves the imaging unit 31 carried in the helmet without moving his / her hands, an image 49 can be obtained in which the hands of the worker 43 appear to be stationary.

[0099] In this way, by correcting image blur using the image blur correction unit 19, even if changes occur in the image 49 due to head movement of the worker 43 carrying the imaging unit 31 on his helmet, it is possible to simplify the motion analysis and prevent a decrease in judgment accuracy due to image blur, making it possible to reliably determine whether the hook 42 has been hung on the hanging point 48.

[0100] Although various exemplary embodiments are described in this disclosure, the various features, aspects, and functions described in one or more embodiments are not limited to the application of a particular embodiment, but may be applied to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are contemplated within the scope of the technology disclosed herein. For example, this includes cases where at least one component is modified, added, or omitted, and even cases where at least one component is extracted and combined with components of another embodiment. [Explanation of symbols]

[0101] 11 Image acquisition unit, 12 Motion area extraction unit, 13 Hook area extraction unit, 14 Hook usage status analysis unit, 15 Hook usage state determination unit, 16 Determination result output unit, 17 Hook state identification unit, 18 Determination necessity determination unit, 19 Image blur correction unit, 20 Correct label input unit, 31 Imaging unit, 32 Alarm unit, 41 Safety belt, 42 Hook, 43 Worker, 44 Right hand, 45 Left hand, 46 Motion area, 47 Hook area, 48 Hanging point, 49 Image, 50 Processor, 51 Storage device, 52 Communication I / F, 53 Recording medium, 54 Program, 55 Identification code, 56 Ascent detection measurement unit, 100 Work monitoring support device, 151 Hook state learning unit, 152 Learning data acquisition unit, 153 Model generation unit, 154 Reference learned model storage unit, 155 Analysis data acquisition unit, 156 hook hook determination unit, 171 hook area acquisition unit, 172 identification processing unit, 173 state learned model storage unit, 181 judgment necessity information acquisition unit, 182 decision processing unit, 183 hook use necessity information storage unit, 201 worker terminal, 202 management terminal, 301 work management device, 1000 work monitoring support system

Claims

1. an image acquisition unit that acquires an image of a work site; a motion area extraction unit that extracts, from frames constituting the video, a range of motion of the joints of the worker's fingers included in the frames as a motion area; a hook region extraction unit that extracts, from the frames constituting the video, a range including a hook connected to a safety belt worn by the worker included in the frame as a hook region; a hook usage status analysis unit that receives the frame in which the motion area and the hook area are extracted, and analyzes motion information related to the positions of the finger joints obtained from the motion area and movement information obtained from the hook area as the usage status of the hook; a hook use state determination unit that determines whether the worker has grasped the hook based on the positional relationship between the joints of the fingers, and determines at least one of the use states of the hook, whether the hook has been hung on a hanging point or whether the hook has been removed from the hanging point, based on the analyzed use state; a determination result output unit that outputs a determination result of the hook use state determination unit; A work monitoring support device comprising:

2. The work monitoring support device according to claim 1 , wherein the hook usage status analysis unit analyzes the coordinates of the finger joints as the motion information, and analyzes the center coordinates of the hook area or the coordinates of a specific position as the movement information.

3. The work monitoring support device according to claim 1, wherein the hook usage status analysis unit analyzes, as the movement information, a movement trajectory based on changes in the coordinates of the finger joints over time, and analyzes, as the movement information, a movement trajectory based on changes in the center coordinates of the hook area or the coordinates of a specific position over time.

4. The work monitoring support device of claim 1, wherein the hook usage status analysis unit analyzes a body movement vector, which is the direction of movement of the movement area, as the movement information, and analyzes a hook movement vector, which is the direction of movement of the hook area, as the movement information.

5. an image acquisition unit that acquires an image of a work site; a motion area extraction unit that extracts, from frames constituting the video, a range in which at least a part of the worker's body included in the frames moves as a motion area; a hook region extraction unit that extracts, from the frames constituting the video, a range including a hook connected to a safety belt worn by the worker included in the frame as a hook region; a hook usage status analysis unit that receives the frame from which the action area and the hook area are extracted, and analyzes the action information acquired from the action area and the movement information acquired from the hook area as the hook usage status; a hook use state determination unit that determines, based on the analysis results of the operation information and the movement information, at least one of whether or not the worker has performed a hooking operation to hook the hook onto a target, and whether or not the worker has performed a hooking release operation to release the hook from the target; a determination result output unit that outputs a determination result of the hook use state determination unit; A work monitoring support device comprising:

6. The work monitoring support device described in claim 1, wherein the hook usage status determination unit determines, based on the analysis results of the operation information and the movement information, at least one of whether the worker has performed a hooking operation to hook the hook onto the hanging point, and whether the worker has performed a hook removal operation to remove the hook from the hanging point.

7. The hook usage state determination unit, based on the analyzed usage state, Whether the hooking operation has been performed or not; Whether or not a hook confirmation operation has been performed to confirm the hook engagement state, and determining whether a hook release action has been performed to release at least a part of the worker's body from the hook; 7. The work monitoring support device according to claim 6, wherein it is determined that the hook has been hooked on the target when all of the hooking operation, the hook checking operation, and the hook releasing operation have been performed.

8. The hook usage state determination unit, based on the analyzed usage state, When the distance between the region of the action and the region of the hook is less than the set grippable distance at which the hook can be gripped by the worker, it is determined whether or not a hook preparation action has been performed in which the hook is gripped by the worker; 8. The work monitoring support device according to claim 7, wherein the device determines that the hook has been hooked on the target when all of the hook preparation operation, the hooking operation, the hook confirmation operation, and the hook releasing operation have been performed.

9. The hook usage state determination unit, based on the analyzed usage state, When the distance between the region of the action and the region of the hook is less than the set grippable distance at which the hook can be gripped by the worker, it is determined whether or not a hook preparation action in which the hook is gripped by the worker has been performed, and whether or not the hook release action has been performed; 7. The work monitoring support device according to claim 6, wherein it is determined that the hook has been released from the hook point when both the hook preparation operation and the hook release operation have been performed.

10. the hook usage status analysis unit analyzes, as the motion information, a body motion vector that is a direction of motion of the motion area, and analyzes, as the movement information, a hook movement vector that is a direction of movement of the hook area; The work monitoring support device described in claim 7 or claim 9, wherein the hook usage status determination unit determines that at least one of the hooking operation and the hook unhooking operation has been performed when the distance between the position of the operation area and the position of the hook area is less than the set graspable distance at which the hook can be grasped by the worker, and the similarity between the time change of the body movement vector and the time change of the hook movement vector is equal to or greater than a set similarity.

11. The hook use state determination unit The work monitoring support device described in claim 7, wherein the hook confirmation operation is determined to have been performed when the position of the action area and the position of the hook area are within a confirmable distance area in which it can be confirmed that the set hook is hung on the hanging point during a confirmable time in which it can be confirmed that the set hook is hung on the hanging point.

12. The hook use state determination unit The work monitoring support device according to claim 7, wherein the hook release operation is determined to have been performed when the position of the hook area is within a hook engagement area where the hook can be engaged, and the distance between the position of the action area and the position of the hook area is equal to or greater than the grippable distance at which the set hook can be grasped by the worker.

13. The hook use state determination unit a learning data acquisition unit that acquires learning data including the motion information, the movement information, and a correct label of the use state of the hook; a model generation unit that generates a reference trained model for determining whether the worker has hooked the hook to the target point based on the operation information and the movement information using the learning data; and 2. The work monitoring support device according to claim 1, further comprising:

14. The hook use state determination unit an analysis data acquisition unit that acquires the analyzed operation information and movement information from the hook usage status analysis unit; a hooking determination unit that determines whether the hook has been hooked to the hooking point by the worker based on the operation information and the movement information using the reference trained model, and outputs the determination result to the determination result output unit; The work monitoring support device according to claim 13, further comprising:

15. a hook region acquisition unit that acquires the hook region from the hook region extraction unit; an identification processing unit that identifies the open / closed state or the hooked state of the hook from the image including the hook region using a state-trained model for identifying the open / closed state or the hooked state of the hook from the hook region, The work monitoring support device according to claim 1 , further comprising a hook state identification unit that outputs the identification result to the determination result output unit.

16. a determination necessity information acquisition unit that receives determination necessity information including at least one of the image acquired by the image acquisition unit, the image of the worker ascending, and ascent detection data measured by an ascent detection measurement unit held by the worker; and a determination processing unit that determines whether or not it is necessary to determine the usage state of the hook based on the determination necessity information and hook usage necessity information corresponding to the determination necessity information, The work monitoring support device according to claim 1 , further comprising a determination necessity determining unit that transmits a determined result to the hook use state determining unit.

17. 2. The work monitoring support device according to claim 1, further comprising an image blur correction unit that corrects image blur caused by the movement of the worker when the image acquisition unit acquires the image from an imaging unit carried by the worker, and outputs the corrected image to the motion area extraction unit and the hook area extraction unit.

18. an imaging unit that images the work site; The work monitoring support device according to claim 1 or 5; an alarm unit that issues at least one of a warning popup, light, sound, and vibration based on the determination result of the work monitoring support device; A work monitoring support system equipped with:

19. An imaging unit comprising a plurality of cameras fixed to the body of a worker working at a work site in order to capture an image of the body of the worker, the imaging unit imaging the work site; a motion area extraction unit that extracts, from frames constituting the video, a range in which at least a part of the worker's body included in the frames moves as a motion area; a hook region extraction unit that extracts, from the frames constituting the video, a range including a hook connected to a safety belt worn by the worker included in the frame as a hook region; a hook usage status analysis unit that receives the frame from which the action area and the hook area are extracted, and analyzes the action information acquired from the action area and the movement information acquired from the hook area as the hook usage status; a hook use state determination unit that determines the use state of the hook based on the analyzed use state, at least one of whether the hook is hung on a hanging point and whether the hook is removed from the hanging point; and a determination result output unit that outputs a determination result of the hook use state determination unit; an alarm unit that issues at least one of a warning popup, light, sound, and vibration based on the determination result; Equipped with The plurality of cameras include a right camera that photographs the worker's right arm and a left camera that photographs the worker's left arm.

20. an image acquisition unit that acquires an image of a work site; a motion area extraction unit that extracts a range of motion of the joints of the worker's fingers included in the video as a motion area; a hook region extraction unit that extracts, as a hook region, a range of a hook connected to a safety belt worn by the worker and included in the video; a hook usage status analysis unit that analyzes motion information related to the positions of the finger joints of the worker included in the motion area extracted by the motion area extraction unit and movement information including at least one of the position and movement direction of the hook included in the hook area extracted by the hook area extraction unit, and analyzes the relationship between the motion information and the movement information as a usage status; a hook use state determination unit that determines whether the worker has grasped the hook based on the positional relationship between the joints of the fingers, and determines at least one of the use states of the hook, whether the hook has been hung on a hanging point or whether the hook has been removed from the hanging point, based on the analyzed use state; a determination result output unit that outputs a determination result of the hook use state determination unit; A work monitoring support terminal equipped with the above.

21. acquiring an image of a work site; extracting a region of the joint movement of the worker's fingers included in the video; extracting a range of a hook connected to a safety belt worn by the worker included in the video as a hook area; analyzing motion information relating to the positions of the joints of the fingers of the worker included in the motion area and movement information including at least one of the position and movement direction of the hook included in the hook area, and analyzing the relationship between the motion information and the movement information as a usage situation; determining that the worker has grasped the hook based on the positional relationship between the positions of the joints of the fingers, and determining at least one of the usage states of the hook, whether or not the hook has been hung on a hanging point, and whether or not the hook has been removed from the hanging point, based on the analyzed usage state; outputting a result of the determination of the usage state of the hook; A work monitoring support method including:

Images