Behavior recording device, behavior recording system, and behavior recording method

Abstract

To solve the problem that a work state cannot be recorded before while personal information on a third person is protected.SOLUTION: An action recording device comprises: an image acquisition unit 3 which acquires an image of an instrument or appliance 2 fitted with a marker 1 having a color tone as a complementary color of a skin region of a person together with a part or the whole of a worker; a region extraction unit 4 which extracts a marker region from the acquired image; a defocusing processing unit 5 which performs defocusing processing on the acquired image; a work state determination unit 6 which determines work environment and / or work details as a work state of the worker; a defocusing processing-unrequired state determination unit 7 which determines whether the defocusing processing needs to be executed; and an image composition unit 8 which composes an image after the defocusing processing and an image of the extracted marker region.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an action recording device, an action recording system, and an action recording method for recording the actions of an operator.

Background Art

[0002] An elevator is a device that transports people vertically within a building and is an essential device in urban spaces. Regular maintenance inspections are carried out to ensure the safe operation of the elevator. For example, the part where users board and alight from the elevator is called a car, and maintenance inspection work is performed not only on the inside of the car but also on the equipment that controls the movement of the car within the hoistway where the car moves up and down. In such maintenance inspection work, there is a need to record the work content in video for the purpose of confirming the work content and passing on skills.

[0003] On the other hand, in the maintenance work of elevators, before and after the work, the operator is in the hall area in front of the hoistway. When recording video at that time, if it is difficult to appropriately divide the area, there is a possibility that a person other than the operator (hereinafter referred to as a "third party") may be captured in a part of the imaging range of the camera. Since such information other than the operator is unnecessary for the above purposes, it is desirable to prevent individuals from being identified.

[0004] As a technology related to this, Patent Document 1 discloses a system for monitoring the working state of an operator. In this system, the position of the operator is identified and it is monitored whether the state of the safety belt is appropriate.

[0005] Also, Patent Document 2 discloses a device that can convert an image into an image that does not include a subject or convert only a specific area including the subject into an image different from the original image according to the intention of the subject in the image.

Prior Art Documents

[0007] Patent Document 1, mentioned above, monitors workers and safety harnesses to ensure that safety harnesses are used appropriately. While workers' actions related to safety harnesses are recorded, a variety of tasks exist in maintenance work within elevator shafts. For example, the location where safety harnesses are attached changes depending on the work being done in the elevator shaft. Therefore, in order to determine whether safety harnesses are attached in the appropriate position for each task, it is necessary to simultaneously understand the condition of the safety harness itself and the condition of the elevator to which the safety harness is attached. However, the elevator to which the safety harness is attached is customer property, and modifications to understand the condition of the elevator to which the safety harness is attached are not easy.

[0008] Patent Document 2 described above involves placing a marker on the subject to be protected, and processing the image so that the subject is protected when the marker is detected in the image. However, during elevator maintenance work, people other than the workers who are concerned about being captured on video may not know that video is being recorded at that location. Furthermore, even if they were aware that video was being recorded, it would be troublesome for them to attach a protective marker to themselves after filming. For this reason, it is desirable to have video in which people other than workers can be identified without any additional action.

[0009] Given the circumstances described above, there was a need for a method that could record work status while protecting the personal information of third parties. [Means for solving the problem]

[0010] To solve the above problems, an action recording device according to one aspect of the present invention is an action recording device that photographs the actions of a worker and records the photographed images, comprising: an image acquisition unit that acquires images of a part or the whole of the worker, along with an instrument or device to which a marker having a hue that is the complementary color of the human skin area is attached; a region extraction unit that extracts a marker region corresponding to the marker from the image acquired by the image acquisition unit; a blurring processing unit that performs blurring on the image acquired by the image acquisition unit; a work state determination unit that determines the work environment and / or work content as the worker's work state; and a blurring unnecessary state determination unit that determines whether or not blurring processing is necessary. The action recording device also comprises an image synthesis unit that synthesizes the image that has been blurred by the blurring processing unit and the image of the marker region extracted by the marker region extraction unit; and a recording unit that records the image output from the image synthesis unit. [Effects of the Invention]

[0011] According to at least one aspect of the present invention, it is possible to record work status while protecting the personal information of third parties. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example of the functional configuration of an action recording device according to the first embodiment of the present invention. [Figure 2] This figure shows an example of the hardware configuration of an action recording device according to the first embodiment of the present invention. [Figure 3] This figure shows an example of an image taken during maintenance work on an elevator targeted by the first embodiment of the present invention. [Figure 4] This figure shows an example of blurring applied to an image taken during maintenance work on an elevator, which is the subject of the first embodiment of the present invention. [Figure 5] This figure shows another example of an image taken during maintenance work on an elevator targeted in the first embodiment of the present invention. [Figure 6]This is a diagram showing another example in which blurring processing is applied to an image during maintenance work on an elevator targeted in the first embodiment of the present invention. [Figure 7] This is a flowchart showing an example of the procedure of marker area extraction processing according to the first embodiment of the present invention. [Figure 8] This is a flowchart showing an example of the procedure of blurring processing unnecessary state determination processing according to the first embodiment of the present invention. [Figure 9] This is a diagram showing an example of the functional configuration of an action recording device according to the second embodiment of the present invention. [Figure 10] This is a diagram showing an example of the configuration of an action recording system including an action recording device according to the third embodiment of the present invention.

Modes for Carrying Out the Invention

[0013] Hereinafter, examples of modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. In this specification and the accompanying drawings, the same reference numerals are given to the same components or components having substantially the same functions, and redundant descriptions are omitted.

[0014] <First Embodiment> First, an example of the internal configuration of an action recording device according to the first embodiment of the present invention will be described with reference to FIG. 1. In this embodiment, an image processing device having a photographing function is assumed as the action recording device. For example, as the image processing device, a smartphone, a wearable terminal, or the like having at least a camera can be used. When performing work on an elevator, the operator adjusts the position so that what he / she sees can be understood, attaches the action recording device 10 to the head or near the head, and turns on the photographing function to perform the work. In this specification, an elevator is taken as an example of the elevator, but other types of elevators such as escalators may also be used.

[0015] FIG. 1 is a diagram showing an example of the functional configuration of an action recording device according to the first embodiment. The action recording device 10 shown in FIG. 1 includes an image acquisition unit 3, a marker region extraction unit 4, a blurring processing unit 5, a working state determination unit 6, a blurring processing unnecessary state determination unit 7, an image synthesis unit 8, and a recording unit 9.

[0016] A marker 1 is attached to the tool / equipment 2 used by the operator. This marker 1 is imaged by the image acquisition unit 3. The marker 1 is a retroreflective tape using the complementary color of the color of the human skin region (for example, the hue from blue to green), or a member having the same visibility, and is attached to a part or the whole of the tool or equipment used by the operator.

[0017] For example, the equipment with the marker 1 attached is the hook of the safety belt carried by the operator and / or the fixed part on the lift car or in the hoistway where the hook of the safety belt is hung. Also, examples of the equipment with the marker 1 attached include work clothes, gloves, helmets, etc. worn by the operator.

[0018] The image acquisition unit 3 is an image sensor such as a CMOS sensor capable of imaging a subject. The image acquisition unit 3 is set so that what the operator is looking at fits within the angle of view. The image acquisition unit 3 sends the image (video or still image) acquired by imaging to the marker region extraction unit 4, the blurring processing unit 5, the working state determination unit 6, the blurring processing unnecessary state determination unit 7, and the image synthesis unit 8. Also, in the image acquisition unit 3, at the same time as imaging the image, information on the position of the operator and the direction in which the action recording device 10 is facing is acquired. In the present embodiment, what the operator is looking at is assumed to be a part or the whole of the tool / equipment 2 used by the operator and the objects existing in its background and periphery. The image acquisition unit 3 corresponds to a photographing function.

[0019] The marker region extraction unit 4 extracts the region corresponding to the marker 1 from the image obtained by the image acquisition unit 3. The marker region extraction unit 4 sends information on the extracted marker region to the image synthesis unit 8.

[0020] The blurring processing unit 5 applies a blurring process to the entire image acquired by the image acquisition unit 3. The blurring processing unit 5 then sends the blurred image to the image synthesis unit 8. This blurring process is an image processing method that prevents the identification of individuals from the image when a human face is included in the image. For example, one example of blurring is to filter areas of hue that do not affect marker detection (for example, red-yellow tones corresponding to human skin) using an HSV threshold and perform an averaging mask process. HSV stands for hue (H), saturation (S), and lightness (V).

[0021] When applying blurring using a smoothing filter, the kernel size of the smoothing filter should be set based on the distance at which ordinary passersby other than the worker can approach the worker wearing the camera (action recording device 10). For example, it is advisable to set the kernel size of the smoothing filter considering the camera's resolution so that individuals cannot be identified even when passersby are at their closest point to the worker.

[0022] The work status determination unit 6 determines the work environment and / or work content as the worker's work status. For example, the work status determination unit 6 determines from the image whether the worker is working (for example, where the worker is located) and, if working, which direction the worker is facing relative to the elevator. The work status determination unit 6 sends the determination result regarding the work status as work status information to the blur-free state determination unit 7.

[0023] The blurring unnecessary state determination unit 7 determines whether blurring is unnecessary based on the brightness and saturation of the image acquired by the image acquisition unit 3 and the work state information determined by the work state determination unit 6. The determination result of the blurring unnecessary state determination unit 7 is input to the image synthesis unit 8.

[0024] The image synthesis unit 8 synthesizes the image that has been blurred by the blurring processing unit 5 and the image that has been processed by the marker region extraction unit 4. The image synthesis unit 8 performs image synthesis based on the determination result of the blurring unnecessary state determination unit 7. If blurring is determined to be "necessary", the image synthesis unit 8 extracts the image (original image) acquired by the image acquisition unit 3 corresponding to the marker region obtained by the marker region extraction unit 4 from the image that has been blurred by the blurring processing unit 5, synthesizes an image in which the extracted image replaces the corresponding region of the image processed by the blurring processing unit 5, and outputs it to the recording unit 9.

[0025] On the other hand, if blurring is deemed "unnecessary," the image synthesis unit 8 does not perform image synthesis and outputs the image acquired by the image acquisition unit 3 directly to the recording unit 9. The data of the image output from the image synthesis unit 8 is then recorded in the recording unit 9.

[0026] [Hardware configuration of the activity recording device] Next, the hardware configuration of the behavior recording device 10 according to this embodiment will be described with reference to Figure 2.

[0027] Figure 2 shows an example of the hardware configuration of the activity recording device 10. The activity recording device 10 includes, as hardware, a CPU (Central Processing Unit) 21, ROM (Read Only Memory) 22, RAM (Random Access Memory) 23, non-volatile storage 24, a camera 25, a microphone 26, a positioning sensor 27, a direction sensor 28, and a network interface 29. Each part of the activity recording device 10 is connected to each other via a system bus, enabling data transmission and reception.

[0028] The CPU 21, ROM 22, RAM 23, and non-volatile storage 24 constitute a control unit. This control unit is used as an example of a computer that controls the operation of the behavior recording device 10. The CPU 21 reads the software program that realizes each function of the behavior recording device according to this embodiment from the ROM 22, loads the program into the RAM 23, and executes it.

[0029] ROM22 is used as an example of non-volatile memory (recording medium). ROM22 stores the OS (Operating System), various parameters, and programs for operating the action recording device 10. ROM22 may also be a rewritable non-volatile memory such as EEPROM (Electrically Erasable and Programmable Read Only Memory). RAM23 is used as an example of volatile memory. Variables and parameters generated during the calculation process of the CPU21 are temporarily written to RAM23. Instead of the CPU21, other processors such as an MPU (Micro Processing Unit) may be used as the calculation processing unit.

[0030] Non-volatile storage 24 is an example of a recording medium that can store data used by a program or data obtained by executing a program. For example, information such as thresholds used in the judgment process described later can be stored in non-volatile storage 24. Alternatively, the OS (Operating System) or programs executed by the CPU 21 may be recorded in non-volatile storage 24. Examples of non-volatile storage 24 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), disk media that utilize light or magnetism, or semiconductor memory cards.

[0031] The camera 25 consists of an optical system including multiple lenses and an image sensor. The microphone 26 converts sound into electrical signals.

[0032] The positioning sensor 27 acquires location information of the positioning sensor 27 itself, i.e., the worker wearing the activity recording device 10. The positioning sensor 27 can utilize the Global Navigation Satellite System (GNSS). For example, a GPS (Global Positioning System) receiver is an example of a positioning sensor that utilizes the Global Navigation Satellite System.

[0033] The direction sensor 28 acquires the direction that the direction sensor 28 itself, i.e., the worker wearing the action recording device 10, is facing. A magnetic sensor, an acceleration sensor, or the like can be used as the direction sensor 28.

[0034] The network interface 29 is a communication device that can send and receive various types of data to and from external devices via a communication network.

[0035] The activity recording device 10 may be assumed to include a display unit and an operation unit, which are not shown in the diagram. The display unit is a monitor such as a liquid crystal display, and displays the GUI screen and the results of processing performed by the CPU 21. The operation unit generates input signals in response to user operations and outputs them to the CPU 21. Users can input information and instructions by operating the operation unit. The display unit and the operation unit may be integrated as a touch panel.

[0036] [Example of an image during maintenance work (1)] Figures 3 and 4 show schematic diagrams of the images obtained by the image synthesis process described above. Figure 3 shows an example of an image taken during maintenance work on the elevator in question. Figure 4 shows an example of blurring applied to an image taken during maintenance work on the elevator in question.

[0037] Figure 3 shows an example of an image 30 taken by a camera (action recording device 10) mounted on the worker's head, showing the elevator shaft 32 with the door 31 open. The image shows the elevator shaft 32 and the top of the elevator car 33, as well as the safety harness hook 35 held by the worker 34. A green retroreflective tape 36 is wrapped around the safety harness hook 35, and this hook 35 is attached to a predetermined part (fixing part 37) of the elevator car 33. In such a scene, when the image is composited by marker area extraction and blurring, everything except the marker area, i.e., the part of the hook 35 not wrapped with the retroreflective tape 36, is subject to blurring. Therefore, as shown in Figure 4, the area indicated by hatching in image 30 is subject to blurring.

[0038] [Example of an image during maintenance work (2)] Figure 5 shows another example of images taken during maintenance work on the elevator in question. Figure 6 shows another example of blurring applied to an image taken during maintenance work on the elevator in question.

[0039] As a similar example, Figures 5 and 6 show schematic diagrams of images 40 obtained when a worker 34 is on the elevator car and facing the door 31. During work, ordinary passersby 43 cannot approach the vicinity of the door 31, which is inside the work boundary 41 defined by a fence, etc. (closer to the worker). Therefore, if there are ordinary passersby 43, they are located at a certain distance from the door 31. Since there is a possibility that other workers 42 are near the door 31, other workers 42 closer to the door 31 will appear larger in the image 40, while ordinary passersby 43 will appear smaller. Figure 5 shows this state, and since there is nothing that can be considered a marker in the image 40 of Figure 5, the entire area of ​​the image 40 is blurred. In Figure 6, the area to be blurred is shown with hatching, but the entire area of ​​the image 40 is subject to blurring.

[0040] Considering the scenes shown in Figures 3 and 5, in the scene in Figure 3, worker 34 is facing the elevator shaft 32, and in this state, it is clearly impossible for ordinary passersby to be captured within the frame. Even if the image is blurred, the work can still be understood to some extent. However, if blurring is clearly unnecessary, an image without blurring is more suitable for recording the work.

[0041] On the other hand, in the scene shown in Figure 5, although work is in progress, worker 34 is facing the door 31. Therefore, when the door 31 is open, there is a possibility that ordinary passersby 43 may be captured in the image. In such cases, it is preferable to apply a blurring effect to the entire image.

[0042] Thus, even during work, there is a possibility that a third party may appear in the image. For this reason, it is generally preferable to blur the image even during work, and blurring should only be deemed unnecessary when it is determined that there is clearly no possibility of a third party appearing in the image. The blurring unnecessary state determination unit 7 is a functional block that determines the work state and, based on the work state, determines whether blurring is unnecessary or not.

[0043] [Marker region extraction] Here, the marker region extraction process by the marker region extraction unit 4 will be explained using Figure 7. Figure 7 is a flowchart showing an example of the procedure for marker region extraction processing by the marker region extraction unit 4.

[0044] First, the marker region extraction unit 4 performs HSV conversion on the image acquired by the image acquisition unit 3 (S1). In HSV conversion, the GBR component of the image is converted to an HSV component.

[0045] Next, the marker region extraction unit 4 creates a mask (S2) for extracting regions where the hue (H) falls within a predetermined range set for marker extraction (in this embodiment, hues from blue to green). This mask sets pixels whose hue falls within the predetermined range to "True" and others to "False".

[0046] Similarly, the marker region extraction unit 4 creates a mask for extracting regions where the saturation (S) is greater than or equal to a predetermined value (S3). Furthermore, the marker region extraction unit 4 creates a mask for extracting regions where the brightness (V) is greater than or equal to a predetermined value (S4).

[0047] The logical OR of the three masks created in this way is calculated to obtain the marker region extraction mask. The marker region extraction unit 4 uses the marker region extraction mask to extract the region that is the logical OR of the three masks from the image (original image) acquired by the image acquisition unit 3 (S5). After the processing in step S5 is completed, the marker region extraction process is terminated.

[0048] [Determining whether blurring is unnecessary] Next, the process of determining whether blurring is unnecessary by the blurring unnecessary state determination unit 7 will be explained using Figure 7. Figure 8 is a flowchart showing an example of the procedure for determining whether blurring is unnecessary by the blurring unnecessary state determination unit 7.

[0049] First, the blurring-free state determination unit 7 performs HSV conversion on the image output from the image acquisition unit 3 (S11).

[0050] Next, the blurring-free state determination unit 7 determines whether the elevator to be worked on is a panoramic elevator (S12). Whether the elevator to be worked on is a panoramic elevator is either stored in advance as elevator information by the blurring-free state determination unit 7 or stored in the ROM 22 or non-volatile storage 24. The elevator information may also be written to the program. If the elevator to be worked on is a panoramic elevator (YES determination in S12), the blurring-free state determination unit 7 determines that blurring is necessary (S19).

[0051] If the elevator being worked on is not a panoramic elevator (NO determination in S12), the blurring unnecessary state determination unit 7 then determines whether work was being performed inside the elevator shaft at the time the image was created (S13). If the work being performed at the time the image was created was not being performed inside the elevator shaft (NO determination in S13), the blurring unnecessary state determination unit 7 determines that blurring is necessary (S19).

[0052] If the work performed when creating the image is performed inside the elevator shaft (YES determination in S13), the blurring unnecessary state determination unit 7 determines whether the amount of horizontal movement of the camera's shooting direction is less than a predetermined angle from the direction at the start of the work (S14). Here, the start of the work is assumed to be the timing when the elevator shaft door is opened. The illumination inside the elevator shaft is low, and when the door is opened, the average brightness of the image decreases, so this large change in brightness is used to detect the start of the work.

[0053] When opening the door, the worker is always facing the door, so as long as they are facing that direction, ordinary passersby will not be captured by the camera. If the camera's shooting direction shifts horizontally by more than a predetermined angle from that point, passersby may be captured. For example, if the camera's shooting direction is 180 degrees in the opposite direction, it will be facing the hall. The extent to which the horizontal angle must be changed from the camera's shooting direction at the start of work before the hall may be captured depends on the camera's field of view. Therefore, the threshold value (predetermined angle) used in the judgment process in step S14 is set according to the camera's specifications, and this value is either stored in the blur-free state determination unit 7 or stored in the ROM 22 or non-volatile storage 24. The threshold value may also be written into the program.

[0054] In this embodiment, the blurring unnecessary state determination unit 7 uses the direction acquisition unit (direction sensor 28) to acquire the direction the worker was facing at the start of the work (camera shooting direction) and the direction the worker is facing after the start of the work (camera shooting direction). The blurring unnecessary state determination unit 7 then determines whether or not blurring is necessary based on the difference between the direction the worker was facing at the start of the work and the direction the worker is facing after the start of the work.

[0055] If the amount of horizontal movement of the camera's shooting direction is less than a predetermined angle from the starting direction of the work (YES determination in S14), there is no possibility of ordinary passersby appearing in the image, so the blurring unnecessary state determination unit 7 determines that blurring is unnecessary (S18).

[0056] On the other hand, if the camera's shooting direction is shifted by more than a predetermined angle horizontally from the camera's shooting direction at the start of the work (NO judgment in S14), there is a possibility that people on the hall side may be captured in the image, but the hall door is often closed during the work. In this case, even if the camera is facing the hall side, there is no possibility that ordinary passersby will be captured in the image.

[0057] To determine the above situation, the blur-free state determination unit 7 determines whether the average brightness of the image is less than or equal to a predetermined value A (S15). If the average brightness of the image exceeds the predetermined value A and the entire image is bright (NO determination in S15), the blur-free state determination unit 7 determines that blur processing is necessary because the holder door may be open and the camera shooting direction may be facing that way (S19).

[0058] If the average brightness of the image is less than or equal to a predetermined value A (YES judgment in S15), the blurring unnecessary state determination unit 7 then determines whether the average saturation of the image is less than or equal to a predetermined value B (S16). Since the saturation is higher on the hall side than inside the elevator shaft, if the overall saturation of the image is high (NO judgment in S16), the blurring unnecessary state determination unit 7 determines that blurring is necessary because the hall door may be open and the camera shooting direction may be facing that way (S19).

[0059] If the average brightness and average saturation of the image are less than or equal to predetermined values ​​A and B, respectively (YES determination in S16), the blurring unnecessary state determination unit 7 then determines whether the size of the area in the image where the brightness exceeds predetermined value C is less than or equal to predetermined value D (S17). This determination process assumes a marker area. If the size of the area in the image where the brightness exceeds predetermined value C is greater than predetermined value D (NO determination in S17), the blurring unnecessary state determination unit 7 determines that blurring is necessary (S19).

[0060] This is because elevators installed in apartment buildings and other similar buildings sometimes have glass panels in the elevator doors. The glass panels in the elevator doors allow people inside the car to be seen from the hall side when ascending or descending. When workers are working on top of the elevator car, even when the elevator doors are closed, there is a possibility that ordinary passersby may be reflected in the image through the glass panels in the elevator doors. Therefore, in step S17, the system is configured to make a determination from the perspective of how much of the pixel area size is acceptable (for example, by setting a predetermined value D) in the image where the brightness is greater than a predetermined value C, taking into account the size of the glass panels in the elevator doors.

[0061] Through this process, if the above determination based on brightness and saturation does not determine that blurring is necessary (YES in S17), the blurring unnecessary state determination unit 7 determines that blurring is unnecessary (S18).

[0062] Thus, in this embodiment, the blurring unnecessary state determination unit 7 determines whether the brightness and / or saturation of a part or the whole of the image obtained by the image acquisition unit 3 is below a predetermined value during the operator's work, and determines that blurring is unnecessary if it is below the predetermined value.

[0063] In the blurring-free state determination process shown in Figure 8, the target for determining that blurring is unnecessary is assumed to be a typical elevator shaft. Step S12 first determines that the target is not a panoramic elevator with glass-covered elevator shaft walls. Furthermore, since elevator shafts with panoramic features are often painted in bright colors, the target is initially determined not to be a panoramic elevator.

[0064] In the embodiment described above, blurring is performed using a smoothing filter, but other image processing methods are acceptable as long as they prevent individuals from being identified when a third party is captured in the image. However, in this embodiment, since all areas except those excluded by the marker are processed, a coarse mosaic effect, for example, would make it difficult to even determine what kind of work is being done. Therefore, the image processing must be such that the content and procedure of the work can be understood to some extent, but people's faces become unrecognizable. Other processing methods are acceptable as long as they satisfy this condition.

[0065] Furthermore, in this embodiment, a marker was attached to the device / equipment 2 held by the worker, but this marker can be anything as long as it has a blue to green hue. For example, multiple spherical markers may be attached to the device or equipment, or the device or equipment may be colored by painting. Alternatively, a film or cover that retains the shape of the device or equipment to a certain extent may be used so that the device or equipment can be detected. In addition, the equipment may include the worker's work clothes, gloves, helmet, etc., and these may also be made of materials with a blue to green hue.

[0066] Furthermore, in this embodiment, the marker was attached to the equipment / gear 2 held by the worker, but the marker may also be attached to the equipment on the elevator side. For example, on equipment on the elevator car or equipment in the elevator shaft, the parts to which the safety harness hook is attached may be marked with a blue to green hue, or these objects may be painted or otherwise made to have a blue to green hue.

[0067] As described above, the behavior recording device (behavior recording device 10) according to the first embodiment is a behavior recording device that photographs the actions of a worker and records the photographed images. The behavior recording device 10 includes an image acquisition unit 3 that acquires images of a part or the whole of the worker, along with equipment or devices to which markers having a hue that is the complementary color of the human skin area are attached; a marker area extraction unit 4 that extracts marker areas corresponding to the markers from the images acquired by the image acquisition unit 3; a blurring processing unit 5 that performs blurring on the images acquired by the image acquisition unit 3; a work state determination unit 6 that determines the work environment and / or work content as the worker's work state; and a blurring unnecessary state determination unit 7 that determines whether or not blurring processing is necessary. The behavior recording device 10 also includes an image synthesis unit 8 that synthesizes the image that has been blurred by the blurring processing unit 5 and the image of the marker area extracted by the marker area extraction unit 4; and a recording unit 9 that records the image output from the image synthesis unit 8.

[0068] According to the behavior recording device of this embodiment with the above configuration, it is possible to identify unsafe behavior by using images in which human faces cannot be identified, and by confirming the equipment / gear and the work performed using them based on markers attached to the equipment / gear used by the worker. Furthermore, with this behavior recording device, if it is determined that blurring is unnecessary because there is clearly no possibility of a third party appearing in the image, such as when working in an elevator shaft, based on the worker's work status, blurring can be omitted. Therefore, it is possible to easily record work status while protecting personal information.

[0069] <Second Embodiment> Next, the functional configuration of the behavior recording device according to the second embodiment of the present invention will be described with reference to Figure 9. In the following, the differences between the second embodiment and the first embodiment will be described in detail, and redundant explanations will be omitted.

[0070] Figure 9 shows an example of the functional configuration of the behavior recording device according to the second embodiment. The activity recording device 10A shown in Figure 9 is configured by adding a work status information acquisition unit 51, a work start detection unit 52, and a work end detection unit 53 to the activity recording device 10 according to the first embodiment. Specifically, the activity recording device 10A includes an image acquisition unit 3, a marker area extraction unit 4, a blurring processing unit 5, a work status determination unit 6, a blurring unnecessary state determination unit 7, an image synthesis unit 8, and a recording unit 9, as well as a work status information acquisition unit 51, a work start detection unit 52, and a work end detection unit 53.

[0071] The image acquisition unit 3 sends the image (video or still image) acquired by imaging to the marker area extraction unit 4, the blurring processing unit 5, the work state determination unit 6, the blurring unnecessary state determination unit 7, and the image synthesis unit 8, as well as to the work start detection unit 52 and the work end detection unit 53.

[0072] The work status information acquisition unit 51 acquires information (work status information) necessary for the work status determination unit 6 to determine the worker's work status, and provides this information to the work status determination unit 6. For example, the work status information acquisition unit 51 can be the microphone 26, positioning sensor 27, or direction sensor 28 shown in Figure 2. This allows the work status determination unit 6 to determine the work status using information other than the image acquired by the image acquisition unit 3.

[0073] For example, if a microphone 26 (an example of an audio recording unit) is applied to the work status information acquisition unit 51, the work status determination unit 6 can determine the work status based on the image acquired by the image acquisition unit 3 and / or the signal from the microphone 26. For example, if the hoist door 31 opens and the brightness of the captured image suddenly increases, it can be inferred that the worker is about to start work. In addition, by detecting a worker's call such as "I will now start working in the elevator shaft" from the audio picked up by the microphone 26, the work status, such as the start of work, can be directly grasped.

[0074] Furthermore, when a positioning sensor 27 (an example of a location information acquisition unit) is applied to the work status information acquisition unit 51, the work status determination unit 6 can determine the work status based on the location information acquired by the positioning sensor 27. For example, from the location information acquired by the positioning sensor 27, it is possible to infer the work status of the worker, such as whether the worker is in the hall, in the elevator shaft, or on the elevator car.

[0075] Furthermore, when a direction sensor 28 (an example of a direction acquisition unit) is applied to the work status information acquisition unit 51, the work status determination unit 6 can determine the work status based on the position information acquired by the positioning sensor 27 and the direction the worker is facing, as acquired by the direction sensor 28. If the worker's position and the direction the worker is facing (camera shooting direction) are known, it is possible to infer the worker's work status because it is determined what equipment of the elevator is in the direction the worker is facing.

[0076] The work start detection unit 52 detects when an worker starts work and sends the detection result to the blurring-free state determination unit 7. The work end detection unit 53 detects when an worker finishes work and sends the detection result to the blurring-free state determination unit 7.

[0077] For example, the work start detection unit 52 and the work end detection unit 53 detect the start of work based on the feature quantities (e.g., changes in brightness) of the image recorded by the recording unit 9. For example, when a worker is in the hall and the hall door 31 is opened, the brightness value of the image obtained by the image acquisition unit 3 suddenly decreases, so this change in brightness can be used to determine that work has started. In addition, by monitoring changes in the brightness of the image, it is also possible to detect, for example, whether a worker is in the hall of an elevator (e.g., before work starts or after work ends), or whether they are moving on the elevator car or inside the elevator shaft (during work).

[0078] Alternatively, the work start detection unit 52 and the work end detection unit 53 detect the start of work based on a signal indicating the status of a work / stop switch (not shown) operated by a worker during maintenance work on the elevator. Generally, the work / stop switch is installed in the ceiling of the elevator car, and a worker can manually switch the work start or work stop state of the switch. The signal indicating the status of the work / stop switch is transmitted to the activity recording device 10 via wireless communication through the elevator control device or the like.

[0079] <Third Embodiment> Next, the configuration of the behavior recording system including the behavior recording device according to the third embodiment of the present invention will be described with reference to Figure 10. In the following, the differences between the third embodiment and the first embodiment will be described in detail, and redundant explanations will be omitted.

[0080] Figure 10 shows an example configuration of an action recording system including an action recording device according to the third embodiment. The difference between the third embodiment and the first embodiment is that the functions of the action recording device 10 in the first embodiment are divided into an imaging device 60 and an action recording device 10B in the third embodiment. Images obtained by the image acquisition unit 3 of the imaging device 60 are recorded by the recording unit 61, and these images are transmitted to the action recording device 10B by the transmission processing unit 62 before further processing is carried out.

[0081] The behavior recording system 100 according to this embodiment consists of a shooting device 60 and a behavior recording device 10B that transmits and receives data to and from the shooting device 60 via a communication network. The shooting device 60 includes an image acquisition unit 3, a recording unit 61, and a transmission processing unit 62.

[0082] The image acquisition unit 3 sends the image (video or still image) acquired by imaging to the recording unit 61. The recording unit 61 has the same configuration and function as the recording unit 9 (see Figure 1). The transmission processing unit 62 performs the process of transmitting the image recorded by the recording unit 61 to the action recording device 10B via the communication network.

[0083] The behavior recording device 10B has the same configuration as the behavior recording device 10 shown in Figure 1, but with the image acquisition unit 3 replaced by the receiving processing unit 101. That is, the behavior recording device 10B comprises a receiving processing unit 101, a marker area extraction unit 4, a blurring processing unit 5, a work state determination unit 6, a blurring unnecessary state determination unit 7, an image synthesis unit 8, and a recording unit 9.

[0084] The receiving processing unit 101 processes the image transmitted from the imaging device 60. The receiving processing unit 101 then sends the received image to the marker area extraction unit 4, the blurring processing unit 5, the work state determination unit 6, the blurring unnecessary state determination unit 7, and the image synthesis unit 8.

[0085] As described above, the behavior recording system (behavior recording system 100) according to this embodiment is a behavior recording system that photographs the actions of a worker and records the images taken. The behavior recording system 100 includes an image acquisition unit 3 that acquires images of a part or the whole of the worker, along with equipment or devices to which markers having a hue that is the complementary color of the human skin area are attached; a first recording unit (recording unit 61) that records the images acquired by the image acquisition unit 3; and a transmission unit (transmission processing unit 62, reception processing unit 101) that transmits the images recorded by the first recording unit. The behavior recording system 100 also includes a marker area extraction unit 4 that extracts marker areas corresponding to markers from the images transmitted by the transmission unit; a blurring processing unit 5 that performs blurring processing on the images transmitted by the transmission unit; a work state determination unit 6 that determines the work environment and / or work content as the worker's work state; and a blurring processing unnecessary state determination unit 7 that determines whether or not blurring processing is necessary. Furthermore, the action recording system 100 includes an image synthesis unit 8 that synthesizes an image that has been blurred by the blur processing unit 5 and an image of a marker region extracted by the marker region extraction unit 4, and a second recording unit (recording unit 9) that records the image output from the image synthesis unit 8.

[0086] The camera carried by the worker for image acquisition (corresponding to the shooting device 60 in Figure 10) may have limitations in memory capacity, making long-term recording difficult, or it may not have high processing power. In such cases, the images temporarily stored in the camera are transmitted to another terminal device or server, where the subsequent blurring process is performed. The activity recording device 10B is an example of another terminal device or server. With this configuration, it becomes possible to record the worker's activities even if the shooting device 60 carried by the worker has limitations in processing power or memory capacity.

[0087] The hardware of the activity recording device 10B can be configured, for example, using the CPU 21, ROM 22, RAM 23, non-volatile storage 24, and network interface 29 shown in Figure 2. Furthermore, if a terminal device such as a smartphone is used as the activity recording device 10B, the activity recording device 10B may also be equipped with a microphone 26, a positioning sensor 27, and a direction sensor 28. The imaging device 60 can be configured using at least a camera 25, CPU 21, ROM 22, RAM 23, non-volatile storage 24, and network interface 29.

[0088] Furthermore, the configuration of the behavior recording system by separating the functions of the behavior recording device 10 in the first embodiment into a shooting device and a behavior recording device, as shown in Figure 10, is also applicable to the second embodiment.

[0089] The present invention is not limited to the embodiments described above, and of course, various other applications and modifications can be taken as long as they do not depart from the gist of the invention as described in the claims. For example, the embodiments described above are described in detail and specifically in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those comprising all the components described. Furthermore, it is possible to replace a part of the configuration of one embodiment with a component of another embodiment. It is also possible to add a component of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, replace, or delete other components for a part of the configuration of each embodiment.

[0090] Furthermore, some or all of the above configurations, functions, and processing units may be implemented in hardware, for example, by designing them as integrated circuits. Broadly defined processor devices such as FPGAs (Field Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits) may be used as hardware.

[0091] Furthermore, in this specification, processing steps describing chronological processing include not only processing performed chronologically in the order described, but also processing that is not necessarily performed chronologically but is executed in parallel or individually (for example, processing by objects). In addition, the order of processing steps may be changed as long as it does not affect the processing result.

[0092] Furthermore, in the embodiments described above, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In practice, it can be assumed that almost all components are interconnected.

[0093] Furthermore, although the term "horizontal" is used in this specification, the term does not mean only strictly "horizontal," but rather includes "horizontal" in the strict sense, as well as "approximately horizontal," which is within the range in which it can perform its function. [Explanation of Symbols]

[0094] 1...Marker, 2...Device / Orthotics, 3...Image acquisition unit, 4...Marker area extraction unit, 5...Blur processing unit, 6...Work state determination unit, 7...Blur processing unnecessary state determination unit, 8...Image synthesis unit, 9...Recording unit, 10,10A,10B...Action recording device, 21...CPU, 24...Non-volatile storage (recording unit), 25...Camera (image acquisition unit), 26...Microphone (voice recording unit), 27...Positioning sensor (location information acquisition unit), 28...Direction sensor (direction acquisition unit), 29...Network interface (transmission processing unit, reception processing unit), 30...Image, 31...Hold door, 32...Elevator shaft, 33...Elevator car, 34...Worker, 35...Hook, 36...Marker, 37...Fixing unit, 40...Image, 41...Work boundary, 42...Pedestrian 43...Worker, 51...Work status information acquisition unit, 52...Work start detection unit, 53...Work end detection unit, 60...Photography device, 61...Recording unit, 62...Transmission processing unit, 101...Receiving processing unit

Claims

1. An action recording device that photographs the actions of a worker and records the images that are taken, An image acquisition unit acquires images of a part or all of the worker, along with equipment or devices to which markers with hues that are complementary to the human skin area are attached. A region extraction unit extracts a marker region corresponding to the marker from the image acquired by the image acquisition unit, A blurring processing unit performs blurring on the image acquired by the image acquisition unit, A work status determination unit that determines the work status of the worker, A blurring unnecessary state determination unit determines whether or not the blurring process should be performed based on the image acquired by the image acquisition unit and the work state determined by the work state determination unit, If it is determined that the blurring process described above is necessary, an image synthesis unit synthesizes the image that has been blurred by the blurring processing unit and the image of the marker region extracted by the region extraction unit. The system includes a recording unit that records the image output from the image synthesis unit. Activity recording device.

2. It includes an audio recording unit that converts sound into electrical signals, The work status determination unit determines the work status based on the image acquired by the image acquisition unit and / or the signal from the audio recording unit. The behavior recording device according to claim 1.

3. The unit includes a location information acquisition unit that acquires the location information of the worker, The work status determination unit determines the work status based on the location information acquired by the location information acquisition unit. The behavior recording device according to claim 1.

4. The unit includes a direction acquisition unit that acquires the direction the worker is facing, The work status determination unit determines the work status based on the location information acquired by the location information acquisition unit and the direction the worker is facing, which is acquired by the direction acquisition unit. The behavior recording device according to claim 3.

5. It includes a work start detection unit that detects the start of work by the worker, The work start detection unit detects the start of work based on a change in the brightness of the image recorded by the recording unit. The behavior recording device according to claim 1.

6. It includes a work start detection unit that detects the start of work by the worker, The work start detection unit detects the start of work based on a signal indicating the status of the work / stop switch operated by the worker during work on the elevator. The behavior recording device according to claim 1.

7. The direction acquisition unit acquires the direction the worker was facing at the start of the work and the direction the worker is facing after the start of the work. The blurring unnecessary state determination unit determines whether or not the blurring process is necessary based on the difference between the direction the worker was facing at the start of the work and the direction the worker is facing after the start of the work. The behavior recording device according to claim 4.

8. During the operator's work, the blurring unnecessary state determination unit determines that blurring is unnecessary if the brightness and / or saturation of part or the whole of the image obtained by the image acquisition unit is below a predetermined value. The behavior recording device according to claim 5 or 6.

9. The aforementioned marker has a hue ranging from blue to green, which is the complementary color to the human skin area. The behavior recording device according to claim 1.

10. An action recording system that photographs the actions of workers and records the images taken, An image acquisition unit acquires images of a part or all of the worker, along with equipment or devices to which markers with hues that are complementary to the human skin area are attached. A first recording unit that records the image acquired by the image acquisition unit, A transmission unit that transmits the image recorded in the first recording unit, A region extraction unit extracts a marker region corresponding to the marker from the image transmitted by the transmission unit, A blurring processing unit performs blurring on the image transmitted by the transmission unit, A work status determination unit that determines the work status of the worker, A blurring unnecessary state determination unit determines whether or not the blurring process should be performed based on the image acquired by the image acquisition unit and the work state determined by the work state determination unit, If it is determined that the blurring process described above is necessary, an image synthesis unit synthesizes the image that has been blurred by the blurring processing unit and the image of the marker region extracted by the region extraction unit. The system includes a second recording unit that records the image output from the image synthesis unit. Activity recording system.

11. A method for recording actions using an action recording device that photographs the actions of a worker and records the captured images, A process to acquire images of equipment or devices with markers attached that have a hue that is the complementary color to the human skin area, along with part or all of the worker. A process to extract the marker region corresponding to the marker from the acquired image, The acquired image is subjected to a blurring process, A process for determining the work status of the aforementioned worker, A process to determine whether or not the blurring process needs to be performed based on the acquired image and the work status, If it is determined that the blurring process described above is necessary, the process involves combining the blurred image and the extracted image of the marker region. Includes a process for recording the synthesized image. Method for recording activities.

Images