Discrimination device, discrimination method, and discrimination program

Abstract

The system calculates a score in real time indicating the degree to which work equipment or clothing is being worn correctly, thereby improving safety. [Solution] The discrimination device includes a processor, which extracts partial images representing each element of the equipment or clothing from an image of a worker wearing work equipment or clothing, and for each element, uses a discrimination model that takes the partial image of the element as input to determine whether or not the element is properly worn, to obtain the output of the discrimination model, calculates a score indicating the degree to which the work equipment or clothing is properly worn from the output of the discrimination model for each element, and outputs the score to the display unit, repeating this process.

Description

Technical Field

[0001] The disclosed technology relates to a discrimination device, a discrimination method, and a discrimination program.

Background Art

[0002] Skeleton detection technology and image processing technology can be easily utilized and procured on the market by applying publicly available technologies such as Openpose, Posenet, and OpenCV. Proposals for application technologies to standard equipment such as protective gear and clothing management for ensuring the safety of workers have been made.

[0003] There is also known a monitoring device for monitoring whether the appearance of a subject is in an appropriate state according to the working environment or the work content, which includes an image input unit for inputting an image of the subject, a detection unit for analyzing the image input by the image input unit to detect a predetermined part of the body of the subject, a state estimation unit for estimating the state of the predetermined part detected by the detection unit, an appropriate state acquisition unit for acquiring appropriate state information representing an appropriate state that the predetermined part should take according to the working environment or the work content, and a control unit for comparing the estimation result by the state estimation unit with the appropriate state information to determine whether the current state of the predetermined part of the subject is appropriate and performing control according to the determination result (for example, Patent Document 1).

[0004] There is also known a person management device for performing person management based on a person image of a photographed person, which includes image storage means for storing and managing a reference image for determining the posture of a person, determination means for determining whether the posture of a person conforms to the reference by analyzing and comparing the person image of the photographed person with the reference image, and control means for performing processing according to the determination result by the determination means (for example, Patent Document 2).

[0005] A person management device is known that includes an image input unit that inputs an image taken by a camera directed towards the worker's line of sight; an object recognition unit that image-recognizes a work object within the image area of ​​the image; a hand recognition unit that image-recognizes the worker's hand within the image area of ​​the image; an action determination unit that determines whether the work object and the worker's hand overlap within the image area of ​​the image; and, if the action determination unit determines that the work object and the worker's hand overlap, a condition determination unit that determines whether the worker has met predetermined conditions indicating that work safety is not ensured in relation to the work object (for example, Patent Document 3).

[0006] A work safety management system is known that comprises a terminal device and a server device, wherein the terminal device comprises an acquisition unit for acquiring images of workers, a first transmission unit for transmitting image data of the images acquired by the acquisition unit to the server device, a first receiving unit for receiving caution information transmitted by the server device, and a processing unit for notifying the worker of the caution information received by the first receiving unit; and the server device comprises a second receiving unit for receiving the image data transmitted by the terminal device, a determination unit for determining the safety of the worker based on the image data received by the second receiving unit, a creation unit for creating the caution information if the determination unit determines that there is a safety problem, and a second transmission unit for transmitting the caution information created by the creation unit to the terminal device (for example, Patent Document 4).

[0007] A person management device is known that checks whether a detection target is properly equipped based on a captured image of a person, and comprises: an image acquisition means for acquiring the captured image; a storage means for storing information of the detection target; a face detection means for detecting a person's face from the captured image acquired by the image acquisition means; a region setting means for setting a region in which the detection target should be located based on the coordinates of the face detected by the face detection means; a determination means for determining whether the detection target is located in the region set by the region setting means; and a control means for performing processing according to the determination result of the determination means (for example, Patent Document 5). [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2015-95209 [Patent Document 2] Japanese Patent Publication No. 2005-30861 [Patent Document 3] Japanese Patent Publication No. 2019-144861 [Patent Document 4] Japanese Patent Publication No. 2021-64103 [Patent Document 5] Japanese Patent Publication No. 2012-237569 [Overview of the project] [Problems that the invention aims to solve]

[0009] However, while conventional technologies are designed as management and monitoring technologies, in practice, due to variations in field of view, clothing, and tools, individual customization is often insufficient, making it difficult to eliminate inadequate management and missed monitoring. Furthermore, misjudgments can lead to the commencement of work in unsafe conditions, creating another danger by failing to maintain the original purpose of worker safety.

[0010] In traditional human-to-human communication, concerns about the impact on interpersonal relationships have recently become apparent, hindering improvements in safety. Therefore, AI (Artificial Intelligence)-based judgments are being sought. However, when AI judgments are made, inaccuracies remain, leading to psychological effects such as resistance and distrust among users, further hindering safety assurance and revealing the limitations of the technology.

[0011] The disclosed technology was developed in view of the above points and aims to provide a discrimination device, discrimination method, and discrimination program that can improve safety by calculating a score in real time indicating the degree to which work equipment or clothing is properly worn. [Means for solving the problem]

[0012] A first aspect of the present disclosure is a discrimination device including a processor, the processor extracts partial images representing each element of the equipment or clothing from an image of a worker wearing work equipment or clothing, and for each element, uses a discrimination model that takes the partial image of the element as input to determine whether or not the element is properly worn, to obtain the output of the discrimination model, calculates a score indicating the degree to which the work equipment or clothing is properly worn from the output of the discrimination model for each element, and outputs the score to a display unit, and repeats this process.

[0013] According to the discrimination device of the first embodiment, safety can be improved by calculating a score in real time that indicates the degree to which work equipment or clothing is properly worn.

[0014] A second aspect of this disclosure is a discrimination device of the first aspect, wherein the processor further displays the image by superimposing a predetermined mark on the portion of the image representing the element if the output of the discrimination model satisfies predetermined conditions indicating that the element is not properly installed.

[0015] According to the discrimination device of the second embodiment, it is possible to notify in real time of any elements of work equipment or clothing that are not properly worn.

[0016] A third aspect of this disclosure is the discrimination device of the first aspect, wherein the processor repeatedly performs the process each time it is determined that the worker is in a predetermined posture.

[0017] According to the third type of discrimination device, the worker can instruct the timing for determining whether work equipment or clothing is being worn correctly.

[0018] A fourth aspect of this disclosure is a discrimination device according to the first aspect, wherein the processor performs a skeletal detection process on the image to detect predetermined joint points of a person when extracting the partial image, and for each element, extracts a predetermined range from the joint point corresponding to the element as a partial image representing the element.

[0019] According to the fourth type of discrimination device, partial images representing each element of equipment or clothing can be accurately extracted.

[0020] A fifth aspect of this disclosure is a discrimination method, in which a computer extracts partial images representing each element of the equipment or clothing from an image of a worker wearing work equipment or clothing, and for each element, uses a discrimination model that takes the partial image of the element as input to determine whether or not the element is properly worn, obtains the output of the discrimination model, calculates a score indicating the degree to which the work equipment or clothing is properly worn from the output of the discrimination model for each element, and outputs the score to a display unit, and repeatedly performs the following process.

[0021] According to the determination method of the fifth embodiment, safety can be improved by calculating a score in real time that indicates the degree to which work equipment or clothing is properly worn.

[0022] A sixth aspect of the present disclosure is a discrimination program that causes a computer to cut out partial images representing each element of equipment or clothing from an image of a worker wearing work equipment or clothing, and for each element, use a discrimination model that discriminates whether the element is correctly worn, taking the partial image of the element as an input, to obtain the output of the discrimination model, calculate a score indicating the degree to which the work equipment or clothing is correctly worn from the output of the discrimination model for each element, and repeatedly execute a process of outputting the score to a display unit.

[0023] According to the discrimination program of the sixth aspect, a score indicating the degree to which work equipment or clothing is correctly worn can be calculated in real time, improving safety.

Advantages of the Invention

[0024] According to the disclosed technology, a score indicating the degree to which work equipment or clothing is correctly worn can be calculated, improving safety.

Brief Description of the Drawings

[0025] [Figure 1] It is a schematic block diagram of an example of a computer that functions as the discrimination device of the present embodiment. [Figure 2] In the discrimination device of the present embodiment, it is a diagram showing a state in which a worker wearing work equipment or clothing is photographed. [Figure 3] It is a block diagram showing the configuration of the discrimination device of the present embodiment. [Figure 4] It is a diagram showing a state in which skeleton detection processing is performed on a worker. [Figure 5] (A) A diagram showing an example of a partial image representing a helmet, (B) A diagram showing an example of a partial image representing glasses, (C) A diagram showing an example of a partial image representing gloves, (D) A diagram showing an example of a partial image representing gauntlets. [Figure 6] It is a diagram showing an example of a screen for displaying a score. [Figure 7] It is a flowchart showing the flow of the discrimination process of the discrimination device of the present embodiment. [Modes for carrying out the invention]

[0026] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals. Furthermore, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from actual ratios.

[0027] <Summary of this embodiment> In this embodiment, even if the output of the discrimination model does not reach a 100% correct judgment, a means is provided to obtain a safer state from a practical standpoint. The discrimination model appears to the user as outputting an OK / NG judgment. However, in many cases, internally, the discrimination model outputs a continuous value between 0 and 1 for each individual element, with 0 representing a 100% OK judgment and 1 representing a 100% NG judgment. For example, by setting a threshold of 0.5, values ​​less than 0.5 are categorized as OK, and values ​​0.5 or greater are categorized as NG.

[0028] As a result, something that should be considered NG (not acceptable) is incorrectly judged as OK with a threshold of 0.49, creating a new safety risk for workers who perform tasks based on that judgment. While attempts are often made to adjust the threshold to 0.4, this imposes a burden on workers, as they may not receive an OK judgment even if they are wearing the equipment correctly. Furthermore, it can lead to a decrease in motivation to perform the work, a loss of trust in management techniques, and ultimately contribute to unsafe working conditions that hinder a safe and secure work environment.

[0029] In this embodiment, the fundamental motivations of the workers themselves—to improve, to seek advice for improvement, and to receive thoughtful feedback—are stimulated. Furthermore, it not only aims to work more safely, but also to draw out the workers' fundamental desire to work in a safe environment, thereby improving the degree to which a safe environment is achieved.

[0030] More specifically, the process involves extracting partial images using skeletal detection, utilizing a pre-trained discrimination model for each extracted element, and outputting a score to the operator based on the aggregated output of the discrimination model.

[0031] The output of the discrimination model, which ranges from 0 to 1, is treated as similarity and aggregated to provide a comprehensive index (score). Furthermore, when the threshold is 0.5, subtle areas (for example, 0.4 to 0.6) are highlighted with a red circle in the real-time image to draw the worker's attention, providing additional advice.

[0032] The essence of this method is to stimulate and sustain a person's desire for higher scores by presenting them with challenges. It's about using AI to establish a system that influences human motivation, also known as gamification.

[0033] <Configuration of the discrimination device according to this embodiment> Figure 1 is a block diagram showing the hardware configuration of the discrimination device 10 of this embodiment.

[0034] As shown in Figure 1, the discrimination device 10 includes a CPU (Central Processing Unit) 11A, a GPU (Graphics Processing Unit) 11B, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to the others via a bus 19 so that they can communicate with each other.

[0035] The CPU 11A is a central processing unit that executes various programs and controls various parts. Specifically, the CPU 11A reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a working area. The CPU 11A and GPU 11B control each of the above components and perform various calculations according to the program stored in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores a discrimination program. The discrimination program may be a single program or a group of programs composed of multiple programs or modules.

[0036] ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. Storage14 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs, including the operating system, and various data.

[0037] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to input various types of data, including images of workers wearing work equipment or clothing. A camera 20, as shown in Figure 2, can be used to capture images of workers wearing work equipment or clothing.

[0038] The display unit 16 is, for example, a liquid crystal display and displays various information, including a score indicating the degree to which work equipment or clothing is properly worn. The display unit 16 may also function as an input unit 15 by employing a touch panel system. Figure 2 shows an example in which a worker 30 stands facing the display unit 16 and camera 20 within the discrimination area indicated by the dashed line, and checks the score determined in real time on the display unit 16.

[0039] The communication interface 17 is an interface for communicating with other devices, and standards such as Ethernet®, FDDI, and Wi-Fi® can be used.

[0040] Next, the functional configuration of the discrimination device 10 will be described. Figure 3 is a block diagram showing an example of the functional configuration of the discrimination device 10.

[0041] Functionally, the discrimination device 10 comprises an acquisition unit 101, a cutting unit 102, a discrimination unit 103, a calculation unit 104, and an output unit 105, as shown in Figure 3.

[0042] The acquisition unit 101 acquires images of workers wearing work equipment or clothing, captured by the camera 20.

[0043] The cropping unit 102 determines from an image of a worker wearing work equipment or clothing whether the worker is in a specific pose. Specifically, as shown in Figure 4, the cropping unit 102 performs skeletal detection processing (e.g., posenet) to detect predetermined joint points of the person (coordinates of the eyes, elbows, wrists, etc.) and determines whether the worker is in a specific pose. Figure 4 shows an example in which the skeletal detection processing determines that the worker is in a specific pose, with both hands raised.

[0044] Furthermore, when the cropping unit 102 determines that a worker is in a specific pose, it extracts partial images representing each element of the equipment or clothing from an image of the worker wearing work equipment or clothing. Specifically, when the cropping unit 102 extracts partial images, it performs skeletal detection processing on the image and extracts a predetermined range from the joint point corresponding to each element as a partial image representing the element (see Figures 5(A) to (D)). Figure 5(A) shows an example of a partial image representing a helmet. Figure 5(B) shows an example of a partial image representing glasses. Figure 5(C) shows an example of a partial image representing gloves. Figure 5(D) shows an example of a partial image representing a hand guard.

[0045] The discrimination unit 103 uses a discrimination model that takes a partial image of the element as input to determine whether or not the element is correctly attached, and obtains the output of the discrimination model for each element.

[0046] For example, as a discrimination model, PyTorch is used to perform machine learning on each element using a normal image, which is a partial image of the element when it is correctly attached. Then, for each element, the partial image of that element is input into the discrimination model that was trained on that element, and the output of the discrimination model (for example, a value between 0 and 1 indicating the distance from the normal image) is obtained.

[0047] The calculation unit 104 calculates a score indicating the degree to which work equipment or clothing is correctly worn, based on the output of the discrimination model for each element.

[0048] For example, calculate the weighted average of the outputs of the discrimination model for each element, multiply the resulting value by 100 to convert it to a value between 0 and 100, and use that as the score.

[0049] The output unit 105 outputs the score to the display unit 16 (Figure 6). Figure 6 shows an example in which the judgment results for each element and the score are superimposed on the image taken by the operator.

[0050] Furthermore, for each element, if the output of the discrimination model satisfies a predetermined condition indicating that the element is not properly installed (for example, if the output of the discrimination model is below a threshold), the image may be displayed with a predetermined mark superimposed on the part of the image representing that element. Alternatively, the image may be displayed with a message indicating that the element is not properly installed.

[0051] <Operation of the discrimination device according to this embodiment> Next, the operation of the discrimination device 10 will be explained.

[0052] Figure 7 is a flowchart showing the flow of the discrimination process by the discrimination device 10. The CPU 11A reads the discrimination program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it to perform the discrimination process. It is assumed that images captured by the camera 20 are input sequentially. Note that the discrimination process is an example of a discrimination method.

[0053] First, in step S100, the CPU 11A and GPU 11B, acting as the acquisition unit 101, acquire images of workers wearing work equipment or clothing using the camera 20.

[0054] In step S101, the CPU 11A and GPU 11B determine from the image of the worker wearing work equipment or clothing, which is the cropping unit 102, whether the worker is in a specific pose. If the worker is in a specific pose, the process proceeds to step S102. On the other hand, if the worker is not in a specific pose, the process returns to step S100.

[0055] In step S102, the CPU 11A and GPU 11B extract partial images representing each element of the equipment or clothing from an image of a worker wearing work equipment or clothing, which is then used as the cropping unit 102.

[0056] In step S104, the CPU 11A and GPU 11B, acting as a discrimination unit 103, use a discrimination model that takes a partial image of the element as input to determine whether the element is correctly installed, and obtain the output of the discrimination model.

[0057] In step S106, the CPU 11A, GPU 11B, and calculation unit 104 calculate a score indicating the degree to which work equipment or clothing is correctly worn, based on the output of the discrimination model for each element.

[0058] In step S108, the CPU 11A and GPU 11B output the score to the display unit, and the process returns to step S100.

[0059] Through the above discrimination process, a score indicating the degree to which the work equipment or clothing is correctly worn is repeatedly calculated and displayed in real time each time a specific pose is taken. After checking the score, the worker can adjust the fit of their work equipment or clothing, take the specific pose again, and check the score in real time.

[0060] As described above, the discrimination device according to this embodiment extracts partial images representing each element of the equipment or clothing from an image of a worker wearing work equipment or clothing. For each element, the discrimination device uses a discrimination model that determines whether or not the element is correctly worn, obtains the output of the discrimination model, calculates a score indicating the degree to which the work equipment or clothing is correctly worn from the output of the discrimination model for each element, and outputs the score to the display unit. These processes are repeated each time it is determined that the worker is in a predetermined posture. This makes it possible to calculate a score indicating the degree to which the work equipment or clothing is correctly worn in real time, thereby improving safety.

[0061] Furthermore, the discrimination device can calculate and display scores in real time, making it more like a game rather than requiring human instruction. It can also stimulate motivation and a desire for improvement by addressing the challenges involved, thereby fostering continued effort. By utilizing a mechanism that influences human motivation, also known as gamification, it can help establish the correct wearing of work equipment or clothing. It can also direct workers' attention towards finding ways to wear equipment or clothing that yield higher scores. Additionally, it can facilitate new employee training and improve safety through continued use.

[0062] Furthermore, by realizing the discrimination device according to this embodiment using an edge computer, discrimination can be performed at the work site.

[0063] <Variation> It should be noted that the present invention is not limited to the embodiments described above, and various modifications and applications are possible without departing from the spirit of the invention.

[0064] For example, the various processes that the CPU reads and executes in each of the above embodiments may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processes, such as ASICs (Application Specific Integrated Circuits). Furthermore, the discrimination process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0065] Furthermore, although the above embodiments describe a configuration in which the discrimination program is pre-stored (installed) in the storage 14, the invention is not limited thereto. The program may be provided in a form stored on a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), and USB (Universal Serial Bus) memory. The program may also be provided in a form that is downloaded from an external device via a network. The present invention may also be applied to a program product. [Explanation of symbols]

[0066] 10 Discrimination device 11A CPU 11B GPU 14 Storage 15 Input section 16 Display section 20 cameras 30 workers 101 Acquisition Department 102 Cutting section 103 Discrimination part 104 Calculation section 105 Output section

Claims

1. Including the processor, The aforementioned processor, From an image of a worker wearing work equipment or clothing, extract partial images representing each element of the equipment or clothing. For each element, a discrimination model is used that takes a partial image of the element as input to determine whether or not the element is correctly attached, and the output of the discrimination model is obtained. From the output of the discrimination model for each element, a score indicating the degree to which work equipment or clothing is correctly worn is calculated. Output the score to the display unit. The process is repeated. Discrimination device.

2. The aforementioned processor further, For each element, if the output of the discrimination model satisfies predetermined conditions indicating that the element is not properly installed, a predetermined mark is superimposed on the portion of the image representing the element, and the image is displayed. The discrimination device according to claim 1.

3. The aforementioned processor, The determination device according to claim 1, which repeats the process each time it is determined that the worker is in a predetermined posture.

4. The aforementioned processor, The discrimination device according to claim 1, wherein when extracting the aforementioned partial image, a skeletal detection process is performed on the image to detect predetermined joint points of a person, and for each element, a predetermined range from the joint point corresponding to the element is extracted as a partial image representing the element.

5. Computers From an image of a worker wearing work equipment or clothing, extract partial images representing each element of the equipment or clothing. For each element, a discrimination model is used that takes a partial image of the element as input to determine whether or not the element is correctly attached, and the output of the discrimination model is obtained. From the output of the discrimination model for each element, a score indicating the degree to which work equipment or clothing is correctly worn is calculated. Output the score to the display unit. A method for determining whether to repeatedly execute a process.

6. On the computer, From an image of a worker wearing work equipment or clothing, extract partial images representing each element of the equipment or clothing. For each element, a discrimination model is used that takes a partial image of the element as input to determine whether or not the element is correctly attached, and the output of the discrimination model is obtained. From the output of the discrimination model for each element, a score indicating the degree to which work equipment or clothing is correctly worn is calculated. Output the score to the display unit. A discrimination program that repeatedly executes a process.

Images