Processing system, processing method, program, and storage medium

Abstract

To provide a processing system, a processing method, a program, and a storage medium capable of determining a fatigue state more accurately.SOLUTION: A processing system determines whether an operator is moving from an image taken of the operator performing work. When it is determined that the operator is not moving, the processing system determines a fatigue state of the operator using biological data acquired from a sensor attached to the operator. It is preferable that the processing system tracks the operator from the image taken of the operator, and determines whether the operator is moving on the basis of a result of the tracking.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] FIELD An embodiment of the present invention relates to a processing system, a processing method, a program, and a storage medium. [Background technology]

[0002] At a workplace, various tasks are performed. If a worker becomes physically or mentally fatigued due to the type of work or the worker's physical condition, a fatigue-related accident may occur. In order to prevent accidents from occurring, there is a demand for technology that can more accurately determine whether a worker is fatigued or not. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-86488 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a processing system, a processing method, a program, and a storage medium that are capable of determining a fatigue state more accurately. [Means for solving the problem]

[0005] A processing system according to an embodiment determines whether a worker performing a task is moving from an image of the worker. When the processing system determines that the worker is not moving, the processing system determines the fatigue state of the worker using biometric data acquired from a sensor attached to the worker. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a processing system according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram illustrating a work site. [Diagram 3] FIG. 3 is a table illustrating the sensors, the biometric data, and the process data. [Figure 4] FIG. 4 is a table illustrating the relationship between fatigue symptoms and sensors. [Diagram 5] 5(a) and 5(b) are schematic diagrams showing an example of output by the processing system according to the embodiment. [Figure 6] 6(a) and 6(b) are schematic diagrams showing an example of output by the processing system according to the embodiment. [Figure 7] Figure 7 is a table showing the relationship between fatigue state and possible accidents. [Figure 8] FIG. 8 is a schematic diagram showing an example of output by the processing system according to the embodiment. [Figure 9] FIG. 9 is a schematic diagram showing an example of output by the processing system according to the embodiment. [Figure 10] 10(a) to 10(c) are schematic diagrams showing examples of outputs from the processing system according to the embodiment. [Figure 11] FIG. 11 is a flowchart illustrating a processing method according to an embodiment. [Figure 12] FIG. 12 is a schematic diagram showing a hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, elements similar to those already described are designated by the same reference numerals, and detailed description thereof will be omitted as appropriate.

[0008] Fig. 1 is a schematic diagram showing a configuration of a processing system according to an embodiment. Fig. 2 is a schematic diagram showing a work site. Fig. 3 is a table showing examples of sensors, biological data, and processing data. Fig. 4 is a table showing examples of relationships between fatigue symptoms and sensors. The processing system 1 according to the embodiment includes an imaging unit 10, a detection unit 11, a recognition unit 12, a movement determination unit 13, a sensor 14, a state determination unit 15, a generation unit 16, and an output unit 17.

[0009] The image capturing unit 10 captures images of a work site and acquires images. At the work site, a worker performs a predetermined task. The image capturing unit 10 repeatedly acquires images at a predetermined interval. For example, the image capturing unit 10 is a camera that captures moving images.

[0010] For example, as shown in Fig. 2, it is preferable that the image capturing unit 10 is attached at a position higher than the height of a person and captures the work site from above. By capturing an image of the work site from above, a person is less likely to overlap with an object or another person in the acquired image. The image capturing unit 10 may capture an image of the work site from an obliquely upward direction, or may be attached to the ceiling or the like and capture an image of the work site in a vertical direction.

[0011] The detection unit 11 acquires a captured image and detects people appearing in the image. A person detection model including a neural network can be used for the detection. The detection unit 11 inputs the image to the person detection model, and the person detection model detects people appearing in the image. The detection unit 11 also acquires the coordinates of the detected people on the image.

[0012] For example, the photographing unit 10 photographs a work site 100 shown in FIG. 2. People 111-116 appear in the photographed image. The detection unit 11 detects each of the people 111-116 appearing in the image. The detection unit 11 also acquires the coordinates of each of the people 111-116. The detection unit 11 transmits the person detection results and the acquired coordinate results to the recognition unit 12.

[0013] The identification unit 12 identifies a person. Furthermore, the identification unit 12 acquires the person's identification information from a database or the like based on the identification result. The identification information includes a unique character string (ID) for identifying an individual, a name, and the like. The identification unit 12 links the identification information to the detection result by the detection unit 11.

[0014] When identifying people, the identification unit 12 cuts out an area from an image where a person is detected. The identification unit 12 inputs the cut-out image to a discrimination model including a neural network. In response to an input image, the discrimination model outputs a discrimination result of people appearing in the image. The identification unit 12 identifies people appearing in the image based on the output result from the discrimination model.

[0015] Alternatively, the identification unit 12 may identify people from their skeletons in an image. The identification unit 12 inputs an image into a skeleton extraction model including a neural network. The skeleton extraction model outputs the skeleton of the person in the image in response to the input of the image. The dimensions of each part of the skeleton are unique values ​​for each person. The identification unit 12 identifies people in the image based on the skeleton output from the skeleton extraction model.

[0016] Alternatively, the identification unit 12 may include an identifier reader. For example, the reader is provided at the entrance and exit of the work site. When entering and leaving the work site, a person has the reader read an identifier linked to his or her own data. The identifier may be a one-dimensional code (barcode), a two-dimensional code (QR Code (registered trademark)), a Radio Frequency Identification (RFID) tag, an IC chip, or the like. For example, the person holds up an employee ID card to which an identifier is attached over the reader. The reader reads the identification information from the identifier and identifies the person. Instead of the identifier, biometric data such as a person's fingerprint, vein, iris, or voice may be read, and the person may be identified based on any of the data.

[0017] When a reading device is used to identify a person, the identification information is linked to the person detected near the reading device at the time the identifier or biometric data is read.

[0018] In addition to workers, there may be supervisors supervising the work, visitors observing the work, etc. at the work site. As a result of the identification, it is determined whether the person in the image is a worker or not. In the following, the processing related to the worker in the image will be described.

[0019] The movement determination unit 13 tracks the worker using the detection result of the worker and the coordinates of the worker. Specifically, the movement determination unit 13 calculates the feature amount of the detected worker in an image acquired in a certain frame among a plurality of images acquired continuously. Next, the movement determination unit 13 calculates the feature amount of the detected worker in an image acquired in the immediately following frame. The movement determination unit 13 calculates the difference in the feature amount and the difference in coordinates (distance) between the worker detected in a certain frame and the worker detected in the immediately following frame. If the difference in the feature amount is smaller than a predetermined threshold and the distance is smaller than a predetermined threshold, the movement determination unit 13 determines that the workers shown in those images are the same. If multiple workers are shown in an image, tracking is performed for each worker.

[0020] The movement determination unit 13 performs tracking every time an image is acquired. In addition, the identification unit 12 associates the worker detection result with identification information. Therefore, the worker can be tracked in a state where the worker is identified.

[0021] The movement determination unit 13 determines whether each worker is moving or not based on the tracking result. For example, the movement determination unit 13 calculates the movement speed of the worker in a predetermined period. The movement speed is calculated from the difference in time when two images are acquired and the change in the coordinates of the worker in the two images. If the movement speed exceeds a predetermined threshold, the movement determination unit 13 determines that the worker is moving. As an example, if the average movement speed of the worker in a 5-second period exceeds 3 km / h, the movement determination unit 13 determines that the worker is moving. The movement determination unit 13 transmits the tracking result and the judgment result regarding the movement to the state determination unit 15.

[0022] The sensor 14 is attached to the worker and acquires biometric data of the worker. For example, the sensor 14 is a wearable biosensor that measures the biometric data. The acquired biometric data is one or more selected from cardiac potential, skin temperature, electroencephalogram, eye movement, and myoelectric potential. The sensor 14 transmits the acquired biometric data to the state determination unit 15.

[0023] The state determination unit 15 determines the fatigue state of the worker using the biometric data. The biometric data acquired when the movement determination unit 13 determines that the worker is not moving is used to determine the fatigue state. For example, as shown in FIG. 3, the biometric data acquired by the sensor 14 is processed to obtain processed data. The fatigue state is determined based on the processed data. The biometric data may be processed by the sensor 14 or may be processed by the state determination unit 15. The fatigue state is a state in which drowsiness, lethargy, tachycardia, decreased concentration, muscle pain, muscle tremors, stiff shoulders, or the like occurs. The state determination unit 15 transmits the determination result of the fatigue state to the generation unit 16.

[0024] For example, heart rate and heartbeat interval are calculated from cardiac potential data obtained by an electrocardiograph. Furthermore, low frequency (LF), high frequency (HF), and the ratio of LF to HF magnitudes are calculated from the heartbeat interval. LF is a fluctuation wave with a period of about 3 to 4 seconds, whose signal source is respiration. LF is a fluctuation wave with a period of about 10 seconds, whose signal source is blood pressure changes. The magnitude of the HF fluctuation wave and LF fluctuation wave in heart rate fluctuation changes depending on the balance of tension between the sympathetic and parasympathetic nerves. If the sympathetic nerves are in a state of tension, the worker can be determined to be in a "stressed state." If the parasympathetic nerves are in a state of tension, the worker can be determined to be in a "relaxed state."

[0025] The data measured by the skin thermometer gives the body temperature. When the skin is warm due to the heat inside the body, people tend to feel sleepy.

[0026] Alpha waves and beta waves are extracted from the brainwave data obtained by the electroencephalograph. Alpha waves are a waveform seen during sleep or relaxation. Alpha waves attenuate when the eyes are open or mental activity is performed. Beta waves are a waveform seen during alertness or attention.

[0027] When the eyeball is photographed by the eye camera, eye movement is calculated from the photographed result. For example, the coordinates of the gaze, the angle of the gaze, the gaze point, the acceleration of the eyeball, the blink frequency, etc. are calculated. The gaze point indicates the coordinates where the gaze is concentrated.

[0028] From the myoelectric potential obtained by the electromyograph, the presence or absence of muscle contraction, the strength of muscle contraction, etc. can be calculated. When a muscle (muscle fiber) contracts and exerts muscle force, the myoelectric potential oscillates. Also, the greater the strength of muscle contraction, the greater the amplitude.

[0029] The state determination unit 15 determines the fatigue state of the worker by using the processed data obtained from any of the biological data. For example, as shown in Fig. 4, the fatigue state can be determined from each processed data.

[0030] Specifically, when the worker is judged to be in a "relaxed state" based on the processed data of the cardiac potential, the state judgment unit 15 judges the worker to be in a fatigue state with drowsiness or fatigue, or a fatigue state with reduced concentration. When the heart rate is higher than the average value and the difference between them exceeds a predetermined threshold, the state judgment unit 15 judges the worker to be in a fatigue state with tachycardia.

[0031] When the skin temperature is higher than the average value and the difference therebetween exceeds a predetermined threshold, state determining unit 15 determines that the worker is in a fatigued state with drowsiness.

[0032] When alpha waves longer than a predetermined period are detected from the electroencephalogram data, the state determining unit 15 determines that the worker is in a fatigue state with drowsiness or lethargy, or a fatigue state with reduced concentration.

[0033] If the change in the gaze coordinate, angle, or change in the coordinate of the attention point is smaller than a predetermined threshold, and the eyeball acceleration is smaller than a predetermined threshold, the state determination unit 15 determines that the worker is in a fatigued state with drowsiness.

[0034] If the period of the myoelectric potential fluctuation is shorter than a predetermined threshold, the condition determination unit 15 determines that the worker is in a fatigued state with muscle tremors.If a frequency analysis of the myoelectric potential shows that there is a change in potential in low frequency components that is greater than a predetermined threshold, the condition determination unit 15 determines that the worker is in a fatigued state with muscle pain.

[0035] When it is determined that the worker is in a fatigued state, the generating unit 16 generates a notification according to the determination result. The output unit 17 outputs the notification generated by the generating unit 16. For example, the notification is transmitted to a terminal device carried by the worker. When the worker is looking at a monitor, the notification may be displayed on the monitor.

[0036] 5(a), 5(b), 6(a), and 6(b) are schematic diagrams showing examples of outputs from the processing system according to the embodiment. For example, the generating unit 16 generates a notification including an instruction to alleviate or eliminate a fatigue state. As a specific example, when it is determined that the worker is in a fatigue state in which drowsiness, fatigue, or reduced concentration occurs, the generating unit 16 generates a notification encouraging the worker to do exercises, stretching, or the like. The output unit 17 transmits the notification to a terminal device 20 carried by the worker, and the notification is output from the terminal device 20 as shown in FIG. 5(a). When it is determined that the worker is in a fatigue state in which muscle pain, muscle tremors, or stiff shoulders occur, the generating unit 16 generates a notification encouraging the worker to take a break as shown in FIG. 5(b).

[0037] When it is determined that the worker is in a fatigued state in which drowsiness, fatigue, or reduced concentration occurs, the generation unit 16 may generate work-related instructions as shown in Fig. 6(a). In a state in which drowsiness, fatigue, or reduced concentration occurs, mistakes in work are likely to occur. By notifying the worker of the work-related instructions, the worker can be alerted and the occurrence of mistakes in work can be suppressed.

[0038] When equipment data indicating information on equipment installed at the work site is prepared in advance, the generating unit 16 may evaluate the risk of a disaster caused by the equipment. The equipment data includes the ID, type, and location of the equipment installed at the work site. The generating unit 16 generates a notification based on the risk evaluation result.

[0039] For example, if a worker is in a fatigued state due to drowsiness, lethargy, or reduced concentration, he or she may accidentally come into contact with the equipment and suffer injuries such as being pinched, entangled, cut, scraped, electric shock, or burn. When it is determined that the worker is in a fatigued state, the generation unit 16 calculates the distance between the worker and the equipment. When the distance is below a predetermined threshold, the generation unit 16 generates a notification to call attention to the equipment, as shown in FIG. 6(b).

[0040] The state determination unit 15 may determine the fatigue state of the worker using a plurality of pieces of biodata. For example, the state determination unit 15 determines whether the worker is in a fatigue state (an example of a first fatigue state) in which drowsiness, fatigue, or reduced concentration occurs, based on a cardiac potential (an example of first biodata) obtained by a heart rate monitor (an example of a first sensor). If it is determined that the worker is in the first fatigue state, a first notification corresponding to the first fatigue state is output. The notification shown in FIG. 5(a) is an example of the first notification. The state determination unit 15 determines whether the worker is in a fatigue state (an example of a second fatigue state) in which the worker has muscle pain, muscle tremors, or stiff shoulders, based on data of myoelectric potential (an example of second biodata) obtained by an electromyograph (an example of a second sensor). If it is determined that the worker is in the second fatigue state, a second notification corresponding to the second fatigue state is output. The notification shown in FIG. 5(b) is an example of the second notification.

[0041] Figure 7 is a table showing the relationship between fatigue state and possible accidents. The generating unit 16 may generate a notification (third notification) to alert the worker to a disaster. When a worker is in a fatigued state, disasters are more likely to occur than when the worker is in a healthy state. For example, as shown in FIG. 7, when a worker is in a fatigued state with drowsiness, lethargy, stiff shoulders, or reduced concentration, there is a possibility that the worker may fall from a high place, trip over something, or come into contact with a falling object. By notifying the worker of a disaster warning according to the fatigue state, the occurrence of an accident can be suppressed.

[0042] As long as the notification can be conveyed to the worker, the specific output method of the notification is arbitrary. The notification may be a text message output from the terminal device 20, or a voice notification may be read out by the terminal device 20.

[0043] 8, 9, and 10(a) to 10(c) are schematic diagrams showing examples of outputs from the processing system according to the embodiment. When a worker is in a fatigued state, the generation unit 16 may modify the task time of the worker. In the example shown in Fig. 8, a certain worker is scheduled to perform task n, task n+1, and task n+2. A standard time required for each task is defined in a manual such as a task procedure. As shown on the left side of Fig. 8, a target time for the worker to finish the task is defined in advance based on the standard time.

[0044] When the worker is in a fatigued state, the generation unit 16 corrects the predicted work time by extending the standard time, as shown on the right side of Fig. 8. By automatically correcting the work time according to the fatigue state, it is possible to reduce the discrepancy between the pre-planned work time and the actual work time. The generation unit 16 may generate a chart shown in Fig. 8. The output unit 17 outputs the corrected work time or the chart shown in Fig. 8.

[0045] The generating unit 16 may generate an image showing the state of the worker. The generating unit 16 generates, for example, an image IMG shown in FIG. 9. The image IMG shows the work site, the position of each worker, and the state of each worker. In the image IMG, objects showing fatigued workers are displayed in different colors so that the fatigued workers can be distinguished from the non-fatigued workers. As long as the state of the worker can be determined, the manner of displaying the state is arbitrary. The state of the worker may be shown by the shape, size, pattern, etc. of the object other than the color.

[0046] The output unit 17 may output the image IMG and its legend. For example, the output unit 17 displays the image IMG and the legend on a graphical user interface (GUI) of a monitor so that a worker or a manager of the work can visually confirm the image IMG and the legend.

[0047] The generating unit 16 may modify the work plan. As shown in Fig. 10(a), when it is determined that the worker A is in a fatigued state, the generating unit 16 refers to the work plan and checks the work to be performed by the worker A. Next, the generating unit 16 refers to the work procedure. As shown in Fig. 10(b), the work procedure registers the priority of each task, the workers who are qualified to perform each task, and the like.

[0048] The generation unit 16 refers to the priority of "task n" performed by worker A and the priority of "task n+1" and "task n+2" performed by other workers. If the priority of "task n" is higher than the priority of the other tasks, the generation unit 16 assigns "task n" to the other workers. In other words, the higher the priority of a task, the higher the assignment priority.

[0049] In the illustrated example, the priority of task n+1 is higher than the priority of task n+2 and lower than the priority of task n. As shown in FIG. 10(c), task n is assigned to worker B instead of worker A who is in a fatigued state. Task n+1 is assigned to worker C instead of worker B who has been assigned task n. Task n+2, which was scheduled to be performed by worker C, is interrupted. The generation unit 16 outputs the revised work plan.

[0050] If a worker in a fatigued state continues working, it may lead to an accident. By modifying the work plan and changing the work to be performed by multiple workers, the possibility of a worker in a fatigued state suffering an accident can be reduced. Furthermore, since the work plan is modified by the generation unit 16, it is not necessary for a work manager or the like to modify the work plan. By using the processing system 1 according to the embodiment, the work plan can be modified more easily.

[0051] FIG. 11 is a flowchart illustrating a processing method according to an embodiment. First, the photographing unit 10 acquires an image of the work site (step S1). The detection unit 11 detects people from the acquired image (step S2). The identification unit 12 identifies the detected people (step S3). The movement determination unit 13 tracks the people identified as workers (step S4). Images are acquired repeatedly, and detection, identification, and tracking are performed each time an image is acquired.

[0052] The movement determination unit 13 determines whether the worker is moving or not based on the tracking result (step S5). If it is determined that the worker is not moving, the state determination unit 15 determines whether the worker is in a fatigued state using the biometric data (step S6).

[0053] If it is determined that the worker is in a fatigued state, the generation unit 16 generates information (step S7). For example, one or more selected from a notification shown in any one of Figs. 5(a) to 6(b), a corrected work time, a chart shown in Fig. 8, an image shown in Fig. 9, and a corrected work plan shown in Fig. 10(c) are generated. The output unit 17 outputs the information generated by the generation unit 16 (step S8).

[0054] After the information is output, if it is determined in step S5 that the worker is moving, or if it is determined in step S6 that the worker is not in a fatigued state, the movement determination unit 13 determines whether the work has been completed (step S9). If it is determined that the work has not been completed, the movement determination unit 13 executes step S5 again.

[0055] Advantages of the embodiment will be described. In order to determine whether a worker is fatigued, a method can be considered in which a sensor capable of acquiring biometric data is attached to the worker. In this method, the worker's fatigue state is determined based on the biometric data. Since biometric data can be easily collected, there is no need to ask the worker to provide data for determining the fatigue state. With this method, the worker's fatigue state can be determined easily.

[0056] On the other hand, when working, a lot of noise occurs in the biometric data. Therefore, when trying to determine the state of the worker based on the biometric data, an erroneous determination result is likely to be obtained. Regarding this problem, in the processing system 1 according to the embodiment, first, it is determined whether the worker is moving or not. When the worker is moving, the body movement is large and more noise occurs in the biometric data. On the other hand, when the worker is not moving, the body movement is small and less noise occurs in the biometric data. Therefore, in the processing system 1, the fatigue state of the worker is determined using the biometric data acquired when it is determined that the worker is not moving. According to the invention according to the embodiment, the fatigue state can be determined more accurately by using biometric data with less noise.

[0057] Furthermore, in the processing system 1, a worker is tracked from an image in which the worker is captured. Whether or not the worker is moving is determined based on the tracking result. In tracking using an image, the position of the worker is estimated from the image, and whether or not the worker is moving is determined from a change in that position. By accurately estimating the position of the worker from the image, it is also possible to accurately determine whether or not the worker is moving.

[0058] One method for determining whether or not a worker is moving is to use acceleration detected by an acceleration sensor. However, it is difficult to accurately estimate a position from acceleration. Therefore, it is also difficult to determine whether or not a worker is moving from a change in the estimated position. Tracking using images can more accurately determine whether or not a worker is moving.

[0059] When it is determined that the worker is in a fatigued state, the processing system 1 outputs a notification according to the determined fatigue state. For example, the worker can recognize that he or she is in a fatigued state from the output notification. The worker can be alerted to prevent accidents from occurring.

[0060] In the processing system 1, identification information of the worker in the image is acquired during tracking. Then, each worker in the image is tracked in a state where the worker is identified. By using the identification result, if any worker is determined to be in a fatigued state, a notification can be sent to a terminal device linked to the identification information of that worker.

[0061] When determining the fatigue state, the movement determination unit 13 may further determine the posture of the worker. Specifically, the movement determination unit 13 determines whether the worker is standing or sitting. The sitting state includes kneeling, squatting, and the like. The posture can be determined from the skeleton of the worker. The movement determination unit 13 inputs an image into a skeleton extraction model and acquires the skeleton of the person depicted in the image. If skeleton data is obtained in the identification by the identification unit 12, that data may be used. When a person is standing, the waist is straight. When a person is sitting, the waist is bent. The movement determination unit 13 determines whether the person is in a standing or sitting state from the angle of the skeleton of the waist. Alternatively, the state of the person may be determined based on the length from the foot to the head, the position of the skeleton of the head, or the like.

[0062] When it is determined that the worker is in a fatigued state, the generation unit 16 may change the notification according to the posture of the worker. In the table shown in FIG. 7, it is defined that the worker may fall if the worker is in a fatigued state of muscle pain, muscle tremors, drowsiness, fatigue, stiff shoulders, or reduced concentration. On the other hand, the possibility of a worker who is sitting falling is much smaller than the possibility of a worker who is standing falling. For example, the generation unit 16 does not generate a notification notifying the risk of falling if the worker is sitting, even if the worker is in a fatigued state of muscle tremors, drowsiness, fatigue, stiff shoulders, or reduced concentration.

[0063] When a worker is sitting, a notification to warn about a disaster is not output, so that it is possible to prevent unnecessary output of notifications. For example, it is possible to prevent the worker's attention from being distracted by notifications, which leads to a decrease in work efficiency. On the other hand, notifications to alleviate or eliminate a fatigue state, as shown in FIG. 5(a) or FIG. 5(b), and notifications related to work, as shown in FIG. 6(a), are generated regardless of the worker's posture.

[0064] FIG. 12 is a schematic diagram showing a hardware configuration. The processing system 1 according to the embodiment uses, for example, a computer 90 shown in Fig. 12. The computer 90 includes a CPU 91, a ROM 92, a RAM 93, a storage device 94, an input interface 95, an output interface 96, and a communication interface 97.

[0065] The ROM 92 stores a program for controlling the operation of the computer 90. The ROM 92 stores a program necessary for causing the computer 90 to realize each of the above-mentioned processes. The RAM 93 functions as a storage area in which the programs stored in the ROM 92 are expanded.

[0066] The CPU 91 includes a processing circuit. The CPU 91 uses a RAM 93 as a working memory and executes a program stored in at least one of a ROM 92 and a storage device 94. During execution of the program, the CPU 91 controls each component via a system bus 98 and executes various processes.

[0067] The storage device 94 stores data necessary for executing the programs and data obtained by executing the programs.

[0068] The input interface (I / F) 95 can connect the computer 90 and the input device 95a. The input I / F 95 is, for example, a serial bus interface such as USB. The CPU 91 can read various data from the input device 95a via the input I / F 95.

[0069] The output interface (I / F) 96 can connect the computer 90 and the output device 96a. The output I / F 96 is, for example, a video output interface such as a Digital Visual Interface (DVI) or a High-Definition Multimedia Interface (HPMI (registered trademark)). The CPU 91 can transmit data to the output device 96a via the output I / F 96 and cause the output device 96a to display an image.

[0070] The communication interface (I / F) 97 can connect the computer 90 to a server 97a outside the computer 90. The communication I / F 97 is, for example, a network card such as a LAN card. The CPU 91 can read various data from the server 97a via the communication I / F 97.

[0071] The storage device 94 includes one or more selected from a hard disk drive (HDD) and a solid state drive (SSD). The input device 95a includes one or more selected from a mouse, a keyboard, a microphone (audio input), and a touch pad. The output device 96a includes one or more selected from a monitor, a projector, a printer, and a speaker. A device having the functions of both the input device 95a and the output device 96a, such as a touch panel, may be used.

[0072] One computer 90 may function as the detection unit 11, the identification unit 12, the movement determination unit 13, the state determination unit 15, and the generation unit 16, or two or more computers 90 may cooperate to realize the functions of the detection unit 11, the identification unit 12, the movement determination unit 13, the state determination unit 15, and the generation unit 16. Each of the detection unit 11, the identification unit 12, the movement determination unit 13, the state determination unit 15, and the generation unit 16 may be configured as a module equipped with the computer 90.

[0073] The various data processing operations described above may be recorded as a program that can be executed by a computer on a magnetic disk (such as a flexible disk or a hard disk), an optical disk (such as a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW), a semiconductor memory, or other non-transitory computer-readable storage medium.

[0074] For example, information recorded on a recording medium can be read by a computer (or an embedded system). The recording medium may have any recording format (storage format). For example, the computer reads a program from the recording medium and causes a CPU to execute instructions described in the program based on the program. The computer may acquire (or read) the program via a network.

[0075] Embodiments of the invention include the following features. (Feature 1) determining whether or not a worker is moving based on an image showing the worker performing a task; A processing system that, when it is determined that the worker is not moving, determines a fatigue state of the worker using biometric data acquired from a sensor attached to the worker. (Feature 2) The processing system according to feature 1, further comprising: tracking the worker from an image showing the worker; and determining whether the worker is moving or not based on a result of the tracking. (Feature 3) Acquire identification information of the worker; Tracking the worker while the worker is identified; The processing system according to feature 2, wherein when the worker is determined to be in a fatigued state, a notification is output to a terminal device associated with the identification information. (Feature 4) outputting a first notification when the worker is determined to be in a first fatigue state; outputting a second notification when the worker is determined to be in a second fatigue state; 4. The processing system according to any one of features 1 to 3, wherein, when it is determined that the worker is not in a fatigued state, no notification is output. (Feature 5) acquiring first biometric data and second biometric data from a first sensor and a second sensor attached to the worker, respectively; Using the first biological data, determine whether the worker is in the first fatigue state; The processing system according to feature 4, further comprising: a processing unit configured to determine whether the worker is in the second fatigue state using the second biometric data. (Feature 6) The first fatigue state is a state in which any one selected from the group consisting of muscle pain, muscle tremors, drowsiness, lethargy, tachycardia, and decreased concentration occurs; 6. The treatment system of feature 5, wherein the second fatigue state is a state in which any other state selected from the group occurs. (Feature 7) Acquire a pre-registered task time for the task performed by the worker; 7. The processing system according to any one of features 1 to 6, wherein the operation time is corrected when it is determined that the worker is in a fatigued state. (Feature 8) Determining a fatigue state of a plurality of said workers; The processing system according to any one of features 1 to 7, further comprising: changing the work performed by the plurality of workers when it is determined that one or more of the plurality of workers is in a fatigued state. (Feature 9) Determining the posture of the worker; outputting a third notification to warn the worker about a disaster when the worker is in a fatigued state and the worker is standing; The processing system according to any one of features 1 to 8, wherein the third notification is not output when the worker is in a fatigued state and the worker is sitting. (Feature 10) The processing system determining whether or not a worker is moving based on an image showing the worker performing a task; When it is determined that the worker is not moving, a fatigue state of the worker is determined using biological data acquired from a sensor attached to the worker. Processing methods. (Feature 11) On the computer, determining whether or not a worker is moving based on an image of the worker performing a task; When it is determined that the worker is not moving, a fatigue state of the worker is determined using biological data acquired from a sensor attached to the worker. program. (Feature 12) A storage medium storing the program according to feature 11.

[0076] According to the embodiment described above, a processing system, a processing method, a program, and a storage medium are provided that are capable of more accurately determining a fatigue state of a worker.

[0077] Although some embodiments of the present invention have been illustrated above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, etc. can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. In addition, the above-mentioned embodiments can be implemented in combination with each other. [Explanation of symbols]

[0078] 1: processing system, 10: photographing unit, 11: detection unit, 12: identification unit, 13: movement determination unit, 14: sensor, 15: state determination unit, 16: generation unit, 17: output unit, 20: terminal device, 90: computer, 91: CPU, 92: ROM, 93: RAM, 94: storage device, 95: input interface, 95a: input device, 96: output interface, 96a: output device, 97: communication interface, 97a: server, 98: system bus, 100: work site, 111-116: person, IMG: image

Claims

1. determining whether or not a worker is moving based on an image showing the worker performing a task; A processing system that, when it is determined that the worker is not moving, determines a fatigue state of the worker using biometric data acquired from a sensor attached to the worker.

2. The processing system according to claim 1 , further comprising: tracking the worker from an image in which the worker is captured; and determining whether the worker is moving based on a result of the tracking.

3. Acquire identification information of the worker; Tracking the worker while the worker is identified; The processing system according to claim 2 , further comprising: a notification output to a terminal device associated with the identification information when the worker is determined to be in a fatigued state.

4. outputting a first notification when the worker is determined to be in a first fatigue state; outputting a second notification when the worker is determined to be in a second fatigue state; 4. The processing system according to claim 1, wherein when it is determined that the worker is not in a fatigued state, no notification is output.

5. acquiring first biometric data and second biometric data from a first sensor and a second sensor attached to the worker, respectively; Using the first biological data, determine whether the worker is in the first fatigue state; The processing system according to claim 4 , further comprising a step of determining whether or not the worker is in the second fatigue state by using the second biological data.

6. The first fatigue state is a state in which any one selected from the group consisting of muscle pain, muscle tremors, drowsiness, lethargy, tachycardia, and decreased concentration occurs; The treatment system of claim 5 , wherein the second fatigue state is an occurrence of any other state selected from the group.

7. Acquire a pre-registered task time for the task performed by the worker; 4. The processing system according to claim 1, further comprising: a processing unit that corrects the operation time when the worker is determined to be in a fatigued state.

8. Determining a fatigue state of a plurality of said workers; 4. The processing system according to claim 1, further comprising: a processing unit configured to change operations performed by the plurality of workers when one or more of the plurality of workers is determined to be in a fatigued state.

9. Determining the posture of the worker; outputting a third notification to warn the worker about a disaster when the worker is in a fatigued state and the worker is standing; 4. The processing system according to claim 1, wherein the third notification is not output when the worker is in a fatigued state and the worker is sitting.

10. The processing system determining whether or not a worker is moving based on an image showing the worker performing a task; when it is determined that the worker is not moving, determining a fatigue state of the worker using biological data acquired from a sensor attached to the worker; Processing methods.

11. On the computer, determining whether or not a worker is moving based on an image of the worker performing a task; when it is determined that the worker is not moving, a fatigue state of the worker is determined using biological data acquired from a sensor attached to the worker; program.

12. A storage medium storing the program according to claim 11.

Images