Work information management system and work information management method

Abstract

To accurately perform task recognition of a worker based on sensor data received from the worker's wearable sensor even when missing data occurs in the sensor data.SOLUTION: An abnormality determination unit determines whether sensor data includes abnormality data. Even when the abnormality determination unit determines that the sensor data includes abnormality data, the abnormality determination unit does not discard the sensor data including the abnormality data, but registers the sensor data including the abnormality data in a database, in association with the reception time thereof, upon imparting an abnormality flag indicating that abnormality data is included.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present invention relates to a work information management system and a work information management method, and is suitable for application to a work information management system relating to technology for recognizing the physical movements of workers, for example. [Background technology]

[0002] In recent years, Japan's population has been declining due to a declining birthrate and aging population, with the working-age population (ages 15 to 64) declining at a particularly rapid pace. In response to these demographic changes, efforts to incorporate older workers into the labor market are underway. The number of foreign workers is also increasing. Furthermore, the number of women entering the workforce, a trend that began in the 20th century, is also diversifying the domestic workforce. As the workforce changes in Japan, technological development is becoming increasingly important to foster a vibrant society and maintain and improve corporate competitiveness. For example, in a society where the number of workers is declining due to an aging population, domestic manufacturing will need to replace processes previously performed by humans with machines such as robots. Meanwhile, the remaining manual tasks that are difficult to mechanize will likely require greater utilization of female workers, physically challenged older workers, and foreign workers with different languages ​​and values. This will require technologies to effectively utilize workers, such as maintaining and improving productivity, appropriately managing physical loads, and providing effective skills training.

[0003] To solve these problems, methods have been proposed that recognize the work of workers by sensing their physical movements in factories and maintenance sites. For example, Patent Documents 1 and 2 disclose technologies that detect the physical movements of the workers' hands and the sounds that accompany those movements to determine whether the worker's work is good or bad. Furthermore, Patent Document 3 discloses technology that transmits sensor data from multiple sensors to downstream equipment via wireless communication.

[0004] When sensing the physical movements of workers in factories or maintenance sites, particularly in the case of sensors worn by the workers (hereinafter referred to as wearable sensors) as in the technologies disclosed in Patent Documents 1 and 2, the device having the function of analyzing the sensed physical movement information (hereinafter referred to as analytical device) is often configured separately from the sensor itself due to the computational resources required for the analysis, which increases the size of the device and power consumption.Signals are often exchanged between the wearable sensor and analytical device wirelessly rather than by wired communication using wires so as not to interfere with the worker's walking around. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6815378 [Patent Document 2] Patent No. 6798007 [Patent Document 3] Patent No. 6547057 Summary of the Invention [Problem to be solved by the invention]

[0006] However, one problem with this type of wireless communication is the stability of the wireless communication. For example, in a factory environment such as a production line or a maintenance site with many pieces of equipment, wireless communication can be interrupted or normal data cannot be transmitted when a worker walks around the required area, blocking radio waves. Furthermore, sensor devices can transmit momentarily erroneous data (hereinafter referred to as "anomalous data") due to electromagnetic noise from the environment. Because analytical devices extract characteristics of a worker's movements based on signals from wearable sensors, receiving anomalous data can prevent normal task recognition. In this case, if the analytical device discards all anomalous data and attempts to acquire only normal data (hereinafter referred to as "normal data"), the amount of normal data that can be acquired per unit time may decrease depending on the data acquisition environment, potentially preventing normal task recognition.

[0007] The present invention has been made in consideration of the above points, and aims to propose a work information management system and a work information management method that can accurately recognize the work of a worker based on sensor data received from the worker's wearable sensor, even if there is a gap in the sensor data. [Means for solving the problem]

[0008] In order to solve this problem, the present invention provides a system that receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more pieces of sensor data related to physical movements when the body acts on a work object through contact with the work object using the body; a sensor data collection unit that collects the sensor data; a database in which the sensor data collected by the sensor data collection unit is registered; and a work recognition unit that analyzes the sensor data registered in the database to recognize the work movements of the worker, wherein the sensor data collection unit has a time assignment unit that uses the receiver to acquire a reception time of the sensor data, and an abnormality determination unit that determines whether the sensor data contains abnormal data, and even if it is determined that the sensor data contains abnormal data, does not discard the sensor data containing the abnormal data, but assigns an abnormality flag indicating that the sensor data contains abnormal data, and registers the sensor data containing the abnormal data in the database in association with the reception time.

[0009] Furthermore, in the present invention, the method includes a receiving step in which a receiver receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more sensor data related to physical movements when the body acts on a work object through contact with the body using the body; a sensor data collecting step in which a sensor data collecting unit collects the sensor data and registers the sensor data in a database; and a work recognition step in which a work recognition unit analyzes the sensor data registered in the database to recognize the work movements of the worker, and the sensor data collecting step includes a time assigning step in which a time assigning unit of the sensor data collecting unit obtains a reception time of the sensor data using the receiver; and an abnormality determination step in which an abnormality determination unit of the sensor data collecting unit determines whether the sensor data includes abnormal data, and even if it is determined that the sensor data includes abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data includes abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time. [Effects of the Invention]

[0010] According to the present invention, even if there is a defect in the sensor data received from the worker's wearable sensor, the work of the worker can be accurately recognized based on the sensor data. [Brief explanation of the drawings]

[0011] [Figure 1A] 1 is a system configuration diagram showing an example of the configuration of a work information management system according to a first embodiment. [Figure 1B] FIG. 10 is a diagram illustrating an example of a state in which a worker wears a wearable sensor. [Figure 2] FIG. 2 is a conceptual diagram illustrating an example of data to be registered in a database of a work information management system. [Figure 3] FIG. 10 is a diagram illustrating an example of data to be registered in a database of a work information management system. [Figure 4]FIG. 10 is a diagram showing an example of classification of abnormality flags for each sensor to be registered in a database of a work information management system. [Figure 5] FIG. 10 is a diagram showing an example of abnormality classification of abnormality flags registered in the database of the work information management system. [Figure 6A] FIG. 10 is a diagram showing calibration of a glove-type sensor as an example of a sensor registered in a database of a work information management system according to a second embodiment. [Figure 6B] FIG. 10 is a diagram showing calibration of a glove-type sensor as an example of a sensor registered in a database of a work information management system according to a second embodiment. [Figure 7A] FIG. 10 is a diagram illustrating an example of sensor data of a first channel. [Figure 7B] FIG. 10 is a diagram illustrating an example of sensor data of a second channel. [Figure 7C] FIG. 10 is a diagram showing an example of how sensor data of a second channel is restored. [Figure 8A] FIG. 10 is a system configuration diagram showing an example of the configuration of a work information management system according to a third embodiment. [Figure 8B] FIG. 10 is a diagram showing an example of a state in which a worker wears a plurality of sensor devices on both hands. [Figure 9A] FIG. 10 is a diagram illustrating an example of sensor data of a first channel. [Figure 9B] FIG. 10 is a diagram illustrating an example of sensor data of a second channel. [Figure 9C] FIG. 10 is a diagram showing an example of how sensor data of a second channel is restored. [Figure 10] 10 is a diagram showing an example in which the work recognition unit of the work information management system evaluates the recognition results of sensor data that includes abnormal data and sensor data that does not include abnormal data by combining abnormality flags. FIG. [Figure 11] FIG. 10 is a system configuration diagram showing an example of the configuration of a work information management system according to a fifth embodiment. [Figure 12A] 10 is a graph showing an example in which the data from the acceleration sensor built into the wearable sensor is normal, but the data from the gyro sensor contains missing data. [Figure 12B] 10 is a graph showing an example in which the data from the acceleration sensor built into the wearable sensor is normal, but the data from the gyro sensor contains missing data. DETAILED DESCRIPTION OF THE INVENTION

[0012] (1) First embodiment The first embodiment will be described with reference to Figures 1A to 6B. Figure 1A is a system configuration diagram showing an example of the configuration of a work information management system according to the first embodiment, and Figure 1B is a diagram showing an example of a state in which a worker 100 wears a sensor device 200.

[0013] The work information management system according to the first embodiment includes a data collection management unit 104, which is, for example, a computer, and preferably further includes a sensor device 200 worn by the worker 100. The sensor device 200 includes a sensor device 200 worn by the worker 100 and a wireless transmitter 103. The sensor device 200 communicates data with the data collection management unit 104 wirelessly.

[0014] The sensor device 200 acquires its internal state and transmits sensor data including the acquired internal state to the outside. The sensor device 200 also includes, for example, at least one of an acceleration sensor, a gyro sensor, a geomagnetic sensor, a pressure sensor, and one or more microphones.

[0015] The sensor device 200 includes a wearable sensor 102 that is worn on the body of the worker 100 and outputs one or more pieces of sensor data related to the body movement when the body acts on a work object through contact with the work object using the body, and a wireless transmitter 103 as an example of a transmitter that transmits the sensor data output by the wearable sensor 102 to a wireless receiver 105. Note that the wireless transmitter 103 does not have to be included in the sensor device 200, but may be independent outside the sensor device 200.

[0016] As described above, the wearable sensor 102 is worn by the worker 100. The wearable sensor 102 acquires data (hereinafter referred to as sensor data) related to the physical movements of the worker 100 that the worker 100 makes on the sensing target 101, and outputs the sensor data to the data collection management unit 104 via the wireless transmitter 103.

[0017] The sensing target 101 is, for example, the body movement (hereinafter also referred to as body action) of a worker 100 when performing a specific task. The sensing target 101 may also be, for example, a part having a certain shape that is being assembled during an assembly task, or may even be a tool used in the assembly. The sensor data acquired by the sensor device 200 is input into the data collection management unit 104 via the wireless transmitter 103.

[0018] The data collection management unit 104 includes a wireless receiver 105, a sensor data collection unit 106, a database (hereinafter also abbreviated as DB) 110, an activity recognition unit 109, and an external output interface (hereinafter also abbreviated as I / F) 111. Details of the wireless receiver 105, the sensor data collection unit 106, the DB 110, the activity recognition unit 109, and the external output I / F 111 will be described later. The sensor data collection unit 106 includes a time setting unit 107 and an abnormality determination unit 108. Details of the time setting unit 107 and the abnormality determination unit 108 will be described later.

[0019] The data collection management unit 104 may have its functions realized on a single device such as a computer as described above, or may be composed of a combination of multiple devices, and some of its functions may be realized on a cloud system.

[0020] The wireless receiver 105 has a function of receiving sensor data transmitted from the wireless transmitter 103 of the sensor device 200. The wireless receiver 105 is an example of a receiver, and receives sensor data from the wearable sensor device 200, which is worn on the body of the worker 100 and outputs one or more pieces of sensor data related to the body movement when the body acts on a work object through contact with the work object using the body.

[0021] The sensor data collection unit 106 has a function of collecting sensor data. The sensor data collection unit 106 will be described in detail later.

[0022] The DB 110 registers the sensor data collected by the sensor data collection unit 106. Although details will be described later, the DB 110 manages not only normal data (hereinafter also referred to as normal data) among the sensor data, but also abnormal data (hereinafter also referred to as abnormal data), and further manages task recognition results (described later). Details of the DB 110 will be described later.

[0023] The task recognition unit 109 has a function of recognizing the task actions of a worker by analyzing sensor data registered in the database 110. In the following description, recognizing the task actions of a worker is also referred to as "task recognition." Details of the task recognition unit 109 will be described later.

[0024] The external output I / F 111 is an interface for connecting the data collection management unit 104 to the outside.

[0025] Furthermore, the sensor data collection unit 106 has a time setting unit 107 and an abnormality determination unit 108. The time setting unit 107 has a clock function and acquires the time when the sensor data is received by the wireless receiver 105. For example, the system time of the data collection management unit 104 may be assigned as the reception time. Alternatively, standard time may be acquired from an NTP (Network Time Protocol) server and assigned as the reception time. Furthermore, if the wearable sensor 102 has a built-in microcontroller, the reception time may be a time converted from the time using clock information from the microcontroller.

[0026] The abnormality determination unit 108, details of which will be described later, determines whether or not the sensor data contains abnormal data, and even if it determines that the sensor data contains abnormal data, it does not discard the sensor data containing the abnormal data, but rather assigns an abnormality flag indicating that the sensor data contains the abnormal data, and registers the sensor data containing the abnormal data in the database 110 in association with the above-mentioned reception time.

[0027] Specifically, the abnormality determination unit 108 determines whether or not there is a missing portion of the sensor data, in which part of the sensor data is missing due to, for example, wireless communication, by inferring from the sensor data related to the missing portion, and determines whether or not abnormal data has occurred in the sensor data. When it is determined that abnormal data has occurred in the sensor data, the abnormality determination unit 108 does not discard the sensor data including the abnormal data as described above, but assigns an abnormality flag to the portion of the abnormal data as described below, and then registers the sensor data including the abnormal data to which the abnormality flag has been assigned in the database 110 in the same way as normal sensor data (normal data).

[0028] The task recognition unit 109 analyzes one or more pieces of sensor data received from the wearable sensors 102 attached to the worker's body, and performs task recognition based on the analysis results to identify the task movements performed by the worker's body. The task recognition unit 109 reads the sensor data registered in the database 110, performs the task recognition, and registers the task recognition results (hereinafter also referred to as "task recognition results") in the database 110.

[0029] When the abnormality determination unit 108 determines that there is a defect in the sensor data, it classifies the type of abnormal data in which the defect occurs, and registers the sensor data including the abnormal data in the database 110 for each type of abnormal data.

[0030] The work recognition unit 109 refers to the abnormality flags registered in the database 110, and recognizes the work actions of the worker based on the type of defect corresponding to the abnormality flag, based on the sensor data to which the abnormality flag has been assigned, and reflects this in the results of the work recognition.

[0031] The task recognition unit 109 estimates missing parts corresponding to the abnormal data based on predetermined sensor data that does not contain abnormal data among multiple sensor data, repairs the missing parts (hereinafter also referred to as the "repair function"), and registers the sensor data with the repaired abnormal data in the abnormal data type management table 209 of the database 110.

[0032] When the work recognition unit 109 has repaired the defective portion 402, the work recognition unit 109 registers in the database 110 the sensor data from which the abnormal data has been repaired together with an abnormality flag indicating that the defective portion 402 has been repaired.

[0033] The sensor device 200 has a plurality of sensors that each output the above-mentioned sensor data, and the task recognition unit 109 uses sensor data from one of the plurality of sensors to repair sensor data including abnormal data from the other sensor.

[0034] 2 is a diagram showing an example of sensor data registered in DB 110. The sensor data here includes normal data and abnormal data with an abnormality flag attached. In the example shown, seven types of sensor data are shown, for example, sensor A value, sensor B value, sensor C value, sensor D value, sensor E value, sensor F value, and internal state.

[0035] The DB 110 manages, as columns, the reception time of the sensor data assigned by the time assignment unit 107, the various types of sensor data, the internal state, and the abnormality flag. In the illustrated example, the reception time is expressed in the Gregorian calendar as year, month, day, hour, minute, and second, but instead, it may be expressed in Unix time, for example.

[0036] The internal state is data indicating the internal state of the sensor device 200. Alternatively or in addition to the internal state, the internal state may be, for example, at least one of the battery voltage of the sensor device 200 and a counter value derived from a clock of a microcontroller built into the sensor device 200 and indicating the elapsed time since power was turned on.

[0037] The abnormality flag is composed of, for example, 7-bit binary data in accordance with the seven types of sensor data described above. Of these 7 bits, each bit from the most significant bit to the least significant bit corresponds to one of the seven types of sensor data described above. Of these 7 bits, if abnormal data occurs in the sensor data, the bit corresponding to that sensor data is set to "1," whereas if abnormal data does not occur in the sensor data, the bit corresponding to that sensor data is set to "0." That is, in this example, the fourth bit from the least significant bit (the fourth bit) of the abnormality flag is "1" and the remaining bits are "0," indicating that the sensor data of the sensor D value is abnormal data and the remaining sensor data is normal data.

[0038] 3 is a diagram showing another specific example of sensor values ​​registered in DB 110. DB 110 manages eight types of sensor data, for example, pressure sensor values ​​201, 202, and 203 for three channels, x-, y-, and z-axis components 204, 205, and 206 of an acceleration sensor, first microphone sensor data 208, and second microphone sensor data 209. In the example shown in the figure, since there are eight types of sensor data, the abnormality flag is eight bits. However, the registered sensor data is not limited to this, and may be, for example, sensor data from a nine-axis inertial sensor or sensor data from a sensor effective for recognizing the work of other people.

[0039] In this example, the sixth bit from the least significant bit (bit 6) of the abnormality flag is "1" and the remaining bits are "0", which indicates that the sensor data of the sensor D value is abnormal and the remaining sensor data is normal. Therefore, this indicates that the sensor data of the x-axis component 204 (corresponding to "accX" in the figure) of the acceleration sensor is abnormal and the other sensor data is normal.

[0040] As described above, Fig. 2 indicates that the sensor D value is abnormal data, and Fig. 3 indicates that the x component of the acceleration sensor is abnormal data. An example of the abnormality classification of each sensor data at this time will be described in more detail with reference to Fig. 2. In this embodiment, an abnormality flag type indicating the type of abnormal data is used as the abnormality classification.

[0041] 4 shows an example of the abnormal data type management table 207. The abnormal data type management table 207 manages, as types of abnormal data, "0000001" as an abnormal flag corresponding to "sensor A missing" indicating that a portion of the sensor A value shown in FIG. 2 described above is missing, "0000010" as an abnormal flag corresponding to "sensor B missing" indicating that a portion of the sensor B value is missing, "0000100" as an abnormal flag corresponding to "sensor C missing" indicating that a portion of the sensor C value is missing, "0001000" as an abnormal flag corresponding to "sensor D missing" indicating that a portion of the sensor D value is missing, "0010000" as an abnormal flag corresponding to "sensor E missing" indicating that a portion of the sensor E value is missing, "0100000" as an abnormal flag corresponding to "sensor F missing" indicating that a portion of the sensor F value is missing, and "1000000" as an abnormal flag corresponding to "internal state" indicating that a portion of data indicating the internal state of the sensor device 200 is missing.

[0042] 2 shows six types of sensor values ​​and one type of internal state of the sensor device 200. As an example of indicating which value has an abnormality such as a missing value, a seven-digit number is used as shown in FIG. 4, with each digit corresponding to an abnormality in the sensor value or internal state. Note that each digit may be expressed in decimal or hexadecimal notation. The abnormal state of each sensor may be indicated, for example, by a decimal value.

[0043] 5 is a diagram showing an example of abnormality flag types. The abnormality flag types are, for example, "0" if there is no abnormality, "1" if data is missing like the sensor D value in FIG. 2, "2" if the sensor takes an abnormal value that should not be taken, and "3" if the data is abnormal due to an abnormality such as a missing value in the internal data. Examples of this internal state include the internal state of other communication devices (for example, at least one of the wireless receiver 105 and the wireless transmitter 103).

[0044] In this embodiment, when a repair process is performed on abnormal data, the following values ​​may be assigned to indicate that the respective repair processes have been completed. Specific examples include a value of "4" for the abnormality flag type "data loss repaired," a value of "5" for the abnormality flag type "abnormal value repaired," and a value of "6" for the abnormality flag type "communication device abnormality repaired."

[0045] 6A and 6B are diagrams showing specific configuration examples of wearable sensor 102 for a worker's right hand. Fig. 6A shows a configuration example for the palm side of the worker's right hand, and Fig. 6B shows a configuration example for the back side of the worker's right hand.

[0046] In the sensor device 200, the wearable sensor 102, which is an example of at least one sensor that acquires the above-described sensor data, is configured as a glove-type sensor that is worn by the worker. This will be described in more detail.

[0047] Wearable sensor 102 as a glove-type sensor is configured, for example, as pressure sensors 301, 302, and 303 on the fingertips of thumb 301A, index finger 302A, and middle finger 303A on the palm side, respectively, as shown in Fig. 6B. Furthermore, wearable sensor 102 as a glove-type sensor may be configured, for example, as a first microphone 304 and a second microphone 305 on the back side of the worker's hand near the wrist and the inside of the thumb, respectively, as shown in Fig. 6B. Furthermore, wearable sensor 102 as a glove-type sensor may be configured, for example, as an inertial sensor 306, such as an acceleration sensor and a gyro sensor, provided on the back of the hand, as shown in Fig. 6B.

[0048] 7A to 7C are diagrams showing an example of how a missing portion 402 of sensor data is repaired by the repair function. In FIGS. 7A to 7C, the horizontal axis represents time t, and the vertical axis represents an example of the pressure values ​​(of the gripping pressure) of channels CH1 and CH2. The pressure sensors of channels CH1 and CH2 are arranged close to each other, and when an operator grips an object with their hands, they output sensor data with a certain degree of pressure value, although there is a difference between the pressure values ​​of the two.

[0049] In this embodiment, the illustrated sensor data is assumed to be data when a worker grasps an object using, for example, two fingers, the thumb and index finger or the thumb and middle finger. The pressure sensor value with respect to elapsed time t is assumed to be pressure value 400 of pressure sensor 301 of the thumb for first channel CH1 shown in Fig. 7A, and pressure value 401 (including missing portion 402) of pressure sensor of index finger 302A or middle finger 303A for second channel CH2 shown in Fig. 7B.

[0050] Fig. 7A shows an example of normal data of a normal pressure value 400 in which no abnormality such as missing portion 402 has occurred in the sensor data, and Fig. 7B shows an example of abnormal data in which an abnormality such as missing portion 402 has occurred in part of a normal pressure value 401 in the sensor data. Fig. 7C shows an example of how missing portion 402 of the abnormal data shown in Fig. 7B has been repaired by the repair function of abnormality determination unit 108. This will be explained in more detail below.

[0051] Assume that a missing portion 402 occurs in the pressure value of the sensor data from the pressure sensor of the second channel CH2, as shown in FIG. 7B . The missing portion 402 may be a single-point missing portion or a missing portion spanning multiple time points. Based on the direct pressure values ​​from the two pressure sensors, it can be inferred that the worker grasped the object with two fingers. However, because a missing portion occurs in the pressure value 401 of the second channel CH2 as described above, if the sensor data at the time of the missing portion 402 is discarded, it becomes impossible to determine whether the sensor value before and after the missing portion 402 exhibited a convex increase / decrease pattern or a continuous change such as the continuous change 403. Furthermore, no history of an abnormality such as the missing portion 402 in the sensor value of the pressure sensor is recorded. This makes it impossible to distinguish, for example, whether the worker grasped the object once, released it, and then grasped it again, or whether the worker continued to grasp it continuously, which may result in a deterioration in the recognition accuracy of the worker's physical movements.

[0052] Therefore, in this embodiment, by referring to the database 110, it is possible to obtain the sensor data of the pressure sensor of the second channel CH2 that has been registered with an abnormality flag assigned to it, and therefore the abnormality determination unit 108 can determine, for example, that the grip has been continuously held based on the sensor data of both the first channel CH1 and the second channel CH2.

[0053] The abnormality determination unit 108 repairs the sensor data from the pressure sensor of the channel CH2 described above based on the missing portion 402 of the sensor data so that normal pressure values ​​401 continue before and after the missing portion 402, as shown in Fig. 7C. The task recognition unit 109 can recognize the task of the worker based on the repaired sensor data.

[0054] Alternatively, in this embodiment, the abnormality determination unit 108 can use an abnormality flag already registered in the database 110 to determine whether or not to perform the worker's work recognition based on the sensor data to which the abnormality flag has been assigned.

[0055] The sensor data may be registered in DB 110 by registering a single value of sensor data for each time t, or by registering an array of multiple values, such as the sensor data of the first microphone and the sensor data of the second microphone shown in Fig. 3. Whether or not an abnormality, such as a missing portion, exists in the sensor data may be determined for each sensor value, or multiple values ​​may be determined together.

[0056] The data collection management unit 104, which serves as an example of a work information management system according to this embodiment, includes a wireless receiver 105 that receives sensor data from a wearable sensor device 200 that is worn on the body of the worker 100 and outputs one or more pieces of sensor data related to physical movements of the worker 100 when the worker 100 makes contact with the work object using the body to act on the work object; a sensor data collection unit 106 that collects the sensor data; a database 110 in which the sensor data collected by the sensor data collection unit 106 is registered; and a work recognition unit 109 that analyzes the sensor data registered in the database 110 to recognize the work movements of the worker. The sensor data collection unit 106 includes a time assignment unit 107 that acquires the time at which the sensor data was received by the wireless receiver 105; and an abnormality determination unit 108 that determines whether the sensor data contains abnormal data, and, even if it is determined that the sensor data contains abnormal data, does not discard the sensor data containing the abnormal data, but assigns an abnormality flag indicating that the sensor data contains the abnormal data, and registers the sensor data containing the abnormal data in the database 110 in association with the reception time.

[0057] The work information management method according to this embodiment includes a receiving step in which a wireless receiver 105 receives sensor data from a wearable sensor device 200 that is worn on the body of a worker 100 and outputs one or more sensor data related to physical movements when the worker acts on a work object through contact with the body using the body; a sensor data collection step in which a sensor data collection unit 106 collects the sensor data and registers it in a database 110; and a work recognition step in which a work recognition unit 109 analyzes the sensor data registered in the database 110 to recognize the work movements of the worker. The sensor data collection step includes a time assignment step in which a time assignment unit 107 of the sensor data collection unit 106 acquires the time when the sensor data was received by the wireless receiver 105; and an abnormality determination step in which an abnormality determination unit 108 of the sensor data collection unit 106 determines whether the sensor data contains abnormal data, and even if it is determined that the sensor data contains abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data contains abnormal data, and registers the sensor data including the abnormal data in the database 110 in association with the reception time.

[0058] In this way, the task recognition unit 109 can combine not only normal data but also abnormal data from among the multiple sensor data, and utilize more sensor data to perform task recognition that recognizes the physical task movements of the worker 100. Therefore, even if there is a defect in the sensor data received from the worker's wearable sensor, the task recognition unit 109 can accurately perform task recognition based on the sensor data.

[0059] In this embodiment, when the abnormality determination unit 108 determines that a defect has occurred in the sensor data, it classifies the type of abnormal data in which the defect has occurred, and registers the sensor data including the abnormal data for each type of abnormal data in the database 110. In this way, it is possible to refer to the types of abnormal data registered in the database 110 and perform task recognition according to the type of abnormal data after classification, thereby making it possible to minimize erroneous task recognition that may occur when utilizing abnormal data as described above.

[0060] In this embodiment, the task recognition unit 109 refers to the abnormality flags registered in the database 110, and recognizes the task actions of the worker according to the type of defect corresponding to the abnormality flag based on the sensor data to which the abnormality flag has been assigned, and reflects this in the results of the task recognition. In this way, task recognition can be performed according to the type of defect corresponding to the abnormality flag by referring to the abnormality flags registered in the database 110, thereby making it possible to minimize erroneous task recognition that may occur when utilizing abnormality data as described above.

[0061] In this embodiment, the task recognition unit 109 estimates missing parts corresponding to abnormal data based on predetermined sensor data that does not contain abnormal data among the plurality of sensor data, repairs the missing parts, and registers the sensor data from which the abnormal data has been repaired in the database 110. In this way, the task recognition unit 109 recognizes the tasks of the worker based on the repaired sensor data, and therefore can accurately recognize the tasks.

[0062] In this embodiment, when the missing portion 402 is repaired, the task recognition unit 109 registers the sensor data from which the abnormal data has been repaired in the database 110 together with an abnormality flag indicating that the missing portion 402 has been repaired. In this way, the task recognition unit 109 uses the abnormality flag to recognize the task of the worker based on the repaired sensor data, and therefore can accurately recognize the task.

[0063] In this embodiment, the sensor device 200 has multiple sensors that each output the above-mentioned sensor data, and the task recognition unit 109 uses sensor data from one of the multiple sensors to repair sensor data containing abnormal data from the other sensor. In this way, task recognition can be performed more accurately.

[0064] The data collection and management unit 104 as an example of a work information management system according to this embodiment includes not only the data collection and management unit 104 but also a sensor device 200 having a wireless transmitter 103 as an example of a transmitter. As described above, the wireless transmitter 103 is worn on the body of the worker 100, and transmits to the wireless transmitter 103 one or more pieces of sensor data relating to the body movement when the body acts on a work object through contact with the work object using the body. In this way, a work information management system in which the data collection and management unit 104 and the sensor device 200 are provided as an integrated unit can make wireless communication between the data collection and management unit 104 and the sensor device 200 less susceptible to the influence of noise and the like.

[0065] In the sensor device 200, at least one sensor that acquires the above-described sensor data is configured as a glove-type sensor that is worn by the worker. In this way, the work of the worker 100 can be accurately recognized using the sensor device 200 that can be worn on the hand of the worker 100.

[0066] The sensor device 200 includes a wearable sensor 102 and a wireless transmitter 103 as an example of a transmitter that transmits sensor data output by the wearable sensor 102 to a wireless receiver 105. As described above, the wearable sensor 102 is worn on the body of the worker 100 and has the function of outputting one or more pieces of sensor data related to the body movement when the body acts on a work object through contact with the work object using the body. In this way, since the wireless transmitter 103 is built into the sensor device 200, once the worker 100 wears the sensor device 200, there is no need for the worker 100 to wear a separate wireless transmitter 103.

[0067] (2) Second embodiment The work information management system according to the second embodiment has a configuration that is almost the same as that of the work information management system according to the first embodiment, so a description of the similar configuration will be omitted and the following description will focus on the differences.

[0068] The work information management system according to the second embodiment includes, in addition to the work information management system according to the first embodiment, a sensor device 200A having a configuration similar to that of the sensor device 200 in the first embodiment. In the first embodiment, the worker wears the sensor device 200 on his right hand, whereas in the second embodiment, the worker also wears the sensor device 200A on his left hand.

[0069] The work information management system according to this embodiment has sensor devices 200 and 200A as an example of the plurality of sensors that respectively output the above-described sensor data, and the work recognition unit 109 uses the sensor data from one of the sensor devices 200 and 200A to repair sensor data containing abnormal data from the other sensor. In this way, even if the sensor data from the other sensor contains abnormal data, work recognition for the worker 100 can be performed with high accuracy.

[0070] (3) Third embodiment The work information management system according to the third embodiment has a configuration that is almost the same as that of the work information management systems according to the first and second embodiments, so we will omit the explanation of the similar configuration and focus on the differences below.

[0071] Fig. 8A is a system configuration diagram showing an example of the configuration of a work information management system according to the third embodiment. Fig. 8B is a diagram showing an example of a state in which a worker 100 wears a sensor device 200 and a sensor device 200A on both hands.

[0072] In the work information management system of the third embodiment, a data recovery unit 112 is provided in the work recognition unit 109. In this embodiment, as an example, a worker 100 wears glove-type sensor devices 200 and 200A shown in Fig. 6 on both hands. Note that sensor device 200A has the same configuration and functions as sensor device 200 described above, except that it is worn on the opposite hand of worker 100.

[0073] 9A and 9B are diagrams showing examples of sensor data acquired by microphones of glove-type sensors as an example of wearable sensors 102 worn on both hands of worker 100. Fig. 9C is a diagram showing an example of restored sensor data of second channel CH2.

[0074] As shown in FIG. 9A, sensor data 500 of sounds specific to the work performed by worker 100 is detected from the microphone of the glove-type sensor on the left hand of worker 100, while as shown in FIG. 9B, a portion (missing portion 502) of sensor data 501 of sounds that should have been detected at the same time is missing from the microphone of the glove-type sensor on the main body of worker 100's right hand.

[0075] The worker's left and right hands are close to each other, and when one hand performs a task, the microphone on the other hand can also detect sound. However, in this embodiment, as shown in Figures 9A and 9B, the amplitude of the sensor data from the microphone in the glove-type sensor on the left hand is larger than the amplitude of the sensor data from the microphone in the glove-type sensor on the left hand.

[0076] The sensor data from the microphone of the glove-type sensor on the right hand shown in FIG. 8B contains missing part 502, so recognition accuracy cannot be guaranteed by itself, and the correct characteristic sound of the task cannot be obtained unless it is combined with the sensor data from the microphone of the glove-type sensor on the left hand shown in FIG. 8A.

[0077] In the work information management system according to this embodiment, the work recognition unit 109 repairs the missing portion 502 using sensor data from a microphone of a sensor attached to a glove-shaped body on the left hand, as shown in FIG. 9C as the repaired portion 503.

[0078] As a repair method, abnormality determination unit 108 replaces missing portion 502 with sensor data of a portion where a characteristic sound is generated from a microphone of a sensor provided on a glove-shaped body for the left hand, and then registers the abnormality flag shown in Fig. 5 as a corrected portion in database 110. Alternatively, abnormality determination unit 108 may prepare template data for repair in advance and replace missing portion 502 with the template data.

[0079] Furthermore, the abnormality determination unit 108 may update the database 110 so as to replace the pre-repair sensor data with the post-repair sensor data. Note that the abnormality determination unit 108 may use the database 110 to store a backup of the pre-repair sensor data.

[0080] According to this embodiment, the task recognition unit 109 can minimize computational resources when analyzing acquired sensor data to recognize the task actions of the worker 100, because it can make a determination by analyzing one or a small number of pieces of sensor data. Furthermore, because the missing portion 502 of the sensor data is repaired, there is no need to distinguish between the missing portion 502 and other portions of the sensor data, which simplifies the task recognition process.

[0081] (4) Fourth embodiment The work information management system according to the fourth embodiment has a configuration that is almost the same as the work information management systems according to the first, second and third embodiments, so we will omit the explanation of the similar configuration and focus on the differences below.

[0082] The fourth embodiment, like the third embodiment, relates to task recognition using multiple types of sensors. The task information management system according to the fourth embodiment will be described mainly with reference to Fig. 1 or Fig. 8 and Fig. 10.

[0083] The work recognition unit 109 shown in FIG. 1 or FIG. 8 utilizes an abnormality flag registered in the database 110 when the work recognition result indicates that multiple sensors recognize the same work action by the worker 100 as different work actions from each other.

[0084] The task recognition unit 109, for example, expresses a confidence level, which is an index as to whether a specific task performed by the worker 100 can be accurately recognized, as a number between "0" and "1" based on the sensor data of a plurality of sensors serving as the wearable sensor 102, for example, sensor A and sensor B shown in FIG. 10.

[0085] In this embodiment, when the confidence level is close to "1," there is a high probability that the work was actually performed by the worker 100, whereas when the confidence level is close to "0," there is a low probability that the work was actually performed by the worker. In addition, this embodiment assumes a situation in which the confidence level of sensor A is high and the confidence level of sensor B is low for the same work action performed by the worker 100. Furthermore, it is assumed that the sensor data used to calculate the confidence level of sensor A contains abnormal data that includes missing portions.

[0086] In this situation, if the task recognition unit 109 determines that a different task recognition result will be obtained for the same task action of the worker based on multiple sensor data, at least one of which includes abnormal data, the task recognition unit 109 adjusts the confidence level, which indicates the reliability of the one of the sensor data including the abnormal data, to lower it, and performs subsequent task recognition according to the adjusted confidence level, for example. Note that, as a method for adjusting the confidence level, the task recognition unit 109 may uniformly lower the confidence level, or may adjust the confidence level by multiplying it by a fixed rate. In this way, it is possible to effectively utilize sensor data including abnormal data while reducing the probability of erroneous recognition of task actions performed by the worker 100 in subsequent task recognition.

[0087] (5) Fifth embodiment The work information management system according to the fifth embodiment has a configuration that is almost the same as the work information management systems according to the first, second, third and fourth embodiments, so we will omit the explanation of the similar configuration and focus on the differences below.

[0088] The fifth embodiment relates to a function for restoring sensor data. The work information management system of the fifth embodiment will be described with reference to mainly Figs.

[0089] 11 is a system configuration diagram showing an example of the configuration of a work information management system according to the fifth embodiment. In the work information management system according to the fifth embodiment, the sensor data repair function described above differs from that of the previously described embodiments, and a data repair unit 113 is provided in the sensor data collection unit 106. The repair function of the data repair unit 113 in the fifth embodiment is the same as that of the previously described embodiments. In this way, the data collection management unit 104 executes the repair function, which relatively reduces the burden on the work recognition unit 109, allowing the work recognition unit 109 to recognize the work of the worker more quickly.

[0090] 12A shows a sensor value 700 of an acceleration sensor as channel CH1, and Fig. 12B shows a sensor value 701 of a gyro sensor as channel CH1. The illustrated example shows a specific component of sensor data of an acceleration sensor (sensor value 700 shown in Fig. 12A) and a specific component of sensor data of a gyro sensor (sensor value 701 shown in Fig. 12B, etc.) of wearable sensor 102 (e.g., a 9-axis inertial sensor) as a glove-type sensor shown in Fig. 6.

[0091] In this embodiment, it is assumed that the sensor value 701 of the gyro sensor includes, for example, a missing portion 702. Under this assumption, when the task recognition unit 109 refers to the sensor value 700 of the acceleration sensor, it determines that no particularly large acceleration has occurred before or after the time when the missing portion 702 occurred in the sensor data of the gyro sensor, and determines that the missing portion 702 of the gyro sensor can be repaired by interpolating the missing portion 702 using both sensor values ​​at the times before and after the missing portion 702, and repairs the missing portion 702.

[0092] Furthermore, in this embodiment, the sensor data in which the missing portion 702 described above has been repaired by the data repair unit 113 is registered in the database 110 with a repaired flag set as shown in Fig. 5. Here, setting a repaired flag means setting the value of the abnormality flag type "data loss repaired" shown in Fig. 5 to "4," the value of the abnormality flag type "abnormal value repaired" to "5," or the value of the abnormality flag type "communication device abnormality repaired" to "6."

[0093] The sensor data from multiple sensors may be sensor data from multiple sensors of a single sensor device 200, or may be sensor data from multiple sensor devices 200, 200A as shown in Fig. 8A. Also, missing portions may be repaired based on the sensor data sent via different wireless paths from sensor devices 200, 200A each having a built-in wireless transmitter.

[0094] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, the elements described in parallel in the present embodiment may be configured such that at least one of the elements is connected in series to the other elements. [Industrial Applicability]

[0095] The present invention can be applied to a work information management system relating to technology for recognizing the physical movements of a worker. [Explanation of symbols]

[0096] 100...Worker, 102...Wearable sensor, 103...Wireless transmitter, 104...Data collection management unit, 105...Wireless receiver, 106...Sensor data collection unit, 107...Time stamping function, 108...Abnormality determination unit, 109...Work recognition unit, 110...Database, 111...External output I / F, 112...Data recovery unit

Claims

1. a receiver that receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more sensor data related to body movements when the body acts on a work object through contact with the work object using the body; a sensor data collection unit that collects the sensor data; a database in which the sensor data collected by the sensor data collection unit is registered; a task recognition unit that analyzes the sensor data registered in the database and recognizes the task actions of the worker, The sensor data collection unit a time stamping unit that acquires a time when the sensor data was received by the receiver; an abnormality determination unit that determines whether the sensor data includes abnormal data, and even if it determines that the sensor data includes the abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data includes the abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time; A work information management system comprising:

2. The abnormality determination unit When it is determined that a defect occurs in the sensor data, the type of the abnormal data in which the defect occurs is classified, and the sensor data including the abnormal data is registered in the database for each type of abnormal data.

2. The work information management system according to claim 1, wherein:

3. The task recognition unit The abnormality flag registered in the database is referenced, and based on the sensor data to which the abnormality flag is assigned, the work action of the worker is recognized according to the type of defect corresponding to the abnormality flag, and this is reflected in the result of the work recognition.

3. The work information management system according to claim 1 or 2.

4. The task recognition unit When at least one of the sensor data is determined to result in a different task recognition result for the same task action of the worker based on the plurality of sensor data including the abnormal data, the confidence level indicating the reliability of the one of the sensor data including the abnormal data is adjusted to be lowered, and task recognition is performed from the next time onwards.

4. The work information management system according to claim 3.

5. The task recognition unit A missing portion corresponding to the abnormal data is estimated based on predetermined sensor data that does not contain the abnormal data among the plurality of sensor data, and the missing portion is repaired, and the sensor data from which the abnormal data has been repaired is registered in the database.

4. The work information management system according to claim 3.

6. The task recognition unit When the missing portion is repaired, an abnormality flag indicating that the missing portion has been repaired is registered in the database along with the sensor data from which the abnormal data has been repaired.

6. The work information management system according to claim 5,

7. The sensor device includes: a plurality of sensors each outputting the sensor data; The task recognition unit and repairing the sensor data including the abnormal data from another sensor using sensor data from one of the plurality of sensors.

2. The work information management system according to claim 1, wherein:

8. the sensor device has a transmitter that is attached to the body of the worker and transmits to the receiver one or more pieces of sensor data relating to body movements when the body acts on a work object through contact with the work object using the body.

2. The work information management system according to claim 1, wherein:

9. The sensor device includes: At least one sensor for acquiring the sensor data is configured as a glove-type sensor worn by the worker.

2. The work information management system according to claim 1, wherein:

10. The sensor device includes: a wearable sensor that is attached to the body of the worker and outputs one or more sensor data related to a body movement when the body acts on a work object through contact with the work object using the body; a transmitter that transmits the sensor data output by the wearable sensor to the receiver; The work information management system according to claim 1, further comprising:

11. a receiving step of receiving sensor data from a wearable sensor device, the receiver being attached to the body of a worker and outputting one or more sensor data relating to body movements when the body acts on a work object through contact with the work object using the body; a sensor data collection step in which a sensor data collection unit collects the sensor data and registers the sensor data in a database; a task recognition step in which a task recognition unit analyzes the sensor data registered in the database to recognize a task action of the worker, The sensor data collection step includes: a time stamping step in which a time stamping unit of the sensor data collecting unit acquires a reception time of the sensor data by the receiver; an abnormality determination step in which an abnormality determination unit of the sensor data collection unit determines whether or not the sensor data includes abnormal data, and even if it determines that the sensor data includes the abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data includes the abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time; A work information management system comprising:

Images