Work information management system and work information management method

Abstract

An abnormality determination unit determines whether sensor data includes abnormality data. Even when the abnormality determination unit determines that the sensor data includes abnormality data, the abnormality determination unit does not discard the sensor data including the abnormality data, but registers the sensor data including the abnormality data in a database, in association with the reception time thereof, upon imparting an abnormality flag indicating that abnormality data is included.

Description

Work information management system and work information management method

[0001] The present invention relates to a work information management system and a work information management method, and is suitable for application to a work information management system relating to technology for recognizing the physical movements of workers, for example.

[0002] In recent years, Japan's population has been declining due to a declining birthrate and aging population, with the working-age population (those aged 15 to 64) declining at a particularly rapid pace. In response to these demographic changes, efforts to incorporate older workers into the labor market are underway. The number of foreign workers is also increasing. Furthermore, the number of women entering the workforce, a trend that began in the 20th century, is also diversifying the domestic workforce. As the work environment in Japan changes, technological development is becoming increasingly important to create a vibrant society and maintain and improve corporate competitiveness. For example, in a society where the number of workers is declining due to an aging population, domestic manufacturing will need to replace processes previously performed by humans with machines such as robots. Meanwhile, the remaining manual tasks that are difficult to mechanize will likely require greater utilization of female workers, physically challenged older workers, and foreign workers with different languages ​​and values. This will require technologies to effectively utilize workers, such as maintaining and improving productivity, appropriately managing physical loads, and effective skills training.

[0003] To solve these problems, methods have been proposed that recognize the work of workers by sensing their physical movements in factories and maintenance sites. For example, Patent Literature 1 and Patent Literature 2 disclose technologies that detect the physical movements of the workers' hands and the sounds associated with those movements to determine whether the worker's work is good or bad. Furthermore, Patent Literature 3 discloses a technology that transmits sensor data from multiple sensors to downstream equipment via wireless communication.

[0004] When sensing the physical movements of workers in factories or maintenance sites, particularly in the case of sensors worn by the workers (hereinafter referred to as "wearable sensors") as in the technologies disclosed in Patent Documents 1 and 2, the device having the function of analyzing the sensed physical movement information (hereinafter referred to as "analysis device") is often configured separately from the sensor body due to the computational resources required for the analysis, which increases the size of the device and power consumption, etc. Signals are often exchanged between the wearable sensor and the analysis device wirelessly rather than by wired communication using wires so as not to interfere with the worker's walking around.

[0005] Patent No. 6815378 Patent No. 6798007 Patent No. 6547057

[0006] However, one problem with this type of wireless communication is the stability of the wireless communication. For example, in a factory environment such as a production line or a maintenance site with many pieces of equipment, wireless communication can be interrupted or normal data cannot be transmitted due to radio wave blocking caused by workers walking around the required area. Furthermore, sensor devices can transmit abnormal data (hereinafter referred to as "anomalous data") due to environmental electromagnetic noise. Because analytical devices extract characteristics of workers' movements based on signals from wearable sensors, receiving anomalous data can prevent normal task recognition. In this case, if the analytical device discards all anomalous data and attempts to acquire only normal data (hereinafter referred to as "normal data"), the amount of normal data that can be acquired per unit time may decrease depending on the data acquisition environment, potentially preventing normal task recognition.

[0007] The present invention has been made in consideration of the above points, and aims to propose a work information management system and a work information management method that can accurately recognize the work of a worker based on sensor data received from the worker's wearable sensor, even if there is a gap in the sensor data.

[0008] In order to solve this problem, the present invention provides a system that receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more pieces of sensor data related to physical movements when the body acts on a work object through contact with the work object using the body; a sensor data collection unit that collects the sensor data; a database in which the sensor data collected by the sensor data collection unit is registered; and a work recognition unit that analyzes the sensor data registered in the database to recognize the work movements of the worker, wherein the sensor data collection unit has a time assignment unit that uses the receiver to acquire a reception time of the sensor data, and an abnormality determination unit that determines whether the sensor data contains abnormal data, and even if it is determined that the sensor data contains abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data contains abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time.

[0009] Furthermore, in the present invention, the method includes a receiving step in which a receiver receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more sensor data related to physical movements when the body acts on a work object through contact with the body using the body; a sensor data collecting step in which a sensor data collecting unit collects the sensor data and registers the sensor data in a database; and a work recognition step in which a work recognition unit analyzes the sensor data registered in the database to recognize the work movements of the worker, and the sensor data collecting step includes a time assigning step in which a time assigning unit of the sensor data collecting unit obtains a reception time of the sensor data using the receiver; and an abnormality determination step in which an abnormality determination unit of the sensor data collecting unit determines whether the sensor data includes abnormal data, and even if it is determined that the sensor data includes abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data includes abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time.

[0010] According to the present invention, even if there is a defect in the sensor data received from the worker's wearable sensor, the work of the worker can be accurately recognized based on the sensor data.

[0011] 1 is a system configuration diagram showing an example of the configuration of a work information management system according to a first embodiment; FIG. 2 is a diagram showing an example of a state in which a worker wears a wearable sensor; FIG. 3 is a conceptual diagram illustrating an example of data to be registered in a database of the work information management system; FIG. 4 is a diagram showing an example of data to be registered in a database of the work information management system; FIG. 5 is a diagram showing an example of classification of abnormality flags for each sensor to be registered in a database of the work information management system; FIG. 6 is a diagram showing an example of abnormality classification of abnormality flags to be registered in a database of the work information management system; FIG. 7 is a diagram showing the calibration of a glove-type sensor as an example of a sensor to be registered in a database of the work information management system according to a second embodiment; FIG. 8 is a diagram showing the calibration of a glove-type sensor as an example of a sensor to be registered in a database of the work information management system according to the second embodiment; FIG. 9 is a diagram showing an example of sensor data of a first channel; FIG. 10 is a diagram showing an example of sensor data of a second channel; FIG. 11 is a system configuration diagram showing an example of the configuration of a work information management system according to a third embodiment; FIG. 12 is a diagram showing an example of a state in which a worker wears a plurality of sensor devices on both hands; FIG. 13 is a diagram showing an example of sensor data of a first channel; FIG. 14 is a diagram showing an example of sensor data of a second channel; FIG. 15 is a diagram showing an example of sensor data of a second channel; 10 is a diagram showing an example in which a work recognition unit of a work information management system evaluates the recognition results of sensor data that includes abnormal data and sensor data that does not include abnormal data by combining abnormality flags. FIG. 11 is a system configuration diagram showing an example of the configuration of a work information management system according to a fifth embodiment. FIG. 12 is a graph showing an example in which acceleration sensor data built into a wearable sensor is normal and gyro sensor data contains missing data. FIG. 13 is a graph showing an example in which acceleration sensor data built into a wearable sensor is normal and gyro sensor data contains missing data.

[0012] (1) First Embodiment A first embodiment will be described with reference to Figures 1A to 6B. Figure 1A is a system configuration diagram showing an example of the configuration of a work information management system according to the first embodiment, and Figure 1B is a diagram showing an example of a worker 100 wearing a sensor device 200.

[0013] The work information management system according to the first embodiment includes a data collection management unit 104, which is, for example, a computer, and preferably further includes a sensor device 200 worn by the worker 100. The sensor device 200 includes a sensor device 200 worn by the worker 100 and a wireless transmitter 103. The sensor device 200 communicates data with the data collection management unit 104 wirelessly.

[0014] The sensor device 200 acquires its internal state and transmits sensor data including the acquired internal state to the outside. The sensor device 200 also includes, for example, at least one of an acceleration sensor, a gyro sensor, a geomagnetic sensor, a pressure sensor, and one or more microphones.

[0015] The sensor device 200 includes a wearable sensor 102 that is worn on the body of the worker 100 and outputs one or more pieces of sensor data related to physical movements when the worker 100 acts on a work object through contact with the work object using the body, and a wireless transmitter 103 that is an example of a transmitter that transmits the sensor data output by the wearable sensor 102 to a wireless receiver 105. Note that the wireless transmitter 103 does not have to be included in the sensor device 200, but may be independent and located outside the sensor device 200.

[0016] As described above, the wearable sensor 102 is worn by the worker 100. The wearable sensor 102 acquires data (hereinafter referred to as sensor data) related to the physical movements of the worker 100 that the worker 100 makes on the sensing target 101, and outputs the sensor data to the data collection management unit 104 via the wireless transmitter 103.

[0017] The sensing target 101 is, for example, the body movement (hereinafter also referred to as body action) of the worker 100 when performing a specific task. The sensing target 101 may also be, for example, a part having a certain shape being assembled during an assembly operation, or a tool used in assembly. The sensor data acquired by the sensor device 200 is input into the data collection and management unit 104 via the wireless transmitter 103.

[0018] The data collection management unit 104 includes a wireless receiver 105, a sensor data collection unit 106, a database (hereinafter also abbreviated as DB) 110, an activity recognition unit 109, and an external output interface (hereinafter also abbreviated as I / F) 111. Details of the wireless receiver 105, the sensor data collection unit 106, the DB 110, the activity recognition unit 109, and the external output I / F 111 will be described later. The sensor data collection unit 106 includes a time setting unit 107 and an abnormality determination unit 108. Details of the time setting unit 107 and the abnormality determination unit 108 will be described later.

[0019] The data collection management unit 104 may have its functions realized on a single device such as a computer as described above, or may be composed of a combination of multiple devices, and some of its functions may be realized on a cloud system.

[0020] The wireless receiver 105 has a function of receiving sensor data transmitted from the wireless transmitter 103 of the sensor device 200. The wireless receiver 105 is an example of a receiver, and receives sensor data from the wearable sensor device 200, which is worn on the body of the worker 100 and outputs one or more pieces of sensor data related to the body movement when the body acts on a work object through contact with the work object using the body.

[0021] The sensor data collection unit 106 has a function of collecting sensor data. The sensor data collection unit 106 will be described in detail later.

[0022] The DB 110 registers the sensor data collected by the sensor data collection unit 106. As will be described in detail later, the DB 110 manages not only normal data (hereinafter also referred to as normal data) among the sensor data, but also abnormal data (hereinafter also referred to as abnormal data), and further manages task recognition results (described later). The DB 110 will be described in detail later.

[0023] The task recognition unit 109 has a function of recognizing the task actions of a worker by analyzing sensor data registered in the database 110. In the following description, recognizing the task actions of a worker is also referred to as "task recognition." Details of the task recognition unit 109 will be described later.

[0024] The external output I / F 111 is an interface for connecting the data collection management unit 104 to the outside.

[0025] Furthermore, the sensor data collection unit 106 has a time setting unit 107 and an abnormality determination unit 108. The time setting unit 107 has a clock function and acquires the time at which the wireless receiver 105 receives the sensor data. For example, the system time of the data collection management unit 104 may be assigned as the reception time. Alternatively, standard time may be acquired from an NTP (Network Time Protocol) server and assigned as the reception time. Furthermore, if the wearable sensor 102 has a built-in microcontroller, the reception time may be a time converted from the time using clock information from the microcontroller.

[0026] The abnormality determination unit 108, details of which will be described later, determines whether or not the sensor data contains abnormal data, and even if it determines that the sensor data contains abnormal data, it does not discard the sensor data containing the abnormal data, but rather assigns an abnormality flag indicating that the sensor data contains the abnormal data, and registers the sensor data containing the abnormal data in the database 110 in association with the above-mentioned reception time.

[0027] Specifically, the abnormality determination unit 108 determines whether or not there is a missing portion of the sensor data, where part of the sensor data is missing due to, for example, wireless communication, by inferring from the sensor data related to the missing portion, and determines whether or not abnormal data has occurred in the sensor data. If the abnormality determination unit 108 determines that abnormal data has occurred in the sensor data, it does not discard the sensor data including the abnormal data as described above, but instead assigns an abnormality flag to the portion of the abnormal data as described below, and registers the sensor data including the abnormal data with the abnormality flag assigned in the database 110 in the same way as normal sensor data (normal data).

[0028] The task recognition unit 109 analyzes one or more pieces of sensor data received from the wearable sensors 102 attached to the worker's body, and performs task recognition based on the analysis results to determine what task movements the worker's body has performed. The task recognition unit 109 reads the sensor data registered in the database 110, performs the task recognition, and registers the task recognition results (hereinafter also referred to as "task recognition results") in the database 110.

[0029] When the abnormality determination unit 108 determines that there is a missing piece of sensor data, it classifies the type of abnormal data in which the missing piece has occurred, and registers the sensor data including the abnormal data for each type of abnormal data in the database 110.

[0030] The work recognition unit 109 refers to the abnormality flags registered in the database 110, and based on the sensor data to which the abnormality flag has been assigned, recognizes the work actions of the worker according to the type of defect corresponding to the abnormality flag, and reflects this in the results of the work recognition.

[0031] The work recognition unit 109 estimates the missing parts corresponding to the abnormal data based on certain sensor data among multiple sensor data that do not contain abnormal data, repairs the missing parts (hereinafter also referred to as the ``repair function''), and registers the sensor data with the repaired abnormal data in the abnormal data type management table 209 of the database 110.

[0032] When the work recognition unit 109 repairs the defective portion 402, it registers the sensor data from which the abnormal data has been repaired in the database 110 together with an abnormality flag indicating that the defective portion 402 has been repaired.

[0033] The sensor device 200 has a plurality of sensors that each output the above-mentioned sensor data, and the task recognition unit 109 uses sensor data from one of the plurality of sensors to repair sensor data that includes abnormal data from the other sensor.

[0034] 2 is a diagram showing an example of sensor data registered in DB 110. The sensor data here includes normal data and abnormal data to which an abnormality flag is assigned. In the example shown, seven types of sensor data are shown, for example, sensor A value, sensor B value, sensor C value, sensor D value, sensor E value, sensor F value, and internal state.

[0035] The DB 110 manages, as columns, the reception time of the sensor data assigned by the time assignment unit 107, the various types of sensor data, internal states, and abnormality flags. In the illustrated example, the reception time is expressed in the Gregorian calendar as year, month, day, hour, minute, and second, but may instead be expressed in Unix time, for example.

[0036] The internal state is data indicating the internal state of the sensor device 200. Alternatively or in addition to the internal state, the internal state may be, for example, at least one of the battery voltage of the sensor device 200 and a counter value derived from a clock of a microcontroller built into the sensor device 200 and indicating the elapsed time since power was turned on.

[0037] The abnormality flag is composed of, for example, 7-bit binary data in accordance with the seven types of sensor data described above. Of these 7 bits, each bit from the most significant bit to the least significant bit corresponds to one of the seven types of sensor data described above. Of these 7 bits, if abnormal data occurs in the sensor data, the bit corresponding to the sensor data is set to "1," whereas if abnormal data does not occur in the sensor data, the bit corresponding to the sensor data is set to "0." That is, in this example, the fourth bit from the least significant bit (fourth bit) of the abnormality flag is "1" and the remaining bits are "0," indicating that the sensor data of the sensor D value is abnormal data and the remaining sensor data is normal data.

[0038] 3 is a diagram showing another specific example of sensor values ​​registered in DB 110. DB 110 manages eight types of sensor data, for example, pressure sensor values ​​201, 202, and 203 for three channels, x-, y-, and z-axis components 204, 205, and 206 of an acceleration sensor, and first microphone sensor data 208 and second microphone sensor data 209. In the example shown in the figure, since there are eight types of sensor data, the abnormality flag has eight bits. However, the registered sensor data is not limited to this, and may also be, for example, sensor data from a nine-axis inertial sensor or sensor data from a sensor effective for recognizing the work of other people.

[0039] In this example, the sixth bit from the least significant bit of the abnormality flag is "1" and the remaining bits are "0", which indicates that the sensor data for the sensor D value is abnormal and the remaining sensor data is normal. Therefore, this indicates that the sensor data for the x-axis component 204 of the acceleration sensor (corresponding to "accX" in the figure) is abnormal and the other sensor data is normal.

[0040] As described above, Fig. 2 indicates that the sensor D value is abnormal data, and Fig. 3 indicates that the x component of the acceleration sensor is abnormal data. An example of the abnormality classification of each sensor data at this time will be described in more detail with reference to Fig. 2. In this embodiment, an abnormality flag type indicating the type of abnormal data is used as the abnormality classification.

[0041] 4 shows an example of the abnormal data type management table 207. The abnormal data type management table 207 manages, as types of abnormal data, the following: "0000001" as an abnormal flag corresponding to "sensor A missing" indicating that a portion of the sensor A value shown in FIG. 2 is missing; "0000010" as an abnormal flag corresponding to "sensor B missing" indicating that a portion of the sensor B value is missing; "0000100" as an abnormal flag corresponding to "sensor C missing" indicating that a portion of the sensor C value is missing; "0001000" as an abnormal flag corresponding to "sensor D missing" indicating that a portion of the sensor D value is missing; "0010000" as an abnormal flag corresponding to "sensor E missing" indicating that a portion of the sensor E value is missing; "0100000" as an abnormal flag corresponding to "sensor F missing" indicating that a portion of the sensor F value is missing; and "1000000" as an abnormal flag corresponding to "internal state" indicating that a portion of data indicating the internal state of the sensor device 200 is missing.

[0042] 2 shows six types of sensor values ​​and one type of internal state of the sensor device 200. As an example of indicating which value has an abnormality such as a missing value, a seven-digit number is used as shown in FIG. 4, with each digit corresponding to an abnormality in the sensor value or internal state. Note that each digit may be expressed in decimal or hexadecimal notation. The abnormal state of each sensor may be expressed, for example, by distinguishing the value of the decimal number.

[0043] 5 is a diagram showing an example of abnormality flag types. The abnormality flag types are, for example, "0" if there is no abnormality, "1" if data is missing like the sensor D value in FIG. 2, "2" if the sensor takes an abnormal value that should not be taken, and "3" if the data is abnormal due to an abnormality such as a missing internal data. Examples of this internal state include the internal state of other communication devices (e.g., at least one of the wireless receiver 105 and the wireless transmitter 103).

[0044] In this embodiment, when a repair process is performed on abnormal data, the following values ​​may be assigned to indicate that the respective repair processes have been completed. Specific examples include a value of "4" for the abnormality flag type "data loss repaired," a value of "5" for the abnormality flag type "abnormal value repaired," and a value of "6" for the abnormality flag type "communication device abnormality repaired."

[0045] 6A and 6B are diagrams showing specific configuration examples of the wearable sensor 102 for the right hand of a worker. Fig. 6A shows an example of the configuration on the palm side of the worker's right hand, and Fig. 6B shows an example of the configuration on the back side of the worker's right hand.

[0046] The sensor device 200 is configured as a glove-type sensor that is worn by a worker, and includes a wearable sensor 102 as an example of at least one sensor that acquires the above-described sensor data.

[0047] Wearable sensor 102 as a glove-type sensor is configured, for example, as pressure sensors 301, 302, and 303 on the fingertips of thumb 301A, index finger 302A, and middle finger 303A on the palm side, respectively, as shown in Fig. 6B . Furthermore, wearable sensor 102 as a glove-type sensor may be configured, for example, as a first microphone 304 and a second microphone 305 near the wrist on the back side of the worker's hand and on the inside of the thumb, respectively, as shown in Fig. 6B . Furthermore, wearable sensor 102 as a glove-type sensor may be configured, for example, as an inertial sensor 306, such as an acceleration sensor and a gyro sensor, provided on the back of the hand, as shown in Fig. 6B .

[0048] 7A to 7C are diagrams showing an example of how a missing portion 402 of sensor data is repaired by the repair function. In FIGS. 7A to 7C, the horizontal axis represents time t, and the vertical axis represents an example of the pressure values ​​(of the gripping pressure) of channels CH1 and CH2. The pressure sensors of channels CH1 and CH2 are arranged close to each other, and when an operator grips an object with their hands, they output sensor data with a certain degree of pressure value, although there is a difference between the pressure values.

[0049] In this embodiment, the illustrated sensor data is assumed to be data obtained when a worker grasps an object using, for example, two fingers, the thumb and index finger or the thumb and middle finger. The pressure sensor values ​​with respect to elapsed time t are assumed to be pressure value 400 of pressure sensor 301 of the thumb for first channel CH1 shown in Fig. 7A and pressure value 401 (including missing portion 402) of pressure sensor of index finger 302A or middle finger 303A for second channel CH2 shown in Fig. 7B.

[0050] Fig. 7A shows an example of normal data of a normal pressure value 400 in which no abnormality such as missing portion 402 has occurred in the sensor data, and Fig. 7B shows an example of abnormal data in which an abnormality such as missing portion 402 has occurred in part of a normal pressure value 401 in the sensor data. Fig. 7C shows an example of how missing portion 402 of the abnormal data shown in Fig. 7B has been repaired by the repair function of the abnormality determination unit 108. This will be described in more detail below.

[0051] Assume that the sensor data from the pressure sensor of the second channel CH2 has a missing portion 402 in its pressure value, as shown in FIG. 7B . The missing portion 402 may be a single-point missing portion or a missing portion spanning multiple time points. Based on the direct pressure values ​​from the two pressure sensors, it would normally be inferred that the worker grasped the object with two fingers. However, because the missing portion occurred in the pressure value 401 of the second channel CH2 as described above, if the sensor data at the time of the missing portion 402 is discarded, it becomes impossible to determine whether the sensor value before and after the missing portion 402 exhibited a convex increase / decrease or a continuous change such as the continuous change 403. Furthermore, no history of an abnormality such as the missing portion 402 in the sensor value of the pressure sensor is recorded. This makes it impossible to distinguish, for example, whether the worker grasped the object once, released it, and then grasped it again, or whether the worker continued to grasp it continuously, which may result in a deterioration in the recognition accuracy of the worker's physical movements.

[0052] Therefore, in this embodiment, by referring to the database 110, it is possible to obtain the sensor data of the pressure sensor of the second channel CH2 that has been registered with an abnormality flag assigned to it, and therefore the abnormality determination unit 108 can determine, for example, that the hand has been continuously gripped based on the sensor data of both the first channel CH1 and the second channel CH2.

[0053] The abnormality determination unit 108 repairs the sensor data from the pressure sensor of channel CH2 described above based on the missing portion 402 of the sensor data so that normal pressure values ​​401 continue before and after the missing portion 402, as shown in Fig. 7C. The task recognition unit 109 can recognize the task of the worker based on the repaired sensor data.

[0054] Alternatively, in this embodiment, the abnormality determination unit 108 can use an abnormality flag already registered in the database 110 to determine whether or not to recognize the worker's work based on sensor data to which the abnormality flag has been assigned.

[0055] The sensor data may be registered in DB 110 by registering a single value of sensor data for each time t, or by registering an array of multiple values, such as the sensor data of the first microphone and the sensor data of the second microphone shown in Fig. 3. Whether or not an abnormality, such as a missing portion, exists in the sensor data may be determined for each sensor value, or may be determined for multiple values ​​collectively.

[0056] The data collection management unit 104, which serves as an example of a work information management system according to this embodiment, includes a wireless receiver 105 that receives sensor data from a wearable sensor device 200 that is worn on the body of the worker 100 and outputs one or more pieces of sensor data related to physical movements of the worker 100 when the worker 100 makes contact with a work object using the body to act on the work object; a sensor data collection unit 106 that collects the sensor data; a database 110 in which the sensor data collected by the sensor data collection unit 106 is registered; and a work recognition unit 109 that analyzes the sensor data registered in the database 110 to recognize the work movements of the worker. The sensor data collection unit 106 includes a time assignment unit 107 that acquires the time at which the sensor data was received by the wireless receiver 105; and an abnormality determination unit 108 that determines whether the sensor data contains abnormal data, and, even if it is determined that the sensor data contains abnormal data, does not discard the sensor data containing the abnormal data, but assigns an abnormality flag indicating that the sensor data contains the abnormal data, and registers the sensor data containing the abnormal data in the database 110 in association with the reception time.

[0057] The work information management method according to this embodiment includes a receiving step in which a wireless receiver 105 receives sensor data from a wearable sensor device 200 that is worn on the body of a worker 100 and outputs one or more sensor data related to physical movements when the worker's body acts on a work object through contact with the work object using the body; a sensor data collection step in which a sensor data collection unit 106 collects the sensor data and registers it in a database 110; and a work recognition step in which a work recognition unit 109 analyzes the sensor data registered in the database 110 to recognize the work movements of the worker. The sensor data collection step includes a time assignment step in which a time assignment unit 107 of the sensor data collection unit 106 acquires the time when the sensor data was received by the wireless receiver 105; and an abnormality determination step in which an abnormality determination unit 108 of the sensor data collection unit 106 determines whether the sensor data includes abnormal data, and even if it is determined that the sensor data includes abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data includes abnormal data, and registers the sensor data including the abnormal data in the database 110 in association with the reception time.

[0058] In this way, the task recognition unit 109 can combine not only normal data but also abnormal data from among the multiple sensor data, thereby utilizing more sensor data to perform task recognition that recognizes the physical task movements of the worker 100. Therefore, even if there is a defect in the sensor data received from the worker's wearable sensor, the task recognition unit 109 can accurately perform task recognition based on the sensor data.

[0059] In this embodiment, when the abnormality determination unit 108 determines that a defect has occurred in the sensor data, it classifies the type of abnormal data in which the defect has occurred, and registers the sensor data including the abnormal data for each type of abnormal data in the database 110. In this way, it is possible to refer to the types of abnormal data registered in the database 110 and perform task recognition according to the type of abnormal data after classification, thereby making it possible to minimize erroneous task recognition that may occur when utilizing abnormal data as described above.

[0060] In this embodiment, the task recognition unit 109 refers to the abnormality flags registered in the database 110, and recognizes the task actions of the worker according to the type of defect corresponding to the abnormality flag based on the sensor data to which the abnormality flag has been assigned, and reflects this in the results of the task recognition. In this way, task recognition can be performed according to the type of defect corresponding to the abnormality flag by referring to the abnormality flags registered in the database 110, thereby making it possible to minimize erroneous task recognition that may occur when utilizing abnormality data as described above.

[0061] In this embodiment, the task recognition unit 109 estimates missing parts corresponding to the abnormal data based on predetermined sensor data that does not contain abnormal data among the plurality of sensor data, repairs the missing parts, and registers the sensor data from which the abnormal data has been repaired in the database 110. In this way, the task recognition unit 109 recognizes the tasks of the worker based on the repaired sensor data, and therefore can accurately recognize the tasks.

[0062] In this embodiment, when the missing portion 402 is repaired, the task recognition unit 109 registers the sensor data from which the abnormal data has been repaired in the database 110 together with an abnormality flag indicating that the missing portion 402 has been repaired. In this way, the task recognition unit 109 uses the abnormality flag to recognize the task of the worker based on the repaired sensor data, and therefore can accurately recognize the task.

[0063] In this embodiment, the sensor device 200 has multiple sensors that each output the above-mentioned sensor data, and the task recognition unit 109 uses sensor data from one of the multiple sensors to repair sensor data containing abnormal data from the other sensor. In this way, task recognition can be performed more accurately.

[0064] The data collection and management unit 104 as an example of a work information management system according to this embodiment includes not only the data collection and management unit 104 but also a sensor device 200 having a wireless transmitter 103 as an example of a transmitter. As described above, the wireless transmitter 103 is worn on the body of the worker 100 and transmits to the wireless transmitter 103 one or more pieces of sensor data relating to the body movement when the body acts on a work object through contact with the work object using the body. In this way, a work information management system in which the data collection and management unit 104 and the sensor device 200 are provided as an integrated unit can make wireless communication between the data collection and management unit 104 and the sensor device 200 less susceptible to the influence of noise and the like.

[0065] In the sensor device 200, at least one sensor that acquires the above-described sensor data is configured as a glove-type sensor that is worn by the worker. In this way, the work of the worker 100 can be accurately recognized using the sensor device 200 that can be worn on the hand of the worker 100.

[0066] The sensor device 200 includes a wearable sensor 102 and a wireless transmitter 103 as an example of a transmitter that transmits sensor data output by the wearable sensor 102 to a wireless receiver 105. As described above, the wearable sensor 102 is worn on the body of the worker 100 and has the function of outputting one or more pieces of sensor data related to physical movements when the worker 100 acts on a work object through contact with the work object using the body. In this way, because the wireless transmitter 103 is built into the sensor device 200, once the worker 100 wears the sensor device 200, there is no need for the worker 100 to wear a separate wireless transmitter 103.

[0067] (2) Second embodiment The work information management system according to the second embodiment has a configuration that is almost the same as that of the work information management system according to the first embodiment, so we will omit the explanation of the similar configuration and focus on the differences below.

[0068] The work information management system according to the second embodiment includes, in addition to the work information management system according to the first embodiment, a sensor device 200A having a configuration similar to that of the sensor device 200 in the first embodiment. In the first embodiment, the worker wears the sensor device 200 on his right hand, whereas in the second embodiment, the worker also wears the sensor device 200A on his left hand.

[0069] The work information management system according to this embodiment has sensor devices 200 and 200A as an example of a plurality of sensors that respectively output the above-described sensor data, and the work recognition unit 109 uses the sensor data from one of the sensor devices 200 and 200A to repair sensor data containing abnormal data from the other sensor. In this way, even if the sensor data from the other sensor contains abnormal data, work recognition for the worker 100 can be performed with high accuracy.

[0070] (3) Third embodiment The work information management system according to the third embodiment has a configuration that is almost the same as the work information management systems according to the first and second embodiments. Therefore, the same configuration will not be described again, and the following description will focus on the differences.

[0071] Fig. 8A is a system configuration diagram showing an example of the configuration of a work information management system according to the third embodiment. Fig. 8B is a diagram showing an example of a state in which a worker 100 wears a sensor device 200 and a sensor device 200A on both hands.

[0072] In the work information management system of the third embodiment, a data recovery unit 112 is provided in the work recognition unit 109. In this embodiment, as an example, a worker 100 wears glove-type sensor devices 200 and 200A shown in Fig. 6 on both hands. Note that sensor device 200A has the same configuration and functions as sensor device 200 described above, except that it is worn on the opposite hand of worker 100.

[0073] 9A and 9B are diagrams showing examples of sensor data acquired by microphones of glove-type sensors as an example of wearable sensors 102 worn on both hands of worker 100. Fig. 9C is a diagram showing an example of restored sensor data of second channel CH2.

[0074] As shown in FIG. 9A, sensor data 500 of sounds specific to the work performed by worker 100 is detected from the microphone of the glove-type sensor on the left hand of worker 100, while as shown in FIG. 9B, a portion (missing portion 502) of sensor data 501 of sounds that should have been detected at the same time is missing from the microphone of the glove-type sensor on the main body of worker 100's right hand.

[0075] The worker's left and right hands are close to each other, and when one hand performs a task, the microphone on the other hand can also detect sound. However, in this embodiment, as shown in Figures 9A and 9B, the amplitude of the sensor data from the microphone in the glove-type sensor on the left hand is larger than the amplitude of the sensor data from the microphone in the glove-type sensor on the left hand.

[0076] The sensor data from the microphone of the glove-type sensor on the right hand shown in FIG. 8B contains missing part 502, and therefore recognition accuracy cannot be guaranteed by itself. Unless it is combined with the sensor data from the microphone of the glove-type sensor on the left hand shown in FIG. 8A, it will not become the correct characteristic sound of the task.

[0077] In the work information management system according to this embodiment, the work recognition unit 109 repairs the missing portion 502 using sensor data from a microphone of a sensor attached to a glove-shaped body on the left hand, as shown in FIG. 9C as the repaired portion 503.

[0078] As a repair method, abnormality determination unit 108 replaces missing portion 502 with sensor data of a portion where a characteristic sound is generated from a microphone of a sensor provided on a glove-shaped main body for the left hand, and then registers the abnormality flag shown in Fig. 5 as a corrected portion in database 110. Alternatively, abnormality determination unit 108 may prepare template data for repair in advance and replace missing portion 502 with the template data.

[0079] Furthermore, the abnormality determination unit 108 may update the database 110 so as to replace the pre-repair sensor data with the post-repair sensor data. Note that the abnormality determination unit 108 may use the database 110 to store a backup of the pre-repair sensor data.

[0080] According to this embodiment, the task recognition unit 109 can minimize computational resources when analyzing acquired sensor data to recognize the task actions of the worker 100, because it can make a determination by analyzing one or a small number of sensor data. Furthermore, because the missing portion 502 of the sensor data is repaired, there is no need to distinguish between the missing portion 502 and other portions of the sensor data, which simplifies the task recognition process.

[0081] (4) Fourth embodiment The work information management system according to the fourth embodiment has a configuration that is almost the same as the work information management systems according to the first, second, and third embodiments. Therefore, the same configuration will not be described again, and the following description will focus on the differences.

[0082] The fourth embodiment relates to task recognition using multiple types of sensors, similar to the third embodiment. The task information management system according to the fourth embodiment will be described mainly with reference to Fig. 1 or Fig. 8 and Fig. 10 .

[0083] The work recognition unit 109 shown in FIG. 1 or 8 utilizes an abnormality flag registered in the database 110 when the work recognition results indicate that multiple sensors recognize the same work action by the worker 100 as different work actions from each other.

[0084] The task recognition unit 109, for example, expresses the confidence level, which is an index as to whether a specific task performed by the worker 100 can be accurately recognized, as a number between "0" and "1" based on the sensor data of multiple sensors serving as the wearable sensor 102, for example, sensor A and sensor B shown in FIG. 10 .

[0085] In this embodiment, when the confidence level is close to "1," there is a high probability that the work was actually performed by the worker 100, whereas when the confidence level is close to "0," there is a low probability that the work was actually performed by the worker. In addition, this embodiment assumes a situation in which the confidence level of sensor A is high and the confidence level of sensor B is low for the same work action performed by the worker 100. Furthermore, it is assumed that the sensor data used to calculate the confidence level of sensor A contains abnormal data that includes missing portions.

[0086] In this situation, if the task recognition unit 109 determines that a different task recognition result will be obtained for the same task action of the worker based on multiple sensor data, at least one of which includes abnormal data, the task recognition unit 109 adjusts the confidence level, which indicates the reliability of the one of the sensor data including the abnormal data, to lower it, and performs subsequent task recognition according to the adjusted confidence level, for example. Note that, as a method for adjusting the confidence level, the task recognition unit 109 may uniformly lower the confidence level value, or may adjust the confidence level by multiplying it by a fixed rate. In this way, it is possible to effectively utilize sensor data including abnormal data while reducing the probability of erroneous recognition of task actions performed by the worker 100 in subsequent task recognition.

[0087] (5) Fifth embodiment The work information management system according to the fifth embodiment has a configuration that is almost the same as the work information management systems according to the first, second, third and fourth embodiments. Therefore, a description of the similar configuration will be omitted, and the following description will focus on the differences.

[0088] The fifth embodiment relates to a function for restoring sensor data. The work information management system of the fifth embodiment will be described with reference to mainly Figs. 5, 8, 11 and 12.

[0089] 11 is a system configuration diagram showing an example of the configuration of a work information management system according to the fifth embodiment. In the work information management system according to the fifth embodiment, the sensor data repair function described above differs from that of the previously described embodiments, and a data repair unit 113 is provided in the sensor data collection unit 106. The repair function of the data repair unit 113 in the fifth embodiment is the same as that of the previously described embodiments. In this way, the data collection management unit 104 executes the repair function, which relatively reduces the burden on the work recognition unit 109, allowing the work recognition unit 109 to recognize the work of the worker more quickly.

[0090] 12A shows a sensor value 700 of an acceleration sensor as channel CH1, and Fig. 12B shows a sensor value 701 of a gyro sensor as channel CH1. The illustrated example shows a specific component of sensor data of the acceleration sensor (sensor value 700 shown in Fig. 12A ) and a specific component of sensor data of the gyro sensor (sensor value 701 shown in Fig. 12B , etc.) of wearable sensor 102 (e.g., a 9-axis inertial sensor) as a glove-type sensor shown in Fig. 6 .

[0091] In this embodiment, it is assumed that the sensor value 701 of the gyro sensor includes, for example, a missing portion 702. Under this assumption, when the task recognition unit 109 refers to the sensor value 700 of the acceleration sensor, it determines that no particularly large acceleration occurs before or after the time when the missing portion 702 occurs in the sensor data of the gyro sensor, and based on this, determines that the missing portion 702 of the gyro sensor can be repaired by interpolating the missing portion 702 using both sensor values ​​at times before and after the missing portion 702, and repairs the missing portion 702.

[0092] In this embodiment, the sensor data in which the missing portion 702 described above has been repaired by the data repair unit 113 is registered in the database 110 with a repaired flag set as shown in Fig. 5. Here, setting a repaired flag means setting the value of the abnormality flag type "data loss repaired" shown in Fig. 5 to "4," the value of the abnormality flag type "abnormal value repaired" to "5," or the value of the abnormality flag type "communication device abnormality repaired" to "6."

[0093] The sensor data from multiple sensors may be sensor data from multiple sensors of a single sensor device 200, or may be sensor data from multiple sensor devices 200, 200A as shown in Fig. 8A. Also, missing portions may be repaired based on the sensor data sent via different wireless paths from the sensor devices 200, 200A each having a built-in wireless transmitter.

[0094] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, the elements described in parallel in the present embodiment may be configured such that at least one of the elements is connected in series to the other elements.

[0095] The present invention can be applied to a work information management system relating to technology for recognizing the physical movements of a worker.

[0096] 100...worker, 102...wearable sensor, 103...wireless transmitter, 104...data collection management unit, 105...wireless receiver, 106...sensor data collection unit, 107...time assignment function, 108...abnormality determination unit, 109...work recognition unit, 110...database, 111...external output I / F, 112...data recovery unit

Claims

1. A work information management system comprising: a receiver that receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more sensor data related to physical movements when the body acts on a work object through contact with the work object using the body; a sensor data collection unit that collects the sensor data; a database in which the sensor data collected by the sensor data collection unit is registered; and a work recognition unit that analyzes the sensor data registered in the database to recognize the work movements of the worker, wherein the sensor data collection unit has: a time assignment unit that acquires the time when the sensor data was received by the receiver; and an abnormality determination unit that determines whether the sensor data contains abnormal data, and even if it is determined that the sensor data contains abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data contains abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time.

2. The work information management system described in claim 1, characterized in that, when the abnormality determination unit determines that a defect has occurred in the sensor data, it classifies the type of abnormal data in which the defect has occurred, and registers the sensor data including the abnormal data in the database for each type of abnormal data.

3. A work information management system as described in claim 1 or claim 2, characterized in that the work recognition unit refers to the abnormality flag registered in the database, and based on the sensor data to which the abnormality flag has been assigned, recognizes the work actions of the worker according to the type of defect corresponding to the abnormality flag, and reflects this in the results of the work recognition.

4. The work information management system described in claim 3, characterized in that when the work recognition unit determines that a different work recognition result will be obtained for the same work action of the worker based on multiple sensor data, at least one of which includes the abnormal data, it adjusts the confidence level indicating the reliability of the one sensor data including the abnormal data to lower it, and performs work recognition from the next time onwards.

5. The work information management system described in claim 3, characterized in that the work recognition unit estimates missing parts corresponding to the abnormal data based on predetermined sensor data among the plurality of sensor data that does not contain the abnormal data, repairs the missing parts, and registers the sensor data from which the abnormal data has been repaired in the database.

6. The work information management system described in claim 5, characterized in that when the missing portion is repaired, the work recognition unit registers the sensor data from which the abnormal data has been repaired in the database along with an abnormality flag indicating that the missing portion has been repaired.

7. The work information management system described in claim 1, characterized in that the sensor device has a plurality of sensors that each output the sensor data, and the work recognition unit uses sensor data from one of the plurality of sensors to repair sensor data including the abnormal data from the other sensor.

8. The work information management system according to claim 1, characterized in that the sensor device has a transmitter that is worn on the body of the worker and transmits to the receiver one or more sensor data relating to the body movement when the body acts on the work object through contact with the work object using the body.

9. The work information management system according to claim 1, characterized in that the sensor device is configured such that at least one sensor that acquires the sensor data is a glove-type sensor worn by the worker.

10. The work information management system described in claim 1, characterized in that the sensor device comprises: a wearable sensor that is attached to the body of the worker and outputs one or more sensor data related to physical movements when the body acts on a work object through contact with the work object using the body; and a transmitter that transmits the sensor data output by the wearable sensor to the receiver.

11. A work information management system comprising: a receiving step in which a receiver receives sensor data from a wearable sensor device that is worn on the body of a worker and outputs one or more sensor data related to physical movements when the body acts on a work object through contact with the work object using the body; a sensor data collecting step in which a sensor data collecting unit collects the sensor data and registers it in a database; and a work recognition step in which a work recognition unit analyzes the sensor data registered in the database to recognize the work movements of the worker, wherein the sensor data collecting step comprises: a time assigning step in which a time assigning unit of the sensor data collecting unit acquires the time when the sensor data was received by the receiver; and an abnormality determining step in which an abnormality determining unit of the sensor data collecting unit determines whether the sensor data contains abnormal data, and even if it is determined that the sensor data contains abnormal data, does not discard the sensor data including the abnormal data, but assigns an abnormality flag indicating that the sensor data contains abnormal data, and registers the sensor data including the abnormal data in the database in association with the reception time.

Images