Work recognition system and work recognition method

Abstract

To reduce the processing load in task recognition. [Solution] The work recognition system comprises an acquisition unit that acquires sensor data indicating time-series physical quantities related to the worker's actions, a creation unit that creates event data by extracting feature portions from the acquired sensor data, and a recognition unit that recognizes the worker's work using the event data.

Description

Technical Field

[0006] , ,

[0005] , , ,

[0001] The present disclosure relates to a work recognition system and a work recognition method.

Background Art

[0002] Conventionally, a technique for recognizing the work being performed by a worker using time-series sensor data has been known. For example, in the behavior recognition device of Patent Document 1, a video is used as sensor data, and a recognition dictionary is created by machine learning using the video as learning data. Then, the behavior of the subject is recognized using the created recognition dictionary.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, if recognition processing is performed on all of the captured video, for example, as in the above technique, the processing load may become excessive.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to one aspect of the present disclosure, a work recognition system is provided. This work recognition system includes an acquisition unit that acquires sensor data indicating a time-series physical quantity related to the operation of a worker, a creation unit that creates event data by extracting a feature portion from the acquired sensor data, and a recognition unit that recognizes the work of the worker using the event data. In this configuration, the event data used by the recognition unit is data from only a portion of the total data period of the sensor data, thus reducing the amount of data compared to the sensor data. Therefore, the processing load on the recognition unit can be reduced. (2) In the work recognition system of the above form, the sensor data includes first sensor data detected by the first sensor and second sensor data detected by a second sensor of a different type from the first sensor, the creation unit creates first event data from the first sensor data and second event data from the second sensor data, and the recognition unit recognizes the work using the first event data and the second event data. In this configuration, the recognition unit can recognize the task using first and second event data of different types, thereby improving the accuracy of recognition. (3) In the work recognition system of the above form, a first extraction method for extracting the feature portion from the first sensor data and a second extraction method for extracting the feature portion from the second sensor data are predetermined, and the creation unit may create the first event data from the first sensor data using the first extraction method and create the second event data from the second sensor data using the second extraction method. With this configuration, when creating event data for the first sensor data, a first extraction method suitable for the first sensor data is used, and when creating event data for the second sensor data, a second extraction method suitable for the second sensor data is used, thereby enabling accurate extraction of feature portions. (4) In the work recognition system of the above form, a first extraction method for extracting the feature portion from the first sensor data is predetermined, and the creation unit may create the first event data from the first sensor data using the first extraction method, and create the second event data by extracting the physical quantities of the time series of the second sensor data for the same period as the feature portion of the first event data as the feature portion. In this configuration, the data creation unit extracts time-series physical quantities for the second sensor data over the same period as the feature portion of the first sensor data to create event data, thereby reducing the processing load on the data creation unit. (5) In the above-described work recognition system, the first sensor may be a camera set up so that at least a portion of the imaging range overlaps with the field of view of the worker. (6) In the work recognition system of the above form, the second sensor may be a sensor attached to the worker's hand and used to detect the worker's hand movements. This disclosure can be implemented in various forms, and in addition to a work recognition system, it can be implemented in the form of, for example, a work recognition method, a method for controlling a work recognition system, a computer program for causing a computer to execute the control method, or a non-temporary tangible recording medium in which the computer program is recorded in a readable manner. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing the system configuration. [Figure 2] This is a diagram illustrating the sensor. [Figure 3] This is a flowchart showing the procedure for task recognition processing. [Figure 4] This is a diagram explaining globe data. [Figure 5] This diagram illustrates steps S16 and S18. [Figure 6] This is a flowchart showing the procedure for the work recognition process in the second embodiment. [Figure 7] This figure illustrates the event data in the second embodiment. [Modes for carrying out the invention]

[0008] A. First Embodiment: Figure 1 is a block diagram showing the configuration of System 1 in an embodiment. System 1 comprises a work recognition system 100 and a sensor 200. The work recognition system 100 uses sensor data 300 (Figure 2) output from the sensor 200 to recognize the work being performed by worker WK (Figure 2). By recognizing the work being performed by worker WK, it is possible to perform evaluations, for example, whether the work is being performed according to the manufacturing process.

[0009] In this disclosure, “work” includes actions that are directly involved in the manufacturing process and actions that are indirectly involved. Direct actions include, for example, tightening screws, plugging in connectors, and carrying components. Indirect actions include, for example, moving to the next step in the process.

[0010] Figure 2 is a diagram illustrating the sensor 200. In this embodiment, the sensor 200 includes multiple types of sensors. In this disclosure, the types of sensors 200 are not divided by the physical quantity they detect, but by their purpose. Specifically, the sensor 200 may include a camera 210 for detecting the field of view of worker WK as an image, a glove sensor 220 for detecting the hand movements of worker WK, and a workwear sensor 230 for detecting the whole-body movements of worker WK. As will be described later, the glove sensor 220 includes multiple sensors to detect multiple types of physical quantities, such as pressure sensors and acceleration sensors. In this disclosure, the camera 210, the glove sensor 220, and the workwear sensor 230 are each treated as a single sensor. As described above, the camera 210, the glove sensor 220, and the workwear sensor 230 have different purposes, and therefore they are of different types. Various sensors can be used as sensor 200, but in this embodiment, we will explain an example in which sensor 200 includes a camera 210, a glove sensor 220, and a workwear sensor 230.

[0011] The camera 210 is attached to the head of the operator WK so that at least a part of the imaging range overlaps with the field of view of the operator WK. The camera 210 is also called a FPV (First Person View) camera or a first-person camera.

[0012] The glove sensor 220 is a sensor attached to the hand of the operator WK. Specifically, the glove sensor 220 includes, for example, a pressure sensor, a three-axis acceleration sensor, a three-axis gyro sensor, a geomagnetic sensor, and a microphone. There is one or more of these various sensors, and they are attached to the glove worn by the operator WK along the position of the fingertips and the hand skeleton. The combination of the acceleration sensor and the gyro sensor is also called an IMU (inertial measurement unit).

[0013] The workwear sensor 230 is a sensor attached to the whole body of the operator WK. Specifically, the workwear sensor 230 includes, for example, a three-axis acceleration sensor and a three-axis gyro sensor. There is one or more of these various sensors, and they are attached to the workwear worn by the operator WK along the body skeleton.

[0014] The sensor data 300 includes a plurality of sensor data 300 indicating time-series physical quantities related to the work of the operator WK. In the present embodiment, the sensor data 300 includes camera data 310, glove data 320, and workwear data 330.

[0015] The camera data 310 is time-series image data output from the camera 210, that is, video data. The glove data 320 is time-series data output from the glove sensor 220. As described above, the glove sensor 220 includes a plurality of sensors. Therefore, the glove data 320 is, specifically, an aggregate of data output from a plurality of sensors.

[0016] The work clothing data 330 is time-series data output from the work clothing sensor 230. As described above, the work clothing sensor 230 includes a plurality of sensors. Therefore, the work clothing data 330 is, in detail, an aggregate of data output from a plurality of sensors. The work clothing data 330 is data output from sensors attached along the skeleton of the body of the worker WK. For this reason, the work clothing data 330 is data capable of reproducing the skeleton of the worker WK by analyzing the work clothing data 330. For this reason, the work clothing data 330 is also called skeleton data.

[0017] As shown in FIG. 1, the work recognition system 100 is constituted by a computer including a processor 110, a memory 120, an input / output interface 130, a communication interface 140, and an internal bus 101. The processor 110, the memory 120, the input / output interface 130, and the communication interface 140 are connected so as to be capable of two-way communication via the internal bus 101. The memory 120 is realized by a RAM, a ROM, or the like. The input / output interface 130 exchanges data with an external device. The communication interface 140 performs wired or wireless communication with an external device. Specifically, the communication interface 140 performs, for example, LAN communication or short-range wireless communication. The communication interface 140 is realized by a communication module. [[ID=⑤]] [[ID=⑥]]

[0018] [[ID=⑦]] [[ID=⑧]]In the present embodiment, data exchange between the sensor 200 and the work recognition system 100 is performed by wired or wireless LAN communication via a communication device and the communication interface 140 not shown in the drawings. Note that communication between the sensor 200 and the work recognition system 100 may be performed via the input / output interface 130. Specifically, for example, data exchange between the camera 210 and the work recognition system 100 may be performed by LAN communication via the communication interface 140, or may be performed by communication via the input / output interface 130 and a video cable. [[ID=⑨]] [[ID=⑩]]

[0019] [[ID=⑪]] [[ID=⑫]]A display 102 and a speaker 103 are connected to the input / output interface 130.

[0020] The processor 110 functions as an acquisition unit 111, a creation unit 112, and a recognition unit 113 by executing the program PG1 stored in the memory 120. The acquisition unit 111 acquires sensor data 300. The creation unit 112 creates event data by extracting feature portions from the acquired sensor data 300. Here, the feature portion is the data portion that shows a time-series physical quantity for a part of the total data period of the sensor data 300. In the following description, the feature portion may be called an event. As will be described later, the feature portion is data for the period during which it is estimated that worker WK is performing work. The creation unit 112 extracts events from the sensor data 300 according to a predetermined extraction method. The recognition unit 113 recognizes the work being performed by worker WK using the extracted event data.

[0021] Memory 120 stores the program PG1, as well as the first model 121 and the second model 122. The first model 121 includes the first model 121a for cameras, the first model 121b for gloves, and the first model 121c for workwear. The first model 121 is a machine learning model that determines the work of worker WK from sensor data 300. The first model 121a for cameras is a machine learning model that determines the work of worker WK from camera data 310. The first model 121b for gloves is a machine learning model that determines the work of worker WK from glove data 320. The first model 121c for workwear is a machine learning model that determines the work of worker WK from workwear data 330.

[0022] Model 121 is a pre-trained machine learning model that has been trained using sensor data 300. Model 121 is trained using a training dataset in which sensor data 300 is associated with task labels. Examples of task labels include "tightening" which indicates tightening a screw, and "connecting" which indicates plugging in a connector. When sensor data 300 is input, Model 121 outputs results that associate the calculated task labels and probabilities with time.

[0023] The second model 122 is a machine learning model that integrates multiple results obtained using the first model 121 to determine the task of worker WK. By using the second model 122, the task can be inferred integrally using the results of multiple first models 121 corresponding to multiple sensors 200, thereby improving the accuracy of task recognition. For example, a convolutional neural network (CNN) can be used as the first model 121 and the second model 122.

[0024] In this embodiment, the work recognition system 100 recognizes the work of worker WK in real time. This allows for real-time comparison with a predetermined manufacturing process procedure. If the work performed by worker WK differs from the procedure, the system can warn worker WK, for example, by displaying a warning on the display screen 102 or emitting a warning sound from the speaker 103.

[0025] As described above, the work recognition system 100 performs data processing on multiple sensor data 300 using the first model 121. If the work recognition system 100 is implemented on a computer with low processing speed, real-time work recognition may not be possible. Therefore, in this embodiment, the amount of sensor data 300 to be read by the first model 121 is reduced. This reduces the processing load on the work recognition system 100.

[0026] Figure 3 is a flowchart showing the procedure for work recognition processing performed by the processor 110. The work recognition method is realized by performing the work recognition processing. As shown in Figure 3, in step S10, the acquisition unit 111 acquires sensor data 300 from each sensor 200. Specifically, the acquisition unit 111 acquires camera data 310 transmitted from the camera 210, glove data 320 transmitted from the glove sensor 220, and workwear data 330 transmitted from the workwear sensor 230, and stores them in the memory 120.

[0027] In step S12, the creation unit 112 creates event data from the sensor data 300. Here, event data is data extracted from the period during which worker WK is estimated to be performing work. When worker WK is actually performing work in the manufacturing process, in between tasks such as tightening screws, they may perform actions such as walking to check the work area or stopping to look around. Even if the period during which such actions are performed is excluded from the sensor data 300, it is unlikely that the accuracy of work recognition by the first model 121 will decrease. By using event data as input data for the first model 121 instead of sensor data 300, the processing load can be reduced while maintaining recognition accuracy.

[0028] In this embodiment, an event extraction method is predetermined for each sensor data 300, that is, for each of the camera data 310, glove data 320, and workwear data 330. This allows for accurate event extraction for each sensor data 300.

[0029] The method for extracting event data will be explained using globe data 320 as an example. Figure 4 is a diagram illustrating globe data 320. As mentioned above, globe data 320 contains multiple types of data such as pressure and acceleration, but for ease of understanding, the following explanation will use the pressure data and sound data included in globe data 320.

[0030] The glove sensor 220 is worn on the hand of worker WK. Therefore, for example, when worker WK grasps an object, the detected value of the pressure sensor increases. In other words, when the detected value of the pressure sensor increases, it can be determined that worker WK has started work. Also, for example, when worker WK inserts the cable connector into the socket and connects it, a sound is emitted when the latch that prevents the connection from being released on the connector engages. Therefore, when the detected value of the microphone increases, it can be determined that worker WK has started work. In this embodiment, the event start time is set when the detected value of the pressure sensor exceeds a predetermined threshold Pth. Similarly, the event start time is set when the detected value of the microphone exceeds a predetermined threshold Lth.

[0031] In this embodiment, the end time of the event is set to a predetermined event time TD that has elapsed from the start time of the event. The period shown by the hatched lines in Figure 4 is the period for which event data is extracted. The creation unit 112 extracts data from various sensors during the period for which event data is extracted and creates event data. Event data is data in which time and partial sensor data extracted from sensor data 300 are associated.

[0032] The method for extracting event data is not limited to the above. For example, the end time of an event may be the time when the sensor's detected value falls below a predetermined threshold. Alternatively, the start time of an event may be set when the detected values ​​of all multiple sensors exceed a threshold. Furthermore, the start or end time of an event may be set based on the change in the detected value instead of the detected value itself. Specifically, the start time of an event may be set when the change in the detected value exceeds a predetermined baseline change. Event data may also be extracted using artificial intelligence technology. Specifically, event data may be extracted using a machine learning model trained on a training dataset that associates sensor data with the period during which the work was performed. In general, the extraction process for extracting event data has a lower processing load than the recognition process for recognizing the work.

[0033] The creation unit 112 extracts event data from the sensor data 300 for the work clothes sensor 230, similar to the glove sensor 220. In other words, for the various sensors included in the work clothes sensor 230, the creation unit 112 sets the event start time to the time when the detected value exceeds a predetermined threshold. The creation unit 112 sets the event end time to the time when the event time TD has elapsed from the event disclosure time.

[0034] The creation unit 112 extracts event data from the camera data 310 using the change in frames. Specifically, it sets the start time of the event as the time when the difference in density or density histogram between two consecutive frames becomes greater than a predetermined reference difference. The creation unit 112 sets the end time of the event as the time when the event time TD has elapsed from the event disclosure time.

[0035] The method for calculating the difference is not particularly limited. For example, in the case of density, the average density of all pixels in each of the two frames may be calculated, and the difference between these averages may be calculated. Alternatively, the difference in density of each pixel in the two frames may be calculated first, and then the average of these differences may be calculated. Furthermore, the difference may be calculated for a predetermined portion of the image rather than the entire frame. In addition, the creation unit 112 may calculate the density difference between the two frames and set the time when the ratio of the area of ​​the calculated difference in the screen to the total area becomes larger than a predetermined reference ratio as the start time of the event.

[0036] Worker WK switches their viewpoint when performing the next task. For example, worker WK looks at the entire workspace to walk towards the workbench. Then, when worker WK arrives at the workbench, they look at the workbench. Therefore, when the view switches from the image of the entire workspace to the image of the workbench, it can be determined that work has started. When the screen switches, the density and density histogram between two consecutive frames change. For this reason, the change in frame can be used to extract the event.

[0037] Alternatively, instead of switching from an image of the entire workspace to an image of the workbench, the start of work may be determined when the image of the workbench continues for a predetermined standard time. This determination utilizes the fact that worker WK does not move their gaze from the workbench while working at it. Specifically, the creation unit 112 sets the start of the event as the period during which the amount of change between two consecutive frames is smaller than a predetermined standard amount of change, and this period is longer than the standard period. In addition, the creation unit 112 may also use artificial intelligence technology to extract event data from the camera data 310, similar to the globe data 320.

[0038] In step S16 of Figure 3, the recognition unit 113 recognizes the work using the event data. Specifically, the recognition unit 113 inputs camera data 310 into the first camera model 121a to obtain the result of the recognized work. The recognition unit 113 inputs glove data 320 into the first glove model 121b to obtain the result of the recognized work. The recognition unit 113 inputs workwear data 330 into the first workwear model 121c to obtain the result of the recognized work.

[0039] Figure 5 illustrates steps S16 and S18. In this embodiment, event data is extracted for each of the camera data 310, glove data 320, and workwear data 330. Therefore, the periods extracted as event data may not coincide for multiple sensors. For this reason, as illustrated in Figure 5, a "walking" recognized using the event data of the camera data 310 may not exist in the tasks recognized using the event data of the glove data 320. In Figure 5, for each event, the task with the highest probability, as determined by the first model 121, is shown. The first model 121 outputs the recognized tasks in association with their probabilities. Specifically, if the first model 121 recognizes multiple tasks as "fastening" or "joining" for a single event, it outputs results such as a probability of 0.6 for "fastening" and a probability of 0.4 for "joining".

[0040] In step S18 of Figure 3, the recognition unit 113 performs integrated recognition using the results of the first model 121 and the second model 122. Specifically, the recognition unit 113 inputs the results of the first model 121 into the second model 122 and uses the output result as the final result. "Integrated recognition" in Figure 5 refers to the result of step S18. As illustrated in Figure 5, if both the task recognized using camera data 310 and the task recognized using glove data 320 are "fastening," and the task recognized using workwear data 330 is "joining," the result of integrated recognition may be "fastening." Thus, for example, if the correct task is "fastening," even if the task recognized using workwear data 330 is incorrect, the results of the first model 121 using multiple sensors 200 are integrated and recognized by the second model 122, allowing the correct task to be recognized. The inventors have confirmed that the accuracy of task recognition is improved by performing integrated recognition.

[0041] As shown in Figure 3, this processing routine terminates after step S18. This processing routine is repeatedly performed as a subroutine of a processing routine that determines, for example, whether the work being performed by worker WK matches the predetermined manufacturing process procedure. For illustrative purposes, Figure 4 shows event data containing multiple events, but event data does not necessarily contain multiple events. To achieve real-time work recognition, this processing routine may be performed, for example, at least every time the event time TD has elapsed. Also, to facilitate the extraction of event data, the extraction of event data in step S12 and the recognition of work from step S16 onward may be performed in parallel.

[0042] Furthermore, the above describes an example where this processing routine is used to determine whether the work being performed by worker WK matches the predetermined manufacturing process procedures. This processing routine can also be used, for example, for training workers WK to learn the manufacturing process or for evaluating the workers' proficiency.

[0043] Furthermore, the above example illustrates a case where sensor 200 includes a camera 210, a glove sensor 220, and a workwear sensor 230. The types of sensors included in sensor 200 are not limited to those described above. For example, sensor 200 may include a sensor for detecting the position of worker WK, or a microphone for detecting the worker WK's voice and ambient sounds. Also, sensor 200 is not limited to sensors attached to worker WK. For example, sensor 200 may be a camera that images worker WK, or a sensor attached to a tool.

[0044] Camera 210 is also called the first sensor, and camera data 310 is also called the first sensor data. Globe sensor 220 is also called the second sensor, and globe data 320 is also called the second sensor data. Event data created from camera data 310 is also called the first event data. The extraction method for extracting feature portions from camera data 310 is called the first extraction method. Event data created from globe data 320 is also called the second event data. The extraction method for extracting feature portions from globe data 320 is called the second extraction method.

[0045] According to the first embodiment described above, the work recognition system 100 comprises an acquisition unit 111, a creation unit 112, and a recognition unit 113. The creation unit 112 creates event data by extracting feature portions from the sensor data 300 acquired by the acquisition unit 111. The recognition unit 113 uses the event data to recognize the work being performed by worker WK. Since the recognition unit 113 recognizes the work using event data, which has a smaller data volume than the sensor data 300, the processing load on the work recognition system 100 can be reduced.

[0046] Furthermore, the acquisition unit 111 acquires multiple sensor data 300. The creation unit 112 creates multiple event data from the multiple sensor data 300. The recognition unit 113 performs integrated recognition to recognize the task using the multiple event data. This improves the accuracy of recognition compared to recognizing the task using only one event data.

[0047] Furthermore, the acquisition unit 111 creates event data for each sensor data 300 using a corresponding extraction method. This allows for the use of an extraction method suitable for each sensor data 300, enabling accurate extraction of characteristic features.

[0048] B. Second Embodiment: In the above embodiment, the start time of an event is set individually for each of the multiple sensor data 300. In this embodiment, the creation unit 112 creates event data for one of the multiple sensor data 300 using a predetermined extraction method. Then, the creation unit 112 uses the start time of the event of the previously created event data to create event data for the other sensor data 300. The same reference numerals are used for the same components and processing steps as in the first embodiment, and detailed explanations are omitted as appropriate.

[0049] In this embodiment, we will explain the case where the sensor data 300 for setting the start time of an event is camera data 310. The sensor data 300 for setting the start time of an event is referred to as the first sensor data.

[0050] Figure 6 is a flowchart showing the procedure for action recognition processing in this embodiment. As shown in Figure 6, in step S10, the acquisition unit 111 acquires sensor data 300 from the sensor 200. In step S13, the creation unit 112 creates event data from the camera data 310, which is the first sensor data, similar to the first embodiment.

[0051] In step S15, the creation unit 112 creates event data from the sensor data 300 other than the first sensor data. Specifically, the creation unit 112 extracts time-series physical quantities for the same period as the feature portion of the sensor data of the camera data 310, which is the first sensor data, and creates event data for the glove data 320 and work clothes data 330. More specifically, the creation unit 112 uses the event start time and event end time of the event data extracted in step S13 to create event data for the glove data 320 and work clothes data 330. Figure 7 is a diagram illustrating the event data in this embodiment. The hatched areas shown in Figure 7 are the extracted events.

[0052] When worker WK starts working, they often change their gaze. Therefore, when worker WK starts working, it is highly likely that this will be extracted as an event in camera data 310. Consequently, when extracting event data from other sensor data 300 using the event data from camera data 310, work can be extracted without any omissions.

[0053] In step S16 of Figure 6, the recognition unit 113 recognizes the task using the event data. In step S18, the recognition unit 113 performs integrated recognition using the results of the first model 121 and the second model 122.

[0054] According to the second embodiment described above, in step S13, the creation unit 112 creates event data using the camera data 310. In step S15, the creation unit 112 extracts time-series physical quantities from the glove data 320 and work clothes data 330 for the same period as the feature portion of the sensor data of the camera data 310, and creates event data for the glove data 320 and work clothes data 330, respectively. As a result, for sensor data 300 other than camera data 310, event data is created by extracting time-series physical quantities for the same period as the feature portion of the camera data 310, thus reducing the processing load on the creation unit 112.

[0055] C. Other embodiments: (C1) In the first embodiment described above, the recognition unit 113 recognizes the work using the first model 121 with multiple event data, and then further recognizes the work using the result of the first model 121 and the second model 122. In another form, the recognition unit 113 may recognize the work using event data created from a single sensor data 300, and this result may be used as the final result. Even when a single event data is used to obtain the final result, the processing load can be reduced by using event data with a reduced amount of data instead of the sensor data 300.

[0056] (C2) In the second embodiment described above, the event data created first is the event data of the camera data 310. The sensor data 300 for which event data is created first is not limited to the camera data 310. Preferably, the sensor data 300 for which event data is created first is the data of the sensor 200 that can detect all of the work performed by worker WK without fail.

[0057] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features of the embodiments corresponding to the technical features in each form described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of symbols]

[0058] 1...System, 100...Work Recognition System, 101...Internal Bus, 102...Display, 103...Speaker, 110...Processor, 111...Acquisition Unit, 112...Creation Unit, 113...Recognition Unit, 120...Memory, 121...First Model, 121a...First Model for Camera, 121b...First Model for Gloves, 121c...First Model for Workwear, 122...Second Model, 130...Input / Output Interface, 140...Communication Interface, 200...Sensor, 210...Camera, 220...Glove Sensor, 230...Workwear Sensor, 300...Sensor Data, 310...Camera Data, 320...Glove Data, 330...Workwear Data, PG1...Program, WK...Worker

Claims

1. A work recognition system, An acquisition unit that acquires sensor data showing time-series physical quantities related to the worker's movements, A creation unit that creates event data by extracting feature portions from the acquired sensor data, A work recognition system comprising a recognition unit that recognizes the work of the worker using the event data.

2. A work recognition system according to claim 1, The sensor data includes first sensor data detected by the first sensor and second sensor data detected by a second sensor of a different type from the first sensor. The creation unit creates first event data from the first sensor data, and creates second event data from the second sensor data. The recognition unit is a work recognition system that recognizes the work using the first event data and the second event data.

3. A work recognition system according to claim 2, A first extraction method for extracting the feature portion from the first sensor data and a second extraction method for extracting the feature portion from the second sensor data are predetermined. The creation unit is a work recognition system that creates first event data from first sensor data using the first extraction method and creates second event data from second sensor data using the second extraction method.

4. A work recognition system according to claim 2, A first extraction method for extracting the feature portion from the first sensor data is predetermined. The aforementioned creation unit, Using the first extraction method, the first event data is created from the first sensor data. A work recognition system that creates second event data by extracting the time-series physical quantities of the second sensor data for the same period as the feature portion of the first event data as the feature portion.

5. A work recognition system according to any one of claims 2 to 4, The first sensor is a camera set so that at least a portion of the imaging range overlaps with the field of view of the worker, in this work recognition system.

6. A work recognition system according to claim 5, The second sensor is a work recognition system, which is attached to the worker's hand and detects the worker's hand movements.

7. A method for recognizing work, We acquire sensor data that shows time-series physical quantities related to the worker's movements. Event data is created by extracting feature portions from the acquired sensor data. A work recognition method that recognizes the work of the worker using the aforementioned event data.

Images