Work analysis device, work analysis method, and recording medium

Abstract

This work analysis device comprises an acquisition unit and a first generation unit. The acquisition unit acquires sensor information that has been generated by using at least one sensor, and is capable of identifying movement of a robot in a work area. The first generation unit generates, by using the sensor information, work information indicating, in time series, the content of work performed by the robot.

Description

Work analysis device, work analysis method, and recording medium

[0001] The present invention relates to a task analysis device, a task analysis method, and a recording medium.

[0002] Understanding the movements of workers is important for work analysis. For example, Patent Literature 1 describes a method of acquiring an actual work time based on a video of a worker performing an actual work, and updating the fastest work time to the actual work time if the fastest work time is shorter than the fastest work time stored in a storage unit. Patent Literature 1 also describes a method of extracting an optimal work procedure based on the updated fastest work time using a trained model that has learned about the fastest work time and the suitability of the work, and updating the work procedure stored in the storage unit to the extracted optimal work procedure.

[0003] International Publication No. 2022 / 091571

[0004] In recent years, robots have been performing tasks. For example, the performance of a robot may be indicated by the robot's provider. However, this performance is not always demonstrated in the actual task. For this reason, it becomes necessary to evaluate the tasks performed by the robot. The technology described in Patent Document 1 does not take robots into consideration.

[0005] The work analysis device of the present disclosure includes an acquisition means for acquiring sensor information generated using at least one sensor, the sensor information being capable of identifying the movement of the robot within a work area; and a first generation means for using the sensor information to generate work information that indicates the content of the work performed by the robot in chronological order.

[0006] The work analysis method disclosed herein involves a computer acquiring sensor information, which is generated using at least one sensor and is capable of identifying the movement of a robot within a work area, and using the sensor information to generate work information that chronologically indicates the content of the work performed by the robot.

[0007] The recording medium of the present disclosure records a program that causes a computer to have: an acquisition means for acquiring sensor information, which is information generated using at least one sensor and is capable of identifying the movement of the robot within a work area; and a first generation means for using the sensor information to generate work information that indicates the content of the work performed by the robot in chronological order.

[0008] According to the present disclosure, the work performed by a robot can be evaluated.

[0009] FIG. 1 is a diagram illustrating an example of a usage environment of a work analysis device according to the present disclosure. FIG. 2 is a diagram illustrating an example of a usage environment of a work analysis device according to the present disclosure. FIG. 3 is a diagram illustrating an example of a functional configuration of a work analysis device according to the present disclosure. FIG. 4 is a diagram illustrating an example of a hardware configuration of a work analysis device according to the present disclosure. FIG. 5 is a flowchart illustrating an example of processing performed by a work analysis device according to the present disclosure. FIG. 6 is a diagram illustrating an example of a usage environment of a work analysis device according to the present disclosure. FIG. 7 is a diagram illustrating an example of a functional configuration of a work analysis device according to the present disclosure.

[0010] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In addition, in all drawings, similar components are given similar reference numerals and descriptions thereof will be omitted as appropriate.

[0011] As shown in FIG. 1 , the work analysis device 10 is used with a robot 30 and multiple sensors 20. The robot 30 performs work within a work area. The robot 30 is, for example, self-propelled and has a mechanism, such as an arm, for moving an object. An example of the robot 30 is at least one of an AGV (Automated Guided Vehicle) and an AMR (Autonomous Mobile Robot). The robot 30 moves to a location where work is to be performed and then performs the work using a mechanism such as an arm. The sensor 20 generates sensor information that can identify the movement of the robot 30 within the work area and transmits the generated sensor information to the work analysis device 10. The work analysis device 10 generates work information using the sensor information. The work information indicates the content of work performed by the robot 30 in chronological order. However, the robot 30 may not be self-propelled. As an example, the robot 30 may have an arm that processes an object.

[0012] A work area is an area where multiple robots 30 perform work simultaneously. As an example, a work area may be a location where construction or civil engineering work is being carried out, or inside a facility or building such as a factory or warehouse, but is not limited to these. There may be one type of work or multiple types of work being performed in one work area. Furthermore, one type of work may be a collection of multiple types of sub-work. In this way, there are various definitions of one type of work.

[0013] An example of the sensor 20 is an imaging device. When the sensor 20 is an imaging device, the sensor 20 generates an image by capturing an image of the work area. It is preferable that a plurality of imaging devices are provided.

[0014] The frame rate of images generated by the imaging devices is arbitrary, but is, for example, 5 frames per second (5 fps) or higher. The imaging ranges of the multiple imaging devices may partially overlap or may differ. Furthermore, there may be areas in the work area that are not included in the imaging ranges of any of the imaging devices, or there may be no such areas. Note that, for example, when the work area is a warehouse, there may be an obstruction, such as a shelf, within the work area. In this case, it is preferable to determine the positions of the multiple imaging devices so that no areas are hidden by the obstruction.

[0015] Another example of the sensor 20 is a sensor mounted on the robot 30. In this example, the sensor 20 is at least one of a sensor capable of detecting the position of the robot 30, such as a GPS, a sensor capable of detecting the orientation of the robot 30, and a sensor capable of detecting the movement of the arm of the robot 30. Note that the information generated by the sensor 20 may be transmitted to the work analysis device 10 via a control unit of the robot 30. Furthermore, control information for controlling the movement of the robot 30 may be used instead of sensor information.

[0016] It should be noted that both the imaging device and the sensor mounted on the robot 30 may be used as the sensor 20. Furthermore, other sensors may be additionally used as the sensor 20.

[0017] As shown in Figure 2, a person, i.e., a worker, may work in the work area in addition to the robot 30. In this case, sensor information generated by at least one sensor 20, for example, images generated by an imaging device, can also identify the movements of the person in the work area. The work information generated by the work analysis device 10 also indicates the details of the work performed by the person in chronological order. Note that work performed by a person may be performed directly by a person or may be performed using a tool. The tool used here includes equipment such as a forklift.

[0018] As shown in FIG. 3 , the work analysis apparatus 10 includes an acquisition unit 110 and a first generation unit 120 .

[0019] The acquisition unit 110 acquires sensor information. The acquisition unit 110 may acquire the sensor information directly from the sensor 20, or may acquire the sensor information from a storage device that stores the sensor information.

[0020] The first generation unit 120 generates work information using the sensor information. In the example shown in FIG. 1 , the work information indicates the content of work performed by the robot 30 in chronological order. If there are multiple robots 30, the work information indicates the content of work performed by each of the multiple robots 30 in chronological order. On the other hand, in the example shown in FIG. 2 , the work information indicates the content of work performed by the robot 30 in chronological order, as well as the content of work performed by a person in chronological order. If there are multiple people, the work information indicates the content of work performed by each of the multiple people in chronological order.

[0021] For example, taking items from a warehouse, placing them in a tray or cardboard box, and carrying them to a designated location may be defined as one type of work. Also, taking items from a warehouse and placing them in a tray or cardboard box and carrying the container to a designated location may be defined as two different types of work.

[0022] A detailed example of the processing performed by the first generation unit 120 will be described later.

[0023] The work analysis apparatus 10 can use a storage unit 40. The storage unit 40 stores various types of information used by the work analysis apparatus 10. For example, the storage unit 40 stores sensor information, such as images, generated by the sensor 20. The acquisition unit 110 may acquire sensor information from the storage unit 40. The acquisition unit 110 may also store sensor information acquired from the sensor 20 in the storage unit 40.

[0024] The work analysis device 10 has, as a hardware configuration, for example, as shown in FIG. 4, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0025] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0026] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0027] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0028] The storage device 1040 is an auxiliary storage device realized by removable media such as a hard disk drive (HDD), a solid state drive (SSD), a memory card, or a read-only memory (ROM), and includes a recording medium. The recording medium of the storage device 1040 stores program modules that realize each function of the work analysis apparatus 10 (e.g., the acquisition unit 110 and first generation unit 120, as well as the second generation unit 130, third generation unit 140, fourth generation unit 150, and cost calculation unit 160, which will be described later). The processor 1020 loads each of these program modules into the memory 1030 and executes them to realize each function corresponding to the program module. The storage device 1040 may also function as a memory unit (e.g., the memory unit 40) that stores various information.

[0029] The input / output interface 1050 is an interface for connecting the work analysis device 10 to various input / output devices.

[0030] The network interface 1060 is an interface for connecting the work analysis apparatus 10 to a network. This network may be, for example, a local area network (LAN) or a wide area network (WAN). The network interface 1060 may connect to the network wirelessly or via a wired connection. The work analysis apparatus 10 may communicate with the sensor 20, as well as a sensor 22 and a communication device 50 (described later), via the network interface 1060.

[0031] The work analysis device 10 may be realized by a plurality of devices each having the configuration shown in FIG.

[0032] The work analysis apparatus 10 operates, for example, as shown in FIG. 5 . First, the acquisition unit 110 acquires multiple pieces of sensor information generated by multiple sensors 20. For example, the acquisition unit 110 acquires multiple frame images generated by each of multiple imaging devices (step S10). Next, the first generation unit 120 processes the multiple pieces of sensor information to generate work information (step S20). The generated work information is stored, for example, in the storage unit 40. This process is performed, for example, by the first generation unit 120.

[0033] The work analysis device 10 may perform processing in real time, i.e., while the robot 30 is working in the work area, or may perform processing using accumulated sensor information after the robot 30 has completed work in the work area.

[0034] In this way, the work analysis device 10 outputs work information. The work information indicates at least the content of the work performed by the robot 30 in chronological order. Therefore, a manager who manages the work area can use the work information to evaluate the work performed by the robot 30. One example of this evaluation is identifying areas for improvement. Examples of areas for improvement include changes in the division of work among multiple robots 30 and improvements to the work procedures of one robot 30. Hereinafter, information indicating areas for improvement will be referred to as improvement information.

[0035] Furthermore, if the work information also indicates the details of the work performed by a person in chronological order, the manager who manages the work area can also evaluate the work performed by the person.

[0036] A detailed example of the processing performed by the first generating unit 120, and detailed examples of the aligned task information and improvement information will be described below.

[0037] As described above, the first generation unit 120 generates work information by processing sensor information generated by each of the multiple sensors 20, for example, images generated by an imaging device. The work information indicates, in chronological order, the content of work performed by the robot 30. If a person is present in the work area, the work information also indicates, in chronological order, the content of work performed by the person.

[0038] For example, if there are multiple imaging devices as at least a part of the sensor 20, the first generation unit 120 processes the images generated by each of the multiple imaging devices to identify each of the multiple robots 30 and people, and to identify the type of work being performed by each of the multiple robots 30 and each of the multiple people. The first generation unit 120 then integrates the processing results for each imaging device.

[0039] Furthermore, if at least a part of the sensors 20 includes a sensor that detects the movement of the robot 30, the first generation unit 120 uses the detection results of this sensor to identify the movement of the robot 30 in time series, and uses this identification result to identify the type of work being performed by the robot 30. In this case, the sensor information output by this sensor 20 includes identification information that distinguishes the robots 30 from one another. The first generation unit 120 uses this identification information to identify the robot 30.

[0040] In this way, the first generation unit 120 generates work information. The work information includes, for example, work type information indicating the content of the work being performed by a certain robot 30 or person, i.e., the type of work, and the time when the robot 30 or person started the work. If the work has been completed, the work information may further include the time when the robot 30 or person finished the work. In this case, the work information includes information indicating, for each robot 30 or person, the type of work performed by that robot 30 or person in chronological order. Hereinafter, this information will be referred to as time-series information.

[0041] [Identifying robots 30 and people in processing using images] The first generation unit 120 identifies each of the multiple robots 30 and people using, for example, at least one of the following methods (1-1) to (1-6), and determines whether the robots 30 or people included in each of the multiple images generated by the multiple imaging devices are the same robots 30 or people.

[0042] (1-1) The first generation unit 120 identifies the robot 30 and the person using their appearance. For example, in the case of a robot 30, an example of the appearance is at least one of a mark, a color, and a character string (including numbers) attached to the outer surface of the robot 30. In the case of a person, an example of the appearance is at least one of a face, a human figure, and an attached item. An example of an attached item is at least one of clothing and tools. In this case, the first generation unit 120 may identify the robot 30 and the person by recognizing features of the appearance. Using this method, the first generation unit 120 can determine whether the person included in the image generated by the first imaging device is the same as the person included in the image generated by the second imaging device.

[0043] (1-2) If a code such as a barcode or a two-dimensional code is attached to the robot 30 or the person's clothing, the first generation unit 120 identifies the robot 30 and the person by recognizing the code. Examples of clothing include clothing, a bib, or a helmet. Using this method, the first generation unit 120 can determine whether the robot 30 or the person included in the image generated by the first imaging device is the same as the robot 30 or the person included in the image generated by the second imaging device.

[0044] (1-3) The first generation unit 120 identifies the type of work being performed by the robot 30 or the person, and uses this type of work to determine the identity of the robot 30 or the person. As an example, if the type of work being performed by the robot 30 or the person included in the image generated by the first imaging device is the same as the type of work being performed by the robot 30 or the person included in the image generated by the second imaging device, the first generation unit 120 determines that there is a high possibility that the robots 30 or the people included in these images are the same.

[0045] (1-4) The first generation unit 120 identifies the position of the robot 30 or person in the image and converts this position into a position in real space, i.e., a position in the world coordinate system, by converting it according to a predetermined conversion rule. Then, using the identity of this position, it determines whether the robot 30 or person included in the image generated by the first imaging device is the same as the robot 30 or person included in the image generated by the second imaging device. This process is effective when at least a part of the imaging area of ​​the first imaging device overlaps with the imaging area of ​​the second imaging device.

[0046] (1-5) The first generation unit 120 uses a rule based on feasible movement lines in the work area to determine whether the robot 30 or person included in the image generated by the first imaging device is the same as the robot 30 or person included in the image generated by the second imaging device. For example, there is a rule that if there is a passage in the work area and the first imaging device captures an image of the entrance of the passage and the second imaging device captures an image of the exit of the passage, the robot 30 or person entering through the entrance will exit through the exit or entrance. The first generation unit 120 uses this rule to determine whether the robot 30 or person included in the image generated by the first imaging device is the same as the robot 30 or person included in the image generated by the second imaging device.

[0047] (1-6) When multiple robots 30 and people each have a transmitting device, such as a beacon or a wireless communication tag, that transmits different identification information over a short communication distance, and multiple receiving devices are placed in the work area, the first generating unit 120 may use the identification information received by the multiple receiving devices to identify the positions of the multiple robots 30 and people in the world coordinate system.The first generating unit 120 then uses these positions to identify which imaging device generated the image that includes each of the multiple robots 30 and people.

[0048] [Method for Identifying Task Type] The first generating unit 120 generates task type information using, for example, at least one of the postures of the robot 30 and the person and the tools they are using. The first generating unit 120 identifies the task type using, for example, at least one of the following methods (2-1) to (2-3). For example, if the task area is a warehouse, task types include stocking, picking, transporting, inspection, packing, and rest. Note that in each of (2-1) to (2-3), the first generating unit 120 also calculates a value indicating the likelihood that the task type is that of the robot 30 and the person, i.e., a reliability value.

[0049] (2-1) The first generation unit 120 processes multiple images, for example, multiple frame images generated by the same imaging device, to generate information arranging the postures of the robot 30 in chronological order and information arranging the postures of the person in chronological order, i.e., to identify the movements of the robot 30 and the person. The first generation unit 120 may generate information arranging the postures of the robot 30 in chronological order using sensors attached to the robot 30 or control information of the robot 30. The first generation unit 120 then uses this movement to identify the type of work and the start time of the work. For example, information indicating the movements corresponding to each type of work is registered in advance in the work analysis device 10 or an external storage device. The first generation unit 120 then compares this information with the movements of the robot 30 or the person to identify the type of work and the start time of the work. Note that if two or more imaging devices capture the same area but are oriented differently relative to the area, the first generation unit 120 may use images generated by these two or more imaging devices at the same time when identifying the posture.

[0050] (2-2) The first generation unit 120 processes at least one image to identify a tool used by at least one person or at least one robot 30. The first generation unit 120 then determines that the task using this tool is the task being performed by that person or robot 30. For example, information indicating the task using each of multiple tools is registered in advance in the work analysis device 10 or an external storage device. The first generation unit 120 uses this information. Note that if two or more image capture devices capture the same area but are oriented differently relative to the area, the first generation unit 120 may use images generated by these two or more image capture devices at the same time when identifying the tool.

[0051] (2-3) When using both (2-1) and (2-2), the first generation unit 120 differentiates the imaging device to which (2-1) is applied from the imaging device to which (2-2) is applied. For example, when the same area is photographed by two or more imaging devices and the orientations of these two or more imaging devices relative to this area are different, an image that makes it easy to identify the pose and an image that makes it easy to identify the tool may differ. In such a case, it is preferable that the first generation unit 120 differentiates the imaging device to which (2-1) is applied from the imaging device to which (2-2) is applied.

[0052] The first generation unit 120 also uses the sensor information to identify, for each of the multiple people and the multiple robots 30, a time period during which the person or robot 30 is not performing a task. Hereinafter, this time period will be referred to as a non-task time period. A non-task time period may occur due to a waiting time caused by the absence of an item to be worked on, a break between tasks, or the like. For example, the first generation unit 120 determines that the person or robot 30 is not performing a task if the person or robot 30 deviates from the movement that would be expected when the person or robot 30 is performing a task. Here, the first generation unit 120 may further add to the criteria for determination whether the person is not moving infrequently or whether the robot 30 is not moving. For example, the first generation unit 120 determines that the person is not performing a task if the person is conversing with others but has few other movements. The first generation unit 120 includes the non-task time period in at least a portion of the task information.

[0053] [Integrating the type of work for each imaging device] The same robot 30 or person may be captured on two or more imaging devices at the same time. In this case, the type of work that the person is performing is identified for each of the two or more imaging devices. The first generation unit 120 integrates these multiple identification results into a single type of work, for example, using at least one of the following methods (3-1) to (3-3).

[0054] In other words, the first generator 120 identifies the type of work performed by the same person at the same time from each of the images generated by at least two image capture devices, and if at least one of the identified work types is different from the others, generates work type information according to a predetermined rule. For example, this rule may relate to the reliability of the work type or to consistency with external information generated using a sensor used in the work area.

[0055] (3-1) When the types of tasks are different from each other, the first generation unit 120 adopts the type of task with the highest reliability.

[0056] (3-2) When the types of tasks are different from one another, the first generation unit 120 adds up the reliability for each task type and adopts the task type with the highest reliability after addition. At this time, the reliability may be weighted, for example, depending on the image capture device. This weighting is set, for example, depending on the position of the image capture device.

[0057] (3-3) Predetermined sensor information may be generated based on the detection results of the human-operated sensor 20. This sensor information is stored, for example, in an external information storage unit different from the storage unit 40. Using this sensor information, it may be possible to identify the type of work performed in the work area, the start time of that work, and the end time of that work. If the type of work based on at least one imaging device differs from the type of work based on the other imaging devices, the first generation unit 120 adopts the type of work that is consistent with the sensor information stored in the external information storage unit. As an example, if the work area is a warehouse and the sensor 20 is a reading device that reads codes attached to items, the external information storage unit may be, for example, a database used in a warehouse management system (WMS) for inventory management of items. For example, if the first generation unit 120 determines, using information stored in the external information storage unit, that an inspection was being performed at a certain time, it adopts the type of work corresponding to the inspection from among multiple types of work.

[0058] [Generation of Time Series Information of Work Type] When the first generating unit 120 is able to identify the work type information, the time when the person or robot 30 started the work, and the time when the work was completed, the first generating unit 120 generates time series information of the work type as at least a part of the work information. This time series information indicates, in chronological order, the type of work performed by a certain person or robot 30, the time when the person started the work, and the time when the work was completed.

[0059] Depending on the person or robot 30, it may be impossible to identify the type of work performed during a certain time period. This occurs, for example, when there is a time period in which the person is not captured by any of the image capture devices. In this case, the first generation unit 120 estimates the type of work performed during that time period using at least one of the type of work performed before that time period, the type of work performed after that time period, the person's location during that time period, and a predetermined work flow.

[0060] For example, the first generating unit 120 may set the type of work before that time period, such as the type of the immediately preceding work, as the type of work for that time period, or may set the type of work after that time period, such as the type of the immediately succeeding work, as the type of work for that time period. Furthermore, if the type of work before that time period and the type of work immediately succeeding that time period are the same, the first generating unit 120 may set the type of work as the type of work for that time period.

[0061] In addition, the location where each type of work should be performed may be determined. In this case, the first generating unit 120 may identify the location of the person or robot 30 during that time period, and estimate that the type of work corresponding to the identified location is the type of work that the person or robot 30 was performing during that time period.

[0062] In addition, the workflow, i.e., the order of tasks, may be predetermined. In this case, the first generation unit 120 may estimate the type of task for a time period using the type of task that precedes the time period, the type of task that follows the time period, and the workflow. For example, the first generation unit 120 identifies the type of task that precedes the time period. Then, the first generation unit 120 identifies the type of task that should be performed next according to the workflow. The first generation unit 120 estimates this type of task as the type of task for that time period. As another example, the first generation unit 120 identifies the type of task that follows the time period. Then, the first generation unit 120 identifies the type of task that should be performed before the next task according to the workflow. The first generation unit 120 estimates this type of task as the type of task for that time period.

[0063] Next, detailed examples of other processes performed by the first generating unit 120 and detailed examples of improvement information will be described. When the improvement information is generated in real time, i.e., while the work is being performed, the first generating unit 120 may generate the improvement information when it becomes highly likely that the estimated completion time of the work will be significantly delayed from the work plan. The timing for this determination is, for example, when half of the scheduled work time has passed.

[0064] As described above, the improvement information is information for improving a task, and is, for example, at least one of task change information regarding a change in the task and the person or robot 30 in charge, task assignment information for informing each of multiple people or robots 30 of the task that that person or robot 30 will be responsible for before the task starts, information indicating improvements to the task procedure, and information indicating task items to be added. To generate the improvement information, the first generator 120 performs, for example, at least one of the processes shown below.

[0065] [First process for generating person-in-charge change information]

[0066] In this example, multiple people are performing multiple types of work. The first generation unit 120 generates responsibility change information, which is an example of work information, during the work and transmits the generated responsibility change information to, for example, the robot 30 or the communication device 50 shown in FIG. 6 . The communication device 50 is operated by at least one of the person performing the work and the person managing the work. An example of the communication device 50 is a portable communication device such as a smartphone or tablet terminal. The person operating the communication device 50 can recognize the change in the person's responsibility by checking the responsibility change information using the communication device 50. Furthermore, the robot 30 can recognize the changed work content by using the responsibility change information and operate in accordance with the changed work content. Note that, when the communication device 50 receives the responsibility change information, it may notify the person operating the communication device 50 that it has received the responsibility change information by providing a human-perceptible output. Examples of the human-perceptible output include at least one of sound output, vibration generation, light emission from a light-emitting element, and a change in the display state of a display.

[0067] The first generating unit 120 acquires a work plan. This work plan is stored in, for example, the storage unit 40, and indicates, for each of a plurality of work types, the scheduled start time and the scheduled end time of each of the work types.

[0068] The work information also includes information that can identify the type of work that has been completed. For example, if the work information includes the type of work, the start time of the work, and the end time of the work, the first generating unit 120 can identify the type of work that has been completed by a certain time by tallying up the types of work that have been indicated as having been completed by the work information.

[0069] The first generation unit 120 then uses the types of tasks completed up to that time to determine the degree of progress toward the work plan for each type of task at that time. The degree of progress may indicate, for example, the ratio of the amount of work of that type completed up to that time to the amount of work of that type scheduled to be completed by that time in the work plan. The degree of progress may also indicate the ratio of the amount of work of that type completed up to that time to the amount of work of that type to be performed on that day. When packing items in a warehouse, the amount of work can be determined by the number of boxes packed. As another example, when picking a predetermined number of items in a warehouse, the quantity of items picked may be calculated by multiplying the number of picks by a reference value (e.g., average) for the quantity of items picked per pick, or by multiplying the length of time spent picking by a reference value (e.g., average) for the quantity of items picked per unit time.

[0070] The first generating unit 120 then generates responsibility change information using the calculated progress. For example, when the first task is progressing faster than planned and the second task is behind schedule, the first generating unit 120 generates responsibility change information such that at least a part of the people or robots 30 performing the first task will perform the second task.

[0071] The first generating unit 120 may use the calculated progress to calculate a predicted completion time for each of the multiple types of tasks, and generate responsibility change information using these predicted completion times. For example, if the planned completion time of the first task is earlier than the planned completion time of the second task by a reference time or more, the first generating unit 120 generates responsibility change information such that at least a part of the people or robots 30 performing the first task will perform the second task.

[0072] When there are multiple people performing a first task and only some of the people perform a second task, the first generation unit 120 may use skill information to identify the person who should perform the second task. The skill information indicates the skill levels of each person for each of the multiple people and is stored in advance in the storage unit 40. The skill levels indicate the person's proficiency with each of multiple types of tasks. The higher the proficiency level, the faster the person can perform that task. Therefore, the first generation unit 120 selects the person who should perform the second task, for example, in descending order of proficiency in the second task.

[0073] In this case, the first generation unit 120 may identify a communication device 50 operated by a person who is to perform the second task, i.e., a person whose role will be changed if the role change information is followed, and transmit the role change information or information indicating the changes in the role of that person if the role change information is followed to the communication device 50. For example, the storage unit 40 stores information identifying the communication device 50 operated by each of multiple people, such as information corresponding to an SNS account or email address. The first generation unit 120 uses this information to identify the communication device 50 to which information should be sent.

[0074] The first generating unit 120 may generate skill information for a plurality of individuals using task information for a plurality of individuals. For example, the first generating unit 120 may calculate the actual amount of work per unit time for each individual and each type of task, and generate skill information using the amount of work.

[0075] [Second Process for Generating Responsibility Change Information] The first generation unit 120 calculates the length of time during which no work is being performed for each of the multiple people and the multiple robots 30, using the non-work time periods included in the work information. Hereinafter, this length will be referred to as the first time. The first generation unit 120 uses the first time calculated for each of the multiple people and the multiple robots 30 to identify the people and robots 30 whose work responsibilities will be changed. In this case, the first generation unit 120 uses, for example, at least one of the following (4-1) to (4-2). (4-1) and (4-2) are executed, for example, in real time.

[0076] (4-1) The first generating unit 120 generates responsibility change information so that a person or robot 30 whose first time exceeds a first reference value is assigned to another task. The first generating unit 120 may set the task to be assigned to a person or robot 30 whose first time is less than a second reference value after the change. The second reference value is smaller than the first reference value.

[0077] (4-2) When multiple types of work are being performed in parallel in a work area and multiple people or robots 30 are performing each type of work, the first generation unit 120 adds up the first times of the multiple people and robots 30 performing the work for each type of work. The first generation unit 120 then generates work change information so that at least some of the people and robots 30 performing the work whose first time after the addition exceeds a third reference value are assigned to other work. The first generation unit 120 may determine that the work after the change is a work whose first time after the addition is less than a fourth reference value. The fourth reference value is smaller than the third reference value. Note that the area in which each type of work is performed may be determined. In this case, the "type of work" may be replaced with the "area" in this process.

[0078] [Process for generating responsible information]

[0079] As described above, the responsibility information is information used to inform each of multiple people and multiple robots 30 of the tasks that they or the robots 30 are responsible for before the tasks begin. An example of responsibility information is a shift schedule. When responsibility information is generated, multiple people share multiple types of tasks. The first generation unit 120 generates responsibility information, which is an example of task information, when the person performing the task changes, i.e., when the shift changes, and transmits the generated responsibility information to, for example, the robot 30 and the communication device 50 shown in FIG. 6. The robot 30 operates according to the responsibility information. Furthermore, the person operating the communication device 50 can confirm the tasks that the person is responsible for by checking the responsibility information using the communication device 50. When generating responsibility information, the first generation unit 120 uses, for example, at least one of the following methods (5-1) to (5-4).

[0080] (5-1) The first generating unit 120 determines the degree of progress with respect to the work plan for each of multiple types of work when the person performing the work is replaced. Specific examples of the degree of progress are the same as when generating the responsibility change information. The first generating unit 120 then generates responsibility information using the calculated degree of progress. For example, when the first work is progressing faster than planned and the second work is behind schedule, the first generating unit 120 generates responsibility information such that at least some of the people and robots 30 scheduled to perform the first work perform the second work. Here, the first generating unit 120 may use skill information to identify the person who should perform the second work, as in the case of generating the responsibility change information.

[0081] (5-2) The first generation unit 120 processes images captured on or before the previous day to calculate cumulative values ​​of the first time for each type of work during a predetermined period on or before the previous day. The first generation unit 120 then uses these cumulative values ​​of the first time to identify the number of people and robots 30 to be in charge of each type of work, or the people and robots 30 to be in charge. For example, the first generation unit 120 reduces the number of people and robots 30 in charge of work with a large cumulative value of the first time compared to the previous day, and instead increases the number of people and robots 30 in charge of work with a small cumulative value of the first time. The first generation unit 120 further identifies people to be in charge of each type of work according to the revised number of people. Here, the first generation unit 120 may calculate a difference in the amount of work for each type of work between the previous day and the current day, and further use this difference to identify the number of people to be in charge of each type of work, or the people to be in charge.

[0082] When generating the responsibility information, the first generating unit 120 may generate the responsibility information using the skill information stored in the storage unit 40. As described above, the skill information indicates the skill levels of multiple people. In this case, the first generating unit 120 uses, for example, at least one of the following (5-3) to (5-4).

[0083] (5-3) The first generation unit 120 calculates the time required to complete a task for each of multiple people. In this process, the first generation unit 120 uses skill information. For example, the first generation unit 120 shortens the time required by people with high skill levels compared to the time required by people with low skill levels. Then, the first generation unit 120 uses the time calculated using the skill information to identify the number of people to be in charge of each type of task, or the people to be in charge.

[0084] (5-4) In the skill information, skill levels may be set for each person and each type of work. In this case, when determining who will be in charge of a certain type of work, the first generation unit 120 gives priority to assigning people with a high skill level for that type of work.

[0085] [Other Examples of Improvement Information] The first generating unit 120 may further include the following information (6-1) to (6-5) as at least part of the improvement information. Here, (6-1) is preferably performed in real time. On the other hand, (6-2) to (6-5) are preferably performed when creating a work plan including personnel allocation.

[0086] (6-1) If there is a person or robot 30 whose first time exceeds a first reference value, the first generation unit 120 includes information indicating this as at least part of the improvement information. Note that the reference value used here may be different from the first reference value. In this regard, in the output process described below, the first generation unit 120 transmits this information to at least one of the communication device 50 of the person managing the work and the communication device 50 operated by the person whose first time exceeded the first reference value. Note that in the former case, the first generation unit 120 preferably transmits to the communication device 50 information capable of identifying the person or robot 30 whose first time exceeded the first reference value. Furthermore, in the latter case, the first generation unit 120 may transmit to the communication device 50 information indicating that support for another person should be provided.

[0087] (6-2) If there is a person whose first time exceeds a first reference value, the first generation unit 120 performs processing to estimate the cause and uses information indicating the estimated result as at least part of the improvement information. The reference value used here may be a value different from the first reference value. The first generation unit 120 may further generate information indicating a work process improvement proposal to eliminate the cause, and use the information as at least part of the improvement information. Note that if the cause is due to staffing issues, the first generation unit 120 may generate information indicating a staffing improvement proposal, and use the information as at least part of the improvement information.

[0088] If no inferred cause is found in (6-2), it is highly likely that the problem lies with the person in charge, and therefore information recommending that some kind of action be taken against that person may be output to the communication device 50 of the person in charge of the work. This information may also be considered as improvement information.

[0089] (6-3) At least two people may form a group and work on this group basis. In this case, the compatibility between the people in the group may affect the work efficiency. The storage unit 40 stores information indicating this compatibility. An example of this information is information indicating combinations of people who have good compatibility. The first generation unit 120 uses this information when forming groups in the work plan.

[0090] (6-4) The work procedures of people with high processing capabilities often include content that others should emulate. The first generation unit 120 includes information about this work performed by that person, i.e., a specific person, as at least part of the improvement information. This information can also be considered know-how information. For example, the first generation unit 120 may include a video showing the person's movements as at least part of the improvement information. Furthermore, when a single work type includes multiple sub-tasks, the first generation unit 120 may include information indicating the order in which the sub-tasks are performed by that person as at least part of the improvement information. For example, this method is applicable when a single work type in a warehouse involves picking items and storing them in a container, and the picking, transporting them to the storage location, and storing them in the container are each sub-tasks. Furthermore, the first generation unit 120 may identify the person's movement path and include this movement path as at least part of the improvement information.

[0091] The first generating unit 120 may use the content of the instruction that a person with a high skill level gives to another person as at least a part of the improvement information. This content is generated by the first generating unit 120, for example.

[0092] (6-5) Depending on the type of work, a work procedure may be predefined. The first generation unit 120 can detect actions that deviate from the work procedure through image processing. As an example, the storage unit 40 pre-stores information indicating actions that would occur if the work procedure were followed. The first generation unit 120 then uses this information to detect actions that deviate from the work procedure. When the same action that deviates from the work procedure is repeatedly detected in a person with a high skill level, this action may be an action that should be included in the work procedure. Examples of such actions include organizing and cleaning. The first generation unit 120 may include information about this action in the improvement information. As an example, the first generation unit 120 may include an image, such as a video, that includes this action in the improvement information.

[0093] As shown in FIG. 7 , the work analysis device 10 may include, in addition to the acquisition unit 110 and the first generation unit 120, at least one of a second generation unit 130, a third generation unit 140, a fourth generation unit 150, and a cost calculation unit 160.

[0094] The second generation unit 130 uses the task information to generate first manual information related to tasks to be performed by the robot 30. The first manual information is, for example, a manual of the actions and procedures to be performed by the robot 30 for each task. For example, the second generation unit 130 uses the task information and sensor information to generate text indicating task procedures and the content of each procedure for each of the multiple robots 30. This process uses generative AI such as LLM (Large Language Models). For example, the second generation unit 130 processes the task procedures of the robot 30 included in the task information and the video or images of each step using LLM to generate text indicating the task procedures and the content of each step. The second generation unit 130 then processes the text indicating the task procedures and the content of each step using, for example, RPA (Robotic Process Automation) to generate manual data in a predetermined format. This manual data is data in which text describing a task is associated with video or images illustrating the task.

[0095] The third generation unit 140 uses the work information to generate second manual information related to work performed by people. The second manual information is, for example, a manual of the actions and procedures that people should perform for each work. This processing is similar to the processing performed by the second generation unit 130, for example. For example, the third generation unit 140 processes the work procedures of people included in the work information and the video or images of each step using LLM to generate text indicating the work procedures and the content of each step. The third generation unit 140 then processes the text indicating the work procedures and the content of each step using, for example, RPA to generate manual data in a predetermined format. This manual data is data in which text explaining a certain work is associated with video or images showing the work.

[0096] The third generation unit 140 may generate the second manual information by using task information of a person who satisfies a specific criterion. The specific criterion may be, for example, at least one of the following: the skill indicated by the skill information is equal to or exceeds a standard value; the person's actions satisfy a safety standard; and the person's actions satisfy a quality standard.

[0097] Whether or not the person's movements satisfy the safety standards can be determined by, for example, using image processing to determine whether or not the person's movements include movements that deviate from the safety standards. This processing is performed by, for example, the first generator 120.

[0098] Whether the person's movements satisfy the quality standard can be identified, for example, by using image processing to determine whether the person's movements include movements that deviate from the quality standard. An example of a movement that deviates from the quality standard is a movement that applies an impact to an object that is greater than or equal to a predetermined value. This processing is performed, for example, by the first generation unit 120.

[0099] The second generating unit 130 and the third generating unit 140 may be integrated into one functional block.

[0100] The fourth generation unit 150 generates association information indicating the association between the content of the work and the carbon dioxide concentration using carbon dioxide information indicating the change in the concentration of carbon dioxide contained in the air in the work area and work information.

[0101] The carbon dioxide information is acquired, for example, by the acquisition unit 110. For example, if one of the sensors 20 is capable of measuring the concentration of carbon dioxide, the acquisition unit 110 acquires the carbon dioxide information from this sensor 20.

[0102] The fourth generation unit 150 then generates the above-mentioned relevance information. The relevance information indicates, for example, how the carbon dioxide concentration changes depending on the content or procedure of the work. For example, when performing a certain work, if there are multiple detailed procedures from the start to the completion of the work, the relevance information indicates the carbon dioxide concentration for each of the multiple procedures. This allows the manager of the work area to understand which procedure will result in the lowest carbon dioxide concentration emissions.

[0103] The cost calculation unit 160 uses the work information to calculate the cost of the work performed by the robot 30. The cost calculation unit 160 may further use the work information to calculate the cost of the work performed by a person.

[0104] For example, the storage unit 40 stores information indicating the cost of each of a plurality of actions that can be performed by the robot 30. The storage unit 40 also stores information indicating the cost of each of a plurality of actions that can be performed by a person. The cost calculation unit 160 then reads out the cost corresponding to each of the plurality of actions included in the work information from the storage unit 40 and adds up the read costs to calculate the cost of the work performed by the robot 30 and the cost of the work performed by a person.

[0105] For example, when a certain task involves multiple detailed steps from start to finish, the cost calculation unit 160 can calculate the cost for each of the multiple steps, allowing the manager of the task area to determine which step will result in the lowest cost.

[0106] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0107] In addition, although the flowcharts used in the above description show a number of steps (processes) in a sequential order, the order of the steps executed in each embodiment is not limited to the order shown in the flowcharts. In each embodiment, the order of the steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0108] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1. A task analysis device comprising: an acquisition means for acquiring sensor information, which is information generated using at least one sensor and is capable of identifying the movement of a robot within a work area; and a first generation means for using the sensor information to generate task information that chronologically indicates the content of tasks performed by the robot. 2. The task analysis device described in above 1, further comprising: a second generation means for using the task information to generate first manual information related to tasks performed by the robot. 3. The task analysis device described in above 1 or 2, further comprising: the sensor information can also identify the movement of a person within the work area; and the task information also chronologically indicates the content of the tasks performed by the person. 4. The task analysis device described in above 3, further comprising: a third generation means for using the task information to generate second manual information related to tasks performed by a person. 5. The task analysis device described in above 4, further comprising: the third generation means for using the task information to generate second manual information related to tasks performed by a person. 6. The task analysis device described in any one of above 1 to 7, wherein the specific criterion is that the person's movements meet a safety standard. 7. The task analysis device described in any one of above 5, wherein the specific criterion is that the person's movements meet a quality standard. 8. The task analysis device described in any one of above 1 to 7, wherein the task analysis device comprises cost calculation means that calculates the cost of the task performed by the robot using the task information. 9. The task analysis device described in any one of above 1 to 8, wherein the acquisition means acquires carbon dioxide information that indicates a change in the concentration of carbon dioxide in the air in the task area, and the task analysis device comprises fourth generation means that uses the carbon dioxide information and the task information to generate association information that indicates the association between the content of the task and the carbon dioxide concentration. 10. The task analysis device described in any one of above 1 to 9, wherein the first generation means generates the task information for each of the multiple people, and generates skill information that indicates the skill level for each person using the task information for each of the multiple people.11. The work analysis device according to any one of claims 1 to 10 above, wherein the sensor includes a plurality of image capturing devices, and the sensor information includes a plurality of images generated by the plurality of image capturing devices. 12. A work analysis method in which a computer acquires sensor information, the sensor information being information generated using at least one sensor and capable of identifying the movement of a robot within a work area, and uses the sensor information to generate work information that chronologically indicates the content of work performed by the robot. 13. A recording medium having recorded thereon a program that causes a computer to have: an acquisition means for acquiring sensor information, the sensor information being information generated using at least one sensor and capable of identifying the movement of the robot within the work area; and a first generation means for using the sensor information to generate work information that chronologically indicates the content of work performed by the robot. 14. The program according to claim 13 above.

[0109] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 11, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 12, 13, and 14 in the same dependent relationship as Supplementary Notes 2 to 11. Furthermore, not limited to Supplementary Notes 1, 12, 13, and 14, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0110] REFERENCE SIGNS LIST 10 Work analysis device 20 Sensor 30 Robot 40 Memory unit 50 Communication device 110 Acquisition unit 120 First generation unit 130 Second generation unit 140 Third generation unit 150 Fourth generation unit 160 Cost calculation unit

Claims

1. A work analysis device comprising: an acquisition means for acquiring sensor information generated using at least one sensor, the sensor information being capable of identifying the movement of a robot within a work area; and a first generation means for using the sensor information to generate work information that indicates the content of the work performed by the robot in chronological order.

2. A task analysis device according to claim 1, further comprising: second generation means for generating first manual information relating to a task to be performed by a robot using the task information.

3. A work analysis device according to claim 1 or 2, wherein the sensor information is also capable of identifying the movements of people in the work area, and the work information also indicates the content of the work performed by the people in chronological order.

4. A work analysis device according to claim 3, further comprising a third generation means for generating second manual information relating to work to be performed by a person using the work information.

5. A work analysis device according to claim 4, wherein the third generation means generates the second manual information using work information of the person that satisfies a specific criterion.

6. A work analysis device according to claim 5, wherein the specific criterion is that the person's actions satisfy a safety standard.

7. A work analysis device according to claim 5, wherein the specific criterion is that the person's actions meet a quality standard.

8. A task analysis device according to any one of claims 1 to 7, further comprising cost calculation means for calculating the cost of the task performed by the robot using the task information.

9. A work analysis device according to any one of claims 1 to 8, wherein the acquisition means acquires carbon dioxide information indicating changes in the concentration of carbon dioxide contained in the air in the work area, and the work analysis device further comprises a fourth generation means that uses the carbon dioxide information and the work information to generate association information indicating the association between the content of the work and the carbon dioxide concentration.

10. A work analysis device according to any one of claims 1 to 9, wherein the first generation means generates the work information for each of the plurality of people, and generates skill information indicating the skill level for each person using the work information for each of the plurality of people.

11. A work analysis device according to any one of claims 1 to 10, wherein the sensor includes a plurality of image capturing devices, and the sensor information includes a plurality of images generated by the plurality of image capturing devices.

12. A work analysis method in which a computer acquires sensor information generated using at least one sensor, the sensor information being capable of identifying the movement of a robot within a work area, and uses the sensor information to generate work information that indicates the content of the work performed by the robot in chronological order.

13. A recording medium having recorded thereon a program that causes a computer to have: an acquisition means for acquiring sensor information, which is information generated using at least one sensor and is capable of identifying the movement of a robot within a work area; and a first generation means for using the sensor information to generate work information that indicates the content of work performed by the robot in chronological order.

Images