Information processing device, manual management method, and program

Abstract

The objective is to provide an information processing device, a manual management method, and a program that can support the high-level leveling of tasks, even when the tasks requiring high-level leveling are unknown. [Solution] One aspect of the information processing device of the present disclosure includes: a video acquisition unit that acquires multiple videos of the work status of multiple workers; a work recognition unit that analyzes the multiple videos and recognizes multiple types of work performed by each worker; a manual generation unit that generates manuals for the recognized work and stores them in a manual storage unit; and an output unit that outputs a setting screen to an administrator terminal used by an administrator for setting at least one of a first condition for selecting the type of work to be subject to manual generation from among the multiple types of work, and a second condition for deleting manuals from the manual storage unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a manual management method, and a program.

Background Art

[0002] Conventionally, at production sites of various products such as factories, measures have been taken to improve the efficiency of production operations such as product manufacturing, processing, and assembly by workers.

[0003] For example, in Patent Document 1, for operations that require improvement by other workers, the work operations of a best performer, who is a skilled worker, are photographed, the trajectories of both hands for each work process are extracted from the photographed video, and compared with the trajectories of both hands for each work process of other workers. According to the technique disclosed in Patent Document 1, it is expected that by having other workers use the video (trajectories of both hands) of the best performer as a model, improvements such as how to move both hands can be promoted, and improvement of work efficiency can be achieved.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the above-described conventional technology, a best performer cannot be identified unless the operation that requires improvement is known. That is, the above-described conventional technology is premised on the operation that requires improvement being known. Therefore, in a situation where it is necessary to perform a wide variety of operations and new types of operations frequently occur, and it is difficult to identify operations that require high-level leveling in the first place, it is also difficult to apply the above-described conventional technology.

[0006] This disclosure is made in view of the above circumstances and aims to provide an information processing device, a manual management method, and a program that can support the high-level leveling of work, even when the work requiring high-level leveling is unknown. [Means for solving the problem]

[0007] One aspect of the information processing device of the present disclosure includes: a video acquisition unit that acquires multiple videos of the work status of multiple workers; a work recognition unit that analyzes the multiple videos and recognizes multiple types of work performed by each worker; a manual generation unit that generates manuals for the recognized work and stores them in a manual storage unit; and an output unit that outputs a setting screen to an administrator terminal used by an administrator for setting at least one of a first condition for selecting the type of work to be included in the manual generation from among the multiple types of work, and a second condition for deleting the manual from the manual storage unit.

[0008] One aspect of the manual management method of this disclosure includes: a video acquisition step in which a video acquisition unit acquires multiple videos of the work status of multiple workers; a work recognition step in which a work recognition unit analyzes the multiple videos and recognizes multiple types of work performed by each worker; a manual generation unit that generates manuals for the recognized work and stores them in a manual storage unit; and an output unit that outputs a setting screen to an administrator terminal used by an administrator for setting at least one of a first condition for selecting a type of work to be subject to manual generation from among the multiple types of work, and a second condition for deleting manuals from the manual storage unit.

[0009] One aspect of the program of this disclosure involves causing a computer to execute the following steps: an image acquisition step of acquiring multiple images of the work status of multiple workers; a work recognition step of analyzing the multiple images to recognize multiple types of work performed by each worker; a manual generation unit that generates manuals for the recognized work and stores them in a manual storage unit; and an output unit that outputs a setting screen to an administrator terminal used by an administrator for setting at least one of a first condition for selecting the type of work to be included in the manual generation from among the multiple types of work, and a second condition for deleting the manual from the manual storage unit. [Effects of the Invention]

[0010] According to this disclosure, even when the tasks requiring high-level leveling are unknown, it is possible to provide an information processing device, a manual management method, and a program that can support high-level leveling of such tasks. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a block diagram showing an example of the system configuration of the work analysis system of this embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device, administrator terminal, and worker terminal of this embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the functional configuration of the information processing device according to this embodiment. [Figure 4] Figure 4 shows an example of an image captured by the imaging device of this embodiment and acquired by the image acquisition unit. [Figure 5] Figure 5 shows an example of browsing information in this embodiment. [Figure 6] Figure 6 shows an example of the work history in this embodiment. [Figure 7] Figure 7 shows an example of the administrator screen of this embodiment. [Figure 8] Figure 8 shows an example of the manual generation condition setting screen of this embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a manual target person setting screen of the present embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a manual generated by the manual generation unit of the present embodiment. [Figure 11] FIG. 11 is a diagram showing an example of an operator screen of the present embodiment. [Figure 12] FIG. 12 is a diagram showing an example of browsing information in which flag information of the present embodiment is updated. [Figure 13] FIG. 13 is a diagram showing an example of a valid manual condition setting screen of the present embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a work history of the present embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a manual merge condition setting screen of the present embodiment. [Figure 16] FIG. 16 is a flowchart showing an example of a work recognition process performed by the information processing apparatus of the present embodiment. [Figure 17] FIG. 17 is a sequence diagram showing an example of a condition setting process performed by the work analysis system of the present embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of a manual generation process performed by the information processing apparatus of the present embodiment. [Figure 19] FIG. 19 is a sequence diagram showing an example of a manual confirmation process performed by the work analysis system of the present embodiment. [Figure 20] FIG. 20 is a flowchart showing an example of a manual deletion process performed by the information processing apparatus of the present embodiment. [Figure 21] FIG. 21 is a sequence diagram showing an example of a manual merge process performed by the work analysis system of the present embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure (hereinafter simply referred to as "these embodiments") will be described in detail with reference to the drawings. However, this disclosure is not limited to the following embodiments. Furthermore, the following embodiments and their variations can be combined as appropriate.

[0013] In the following section, we will explain the work analysis system of this embodiment using warehouse operations performed at a logistics center (warehouse) that delivers parts (repair parts) as an example.

[0014] Generally, logistics centers that handle parts handle a vast and diverse range of parts and tools, and the parts they handle change frequently. Furthermore, logistics centers need to perform warehouse operations such as receiving, inspection, warehousing, picking, individual packaging, packing, and shipping for each part, and the content of these operations, in particular, often differs depending on the part. Thus, logistics centers that distribute parts face the challenge of not only performing a wide variety of warehouse operations, but also frequently encountering new types of parts, making it difficult to identify tasks that require high-level standardization. In particular, warehouse operations often involve simple tasks that can be performed to a certain extent at the discretion of the worker, making it difficult to identify tasks that require high-level standardization.

[0015] Tasks requiring high-level standardization include, for example, tasks where there is variation in work time depending on the worker, but where the variation in work time converges once the worker understands the optimal method.

[0016] In this embodiment, the work analysis system analyzes video footage of each worker's work, uses the analysis results to estimate tasks that require high-level standardization, and generates manuals for the estimated tasks, thereby supporting the high-level standardization of tasks that were initially unclear (unknown) and required high-level standardization. Furthermore, in this embodiment, the work analysis system analyzes video footage of each worker's work, uses the analysis results to delete manuals that have a low support effect for high-level standardization, and retains (keeps) manuals that have a high support effect for high-level standardization, thereby supporting the high-level standardization of tasks that were initially unclear (unknown) and required high-level standardization.

[0017] The following will specifically describe the work analysis system of this embodiment, using individual packaging work in a logistics center (warehouse) handling parts as an example. However, the work analysis system of this embodiment is not limited to this, and can target any warehouse work as described above. Furthermore, it can target all work in logistics centers, etc., that handle finished products rather than parts.

[0018] Individual packaging refers to the process of individually wrapping each component. For example, this might involve wrapping components in cushioning material and then placing them in an outer box. The type of cushioning material, outer box, tape used for wrapping with cushioning material, or tape used to seal the lid of the outer box will vary depending on the type of component being individually packaged (size, shape, etc.). Generally, components are not individually received at the logistics center, but rather stored in batches of a certain number. Therefore, when shipping components, the components to be shipped are picked, individually packaged, and then bundled together with other individually packaged components before being shipped out.

[0019] First, the configuration of the work analysis system in this embodiment will be described.

[0020] Figure 1 is a block diagram showing an example of the system configuration of the work analysis system 1 of this embodiment. As shown in Figure 1, the work analysis system 1 comprises a plurality of imaging devices 10-1 to 10-n (where n is a natural number), an information processing device 20, an administrator terminal 30, and a plurality of worker terminals 40-1 to 40-m (where m is a natural number).

[0021] Multiple imaging devices 10-1 to 10-n, an information processing device 20, an administrator terminal 30, and multiple worker terminals 40-1 to 40-m are connected via network 2. Network 2 can be implemented by at least one of the following: the Internet and / or a LAN (Local Area Network). Network 2 may be a wired network, a wireless network, or a mixture of wired and wireless networks.

[0022] In the following explanation, if it is not necessary to distinguish between the imaging devices 10-1 to 10-n, they may simply be referred to as imaging device 10. Similarly, if it is not necessary to distinguish between the worker terminals 40-1 to 40-m, they may simply be referred to as worker terminal 40.

[0023] The imaging devices 10-1 to 10-n are installed inside and outside the warehouse to photograph the work of workers performing tasks inside and outside the warehouse, and output the captured images to the information processing device 20. Each imaging device 10 is installed inside or outside the warehouse to photograph, for example, the work area where the worker is performing tasks (for example, a workbench where individual packaging work is performed). The imaging devices 10 may include, for example, video cameras.

[0024] The information processing device 20 analyzes multiple images captured by multiple imaging devices 10 and recognizes the tasks being performed by the worker from each image. The information processing device 20 also manages manuals, such as generating, deleting, and merging manuals for the recognized tasks. The information processing device 20 can be implemented, for example, by a server or cloud service.

[0025] The administrator terminal 30 is a terminal device used by the administrator of the work analysis system 1, and examples include a PC (Personal Computer), a tablet terminal, or a smartphone. Based on the operation input from the administrator, the administrator terminal 30 specifies various conditions for manual management to the information processing device 20. As a result, the information processing device 20 performs manual management according to the conditions specified by the administrator terminal 30.

[0026] The worker terminal 40 is a terminal device used by the worker, and examples include a tablet, smartphone, or PC. The worker terminal 40 is used, for example, for the worker to check a manual generated by the information processing device 20. The worker terminal 40 may be held by the worker themselves, or it may be installed in the workplace where the worker performs their work.

[0027] Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 20, administrator terminal 30, and worker terminal 40 in this embodiment.

[0028] First, the hardware configuration of the information processing device 20 will be described. As shown in Figure 2, the information processing device 20 comprises a control device 21, a main memory 22, an auxiliary storage device 23, a communication device 24, and various buses 29. The control device 21, the main memory 22, the auxiliary storage device 23, and the communication device 24 are connected via the various buses 29. Thus, the information processing device 20 of this embodiment has a general hardware configuration using a normal computer.

[0029] The control device 21 controls the overall operation of the information processing device 20. The control device 21 may be, for example, at least one of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), but is not limited to these. There may be one or more CPUs or GPUs, and they may be single-core or multi-core.

[0030] Examples of main memory 22 include, but are not limited to, ROM (Read Only Memory) and RAM (Random Access Memory). ROM stores various programs, such as programs for controlling the information processing device 20, programs for recognizing the worker's work from video, and programs for managing manuals for the recognized work. RAM is used as a workspace when the control device 21 performs various controls based on the programs stored in ROM.

[0031] The auxiliary storage device 23 stores various data, including the various programs mentioned above and a recognition model (a trained interaction recognition model) for recognizing the worker's work from the video. The various programs mentioned above only need to be stored in at least one of the main storage device 22 and the auxiliary storage device 23. The auxiliary storage device 23 may be, but is not limited to, at least one of existing storage devices capable of magnetic, electrical, or optical storage, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), and DVD (Digital Versatile Disc). The auxiliary storage device 23 may be built into the information processing device 20 or externally connected to the information processing device 20 via an interface such as USB (Universal Serial Bus). Furthermore, the auxiliary storage device 23 may be a NAS (Network Attached Storage) connected via a network such as a LAN or WAN (Wide Area Network).

[0032] The communication device 24 is used to communicate with the administrator terminal 30 and the worker terminal 40, etc., via the network 2. Examples of the communication device 24 include, but are not limited to, a communication device for a wired LAN or a wireless communication device for a wireless LAN.

[0033] In addition to the above configuration, the information processing device 20 may also be equipped with hardwired circuits such as ICs (Integrated Circuits), ASICs (Application Specific Integrated Circuits), and FPGAs (Field-Programmable Gate Arrays) that are specific to the information processing device 20.

[0034] Next, the hardware configuration of the administrator terminal 30 will be described. As shown in Figure 2, the administrator terminal 30 comprises a control device 31, a main memory 32, an auxiliary memory 33, a communication device 34, an input device 35, a display device 36, and various buses 39. The control device 31, main memory 32, auxiliary memory 33, communication device 34, input device 35, and display device 36 are connected via the various buses 39. Thus, the administrator terminal 30 in this embodiment has a general hardware configuration, such as that of a PC or tablet terminal.

[0035] The control device 31 controls the overall operation of the administrator terminal 30. The implementation method for the control device 31 is the same as that for the control device 21.

[0036] The implementation method for the main memory 32 is the same as that for the main memory 22, so a detailed explanation will be omitted. The ROM of the main memory 32 stores various programs, including a program for controlling the administrator terminal 30.

[0037] The auxiliary storage device 33 stores the various programs and data described above. Note that the various programs described above only need to be stored in at least one of the main storage device 32 and the auxiliary storage device 33. The implementation method for the auxiliary storage device 33 is the same as that for the auxiliary storage device 23, so a detailed explanation is omitted.

[0038] The communication device 34 is used to communicate with the information processing device 20 and other devices via the network 2. Since the implementation method for the communication device 34 is the same as that for the communication device 24, a detailed explanation is omitted.

[0039] The input device 35 is used for various inputs, selections, and specifications to define various conditions for manual management, and serves as a user interface between the administrator and the user. Examples of input devices 35 include, but are not limited to, keyboards, mice, and touch panels. The input device 35 may be built into the administrator terminal 30 or it may be externally connected via an interface such as USB.

[0040] The display device 36 displays various screens, such as the setting screen for various conditions for manual management, and serves as a user interface between the administrator and the user. Examples of the display device 36 include, but are not limited to, liquid crystal displays, organic electro-luminescence (OLED) displays, and touch panel displays. The display device 36 may be an internal display built into the administrator terminal 30, or an external display connected to the administrator terminal 30 via a display interface such as HDMI®.

[0041] Next, the hardware configuration of the worker terminal 40 will be described. As shown in Figure 2, the worker terminal 40 comprises a control device 41, a main memory 42, an auxiliary storage device 43, a communication device 44, an input device 45, a display device 46, and various buses 49. The control device 41, main memory 42, auxiliary storage device 43, communication device 44, input device 45, and display device 46 are connected via various buses 49. Thus, the worker terminal 40 in this embodiment has a general hardware configuration, such as that of a PC or tablet terminal.

[0042] The control device 41 controls the overall operation of the worker terminal 40. The implementation method of the control device 41 is the same as that of the control device 21.

[0043] The implementation method for the main memory 42 is the same as that for the main memory 22, so a detailed explanation will be omitted. The ROM of the main memory 42 stores various programs, including a program for controlling the worker terminal 40.

[0044] The auxiliary storage device 43 stores the various programs and data described above. The various programs described above only need to be stored in at least one of the main storage device 42 and the auxiliary storage device 43. The implementation method for the auxiliary storage device 43 is the same as that for the auxiliary storage device 23, so a detailed explanation is omitted.

[0045] The communication device 44 is used to communicate with the information processing device 20 and other devices via the network 2. Since the implementation method for the communication device 44 is the same as that for the communication device 24, a detailed explanation is omitted.

[0046] The input device 45 is used for various inputs, selections, and specifications to check the manual, and serves as a user interface between the operator and the device. The implementation method of the input device 45 is the same as that of the input device 35, so a detailed explanation is omitted.

[0047] The display device 46 displays various screens, such as the manual display screen, and serves as a user interface between the operator and the device. The implementation method of the display device 46 is the same as that of the display device 36, so a detailed explanation is omitted.

[0048] Figure 3 is a block diagram showing an example of the functional configuration of the information processing device 20 of this embodiment. As shown in Figure 3, the information processing device 20 includes an image acquisition unit 201, an image storage unit 203, a browsing information storage unit 205, a recognition model storage unit 207, a work recognition unit 209, a work history storage unit 211, an input / output control unit 221, a management unit 231, a condition storage unit 241, a manual generation unit 243, a manual storage unit 245, and an effect measurement unit 247. The input / output control unit 221 includes a reception unit 223 and an output unit 225. The management unit 231 includes a condition management unit 233, a notification unit 235, and a manual management unit 237.

[0049] The video acquisition unit 201, the work recognition unit 209, the input / output control unit 221, the reception unit 223, the output unit 225, the management unit 231, the condition management unit 233, the notification unit 235, the manual management unit 237, the manual generation unit 243, and the effect measurement unit 247 can be realized, for example, by the control device 21, main memory 22, and communication device 24 described in Figure 2. For example, the control device 21 reads various programs stored in the main memory 22 (ROM) or auxiliary memory 23 and expands them into the main memory 22 (RAM). The control device 21 realizes each of the above-mentioned functional units by executing various processes according to the expanded programs. Here, the case in which each of the above-mentioned functional units is realized as software has been explained as an example, but at least a part of each of the above-mentioned functional units may be realized as hardware. In this case, the functional unit to be realized as hardware can be realized, for example, by the hardwired circuit described above. Alternatively, any of the above-mentioned functional units may be realized through the cooperation of software and hardware.

[0050] The video storage unit 203, browsing information storage unit 205, recognition model storage unit 207, work history storage unit 211, condition storage unit 241, and manual storage unit 245 can be implemented, for example, by the auxiliary storage device 23 described in Figure 2.

[0051] The video acquisition unit 201 acquires multiple videos from multiple shooting devices 10-1 to 10-n, capturing the work status of multiple workers, and stores (accumulates) them in the video storage unit 203. Figure 4 shows an example of a video 501 captured by the shooting device 10 of this embodiment and acquired by the video acquisition unit 201. The video 501 shown in Figure 4 is a video of worker W performing individual packaging work on a workbench 51, captured by a shooting device 10 installed on the ceiling. Specifically, worker W wraps parts in cushioning material using tapes 61 and 63 on the workbench 51 (not shown), packs the parts wrapped in cushioning material into an outer box 71, attaches tape 65 to the lid of the outer box 71, and seals the outer box 71.

[0052] In this embodiment, multiple camera devices 10 are arranged in the warehouse to each photograph a workbench where an employee performs individual packaging work. Therefore, the video acquisition unit 201 acquires images from each camera device 10, as shown in Figure 4, of each employee performing individual packaging work, and stores (accumulates) the acquired images in the video storage unit 203. For this reason, it is assumed that the images stored in the video storage unit 203 include images of each employee sequentially performing individual packaging work on various parts, such as images of employee A performing individual packaging work on part X and then on part Y, or images of employee B performing individual packaging work on part Z and then on part Y.

[0053] The browsing information storage unit 205 stores browsing information for each worker indicating whether or not they have viewed the manual for each task. The manual will be described later. Figure 5 shows an example of the browsing information in this embodiment. In the example shown in Figure 5, the browsing information is information that associates the worker, the type of work, and whether or not the manual has been viewed. In the browsing information shown in Figure 5, for example, worker A has not viewed the manual for the individual packaging work of part X, but has viewed the manual for the individual packaging work of part Y.

[0054] The recognition model storage unit 207 stores a recognition model for recognizing the work being performed by an operator from the video stored in the video storage unit 203. The recognition model is, for example, an operator interaction recognition model, and is a trained model generated by a learning device (not shown) learning from training data.

[0055] Training data includes, for example, work videos such as live-action footage of workers performing individual packaging tasks or CG footage of computer graphics models of workers performing individual packaging tasks, to which annotations (labels) have been added. The annotations indicate, for example, whether or not there is interaction between the worker's hands and an object (tool) for each frame of the work video, and if there is interaction, the name of the object involved and the action taken using that object.

[0056] The learning device (not shown) learns from such training data and generates a recognition model that, for each frame of the input video, outputs recognition results indicating what objects (tools) the worker is using and what actions they are performing in the individual packaging process.

[0057] The work recognition unit 209 analyzes multiple videos acquired by the video acquisition unit 201 (stored in the video storage unit 203) to recognize multiple types of work performed by each worker. Specifically, the work recognition unit 209 analyzes multiple videos and recognizes each of the multiple types of work performed by each worker on a work process basis. The work recognition unit 209 stores the work recognition results as work history in the work history storage unit 211.

[0058] In this embodiment, the work recognition unit 209 acquires video from the video storage unit 203 and inputs the acquired video into the recognition model stored in the recognition model storage unit 207. The work recognition unit 209 acquires recognition results from the recognition model for each frame of the video, indicating what objects (tools) the worker is using and what actions they are performing in the individual packaging work, thereby recognizing the work being performed by the worker in the video on a work process basis. The work recognition unit 209 refers to the browsing information stored in the browsing information storage unit 205, sets a browsing flag indicating whether or not the manual has been viewed in the work recognition result, and stores it as work history in the work history storage unit 211. By performing the above processing on all video stored in the video storage unit 203, the work recognition unit 209 can decompose and recognize each of the multiple types of work performed by each worker down to the work process level.

[0059] Figure 6 shows an example of the work history in this embodiment. The work history shown in Figure 6 shows the recognition result of recognizing worker A's work from the video footage of worker A performing the individual packaging work on part X and then on part Y, as described above, specifically the recognition result for the individual packaging work on part X. In addition, in the work history shown in Figure 6, frames with the same work process are displayed together. In the example shown in Figure 6, the work history includes the worker, work type, viewing flag, start date and time, end date and time, frame, work process, and frame video.

[0060] The worker is the worker performing the task recognized by the work recognition unit 209 (in the example shown in Figure 6, "Worker A"). The worker may be recognized by the work recognition unit 209 from the video using a recognition model, or the worker may be set using information such as who will perform which task and which workbench, which the worker entered into the worker terminal 40 before performing the task. In the latter case, the work recognition unit 209 only needs to obtain the information from the worker terminal 40.

[0061] The work type indicates the type of work recognized by the work recognition unit 209 (in the example shown in Figure 6, "Individual packaging of part X"). The work type may be recognized by the work recognition unit 209 from the video using a recognition model, or it may be set using information such as part code information that the worker inputs into the worker terminal 40 before performing each task. Generally, before performing individual packaging work, the worker terminal 40 reads code information such as barcodes or 2D codes attached to the parts to be individually packaged, and after the individual packaging work for the parts is completed, the worker inputs the completion of the work from the worker terminal 40. In the latter case, the work recognition unit 209 only needs to obtain the information from the worker terminal 40.

[0062] The viewing flag indicates whether or not the worker has already viewed the manual for a given type of work. The work recognition unit 209 sets the viewing flag by referring to the viewing information stored in the viewing information storage unit 205. In the case of the viewing information shown in Figure 5, worker A has not yet viewed the manual for the individual packaging work of part X, so in the example shown in Figure 6, the work recognition unit 209 sets the viewing flag to off.

[0063] The start date and time indicate the date and time the work process began, the end date and time indicate the date and time the work process ended, and the frame number indicates the frame number from the start to the end of the work process.

[0064] The work process shows the work process for individual packaging of part X as recognized by the work recognition unit 209 from the video using the recognition model. Specifically, the work process shows what objects (tools) the worker is using and what actions they are taking during the individual packaging process.

[0065] For example, suppose that during the time period from "5 / 10 12:00:00" to "5 / 10 12:00:05" in the video, worker A is placing cushioning material on the workbench, and an interaction is occurring between worker A's hand and the cushioning material. Furthermore, suppose that the recognition model has learned training data in which frames of video showing the worker placing the cushioning material are annotated with a defined action of placing the cushioning material if the object interacting with the worker's hand is the cushioning material. In this case, the work recognition unit 209, using the recognition model, would recognize the object and action as "placement" for the work process during the above time period in the video.

[0066] For example, suppose that during the time period from "5 / 10 12:00:05" to "5 / 10 12:00:20" in the video, worker A is securing part X, which is placed on cushioning material, with tape α, and interaction is occurring between worker A's hand, tape α, and part X. Furthermore, suppose that the recognition model has learned training data in which, in the video frame in which the worker secures part X, which is placed on cushioning material, with tape α, the object interacting with the worker's hand is tape α and part X, and the action is defined as securing part X with tape α. In this case, the work recognition unit 209, using the recognition model, recognizes the work process during the above time period in the video as object "tape α, part X" and action "securing".

[0067] For example, suppose that during the time period from "5 / 10 12:00:20" to "5 / 10 12:00:30" in the video, worker A wraps part X with cushioning material and secures it with tape β, and interaction occurs between worker A's hand and tape β and cushioning material. Furthermore, suppose that the recognition model has learned training data in which, in the video frame in which the worker wraps part X with cushioning material and secures it with tape β, if the objects interacting with the worker's hand are tape β and cushioning material, the action of wrapping with cushioning material and securing with tape β is defined and an annotation is added. In this case, the work recognition unit 209, using the recognition model, recognizes the work process during the above time period in the video as objects "tape β, cushioning material" and action "wrapping".

[0068] For example, suppose that during the time period from "5 / 10 12:00:30" to "5 / 10 12:00:40" in the video, worker A is assembling the outer box, and an interaction occurs between worker A's hand and the outer box. Furthermore, suppose that the recognition model has learned training data in which frames of video showing the worker assembling the outer box are annotated with a defined action of assembling the outer box when the object interacting with the worker's hand is the outer box. In this case, the work recognition unit 209, using the recognition model, recognizes the work process during the above time period in the video as the object "outer box" and the action "assembly".

[0069] For example, suppose that during the time frame "5 / 10 12:00:40" to "5 / 10 12:00:45" in the video, worker A is attaching a label to the outer box, and an interaction occurs between worker A's hand and the label. Also, suppose that the recognition model has learned training data in which, when the object interacting with the worker's hand in the video frame of the worker attaching the label to the outer box is the label, an annotation is added that defines the action as "attaching a label". In this case, the work recognition unit 209, using the recognition model, recognizes the work process during the above time frame in the video as the object "label" and the action "attaching".

[0070] For example, suppose that during the time frame "5 / 10 12:00:45" to "5 / 10 12:00:50" in the video, worker A is packing the cushioning material that wraps part X into an outer box, and interaction occurs between worker A's hand and the cushioning material and the outer box. Furthermore, suppose that the recognition model has learned training data in which, in the video frame in which the worker is packing the cushioning material that wraps part X into an outer box, if the objects that the worker's hand is interacting with are the cushioning material and the outer box, an annotation is added that defines the action as "packing". In this case, the work recognition unit 209, using the recognition model, recognizes the work process during the above time frame in the video as objects "cushioning material, outer box" and action "packing".

[0071] For example, suppose that during the time period from "5 / 10 12:00:50" to "5 / 10 12:01:00" in the video, worker A is attaching tape γ to the lid of the outer box and sealing the box, and that interaction is occurring between worker A's hand, tape γ, and the outer box. Furthermore, suppose that the recognition model has learned training data in which, in the video frame showing tape γ being attached to the lid of the outer box and the outer box being sealed, if the objects interacting with the worker's hand are tape γ and the outer box, the action defined as sealing is added. In this case, the work recognition unit 209, using the recognition model, recognizes the work process during the above time period in the video as objects "tape γ, outer box" and action "sealing".

[0072] Furthermore, prior to the work process of placing the object "cushioning material" and performing the action "placing," worker A reads the code information assigned to part X using worker terminal 40. After the work process of placing the object "tape γ, outer box" and performing the action "sealing," worker A completes the individual packaging of part X using worker terminal 40. Therefore, the work recognition unit 209 can recognize that the type of work for each work process performed from "5 / 10 12:00:00" to "5 / 10 12:01:00" is individual packaging of part X.

[0073] A frame image is video data corresponding to the frame number indicated by the frame, and is video data extracted by the work recognition unit 209 from the video used for work recognition.

[0074] The input / output control unit 221 controls the input and output of various information between the administrator terminal 30 and the worker terminal 40. The reception unit 223 included in the input / output control unit 221 receives a request from the administrator terminal 30 to output an administrator screen for manual management, based on instructions from the administrator to the administrator terminal 30. When the reception unit 223 receives the request to output an administrator screen, the output unit 225 included in the input / output control unit 221 outputs an administrator screen (an example of a settings screen) to the administrator terminal 30, and the control device 31 of the administrator terminal 30 displays the administrator screen on the display device 36. Note that the display of the administrator screen by the administrator terminal 30 may be contingent on the administrator's login authentication.

[0075] The administrator screen is a settings screen for setting at least one of the following: a first condition for selecting the type of work to be included in manual generation from among multiple types of work recognized by the work recognition unit 209, and a second condition for deleting manuals from the manual storage unit 245, which will be described later. Furthermore, the administrator screen is a settings screen for setting a third condition for selecting the workers to be included in the manual effectiveness measurement.

[0076] Figure 7 shows an example of the administrator screen 601 of this embodiment. In the example shown in Figure 7, the administrator screen 601 includes a manual generation condition setting menu 611, a manual target user setting menu 613, an active manual condition setting menu 615, and a manual merge condition setting menu 617.

[0077] The manual generation condition setting menu 611 is a menu for displaying the manual generation condition setting screen, which sets the first condition for selecting the type of work to be included in manual generation from among multiple types of work recognized by the work recognition unit 209. The manual target user setting menu 613 is a menu for displaying the manual target user setting screen, which sets the third condition for selecting the workers to be included in the effectiveness measurement of the manuals generated by the manual generation unit 243, described later. The valid manual condition setting menu 615 and the manual merge condition setting menu 617 are menus for displaying setting screens for setting the second condition for deleting manuals from the manual storage unit 245, described later.

[0078] When the administrator uses the input device 35 to select the manual generation condition setting menu 611 on the administrator screen 601, the administrator terminal 30 requests the information processing device 20 to output the manual generation condition setting screen. When the reception unit 223 receives the request from the administrator terminal 30 to output the manual generation condition setting screen, the output unit 225 outputs the manual generation condition setting screen to the administrator terminal 30, and the control device 31 of the administrator terminal 30 displays the manual generation condition setting screen on the display device 36.

[0079] Figure 8 shows an example of the manual generation condition setting screen 621 of this embodiment. In the example shown in Figure 8, the manual generation condition setting screen 621 allows setting of work time, work process, work variability, number of items handled, and search period as first conditions for selecting the type of work to be generated as a manual. However, the first conditions are not limited to these.

[0080] In the example shown in Figure 8, the work time is set to 600 seconds or more. In this case, tasks of a type that take 600 seconds or more to complete will be subject to manual generation. Tasks of a type that take 600 seconds or more to complete are, for example, tasks where the average work time of all workers is 600 seconds or more. Also in the example shown in Figure 8, the work process is set to 10 steps or more. In this case, tasks of a type with a work process of 10 steps or more will be subject to manual generation. Furthermore, in the example shown in Figure 8, the work variability is set to the top 10% difference. Among all types of tasks, tasks where the difference in work time between the shortest and longest workers is in the top 10% will be subject to manual generation. Finally, in the example shown in Figure 8, the number of items handled is set to 50 or more. In this case, tasks of a type with 50 or more items handled per month (product-related tasks) will be subject to manual generation. Furthermore, in the example shown in Figure 8, the search period is set to 90 days, and the work history of the most recent 90 days stored in the work history storage unit 211 becomes the population for selecting the type of work to be included in the manual generation. Note that the work time, work process, work variability, number of items handled, and search period mentioned above may be applied using either AND or OR conditions.

[0081] In this embodiment, the first condition includes at least an index for estimating the degree of variation in work methods. Examples of such indices include, but are not limited to, work time, work process, and work variability. As work time and work process increase, the amount of work increases, and as the amount of work increases, the methods of work become more diverse and variability is expected. Therefore, these can be considered indices for estimating the degree of variation in work methods. Work variability is an index used to calculate the degree of variation in work methods.

[0082] The administrator uses the input device 35 to input the first condition on the manual generation condition setting screen 621, and when they select a setting button (not shown), the administrator terminal 30 outputs the set first condition to the information processing device 20. The reception unit 223 receives the first condition set on the manual generation condition setting screen 621 (administrator screen 601) from the administrator terminal 30.

[0083] Furthermore, when the administrator uses the input device 35 to select the manual user setting menu 613 on the administrator screen 601, the administrator terminal 30 requests the information processing device 20 to output the manual user setting screen. When the reception unit 223 receives the request from the administrator terminal 30 to output the manual user setting screen, the output unit 225 outputs the manual user setting screen to the administrator terminal 30, and the control device 31 of the administrator terminal 30 displays the manual user setting screen on the display device 36.

[0084] Figure 9 shows an example of the manual target user setting screen 631 of this embodiment. In the example shown in Figure 9, the manual target user setting screen 631 allows setting the third conditions for selecting workers to be subject to manual effectiveness measurement as: no filtering, first time, and lower work time. However, the third conditions are not limited to these.

[0085] In the example shown in Figure 9, no filtering is set. If no filtering is set, all workers will be included in the manual's effectiveness measurement. Also in the example shown in Figure 9, first time is set, in which case workers who perform the type of work for which a manual has been generated (prepared) for the first time will be included in the manual's effectiveness measurement. Also in the example shown in Figure 9, the bottom 50% of work time is set, in which case workers whose work time for the type of work for which a manual has been generated (prepared) falls in the bottom 50% of all workers will be included in the manual's effectiveness measurement.

[0086] In this embodiment, the third condition includes at least a condition for identifying workers who are expected to benefit from work improvement. Examples of conditions for identifying workers who are expected to benefit from work improvement include, but are not limited to, first-time workers and those with lower work experience. Workers performing a task for the first time can understand the work content and procedures in advance by checking the manual, and are expected to benefit from work improvement compared to those who do not check the manual. Workers with lower work experience can understand the work procedures that need improvement by checking the manual, and are expected to benefit from work improvement compared to those who do not check the manual.

[0087] The administrator uses the input device 35 to input the third condition on the manual target user setting screen 631, and when they select a setting button (not shown), the administrator terminal 30 outputs the set third condition to the information processing device 20. The reception unit 223 receives the third condition set on the manual target user setting screen 631 (administrator screen 601) from the administrator terminal 30.

[0088] The management unit 231 manages various conditions for manual management and the manuals themselves. The condition management unit 233, which is part of the management unit 231, stores the first and third conditions in the condition storage unit 241 when they are received by the reception unit 223, and manages these conditions on the condition storage unit 241.

[0089] The manual generation unit 243 generates a manual for the work recognized by the work recognition unit 209 and stores it in the manual storage unit 245. Specifically, the manual generation unit 243 selects a type of work that satisfies the first condition from among multiple types of work recognized by the work recognition unit 209 and generates a manual for the selected work.

[0090] The method used by the manual generation unit 243 to generate the manual may be anything. For example, the manual generation unit 243 may select an operator who can perform the selected task in the shortest time from the work history stored in the work history storage unit 211, and generate the manual using the work history of the selected operator. To prevent the manual from being generated using the work history of a task that was completed in the shortest time for a specific reason, the manual generation unit 243 may extract a certain number of work histories with short work times for the selected task, and generate the manual using the work history of the work procedure with the most identical work procedures. Alternatively, the manual generation unit 243 may select an expert in the selected task and generate the manual using the work history of the selected expert. An expert may be an operator who has handled the selected task (parts) the most times or an operator with the longest experience in the selected task (parts), but is not limited to these.

[0091] For example, suppose the first condition set on the manual generation condition setting screen 621 shown in Figure 8 is stored in the condition storage unit 241, and the average working time for individual packaging of part X, as explained in Figure 6, is 600 seconds or more. The average working time for individual packaging of part X can be calculated from the work history stored in the work history storage unit 211. In this case, the manual generation unit 243 selects individual packaging of part X as a type of work that satisfies the first condition and generates a manual for individual packaging of part X.

[0092] Figure 10 shows an example of a manual generated by the manual generation unit 243 of this embodiment. As mentioned above, the example shown in Figure 10 shows a manual for individual packaging of part X. For example, the manual generation unit 243 selects the work history of individual packaging of part X to be used for manual generation, and generates the manual shown in Figure 10 using the work process and frame video of the selected work history. MOV001 to MOV007 are, for example, thumbnails of frame video, and when selected by the operator, the frame video is played. Note that MOV001 to MOV007 may be simple thumbnails that do not play frame video. Note that in the example shown in Figure 10, information on objects and actions in the work process is used to create text, but the objects and actions in the work process may be displayed as they are.

[0093] The notification unit 235 selects workers who meet the third condition from among multiple workers for tasks of the type covered by the manual generated by the manual generation unit 243, and notifies at least the selected workers of the manual from the output unit 225. Since the selected workers are the ones subject to manual effectiveness measurement, the notification unit 235 may also notify workers who are not subject to manual effectiveness measurement.

[0094] For example, the reception unit 223 receives a request from the worker terminal 40 to output a worker screen for manual confirmation based on instructions from the worker to the worker terminal 40. When the reception unit 223 receives the request to output a worker screen, the notification unit 235 selects a manual (for work that satisfies the third condition) that the worker on the worker terminal 40 is subject to effectiveness measurement, and, referring to the browsing information stored in the browsing information storage unit 205, generates a worker screen that shows a list of manuals, associating whether the selected manual has been confirmed or not. The output unit 225 outputs the worker screen generated by the management unit 231 to the worker terminal 40, and the control device 41 of the worker terminal 40 displays the worker screen on the display device 46. Note that the display of the worker screen by the worker terminal 40 may be contingent on the worker's login authentication.

[0095] Figure 11 is a diagram showing an example of the worker screen 701 of this embodiment, and shows the worker screen of worker A. In the example shown in Figure 11, the worker screen 701 includes a manual list display area 711. The manual list display area 711 displays information such as whether or not the manual has been reviewed, associated with the name of each manual that the worker is using for effectiveness measurement.

[0096] For example, when worker A uses the input device 45 to select the manual name 713 for part X on the manual list display area 711, the worker terminal 40 requests the information processing device 20 to output the manual for part X. When the reception unit 223 receives the request from the worker terminal 40 to output the manual for part X, the notification unit 235 retrieves the manual for part X shown in Figure 10 from the manual storage unit 245. The notification unit 235 also updates the viewing flag for worker A's part X on the viewing information storage unit 205 from "Not yet viewed" to "Viewed". As a result, the viewing information shown in Figure 5 is updated to the viewing information shown in Figure 12. Figure 12 is a diagram showing an example of viewing information with updated flag information in this embodiment. The notification unit 235 generates a worker screen in which the information 715 associated with whether or not the manual name 713 for part X in the manual list on the worker screen created earlier has been confirmed has been updated from "Confirmed" to "Viewed". The output unit 225 outputs the manual for the individual part X package and the updated worker screen acquired by the notification unit 235 to the worker terminal 40. The control device 41 of the worker terminal 40 displays the updated worker screen on the display device 46, and also displays the manual for the individual part X package shown in Figure 10 on the display device 46. This allows worker A to check the manual for the individual part X package.

[0097] Now, let's return to Figure 7 and explain the Enabled Manual Condition Setting Menu 615. When the administrator selects the Enabled Manual Condition Setting Menu 615 on the administrator screen 601 using the input device 35, the administrator terminal 30 requests the information processing device 20 to output the Enabled Manual Condition Setting Screen. When the reception unit 223 receives the request from the administrator terminal 30 to output the Enabled Manual Condition Setting Screen, the output unit 225 outputs the Enabled Manual Condition Setting Screen to the administrator terminal 30, and the control device 31 of the administrator terminal 30 displays the Enabled Manual Condition Setting Screen on the display device 36.

[0098] Figure 13 shows an example of the effective manual condition setting screen 641 of this embodiment. In the example shown in Figure 13, the effective manual condition setting screen 641 allows setting the effect, number of items handled, and last viewed date as second conditions for deleting manuals from the manual storage unit 245 (in other words, second conditions for keeping manuals in the manual storage unit 245). However, the second conditions are not limited to these.

[0099] In the example shown in Figure 13, the effect is set to 10% seconds or more. In this case, manuals that result in a work improvement effect of 10% seconds or more are eligible to be kept as valid manuals in the manual storage unit 245. In other words, manuals that result in a work improvement effect of less than 10% seconds are eligible to be deleted from the manual storage unit 245. Note that "10% seconds or more / less than 10%" means, for example, that for a given manual, the average work improvement effect of all workers who are subject to effect measurement (meeting the third condition) is 10% seconds or more / less than 10%. Also, in the example shown in Figure 13, the number of items handled is set to 100 or more per 180 days. In this case, manuals for parts that have 100 or more items handled in 180 days are eligible to be kept as valid manuals in the manual storage unit 245. In other words, manuals for parts that have fewer than 100 items handled in 180 days are eligible to be deleted from the manual storage unit 245. Furthermore, in the example shown in Figure 13, the last access date is set to within 365 days. In this case, manuals whose last access date falls within 365 days are eligible to be kept as valid manuals in the manual storage unit 245. In other words, manuals whose last access date exceeds 365 days are eligible to be deleted from the manual storage unit 245. Note that the effects, number of items handled, and last access date described above can be expressed using either AND or OR conditions.

[0100] In this embodiment, the second condition includes at least a condition relating to the work improvement effect. The condition relating to the work improvement effect may include, but is not limited to, the effect. Furthermore, the second condition includes at least an indicator for estimating the frequency of manual use. The indicator for estimating the frequency of manual use may include, but is not limited to, the number of items handled and the last viewing date. It can be assumed that the manual is used less frequently when the number of parts handled is small or when the period since the last viewing date is long, so these can be considered indicators for estimating the frequency of manual use.

[0101] The administrator uses the input device 35 to input the second condition on the active manual condition setting screen 641 and selects a setting button (not shown). The administrator terminal 30 then outputs the set second condition to the information processing device 20. The reception unit 223 receives the second condition set on the active manual condition setting screen 641 (administrator screen 601) from the administrator terminal 30. When the condition management unit 233 receives the second condition from the reception unit 223, it stores it in the condition storage unit 241 and manages this condition on the condition storage unit 241.

[0102] The effectiveness measurement unit 247 measures the work improvement effect of the manual based on the work recognition results of the worker subject to the effectiveness measurement before and after applying the manual. The work improvement effect of the manual for individual packaging of part X will be explained below, using part X packaging as an example.

[0103] Figure 14 shows an example of the work history in this embodiment, illustrating the recognition result of the individual packaging work on part X performed by worker A after reviewing the manual for individual packaging of part X. Therefore, the individual packaging work on part X in the work history shown in Figure 14 was performed one month after the work history shown in Figure 6 (see start and end dates and times), and the viewing flag is turned on. Furthermore, in the work history shown in Figure 14, the time required for the work process of object "tape α, part X" and action "fixing" has been improved from 15 seconds to 5 seconds compared to the work history shown in Figure 6. Possible reasons for this include, but are not limited to, worker A reviewing the video (sample video) of the work process of object "tape α, part X" and action "fixing" in the manual for individual packaging of part X, and making changes such as reducing the placement of tape α and the number of tapes α used to fix part X to the cushioning material.

[0104] The effectiveness measurement unit 247 compares the work history of the individual packaging work of part X shown in Figure 6 with the work history of the individual packaging work of part X shown in Figure 14 to measure the work improvement effect of the manual for individual packaging of part X. In Figure 6, the individual packaging work of part X took 60 seconds, whereas in Figure 14, the individual packaging work of part X took only 50 seconds. Therefore, the effectiveness measurement unit 247 measures that the work efficiency has improved by about 17%, and the work improvement effect is 17 seconds. The effectiveness measurement unit 247 also measures the work improvement effect for other workers who are subject to effectiveness measurement of the manual for individual packaging of part X in the same way, and calculates the average work improvement effect of the manual for individual packaging of part X.

[0105] The manual management unit 237 deletes a manual from the manual storage unit 245 based on the second condition. Specifically, if the work improvement effect measured by the effect measurement unit 247 does not satisfy the second condition, the manual management unit 237 deletes the manual for which the work improvement effect was measured from the manual storage unit 245.

[0106] For example, suppose the second condition set on the effective manual condition setting screen 641 shown in Figure 13 is stored in the condition storage unit 241, and the average work improvement effect of the manual for individual parts X, as determined by the effect measurement unit 247 described above, is 15 seconds. In this case, the manual management unit 237 keeps the manual for individual parts X as an effective manual in the manual storage unit 245 because the work improvement effect of the manual for individual parts X is 10 seconds or more (the second condition is met). Alternatively, suppose the average work improvement effect of the manual for individual parts X, as determined by the effect measurement unit 247 described above, is 5 seconds. In this case, the manual management unit 237 deletes the manual for individual parts X from the manual storage unit 245 because the work improvement effect of the manual for individual parts X is less than 10 seconds (the second condition is not met).

[0107] For example, suppose the number of times part X is handled in the manual for individual part X packaging is less than 100 times over a 180-day period. The number of times part X is handled can be calculated from the work history stored in the work history storage unit 211. In this case, the manual management unit 237 deletes the manual for individual part X packaging from the manual storage unit 245 because the number of times part X is handled is less than 100 times over a 180-day period (the second condition is not met).

[0108] For example, suppose the last viewing date of the manual for individual part X is more than 365 days ago. The last viewing date of the manual can be associated with the manual in the manual storage unit 245 and updated each time the notification unit 235 notifies the manual. In this case, the manual management unit 237 deletes the manual for individual part X from the manual storage unit 245 because the last viewing date of the manual for individual part X is more than 365 days ago (and therefore the second condition is not met).

[0109] Now, let's return to Figure 7 and explain the manual merge condition setting menu 617. When the administrator selects the manual merge condition setting menu 617 on the administrator screen 601 using the input device 35, the administrator terminal 30 requests the information processing device 20 to output the manual merge condition setting screen. When the reception unit 223 receives the request from the administrator terminal 30 to output the manual merge condition setting screen, the output unit 225 outputs the manual merge condition setting screen to the administrator terminal 30, and the control device 31 of the administrator terminal 30 displays the manual merge condition setting screen on the display device 36.

[0110] Figure 15 shows an example of the manual merge condition setting screen 651 of this embodiment. In the example shown in Figure 15, the manual merge condition setting screen 651 allows specifying multiple manuals to be merged as a second condition for deleting manuals from the manual storage unit 245. However, the second condition is not limited to these.

[0111] Many parts handled in a logistics center (warehouse) belong to the same category and have identical work procedures (work processes), such as simply being different in color. If the manuals for each of these parts are stored in the manual storage unit 245, it will strain the data capacity of the manual storage unit 245. For this reason, the manual merge condition setting screen 651 allows the manuals for such parts to be merged (integrated) into a single manual.

[0112] In the manual merge condition setting screen 651 shown in Figure 15, the work procedure (work process) "H→I→J→K→L" is set, and manuals for parts P, Q, R, S, and T whose work procedure is "H→I→J→K→L" are displayed for selection. Specifically, when the administrator uses the input device 35 to specify the work procedure (work process) "H→I→J→K→L" on the manual merge condition setting screen 651, the administrator terminal 30 requests the information processing device 20 to output parts whose work procedure is "H→I→J→K→L". When the reception unit 223 receives the request from the administrator terminal 30 to output parts whose work procedure is "H→I→J→K→L", the management unit 231 searches the work history storage unit 211 for parts whose work procedure is "H→I→J→K→L". The output unit 225 outputs parts P, Q, R, S, and T to the administrator terminal 30 as the search results of the management unit 231, and the control device 31 of the administrator terminal 30 displays the manuals for parts P, Q, R, S, and T on the manual merge condition setting screen 651 so that they can be specified.

[0113] In the example shown in Figure 15, the administrator uses the input device 35 to specify the manuals for parts P and Q from the manuals for parts P, Q, R, S, and T as the targets for merging. When the administrator selects an execution button (not shown), the administrator terminal 30 outputs the configured second condition (merging the manuals for parts P and Q) to the information processing device 20. The reception unit 223 receives the second condition set on the manual merging condition setting screen 651 (administrator screen 601) from the administrator terminal 30.

[0114] The manual management unit 237 deletes manuals from the manual storage unit 245 based on the second condition. Specifically, the manual management unit 237 merges multiple manuals specified in the second condition and stores them in the manual storage unit 245, and then deletes the original manuals from the manual storage unit 245.

[0115] For example, if the second condition is the manuals for parts P and Q set on the manual merge condition setting screen 651 shown in Figure 15, the manual management unit 237 merges the manuals for parts P and Q and stores them in the manual storage unit 245, and deletes the original manuals from the manual storage unit 245. Merging the manuals for parts P and Q and storing them in the manual storage unit 245 means that the manual for part P may be left in the manual storage unit 245, the manual for part Q may be left in the manual storage unit 245, or the combined manual of the manuals for parts P and Q may be stored in the manual storage unit 245. Deleting the original manuals from the manual storage unit 245 means that the manual for part P may be deleted from the manual storage unit 245, the manual for part Q may be deleted from the manual storage unit 245, or the manuals for both parts P and Q may be deleted from the manual storage unit 245.

[0116] Next, the operation of the work analysis system of this embodiment will be described.

[0117] Figure 16 is a flowchart showing an example of the work recognition process performed by the information processing device 20 in this embodiment. The work recognition process shown in Figure 16 may be performed sequentially or periodically.

[0118] First, the video acquisition unit 201 acquires multiple videos from multiple shooting devices 10-1 to 10-n, capturing the work status of multiple workers, and stores (accumulates) them in the video storage unit 203 (step S101).

[0119] Next, the work recognition unit 209 acquires video from the video storage unit 203, inputs the acquired video into the recognition model stored in the recognition model storage unit 207, and obtains the recognition result, thereby recognizing the work being performed by the worker shown in the video on a work process basis (step S103).

[0120] Next, the work recognition unit 209 refers to the browsing information stored in the browsing information storage unit 205 and sets a browsing flag in the work recognition result to indicate whether or not the manual has been viewed (step S105).

[0121] Next, the work recognition unit 209 stores the recognition result of the work for which the viewing flag has been set as work history in the work history storage unit 211 (step S107).

[0122] Figure 17 is a sequence diagram showing an example of the condition setting process performed in the work analysis system 1 of this embodiment. In the following, the condition setting process shown in Figure 17 will be explained using the setting of the first condition for selecting the type of work to be generated as an example, but the condition setting process shown in Figure 17 can also be applied to setting the second condition for deleting manuals and the third condition for selecting the workers to be measured for the effectiveness of the manuals.

[0123] First, the administrator uses the input device 35 to select the manual generation condition setting menu 611 on the administrator screen 601 shown in Figure 7. The administrator terminal 30 then requests the information processing device 20 to output the manual generation condition setting screen (step S201).

[0124] Next, when the reception unit 223 receives a request from the administrator terminal 30 to output the manual generation condition setting screen, the output unit 225 outputs the manual generation condition setting screen to the administrator terminal 30 (step S203), and the control device 31 of the administrator terminal 30 displays the manual generation condition setting screen shown in Figure 8 on the display device 36 (step S205).

[0125] Next, the administrator uses the input device 35 to set the first condition on the manual generation condition setting screen 621 (step S207). When the administrator selects a setting button (not shown), the administrator terminal 30 outputs the set first condition to the information processing device 20 (step S209).

[0126] Next, the reception unit 223 receives the first condition set on the manual generation condition setting screen 621 (administrator screen 601) from the administrator terminal 30, and the condition management unit 233 stores it in the condition storage unit 241 (step S211).

[0127] Figure 18 is a flowchart showing an example of the manual generation process performed by the information processing device 20 of this embodiment.

[0128] First, the manual generation unit 243 selects a type of work that satisfies the first condition from among the multiple types of work recognized by the work recognition unit 209 (Yes in step S301), generates a manual for the selected work (step S303), and stores it in the manual storage unit 245 (step S305).

[0129] Next, the notification unit 235 selects workers who meet the third condition (who are subject to manual effectiveness measurement) from among multiple workers for the type of work covered by the manual generated by the manual generation unit 243 (step S307).

[0130] Note that for types of work that do not meet the first condition, steps S303 to S307 are not performed (No in step S301).

[0131] Figure 19 is a sequence diagram showing an example of the manual verification process performed in the work analysis system 1 of this embodiment.

[0132] First, the reception unit 223 receives a request from the worker terminal 40 to output a worker screen for manual confirmation, based on instructions from the worker to the worker terminal 40 (step S401).

[0133] Next, the notification unit 235 selects a manual (for work that satisfies the third condition) that the worker on the worker terminal 40 is subject to effectiveness measurement, and, referring to the browsing information stored in the browsing information storage unit 205, generates a worker screen that shows a list of manuals, associating whether or not the selected manual has been checked (step S403).

[0134] Next, the output unit 225 outputs the worker screen generated by the management unit 231 to the worker terminal 40 (step S405), and the control device 41 of the worker terminal 40 displays the worker screen on the display device 46 (step S407).

[0135] Next, when the worker uses the input device 45 to select the manual to be viewed, the worker terminal 40 requests the information processing device 20 to output the manual to be viewed (step S409).

[0136] Next, when the reception unit 223 receives a request from the worker terminal 40 to output the manual to be viewed, the notification unit 235 retrieves the manual to be viewed from the manual storage unit 245 (step S411).

[0137] Next, the notification unit 235 updates the viewing flag for the manual that the worker is viewing in the viewing information storage unit 205 from "Not yet viewed" to "Viewed" (step S413).

[0138] Next, the notification unit 235 generates a worker screen in which the information indicating whether or not the manual in the worker screen created earlier has been confirmed has been updated from "Confirmed" to "Completed" (step S415).

[0139] Next, the output unit 225 outputs the manual to be viewed and the updated worker screen acquired by the notification unit 235 to the worker terminal 40 (step S417), and the control device 41 of the worker terminal 40 displays the updated worker screen and the manual to be viewed on the display device 46 (step S419).

[0140] Figure 20 is a flowchart showing an example of a manual deletion process performed by the information processing device 20 of this embodiment.

[0141] First, the effectiveness measurement unit 247 measures the work improvement effect of the manual based on the work recognition results of the worker subject to the effectiveness measurement before and after applying the manual (step S501).

[0142] Next, if the work improvement effect measured by the effect measurement unit 247 does not meet the second condition (No in step S503), the manual management unit 237 deletes the manual for which the work improvement effect was measured from the manual storage unit 245 (step S505).

[0143] If the work improvement effect measured by the effect measurement unit 247 satisfies the second condition (Yes in step S503), the process in step S505 is not performed, and the manual for which the work improvement effect was measured remains in the manual storage unit 245.

[0144] Figure 21 is a sequence diagram showing an example of a manual merge process performed in the work analysis system 1 of this embodiment.

[0145] First, the administrator uses the input device 35 to select the manual merge condition setting menu 617 on the administrator screen 601, and the administrator terminal 30 requests the information processing device 20 to output the manual merge condition setting screen (step S601).

[0146] Next, the output unit 225 outputs the manual merge condition setting screen to the administrator terminal 30 (step S603), and the control device 31 of the administrator terminal 30 displays the manual merge condition setting screen on the display device 36 (step S605).

[0147] Next, when the administrator uses the input device 35 to specify a work procedure on the manual merge condition setting screen, the administrator terminal 30 requests the information processing device 20 to output the components that constitute the specified work procedure (step S607).

[0148] Next, when the reception unit 223 receives an output request for parts corresponding to the specified work procedure from the administrator terminal 30, the management unit 231 searches the work history storage unit 211 for parts corresponding to the specified work procedure (step S609), and the output unit 225 outputs the search results of the management unit 231 to the administrator terminal 30 (step S611).

[0149] Next, the control device 31 of the administrator terminal 30 displays the manual for the searched part on the manual merge condition setting screen 651, making it possible to specify it (step S613).

[0150] The administrator sets the manuals of the parts to be merged using the input device 35 (step S615), and when the administrator selects the execute button (not shown), the administrator terminal 30 notifies the information processing device 20 of the set manuals to be merged (step S617).

[0151] Next, the manual management unit 237 merges the multiple manuals that have been notified and stores them in the manual storage unit 245, and then deletes the original manuals from the manual storage unit 245 (step S619).

[0152] As described above, the work analysis system of this embodiment analyzes video footage of the work performed by each worker, estimates the work requiring high-level standardization using the analysis results, and generates a manual for the estimated work. This enables the high-level standardization of work requiring high-level standardization that was initially unclear (unknown). For example, this enables high-level standardization of work where there is variation in work time among workers, but the variation in work time converges as the workers grasp the optimal method.

[0153] Furthermore, the work analysis system of this embodiment analyzes video footage of the work performed by each worker, and uses the analysis results to delete manuals that have a low support effect on high-level standardization, while retaining (keeping) manuals that have a high support effect on high-level standardization. This makes it possible to support high-level standardization for tasks that were not initially clear (unknown) and required high-level standardization. Manuals that have a high support effect on high-level standardization are expected to be effective for everyone, regardless of personal reasons, and are therefore expected to be useful in training new employees and other workers who are performing warehouse work for the first time.

[0154] Furthermore, in this work analysis system, since the manuals for each part belonging to the same category with identical work procedures can be merged (integrated) into a single manual, the pressure on the data capacity of the memory unit that stores the manuals can be suppressed.

[0155] (Variation 1) In the above embodiment, the manual generation unit 243 selected a type of work that satisfies the first condition from among multiple types of work recognized by the work recognition unit 209 and generated a manual for the selected work. However, it is also possible to generate manuals for all types of work. Even in this case, manuals that do not satisfy the second condition, such as those with low work improvement effects or low usage frequency, will be deleted, so manuals that have a high support effect for high-level standardization can be maintained (retained).

[0156] (Modification 2) In the above embodiment, an example was described in which the measurement results of the work improvement effect of the effect measurement unit 247 are used for manual freshness management (deciding whether to delete or keep). However, the measurement results of the work improvement effect of the effect measurement unit 247 may also be notified to the administrator. Alternatively, the measurement results of the work improvement effect of the effect measurement unit 247 may be notified to the administrator without being used for manual freshness management (deciding whether to delete or keep).

[0157] For example, the output unit 225 may output a work improvement effect display screen to the administrator terminal 30, which displays the work improvement effect measured by the effect measurement unit 247. In this way, the administrator can grasp the degree of work improvement effect for each task and consider further work improvement methods.

[0158] For example, the notification unit 235 may also be configured to notify the administrator terminal 30 of the type of work covered by the manual if the work improvement effect measured by the effectiveness measurement unit 247 does not meet the threshold. Work that does not meet the threshold for work improvement is work for which the manual has a low work improvement effect, but it is not clear whether the cause is the manual or the individual ability of the worker. Therefore, by notifying the administrator of this type of work, the administrator can consider the cause of the low work improvement effect, and if the low work improvement effect is due to the manual, they can take the next step, such as manually rewriting the manual.

[0159] (program) The programs executed by the information processing device 20 in the above embodiments and each of the above modified examples are provided as installable or executable files stored on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD).

[0160] Furthermore, the programs executed by the information processing device 20 in the above embodiments and each of the modified examples may be stored on a computer connected to a network such as the Internet and provided by downloading them via the network. Alternatively, the programs executed by the information processing device 20 in the above embodiments and each of the modified examples may be provided or distributed via a network such as the Internet. Furthermore, the programs executed by the information processing device 20 in the above embodiments and each of the modified examples may be pre-installed in ROM or the like and provided.

[0161] The programs executed in the information processing device 20 of the above embodiments and each of the above modifications are configured as modules for realizing the above-described parts on a computer. In actual hardware, for example, the CPU reads the learning program from the HDD into RAM and executes it, thereby realizing the above-described parts on the computer.

[0162] The above embodiments and their respective modifications are merely examples of how this disclosure may be implemented, and they do not restrict the technical scope of this disclosure. Therefore, this disclosure can be implemented in various ways without departing from its essence or its main features. For example, the above embodiments and their respective modifications may be combined as appropriate on a component basis. Also, for example, some components may be removed from the total components in the above embodiments and their respective modifications.

[0163] This disclosure includes the following aspects:

[0164] (1) A video acquisition unit that acquires multiple videos of the work status of multiple workers, A work recognition unit analyzes the aforementioned multiple videos to recognize multiple types of tasks performed by each worker, A manual generation unit generates a manual for the recognized operation and stores it in the manual storage unit, An output unit outputs a setting screen to an administrator terminal used by the administrator, which sets at least one of the following: a first condition for selecting the type of work to be included in manual generation from among the multiple types of work mentioned above, and a second condition for deleting manuals from the manual storage unit. An information processing device equipped with the following features.

[0165] (2) The system further includes a receiving unit that receives the first condition set on the settings screen from the administrator terminal, The manual generation unit selects a type of work that satisfies the first condition from among the multiple types of work, and generates a manual for the selected work. The information processing device described in (1) above.

[0166] (3) The first condition above includes at least an index for estimating the degree of variation in the way the work is done, The information processing device described in (2) above.

[0167] (4) The system further includes a receiving unit that receives the second condition set on the settings screen from the administrator terminal, The manual management unit further comprises a manual management unit that deletes a manual from the manual storage unit based on the second condition described above. An information processing device as described in any one of (1) to (3) above.

[0168] (5) The system further comprises an effectiveness measurement unit that measures the work improvement effect of the manual based on the work recognition results of the workers subject to the effectiveness measurement before and after the application of the manual, The second condition mentioned above includes at least a condition relating to the work improvement effect, If the work improvement effect measured by the effect measurement unit does not satisfy the second condition, the manual management unit deletes the manual for which the work improvement effect was measured from the manual storage unit. The information processing device described in (4) above.

[0169] (6) The settings screen is further used to set a third condition for selecting the workers who will be subject to the manual's effectiveness measurement, The reception unit further receives the third condition set on the settings screen from the administrator terminal, The manual generation unit further includes a notification unit that, for tasks of the type covered by the manual generated by the manual generation unit, selects workers who meet the third condition from among the multiple workers and notifies at least the selected workers of the manual. The information processing device described in (5) above.

[0170] (7) The third condition above includes at least a condition for identifying workers from whom work improvement effects are expected, The information processing device described in (6) above.

[0171] (8) The second condition above includes at least an index for estimating the frequency of use of the manual, The information processing device described in (4) above.

[0172] (9) The second condition above includes at least the specification of multiple manuals to be merged, The manual management unit merges the multiple manuals specified by the second condition and stores them in the manual storage unit, and deletes the original manuals from which the merge was made. The information processing device described in (4) above.

[0173] (10) The work recognition unit recognizes each of the multiple types of work performed by each worker in terms of work process units. An information processing device as described in any one of (1) to (3) above.

[0174] (11) A video acquisition unit that acquires multiple videos of the work status of multiple workers, A work recognition unit analyzes the aforementioned multiple videos to recognize multiple types of tasks performed by each worker, A manual generation unit generates a manual for the recognized operation and stores it in the manual storage unit, An effectiveness measurement unit measures the work improvement effect of the manual based on the work recognition results of the workers subject to the effectiveness measurement before and after applying the manual, An information processing device equipped with the following features.

[0175] (12) The system further includes an output unit that outputs a work improvement effect display screen, which displays the work improvement effect measured by the effect measurement unit, to an administrator terminal used by the administrator. The information processing device described in (11) above.

[0176] (13) If the work improvement effect measured by the effect measurement unit does not meet the threshold, the system further includes a notification unit that notifies the administrator terminal of the type of work covered by the manual. The information processing device described in (11) above.

[0177] (14) A video acquisition step in which the video acquisition unit acquires multiple videos of the work status of multiple workers, The work recognition unit analyzes the multiple videos and recognizes multiple types of work performed by each worker in the work recognition step, The manual generation step involves the manual generation unit generating a manual for the recognized operation and storing it in the manual storage unit, The output step involves outputting a setting screen to an administrator terminal used by the administrator, which sets at least one of the following: a first condition for selecting a type of work from among the multiple types of work to be subject to manual generation, and a second condition for deleting a manual from the manual storage unit. Manual management methods including those mentioned.

[0178] (15) A video acquisition step in which multiple videos are taken of the work status of multiple workers, A work recognition step involves analyzing the aforementioned multiple videos to recognize multiple types of tasks performed by each worker, A manual generation step that generates a manual for the recognized operation and stores it in the manual storage unit, An output step that outputs a settings screen to an administrator terminal used by the administrator, which sets at least one of the following: a first condition for selecting the type of work to be included in manual generation from among the multiple types of work mentioned above, and a second condition for deleting the manual from the manual storage unit; A program that causes a computer to execute something. [Explanation of Symbols]

[0179] 1. Work Analysis System 2 Network 10, 10-1 to 10-n Multiple imaging devices 20 Information Processing Devices 30 Administrator terminals 40, 40-1~40-m Multiple worker terminals 201 Video Acquisition Unit 203 Video Memory Unit 205 Browsing Information Storage Unit 207 Recognition Model Memory Unit 209 Work Recognition Unit 211 Work history storage unit 221 Input / Output Control Unit 231 Management Department 241 Condition memory section 243 Manual Generation Unit 245 Manual Storage Unit 247 Effectiveness Measurement Department 223 Reception Department 225 Output section 231 Management Department 233 Condition Management Department 235 Notification Department 237 Manual Management Department

Claims

1. A video acquisition unit that acquires multiple videos capturing the work status of multiple workers, A work recognition unit analyzes the aforementioned multiple videos to recognize multiple types of tasks performed by each worker, A manual generation unit generates a manual for the recognized operation and stores it in the manual storage unit, An output unit outputs a setting screen to an administrator terminal used by the administrator, which sets at least one of the following: a first condition for selecting the type of work to be included in manual generation from among the multiple types of work mentioned above, and a second condition for deleting manuals from the manual storage unit. An information processing device equipped with the following features.

2. The system further includes a reception unit that receives the first condition set on the settings screen from the administrator terminal, The manual generation unit selects a type of work that satisfies the first condition from among the multiple types of work, and generates a manual for the selected work. The information processing apparatus according to claim 1.

3. The first condition above includes at least an index for estimating the degree of variability in the way the work is performed. The information processing apparatus according to claim 2.

4. The system further includes a reception unit that receives the second condition set on the settings screen from the administrator terminal, The manual management unit further comprises a manual that deletes a manual from the manual storage unit based on the second condition described above. The information processing apparatus according to any one of claims 1 to 3.

5. The system further includes an effectiveness measurement unit that measures the work improvement effect of the manual based on the work recognition results of the workers subject to the effectiveness measurement before and after the application of the manual. The second condition mentioned above includes at least a condition relating to the work improvement effect, If the work improvement effect measured by the effect measurement unit does not satisfy the second condition, the manual management unit deletes the manual for which the work improvement effect was measured from the manual storage unit. The information processing apparatus according to claim 4.

6. The aforementioned settings screen is further used to set a third condition for selecting the workers who will be subject to the manual's effectiveness measurement. The reception unit further receives the third condition set on the settings screen from the administrator terminal, The manual generation unit further includes a notification unit that, for the type of work covered by the manual generated by the manual generation unit, selects workers who meet the third condition from among the multiple workers and notifies at least the selected workers of the manual. The information processing apparatus according to claim 5.

7. The third condition mentioned above includes at least a condition for identifying workers from whom work improvement effects are expected, The information processing apparatus according to claim 6.

8. The second condition described above includes at least an indicator for estimating the frequency of use of the manual, The information processing apparatus according to claim 4.

9. The second condition above includes at least the specification of multiple manuals to be merged, The manual management unit merges the multiple manuals specified by the second condition and stores them in the manual storage unit, and deletes the original manuals from which the merge was made. The information processing apparatus according to claim 4.

10. The aforementioned work recognition unit recognizes each of the multiple types of work performed by each worker on a work process basis. The information processing apparatus according to any one of claims 1 to 3.

11. A video acquisition unit that acquires multiple videos capturing the work status of multiple workers, A work recognition unit analyzes the aforementioned multiple videos to recognize multiple types of tasks performed by each worker, A manual generation unit generates a manual for the recognized operation and stores it in the manual storage unit, An effectiveness measurement unit measures the work improvement effect of the manual based on the work recognition results of the workers subject to the effectiveness measurement before and after applying the manual, An information processing device equipped with the following features.

12. The system further includes an output unit that outputs a work improvement effect display screen, which displays the work improvement effect measured by the aforementioned effect measurement unit, to an administrator terminal used by the administrator. The information processing apparatus according to claim 11.

13. If the work improvement effect measured by the effect measurement unit does not meet the threshold, the system further includes a notification unit that notifies the administrator terminal of the type of work covered by the manual. The information processing apparatus according to claim 11.

14. The video acquisition unit acquires multiple videos capturing the work status of multiple workers in a video acquisition step, The work recognition unit analyzes the multiple videos and recognizes multiple types of work performed by each worker in the work recognition step, The manual generation step involves the manual generation unit generating a manual for the recognized operation and storing it in the manual storage unit, The output step involves outputting a setting screen to an administrator terminal used by the administrator, which sets at least one of the following: a first condition for selecting a type of work from among the multiple types of work to be subject to manual generation, and a second condition for deleting a manual from the manual storage unit. Manual management methods including those mentioned.

15. A video acquisition step involves acquiring multiple videos capturing the work status of multiple workers, A work recognition step involves analyzing the aforementioned multiple videos to recognize multiple types of tasks performed by each worker, A manual generation step that generates a manual for the recognized operation and stores it in the manual storage unit, Output step to an administrator terminal used by the administrator, which outputs a setting screen for setting at least one of the following: a first condition for selecting the type of work to be subject to manual generation from among the multiple types of work mentioned above, and a second condition for deleting the manual from the manual storage unit; A program that causes a computer to execute something.

Images