Content output control system, content output control method, and program

Abstract

A content output control system (1) comprises: a motion information acquisition means that acquires motion information representing movements of a worker; a motion analysis means that determines, on the basis of the motion information, whether or not a work of a process being performed by the worker has been completed; a display means that displays predetermined content for skill training of the worker at a predetermined position in the worker's work site; and a content output control means that controls the display means to display content corresponding to the next process in response to the determination by the motion analysis means that the work of the current process has been completed.

Description

Content output control system, content output control method, and program

[0001] The present disclosure relates to a content output control system, a content output control method, and a program.

[0002] There is known a technology that utilizes training content for assembly work instructions, skill training, etc. at manufacturing sites. For example, a system disclosed in Patent Literature 1 presents training content that reflects important work movements among model work performed by skilled or experienced workers to trainees.

[0003] Japanese Patent Application Laid-Open No. 2020-3707

[0004] When using training content such as that disclosed in Patent Document 1 for work instructions, skill training, and the like, the usual methods are to play the content continuously and perform the actual work in accordance with the content, or to switch to the work instruction content for the process to be checked by operating a button, keyboard, mouse, or the like. However, with the method of playing the content continuously, the playback pace must be adjusted to match the pace of the slow worker, which slows down the work to match the displayed content. Furthermore, with the method of switching the displayed content to the content for the next process, the work must be stopped to switch the display. Therefore, either method can lead to a decrease in work efficiency.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a content output control system, a content output control method, and a program that support improving work efficiency.

[0006] In order to achieve the above-mentioned object, the content output control system according to the present disclosure includes a motion information acquisition means for acquiring motion information representing the movements of a worker, a motion analysis means for determining whether the work of a process performed by the worker has been completed according to the motion information, a display means for displaying predetermined content for skill training of the worker at a predetermined position at the worker's work site, and a content output control means for controlling the display means to display content corresponding to the next process in response to the motion analysis means determining that the work of the process has been completed.

[0007] According to the present disclosure, when it is determined that the work of a process is completed according to the acquired operation information, content corresponding to the next process is displayed, thereby helping to improve the efficiency of the work.

[0008] FIG. 1 is a diagram showing the configuration of a content output control system according to an embodiment; FIG. 2 is a diagram showing an example of a content projection terminal projecting skill training content; FIG. 3 is a block diagram showing the functional configuration of a content output control device according to an embodiment; FIG. 4 is a diagram for explaining a comparison between standard work time and real-time work time; FIG. 5 is a diagram showing an example of work time for each cycle, with all five processes considered as one cycle; FIG. 6 is a diagram for explaining an example of an incorrect work motion; FIG. 7 is a diagram for explaining an example of work time according to an embodiment; FIG. 8 is a diagram showing an example of the hardware configuration of a content output control device according to an embodiment;

[0009] A content output control system, a content output control method, and a program according to an embodiment of the present disclosure will be described in detail below with reference to the drawings. Note that identical or corresponding parts in the drawings are designated by the same reference numerals.

[0010] A content output control system according to an embodiment of the present disclosure is a system for training workers' skills by displaying instructional and training content at a predetermined location on the work site in accordance with the worker's work movements. Specifically, the system compares acquired data indicating the worker's movements with data indicating standard movements to determine which process the worker is currently working on and further determines when the process has ended. Furthermore, when it determines that the process has been completed, the system switches to and projects content corresponding to the next process. This will be described in detail below.

[0011] 1, a content output control system 1 according to this embodiment includes a content output control device 100 that controls the output of content, a content projection terminal 200 that projects content, an imaging device 300 that captures images of work at a workplace, and a sensing device 400 that acquires motion data indicating the hand movements of a worker. The content is for skill training and is also referred to as "skill training content."

[0012] The content output control device 100, the content projection terminal 200, the image capturing device 300, and the sensing device 400 are communicably connected via a network NW. The content output control device 100 is configured by a computer. The network NW may be wired, wireless, a wide area network (WAN), a local area network (LAN), an intranet, or an extranet.

[0013] The content projection terminal 200 includes, for example, a short-focus projector that projects content. The content projection terminal 200 also has, for example, a wireless communication function. The content projection terminal 200 is installed, for example, on the ceiling of a work site. In accordance with instructions from the content output control device 100, the content projection terminal 200 projects the skill training content SC to such an extent that it overlaps with the real space on the workbench WT where the worker HM is working, as shown in FIG. 2 . The worker HM refers to a person taking a skill training course, a person trained in a skill, a learner, or the like. The content projection terminal 200 is an example of a display means according to the present disclosure.

[0014] The image capturing device 300 is an image capturing device, such as a video camera, that captures images of a work site including the worker HM and the work object, tools, measuring instruments, etc. that are the subject of the work. In this embodiment, the image capturing device 300 is fixed to and integrated with the content projection terminal 200 and installed on the ceiling of the work site. The image capturing device 300 captures images continuously while the worker HM is receiving skill training. The image capturing device 300 is an example of an image capturing means according to the present disclosure.

[0015] The sensing device 400 includes a hand tracking sensor that detects the movement of the worker HM's hand. Coordinates are set in the workspace of the worker HM. For example, an orthogonal coordinate system with X, Y, and Z axes is set with one of the four corners of the workbench WT shown in FIG. 2 as the origin. The sensing device 400 has a function of acquiring data on the coordinate position of the worker HM's hand. The sensing device 400 is installed, for example, on the top of a pillar of a work platform using an erector pipe.

[0016] Next, the configuration of the content output control device 100 will be described with reference to Fig. 3. The content output control device 100 includes a motion information acquisition unit 101 that acquires motion information representing the motion of the worker HM, a motion analysis unit 102 that determines which of the steps the worker HM is responsible for the current motion and generates motion analysis information, a motion evaluation unit 103 that identifies which of the steps the worker HM is responsible for most needs improvement and generates motion evaluation information, a display content generation unit 104 that generates content representing desirable motion, a content output control unit 105 that controls the output of content to be projected by the content projection terminal 200, and a storage unit 120 that stores the acquired motion information, programs, etc.

[0017] The storage unit 120 includes a motion information storage unit 121 that stores the motion information acquired by the motion information acquisition unit 101, a motion analysis information storage unit 122 that stores the motion analysis information generated by the motion analysis unit 102, a motion evaluation information storage unit 123 that stores the motion evaluation information generated by the motion evaluation unit 103, a content information storage unit 124 that stores information on skill training content, and a master information storage unit 125 that stores standard work motions including hand movements as master information. The storage unit 120 also stores programs executed by the motion information acquisition unit 101, the motion analysis unit 102, the motion evaluation unit 103, the display content generation unit 104, the content output control unit 105, etc.

[0018] The motion information storage unit 121 stores motion information acquired by the motion information acquisition unit 101 from the imaging device 300 and the sensing device 400. The motion analysis information storage unit 122 stores motion analysis information generated by the motion analysis unit 102. The motion evaluation information storage unit 123 stores motion evaluation information generated by the motion evaluation unit 103. The content information storage unit 124 stores skill training content SC of predetermined standard work motions. The content information storage unit 124 also stores skill training content SC of optimal motions generated by the display content generation unit 104. Details of the skill training content SC of optimal motions will be described later. The master information storage unit 125 stores standard work motions including hand movements that have been accumulated in the past as master information. The master information is predetermined and saved in the master information storage unit 125.

[0019] The motion information acquisition unit 101 acquires motion information from the imaging device 300 and the sensing device 400 via the network NW. The motion information acquisition unit 101 stores the acquired motion information in the motion information storage unit 121. The motion information acquisition unit 101 acquires a work video as motion information from the imaging device 300, and acquires XYZ coordinate values ​​of the hand position of the worker HM from the sensing device 400. The motion information acquisition unit 101 is an example of a motion information acquisition means according to the present disclosure.

[0020] The motion analysis unit 102 compares the standard motions stored in the past with the real-time motions to determine which step the current motion is working in. When displaying the skill training content for the next step, the motion analysis unit 102 instructs the content output control unit 105 to switch the content.

[0021] An example in which the work of a worker HM is behind the standard work time will be described with reference to Figure 4. In the example of Figure 4, as shown in the first row, steps 1, 2, 3, 4, and 5 are repeated as standard steps. The standard motion in the second row represents a certain point in a predetermined work video showing the movement of the worker HM in each step. The standard work time in the third row represents the predetermined standard work time for each step. The real-time work time in the fourth row indicates the work time of the worker HM during work and the point of the current action. The content in the fifth row shows skill training content for each step that is displayed in accordance with the work of the worker HM.

[0022] The motion analysis unit 102 compares the motion data stored in the motion information storage unit 121, specifically the data on the coordinate positions of the worker HM's hands, with the data indicating hand movements for each process stored as master information in the master information storage unit 125. The motion analysis unit 102 then identifies the end point of the work for process 1. The example in FIG. 4 shows that the real-time work time for process 1 ended later than the standard work time for process 1. When the work for process 1 shifts to the work for process 2, the motion analysis unit 102 identifies the end point of process 1 and sends an instruction to the content output control unit 105 to switch from the skill training content SC1 for process 1 to the skill training content SC2 for process 2. The motion analysis unit 102 calculates the difference between the start and end points of each process for each worker as the work time for each process, and stores this in the motion analysis information storage unit 122. Furthermore, the motion analysis unit 102 accumulates and stores the motion history of each cycle, for example, as shown in FIG. 5 , in the motion analysis information storage unit 122. Here, the motion history corresponds to the work time of each worker for each process. In the example of FIG. 5 , the work time for one cycle is the total work time for processes 1 to 5. The motion analysis unit 102 accumulates and stores, for each worker, the work time for each process and the total work time for one cycle. In the example of FIG. 5 , the work time for each process and the combined work time for one cycle are stored vertically for multiple cycles.

[0023] Furthermore, when the worker HM performs an incorrect action, the motion analysis unit 102 sends an alert signal to the content output control unit 105. When the worker HM performs a hand motion that differs from the hand motion registered as master information, the motion analysis unit 102 determines that the work is non-standard and incorrect. For example, this can occur when the standard is to reach into parts box A and then parts box B to retrieve a part, but the order is different, or when parts box A is not used and the worker does not reach into it. Note that in the master information, process 1 is registered as the period from reaching parts box A to assembling the product, process 2 is registered as part of parts box B, and so on. The motion analysis unit 102 determines which process the real-time movement corresponds to based on which action in the master information matches.

[0024] As illustrated in FIG. 6 , when the motion analysis unit 102 determines that the worker HM is performing step 4 instead of step 3 after step 2 is completed, it determines that the worker is performing an incorrect task at that timing. The motion analysis unit 102 then sends an alert signal to the content output control unit 105. The motion analysis unit 102 compares the motion with registered master information to determine which step the motion most closely resembles. Therefore, as shown in FIG. 6 , the motion analysis unit 102 determines that the incorrect task is being performed during or at the end of step 4, rather than at the start of the next step after step 2. The motion analysis unit 102 is an example of a motion analysis means according to the present disclosure.

[0025] The action evaluation unit 103 uses the action analysis information stored in the action analysis information storage unit 122 to generate a box plot of the work time for each process for each worker, as illustrated in FIG. 7 . The action evaluation unit 103 also identifies problematic processes in the work of the worker HM. The problematic processes correspond to, for example, the processes most in need of improvement among the processes performed by the worker HM. Details will be described later. The action evaluation unit 103 is an example of action evaluation means according to the present disclosure.

[0026] The display content generation unit 104 creates skill training content SC of optimal movements. The display content generation unit 104 creates skill training content SC of optimal movements by connecting work videos of workers HM selected as optimal movements for each process by a team leader or the like on site. Note that the team leader or the like on site selects, as optimal movements, the work movements of workers HM that have the shortest median work time and small variation in work time in each process. Note that the display content generation unit 104 is an example of a display content generation means according to the present disclosure.

[0027] When the content output control unit 105 receives an instruction from the motion analysis unit 102 to switch to content for the next process, the content output control unit 105 executes control to have the content for the next process projected on the content projection terminal 200. Furthermore, when an alert signal is sent from the motion analysis unit 102, the content output control unit 105 determines content for displaying an alert warning and projects it on the content projection terminal 200. The alert warning includes, for example, a warning displayed in text, such as "The work content is incorrect!", "The work process is incorrect!", or "The order of the processes is incorrect!" The content representing the alert warning is stored in the content information storage unit 124. Furthermore, the content output control unit 105 is an example of a content output control means according to the present disclosure.

[0028] Next, an example of the hardware configuration of the content output control device 100 having the above configuration will be described with reference to Fig. 8. The content output control device 100 is realized by a computer such as a personal computer or a microcontroller.

[0029] The content output control device 100 includes a processor 1001 that executes an operating program, a memory 1002 that serves as the main storage area of ​​the processor 1001, an interface 1003 that has a communication function, and a secondary storage device 1004 that stores the operating program for executing processing. The processor 1001, memory 1002, interface 1003, and secondary storage device 1004 are connected to one another via a bus 1000.

[0030] The processor 1001 is, for example, a CPU (Central Processing Unit). The processor 1001 loads an operation program stored in the secondary storage device 1004 into the memory 1002 and executes it, thereby realizing the functions of the content output control device 100, specifically, the functions of the motion information acquisition unit 101, the motion analysis unit 102, the motion evaluation unit 103, the display content generation unit 104, and the content output control unit 105.

[0031] The memory 1002 is a main storage device configured, for example, by RAM (Random Access Memory). The memory 1002 stores the operating program that the processor 1001 reads from the secondary storage device 1004. The memory 1002 also functions as a work memory when the processor 1001 executes the operating program.

[0032] The interface 1003 is an I / O (Input / Output) interface such as a serial port, a USB (Universal Serial Bus) port, a network interface, etc. The interface 1003 realizes the communication function of the content output control device 100.

[0033] The secondary storage device 1004 is, for example, a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The secondary storage device 1004 stores the operation program executed by the processor 1001, the operation information of the worker HM, operation analysis information, operation evaluation information, content information, and master information.

[0034] Next, the operation of the content output control system 1 will be described with reference to Figures 9 to 14. The operation of the content output control system 1 can be broadly divided into three stages. The first stage is a stage in which work information for each process of a skill training recipient is collected and analyzed, and the information is broken down by process. The second stage is a preparatory stage in which the collected work information is processed to determine the work time for each worker and process, and problematic processes to be improved are identified, and further, optimal movement skill training content SC is created. The third stage is a stage in which the generated movement information and work movements are used to support the work of each skill training recipient.

[0035] Next, with reference to FIG. 9 , the motion analysis process for analyzing the work motion of the worker HM, which corresponds to the first stage, will be described. When the worker HM, for example, reads a barcode on an assembly work instruction sheet using a barcode reader connected to the content output control device 100 via wireless communication and starts work, the motion analysis process shown in FIG. 9 is executed. The barcode reader sends information about the work object contained in the barcode to the content output control device 100. Here, the information about the work object includes, for example, information about which assembly workplace code, which product code, and how many units are to be made. The worker HM also inputs the worker's ID. The content output control device 100 causes the imaging device 300 to start imaging the work site of the worker HM and causes the sensing device 400 to start acquiring data on the coordinate positions of the worker HM's hands.

[0036] First, the motion information acquisition unit 101 of the content output control device 100 acquires motion information of the worker HM from the imaging device 300 and the sensing device 400 (step S11). Specifically, the motion information acquisition unit 101 acquires work video data of the worker HM from the imaging device 300 as motion information. The motion information acquisition unit 101 also acquires hand motion data of the worker HM from the sensing device 400 as motion information. The hand motion data refers to data indicating hand motion. The hand motion data includes data on position coordinates of the right and left hands in the X, Y, and Z coordinate space and time data. The motion information acquisition unit 101 stores the acquired motion information in the motion information storage unit 121. The hand motion data is an example of data indicating the motion of a specific body part according to the present disclosure.

[0037] The motion analysis unit 102 analyzes the motion of the worker HM according to the motion information acquired by the motion information acquisition unit 101 (step S12). The motion analysis unit 102 identifies the start and end points of each process by comparing the acquired hand motion data of the worker HM with master data stored as master information in the master information storage unit 125, which represents the trajectory of changes in the X, Y, and Z coordinate positions of the hand. For example, the motion analysis unit 102 may identify the start of a task when the worker HM places his / her hand in a parts box containing a screw, and the end of the task and the start of the next process when the worker places his / her hand in another parts box after tightening the screw. The motion analysis unit 102 stores the results of the work motion analysis in the motion analysis information storage unit 122 (step S13). For example, as shown in FIG. 5, the motion analysis unit 102 collects motion records indicating the work time for each cycle for each worker, and accumulates and stores them in the motion analysis information storage unit 122. The motion analysis process shown in FIG. 9 is repeatedly executed while the worker HM is performing the task.

[0038] Next, with reference to FIG. 10 , the preparatory process corresponding to the second stage will be described. The preparatory process includes a process for creating box-and-whisker plot data that visualizes task times, a process for identifying problematic tasks to be improved, and a process for creating optimal task content. The box-and-whisker plot data is created based on the performance data indicating task times, as shown in FIG. 5 , previously stored in the task analysis information storage unit 122. The problematic task identification process refers to a process for identifying the task that most needs to be improved for each worker. The optimal task content creation process refers to a process for creating optimal task skill training content SC by combining task videos of workers HM who performed optimal tasks, selected for each task. When the content output control device 100 is activated, the preparatory process shown in FIG. 10 begins. The optimal task skill training content SC is an example of instructional task content according to the present disclosure.

[0039] The action evaluation unit 103 creates box-and-whisker plot data of the work time (step S21). As illustrated in FIG. 7, the median work time and the variation in work time for each process are visualized for each worker. From the box-and-whisker plot, on-site team leaders can easily identify which processes require no work time, which processes require work time, and the degree of variation in work time for each worker. Team leaders and other on-site team leaders refer to the box-and-whisker plot illustrated in FIG. 7 to select the optimal work action for each process. While FIG. 7 shows a box-and-whisker plot visualizing the work time of some workers HM, the action evaluation unit 103 creates a box-and-whisker plot visualizing the work time of all workers in the processing of step S21.

[0040] Next, a problematic process identification process is executed (step S22). In this embodiment, five workers, namely, worker A, worker B, worker C, worker D, and worker E, perform all five processes, namely, process 1, process 2, process 3, process 4, and process 5, respectively. First, as shown in FIG. 11, the action evaluation unit 103 identifies one process (step S22a). The action evaluation unit 103 identifies, for example, process 1.

[0041] The action evaluation unit 103 identifies one worker (step S22b). The action evaluation unit 103 identifies, for example, worker A. The action evaluation unit 103 identifies the median of the work time for each worker and each task (step S22c). In the example of FIG. 7 , the action evaluation unit 103 identifies the median of 39 seconds for step 1 for worker A. Note that the median of 39 seconds for step 1 for worker A is an example of the worker median according to the present disclosure.

[0042] The action evaluation unit 103 determines whether or not there are any unprocessed workers remaining (step S22d). Since there are other workers B to E (step S22d: Yes), the process returns to step S22b, and the same process as for worker A is performed on the other workers B to E.

[0043] If it is determined that there are no other unprocessed workers (step S22d: No), the action evaluation unit 103 identifies the smallest median among all workers (step S22e). For example, for process 1, the action evaluation unit 103 identifies the smallest median among workers A to E. The action evaluation unit 103 associates the identified smallest median with the process identification number and stores it in the action evaluation information storage unit 123. Note that information identified by the action evaluation unit 103 is stored in the action evaluation information storage unit 123 each time.

[0044] The action evaluation unit 103 calculates the difference between the median of each worker and the minimum median (step S22f). The action evaluation unit 103 calculates the difference between the median of each worker A to E and the minimum median for process 1. The action evaluation unit 103 stores the calculated difference value in the action evaluation information storage unit 123 in association with the process identification number and the worker identification information.

[0045] The action evaluation unit 103 determines whether there are any other unprocessed steps (step S22g). In this example, since the action evaluation unit 103 determines that there are steps 2 to 5 in addition to step 1 (step S22g: Yes), the process returns to step S22a. The action evaluation unit 103 repeats the same process for steps 2 to 5 as for step 1.

[0046] If it is determined that there are no other processes (step S22g: No), the action evaluation unit 103 identifies one new worker (step S22h). The action evaluation unit 103 identifies worker E, for example.

[0047] The action evaluation unit 103 identifies the process with the largest absolute value of the calculated difference as the problematic process (step S22i). The action evaluation unit 103 identifies the process with the largest absolute value among the absolute values ​​of the differences between processes 1 to 5 for worker E as the problematic process. The process with the largest absolute value among the absolute values ​​of the differences between processes 1 to 5 has the greatest room for improvement in worker E's work, and is therefore suitable as the problematic process. The action evaluation unit 103 stores the identified problematic process in the action evaluation information storage unit 123 in association with the worker's identification information.

[0048] The action evaluation unit 103 determines whether any other unprocessed workers remain (step S22j). Since the action evaluation unit 103 determines that there are workers A to D in addition to worker E (step S22j: Yes), the process returns to step S22h. The same process as for worker E is repeated for workers A to D.

[0049] If it is determined that there are no unprocessed workers remaining (step S22j: No), the process returns to the process shown in FIG. 10, and the optimum action content creation process is executed (step S23).

[0050] 12 starts, the display content generation unit 104 determines whether an optimal action for a process has been selected (step S23a). For example, a team leader or the like on-site selects, with reference to a box-and-whisker plot, the action of a worker with a small median work time and small variation in work time for each process as the optimal action for the process. The optimal action for each process is selected via an operation input to the content output control device 100 by the team leader or the like on-site. If it is determined that an optimal action for the process has not been selected (step S23a: No), the display content generation unit 104 repeats the processing of step S23a.

[0051] If it is determined that the optimum action for the process has been selected (step S23a: Yes), the display content generation unit 104 determines whether there are any other processes for which the optimum action has not been selected (step S23b).If it is determined that there are any other processes for which the optimum action has not been selected (step S23b: Yes), the process returns to step S23a.

[0052] If it is determined that there are no other steps for which the optimal action has not been selected (step S23b: No), the display content generation unit 104 creates content for the optimal action (step S23c). The display content generation unit 104 creates the optimal action skill training content SC by sequentially stitching together the representative video representing the work video of the worker selected for the optimal action for each of steps 1 to 5, with the representative video of step 1, the representative video of step 2, the representative video of step 3, the representative video of step 4, and the representative video of step 5. The display content generation unit 104 creates the optimal action skill training content SC by using, for example, an app that processes the worker's action video captured by the image capture device 300 into an illustration-like image. The display content generation unit 104 stores the created optimal action skill training content SC in the content information storage unit 124 together with identification information representing the optimal action skill training content SC. The optimal action content creation process is then completed, and the advance preparation process is completed as shown in FIG. 10 .

[0053] Next, the content output control process will be described with reference to Fig. 13. The content output control process is a process in which training content is switched for each process in accordance with the work of the worker HM and projected on the content projection terminal 200. As with the motion analysis process shown in Fig. 9, when the worker HM reads, for example, a barcode on an assembly work instruction sheet and a barcode on the worker's ID card using a barcode reader connected to the content output control device 100 by wireless communication, the content output control process shown in Fig. 13 is executed.

[0054] The content output control unit 105 determines whether the next process is a problematic process (step S31). Specifically, the content output control unit 105 first determines whether process 1 corresponds to the problematic process for the worker HM stored in the action evaluation information storage unit 123. If it is determined that process 1 is not a problematic process (step S31: No), the content output control unit 105 causes the content projection terminal 200 to project the skill training content SC for the next process (step S34). The content output control unit 105 causes the content projection terminal 200 to project standard skill training content SC for training the next process, which is stored in advance in the content information storage unit 124, so that it overlaps with the real space. The worker HM performs the work of process 1 while referring to the projected skill training content SC for the next process. Note that FIG. 13 will be described assuming that the problematic process for the worker HM is process 5.

[0055] The projection image will be explained with reference to FIG. 2 , which shows a bird's-eye view of the work site. The content output control unit 105 edits the motion data to improve visibility, and controls the content projection terminal 200 to project the skill training content SC so that it overlaps with the real space on the workbench WT. If the video data were projected as is, it would be difficult to distinguish the actual product from the projected image. Therefore, the display content generation unit 104 processes the skill training content SC, increasing the contrast ratio and creating an illustration-like effect. This prevents the actual product from becoming difficult to distinguish from the projected image. The content output control unit 105 plays the skill training content SC in slow motion for new workers, and once the new workers understand the procedures, projects it in real time for each process at the standard working time. If the projection were to continue without stopping at each process, a difference would arise between the actual worker's speed and the projected image, so the content output control unit 105 automatically tracks the projected image. For example, if the working time for the video content of step 1 is 10 seconds and the repetitive working time of the new employee is 20 seconds, the content output control unit 105 projects the content at a playback speed of 0.5 times the normal speed.

[0056] The motion analysis unit 102 determines whether or not there is an abnormal motion (step S35). In this embodiment, for example, two cases of motion can be cited as abnormal motions. In the first case, a hand movement that differs from the motion registered in the master information is assumed. Specifically, when parts boxes A, B, and C are present, parts from parts boxes A and B are used, so the motion of reaching out to parts boxes A and B to pick up parts is registered as master information. However, when the worker HM reaches out to pick up parts box C, which is not registered, the motion analysis unit 102 determines this to be an abnormal motion. In the second case, a case where the process order is different is assumed. Specifically, in process 1, parts are picked up from parts box A and assembled, and in process 2, parts are picked up from parts box B, if the worker HM performs the motion for process 2 before the motion for process 1, the motion analysis unit 102 determines this to be an abnormal motion.

[0057] If it is determined that an abnormal operation has occurred (step S35: Yes), the content output control unit 105 issues a warning of the incorrect operation (step S37). Specifically, first, when the operation analysis unit 102 determines that an abnormal operation has occurred, it sends an alert signal to the content output control unit 105. As described above, the content output control unit 105 determines content for which an alert warning is to be displayed and causes the content projection terminal 200 to project the content. The alert warning includes, for example, a warning displayed in text, such as "The work content is incorrect!" or "The work process is incorrect!" The content for which an alert warning is to be displayed is stored in the content information storage unit 124. After the processing of step S37, the content output control process ends.

[0058] If it is determined that there is no abnormal motion (step S35: No), the motion analysis unit 102 determines whether or not the completion motion of the current process has been detected (step S36). For example, if the master information indicates that process 1 is a hand motion indicating the motion of attaching part A to the right side of the product, the motion analysis unit 102 detects the end of the process when the hand that placed part A in the box moves to the right side of the product and attaches it. Note that coordinate position data indicating the hand motion in process 1 is stored in the master information storage unit 125 as master information. If it is determined that the completion motion of the current process has not been detected (step S36: No), the process returns to step S35.

[0059] When it is determined that the completion motion of the current process has been detected (step S36: Yes), the motion analysis unit 102 determines whether or not there is a next process (step S38). When it is determined that there is a next process (step S38: Yes), the process returns to step S31. Since there are processes 2 to 5 in addition to process 1, the process returns to step S31. The same process as for process 1 is repeated for processes 2 to 4. Next, the case where the next process is process 5, which is the problematic process, will be described.

[0060] If the next process, process 5, is the problem process (step S31: Yes), the content output control unit 105 causes the content projection terminal 200 to project a box-and-whisker plot representing the work time of the worker HM for process 5 and the median work time of the optimal operation (step S32). As shown in FIG. 14, for example, in the case of worker A, the content output control unit 105 causes the content projection terminal 200 to project a box-and-whisker plot for process 5 and a dashed-dotted line Li indicating the median work time of the optimal operation. This allows the worker HM to check the wasted work time in the next process. Note that the box-and-whisker plot and line Li indicating the median work time of the optimal operation shown in FIG. 14 are merely examples.

[0061] The content output control unit 105 projects the skill training content SC for the optimal movement of step 5, which corresponds to the next step (step S33). The content output control unit 105 projects the skill training content SC for the optimal movement of the next step, which was created by the display content generation unit 104 and stored in the content information storage unit 124, onto the content projection terminal 200 so that it overlaps with the real space on the workbench WT, as shown in Fig. 2. Thereafter, in steps S35 and S36, the same processing as in step 1 is executed.

[0062] If it is determined that there is no next process (step S38: No), the operation analysis unit 102 determines whether the specified number of devices have been worked on (step S39). If it is determined that the specified number of devices have not been worked on (step S39: No), the process returns to step S31. If it is determined that the specified number of devices have been worked on (step S39: Yes), the content output control process ends.

[0063] As described above, the content output control system 1 according to the embodiment determines whether a task is complete based on the acquired motion data indicating hand movements. If it is determined that the task is complete, the system switches to the skill training content SC corresponding to the next task and projects it at a predetermined position on the work site. This helps improve the efficiency of the work.

[0064] Furthermore, the content output control system 1 according to the embodiment creates skill training content SC for optimal operations by stitching together videos of workers who performed optimal work, selected by a team leader or other such person on-site, for each process. Furthermore, the content output control system 1 according to the embodiment identifies the process for each worker that needs the most improvement as the problem process, and if the next process is the problem process, projects the skill training content SC for optimal operations. This allows the worker HM to perform the work in the problem process while referring to the optimal work movements, thereby further supporting the improvement of work efficiency.

[0065] In the above embodiment, a process and its divisions are identified based on the coordinate values ​​of the hand position and their history, but depending on the work, a process and its divisions may be identified based on body parts other than the worker HM's hands, for example, the position and orientation of the head and their history, or the coordinate values ​​of the foot position and their history. A process and its divisions may also be identified based on the coordinate values ​​of multiple parts of the worker HM's body and their history.

[0066] Furthermore, the end, completion, division, etc. of a process are not strictly defined, and variations are permitted to the extent that the worker HM can recognize them as divisions of the process and do not feel uncomfortable.

[0067] In the above embodiment, processes and their boundaries are identified based on the coordinate values ​​of the hand position of the worker HM acquired by the sensing device 400 and the history of the coordinate values, but processes and their boundaries can also be identified based on video captured by the image capture device 300. For example, it is also possible to identify processes and their boundaries during processing by pattern matching between the video of the worker HM in the video captured by the image capture device 300 and a video of a preset standard task.

[0068] Furthermore, the reliability may be increased by combining the coordinate values ​​of the body parts of the worker HM, the process and its divisions identified from the history, and the process and its divisions identified from the video.

[0069] In the above embodiment, an example was shown in which a box plot was created in step S21, but it is not necessary to create the plot, and the required statistical processing may be performed and presented.

[0070] In the above embodiment, the on-site team leader, supervisor, etc. identified the optimal action from the video footage they filmed. This disclosure is not limited to this, and the display content generation unit 104 of the content output control device 100 may perform this processing. For example, based on the evaluation by the action evaluation unit 103, an action that meets preset criteria may be selected as the optimal action from among the processes of multiple workers whose footage has been accumulated. Examples of the criteria include the work time being in the top p% and the inspection pass rate separately provided from the inspection process being q% or higher.

[0071] In the embodiment, the various information in the storage unit 120 is provided in the content output control device 100, but it may be provided outside the content output control device 100. For example, the various information in the storage unit 120 may be provided in a cloud-based server outside the content output control device 100.

[0072] In the embodiment, the action evaluation unit 103 calculates the difference between the median work time of each worker and the minimum median work time for each worker, and identifies the work process with the largest absolute value of the calculated difference as the problematic work process for each worker. However, the present disclosure is not limited to this. For example, the action evaluation unit 103 may also identify the work process with the largest rectangular area in the box-and-whisker plot, in other words, the work process with the largest variation in work time, as the problematic work process. For example, in the case of Figure 7, for both workers A and B, the action evaluation unit 103 may identify work process 5, which has the largest variation in work time, as the problematic work process.

[0073] In the embodiment, the content output control process is executed when the worker HM reads the barcode in the assembly work instruction sheet and the barcode on the worker HM's ID card with a barcode reader wirelessly connected to the content output control device 100, but the present disclosure is not limited to this. The content output control process may also be executed by the worker HM operating a terminal provided at the work site to input the code number in the assembly work instruction sheet and his or her own ID number.

[0074] In the embodiment, the image capturing device 300 is fixed to and integrated with the content projection terminal 200 and installed on the ceiling of the work site, but the present disclosure is not limited to this. As long as the functions and effects of the embodiment are achieved, the image capturing device 300 may be fixed to the top of a pillar of an erector work cart with a jig, for example.

[0075] In the embodiment, the content projection terminal 200 is used as an example of a display device, but the display device is not limited to a projection device and may be a general display device such as a liquid crystal display device. However, a projection device is preferable because it can display an image superimposed on the workspace.

[0076] Although the preferred embodiments have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the claims.

[0077] It should be noted that various embodiments and modifications of the present disclosure are possible without departing from the broad spirit and scope of the present disclosure. Furthermore, the above-described embodiments are intended to explain the present disclosure and do not limit the scope of the present disclosure. That is, the scope of the present disclosure is defined by the claims, not the embodiments. Various modifications made within the scope of the claims and within the meaning of equivalent disclosures are considered to be within the scope of the present disclosure.

[0078] This application is based on Japanese Patent Application No. 2024-097171, filed on June 17, 2024. The entire specification, claims, and drawings of Japanese Patent Application No. 2024-097171 are incorporated herein by reference.

[0079] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) A content output control system comprising: motion information acquisition means for acquiring motion information representing the motion of a worker; motion analysis means for determining whether a task in a process performed by the worker has been completed according to the motion information; display means for displaying predetermined content for skill training of the worker at a predetermined position at the worker's work site; and content output control means for controlling the display means to display content corresponding to a next process in response to the motion analysis means determining that the task in the process has been completed. (Appendix 2) The content output control system described in Appendix 1, wherein the motion information includes data indicating the movement of at least a specific part of the worker's body for each process, and the motion analysis means determines the completion of the task based on the motion information and a predetermined criterion. (Appendix 3) The content output control system described in Appendix 1 or 2, wherein the content output control means controls the display means to display instructional motion content indicating a desirable motion in response to determining that the task has been completed if the next process corresponds to a predetermined problematic process to be improved. (Supplementary Note 4) The content output control system according to Supplementary Note 3, further comprising: a filming means that films the work of the worker and generates a work video, and a display content generation means that generates the instructional action content for each process from the work video generated by the filming means. (Supplementary Note 5) The content output control system according to Supplementary Note 3, further comprising: a motion analysis means that calculates the work time of the worker for each process, identifies for each process a worker median that represents the median of the work time of each worker, identifies for each process a minimum median that represents the smallest value of the identified worker medians across all workers, calculates the difference between the identified worker median and the minimum median from the work times for each process of all workers, and identifies for each worker the process with the largest absolute value of the calculated difference as the problematic process.(Supplementary Note 6) The content output control system according to Supplementary Note 5, wherein the action evaluation means generates a box-and-whisker plot for each worker and each process according to the task time calculated by the action analysis means, and the content output control means, when the next process corresponds to the problematic process, displays a diagram showing the box-and-whisker plot for the problematic process and the minimum median for the problematic process. (Supplementary Note 7) The content output control system according to Supplementary Note 3, wherein the action analysis means calculates the task time for each process of the worker, and further comprises the action evaluation means for identifying the process with the largest variation in the task time calculated by the action analysis means as the problematic process. (Supplementary Note 8) A content output control method comprising: acquiring action information representing the movements of a worker; determining whether the task of the process performed by the worker has been completed according to the action information; displaying predetermined content at a predetermined position; and controlling to display content corresponding to the next process according to the determination of the completion of the task. (Supplementary Note 9) A program that causes a computer to execute the following process: acquire motion information representing the movements of a worker; determine whether or not the work of a process performed by the worker has been completed according to the motion information; display predetermined content at a predetermined position; and control the computer to display content corresponding to the next process according to the determination of the completion of the work.

[0080] 1 Content output control system, 100 Content output control device, 101 Motion information acquisition unit, 102 Motion analysis unit, 103 Motion evaluation unit, 104 Display content generation unit, 105 Content output control unit, 120 Memory unit, 121 Motion information storage unit, 122 Motion analysis information storage unit, 123 Motion evaluation information storage unit, 124 Content information storage unit, 125 Master information storage unit, 200 Content projection terminal, 300 Imaging device, 400 Sensing device, 1000 Bus, 1001 Processor, 1002 Memory, 1003 Interface, 1004 Secondary storage device, SC Skill training content, WT Workbench, HM Worker, Li Line.

Claims

1. A content output control system comprising: a motion information acquisition means for acquiring motion information representing the movements of a worker; a motion analysis means for determining whether the work of a process performed by the worker has been completed according to the motion information; a display means for displaying predetermined content for skill training of the worker at a predetermined position at the worker's work site; and a content output control means for controlling the display means to display content corresponding to the next process in response to the motion analysis means determining that the work of the process has been completed.

2. The content output control system of claim 1, wherein the motion information includes data indicating the movement of at least a specific part of the worker's body for each process, and the motion analysis means determines the completion of the work based on the motion information and a predetermined standard.

3. A content output control system as described in claim 1 or 2, wherein the content output control means controls the display means to display instructional action content showing desirable actions in response to a determination that the work is complete when the next process corresponds to a problematic process that is a preset target for improvement.

4. The content output control system according to claim 3, further comprising: a filming means for filming the work of the worker and generating a work video; and a display content generation means for generating the instructional action content for each process from the work video generated by the filming means.

5. The content output control system of claim 3, further comprising an action evaluation means for: determining the working time of each worker for each process; identifying, for each process, a worker median representing the median of the working time of each worker; determining, for each process, a minimum median representing the smallest value of the identified worker medians across all workers; determining the difference between the identified worker median and the minimum median from the working times of all workers for each process; and identifying, for each worker, the process with the largest absolute value of the determined difference as the problematic process.

6. The content output control system according to claim 5, wherein the action evaluation means generates box-and-whisker plots for each worker and process according to the working time calculated by the action analysis means, and the content output control means, when the next process corresponds to the problem process, displays a diagram showing the box-and-whisker plot for the problem process and the minimum median for the problem process.

7. The content output control system according to claim 3, further comprising an action evaluation means for determining the working time of the worker for each process and identifying the process with the greatest variation in the working time for each process determined by the action analysis means as the problematic process.

8. A content output control method, comprising: acquiring motion information representing the movements of a worker; determining whether or not a task in a process performed by the worker has been completed according to the motion information; displaying predetermined content at a predetermined position; and controlling the display of content corresponding to the next process according to the determination of the completion of the task.

9. A program that causes a computer to execute the following process: acquire motion information representing the movements of a worker; determine whether the work of a process performed by the worker has been completed according to the motion information; display predetermined content at a predetermined position; and control the computer to display content corresponding to the next process according to the determination of the completion of the work.

Images