Training support system, training support method, and program

The training support system addresses the issue of posture evaluation in VR and AR training by generating and comparing three-dimensional posture models, improving work accuracy, quality, and safety through posture evaluation and feedback.

JP2025004402AInactive Publication Date: 2025-01-15NS SOLUTIONS CORPORATION
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023104069
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-06-26
Publication Date
2025-01-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Conventional VR and AR training systems fail to capture and evaluate the worker's posture during work, leading to discrepancies in work accuracy, finish quality, efficiency, and safety between skilled and unskilled workers due to differences in tool handling and posture.

Method used

A training support system that includes a display section, operation section, posture measurement, and comparison means to generate and compare three-dimensional posture models using machine learning, enabling evaluation of appropriate work posture.

Benefits of technology

Enables evaluation of appropriate work posture, improving skill acquisition by comparing and providing feedback on tool operation and posture, enhancing work accuracy, finish quality, and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2025004402000001_ABST
    Figure 2025004402000001_ABST
Patent Text Reader

Abstract

To provide a training support system which enables evaluation of work including appropriate postures during the work.SOLUTION: A training support system is provided, comprising: work training means having a display unit and an operating unit; measurement means for measuring postures of a user as three-dimensional data; learning means configured to machine-learn the posture data measured by the measurement means during actual work done by a first user to build a first working posture model; and comparison means configured to compare the first working posture model built by the learning means with the posture data measured by the measurement means during virtual work done by a second user using the work training means and output a comparison result.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a training support system, a training support method, and a program for supporting work training. [Background technology]

[0002] Simulation methods using VR (Virtual Reality) and AR (Augmented Reality) are becoming more common in training to master skills. Patent Document 1 discloses a technique in which a user undergoing training wears a helmet equipped with a small display device and manipulates a simulated welding object (welding coupon) and mock welding tool, and a simulated welding process is displayed on the display device in a virtual reality space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-223966 A Summary of the Invention [Problem to be solved by the invention]

[0004] By the way, it is known that in actual work, the accuracy of work, the beauty of the finished product, the efficiency of work, or the safety of work greatly differs between skilled and unskilled workers depending on the posture during work. In work using tools, the way the tool is held and the posture when holding the tool also affect the accuracy and safety of the work. However, in conventional training using VR or AR, it was not possible to capture the posture of the worker holding the tool and evaluate the work results, for example, the finished product. In particular, the shape and weight of the actual tool and the pseudo tool used in training with VR or AR are often different, and once a person becomes accustomed to operating the tool in VR or AR, the operation may not be reproduced with the actual tool. The present invention aims to provide a training support system that enables evaluation including the appropriate posture during work. [Means for solving the problem]

[0005] The training support system of the present invention is characterized by having a work training means having a display unit and an operation unit, a measurement means for measuring a user's posture as three-dimensional data, a learning means for applying machine learning to the posture data measured by the measurement means during actual work by a first user to generate a first working posture model, and a comparison means for comparing the first working posture model generated by the learning means with posture data measured by the measurement means during virtual work performed by a second user using the work training means, and outputting the comparison result. Effect of the Invention

[0006] According to the present invention, it is possible to provide a training support system that makes it possible to evaluate even the appropriate posture during work. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a hardware configuration of a training support system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a functional configuration of a training assistance system according to an embodiment of the present invention. [Diagram 3] 10 is a flowchart illustrating an example of a model generation process in the present embodiment. [Figure 4] FIG. 2 is a diagram for explaining generation of each model in the present embodiment. [Diagram 5] 4A to 4C are diagrams illustrating an example of posture data in the present embodiment. [Figure 6] 4A to 4C are diagrams illustrating an example of tool operation data in the present embodiment. [Figure 7] 10 is a flowchart showing an example of a process related to a virtual task in the present embodiment. [Figure 8] FIG. 4 is a diagram illustrating an evaluation process in the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. 1 is a diagram showing an example of a hardware configuration of a training support system 100 according to an embodiment of the present invention. The training support system 100 includes a CPU 101, a main memory device 102, an auxiliary memory device 103, an output device 104, an HMD 105, a pseudo tool 106, a motion sensor 107, a camera 108, and a tool operation detection device 109. The CPU 101, the main memory device 102, the auxiliary memory device 103, the output device 104, the HMD 105, the pseudo tool 106, the motion sensor 107, the camera 108, and the tool operation detection device 109 are connected to each other so as to be able to communicate with each other.

[0009] The CPU (Central Processing Unit) 101 is a central processing unit that controls the training support system 100. The main memory 102 is a memory device that functions as a work area for the CPU 101 and a temporary storage location for data. The main memory 102 is implemented using, for example, a RAM (Random Access Memory) or the like. The auxiliary memory 103 is a memory device that stores various data, various programs, and the like. The auxiliary memory 103 is implemented using, for example, a ROM (Read Only Memory), a hard disk drive (HDD), a solid state drive (SSD), and the like. The CPU 101 executes processing based on a program stored in, for example, the auxiliary memory 103, thereby realizing various processing operations in the training support system 100. The output device 104 is a device that outputs various information such as an evaluation result of a task, and is used to present various information to a user, etc.

[0010] The HMD (Head Mounted Display) 105 is a head-mounted display device that displays images (VR images) of a virtual space. The HMD 105 is an example of a display unit of a work training means. The HMD 105 may be a non-transparent HMD that does not allow the user to see the outside when worn, or a transparent (see-through) HMD that allows the user to see the outside when necessary by projecting the outside even when worn. The HMD 105 may also display a generated image superimposed on the real world using AR technology. Instead of the HMD 105, for example, eyeglass-type smart glasses may be used.

[0011] The pseudo tool 106 is a pseudo tool operated by a worker when performing a virtual task (virtual task) performed in a simulated virtual space. The pseudo tool 106 is an example of an operation unit of a task training means. In the training assistance system 100, the pseudo tool 106 is configured to detect the motion (speed, position, and angle) of the pseudo tool 106 and the operation on the pseudo tool 106 (the amount of operation on the operating member, etc.). For example, the motion of the pseudo tool 106 may be detected based on a marker attached to the pseudo tool 106 and photographed, or an IMU sensor (inertial measurement unit) consisting of an acceleration sensor and a gyro sensor may be provided on the pseudo tool 106 to detect the motion. Similarly, the operation on the pseudo tool 106 may be detected using a marker, a detection sensor, or the like.

[0012] The motion sensor 107 is a sensor for three-dimensionally measuring (motion capturing) the posture (movement) of a worker. The motion sensor 107 is only required to measure the posture, etc., of a worker, and the type of motion capture is not limited. The motion sensor 107 may be, for example, a wearable motion capture suit type motion sensor incorporating an IMU sensor (inertial measurement unit), also called smart apparel, or a wearable suit type motion sensor with a marker that is detected by a tracker. The motion sensor 107 may also be, for example, a photographing type (camera type) that captures an image of a subject and detects the distance, position, etc., to each part of the subject.

[0013] The camera 108 is, for example, a camera that captures an object to be worked on in actual work performed in the real world (on-site environment). The tool operation detection device 109 detects the motion (speed, position, and angle) of an actual tool used in the work and the amount of operation corresponding thereto in the actual work performed in the real world. Note that the tool operation detection device 109 only needs to detect the motion and amount of operation of the tool in the actual work, and there is no limitation on the detection method.

[0014] In a virtual task performed in a virtual space (virtual task), various data during the task are acquired, for example, using a motion sensor 107 and a dummy tool 106. In an actual task performed in the real world (on-site environment), various data during the task are acquired, for example, using a motion sensor 107, a camera 108, and a tool operation detection device 109.

[0015] 2 is a diagram showing an example of the functional configuration of the training support system 100 in this embodiment. The training support system 100 includes an acquisition unit 201, a storage unit 202, a display unit 203, an operation unit 204, an analysis unit 205, an evaluation unit 206, a learning processing unit 207, an image generation unit 208, a display control unit 209, and a control unit 210.

[0016] The acquisition unit 201 acquires various data used for training assistance in the training assistance system 100. For example, the acquisition unit 201 acquires various data related to the movement and operation of an actual tool, the worker's posture, the work target, and the like in actual work performed in the real world (on-site environment). The various data related to the actual work are acquired, for example, from the tool operation detection device 109, the motion sensor 107, the camera 108, and the like. In addition, for example, the acquisition unit 201 acquires various data related to the movement and operation of a pseudo tool and the worker's posture, and the like in virtual work (virtual work) performed in a virtual space. The various data related to the virtual work are acquired, for example, from the pseudo tool 106, the motion sensor 107, and the like.

[0017] The storage unit 202 stores various data acquired by the acquisition unit 201 and various models generated by the learning processing unit 207. The storage unit 202 may also store data such as processing results acquired by the analysis unit 205 and the evaluation unit 206. The display unit 203 is a device that outputs various information to a worker, etc., and displays various information to be presented to the worker, etc. according to the control of the display control unit 209. For example, the display unit 203 displays a result screen showing the evaluation result of the work by the evaluation unit 206, and displays various information when performing virtual work and VR images showing the state of the work. The operation unit 204 is used to perform operations in a virtual space, for example, when performing virtual work in a virtual space.

[0018] The analysis unit 205 analyzes the movement and operation of the tool during work, the posture of the worker, etc., based on various data acquired by the acquisition unit 201. For example, the analysis unit 205 analyzes the movement (speed, position, and angle) of the tool during work, the amount of operation of the tool, etc., based on data acquired from the tool operation detection device 109 during actual work performed in the real world, and data acquired from the pseudo tool 106 during virtual work (virtual work) performed in virtual space. In addition, the analysis unit 205 analyzes the posture (movement) of the worker during work, based on data acquired from the motion sensor 107 during actual work and virtual work.

[0019] The evaluation unit 206 compares the movement and operation amount of the tool, the posture during the work, and the work object between the expert and the trainee in the target work, and evaluates the work. The evaluation unit 206 is an example of a comparison means and a value comparison means. For example, the evaluation unit 206 compares the movement and operation amount of the tool in the work by the trainee with a tool operation model (described later) generated in advance regarding the movement and operation amount of the tool in the work by the expert, to evaluate the movement of the tool of the trainee during the work. The evaluation unit 206 also compares the posture of the trainee during the work based on the three-dimensional measurement result with a work posture model (described later) generated in advance regarding the posture of the expert during the work, to evaluate the posture of the trainee during the work. The evaluation unit 206 also compares the state of the work object simulated based on the movement and operation amount of the tool in the work by the trainee with an object model (described later) generated in advance regarding the work object in the work by the expert, to evaluate the work result of the work by the trainee. In addition, the evaluation unit 206 compares the measurement value of any item measured for the work results performed by the trainee on the work object in the virtual work with an ideal value set in advance for that item, thereby evaluating the work performed by the trainee.

[0020] The learning processing unit 207 receives various data obtained during work by an expert, performs machine learning using the input data as training data, and generates a model corresponding to the data. For example, the learning processing unit 207 generates a tool operation model related to the motion and operation amount of a tool used in the work, a working posture model related to the posture during work, and an object model related to the work target, based on the data obtained during work by the expert.

[0021] The image generating unit 208 generates an image to be displayed in the training support system 100. For example, the image generating unit 208 generates a VR image to be displayed when performing a virtual task (virtual task) and an image showing an evaluation result of the task. For example, the image generating unit 208 generates an image of a task object to be displayed when performing a virtual task (virtual task) based on a virtual image generated by computer graphics or an image of an object actually photographed. The display control unit 209 controls the display on the display unit 203. For example, the display control unit 209 controls the display position, size, etc. of the image generated by the image generating unit 208 on the display unit 203. The control unit 210 controls each functional unit of the training support system 100.

[0022] The training support system 100 in this embodiment evaluates the movement and operation of tools in the work, the posture during the work, the work result (work object), etc., based on data acquired when a worker performs a virtual work (virtual work) in a simulated virtual space. The operation of the training support system 100 related to this evaluation will be described below. An example of evaluating the work of a trainee by comparing the work of a skilled person with the work of a trainee will be described below. In addition, as a work example, an example of airbrush painting in which an object is painted with a specified color will be shown as appropriate.

[0023] The training support system 100 generates various models related to work by an expert, which are used to evaluate virtual work (virtual work) performed in a virtual space. The training support system 100 generates a tool operation model related to the motion and operation amount of a tool used in the work, a work posture model related to the posture during work, and an object model related to the work object, based on various data acquired during the work by the expert. Note that the models to be generated are not limited to these, and other models may also be generated.

[0024] 3 is a flowchart showing an example of the model generation process in this embodiment. In the model generation process, in step S301, the acquisition unit 201 of the training support system 100 acquires various work data when a skilled worker performs work. When generating a work posture model, a tool operation model, and an object model for an actual work performed in the real world (on-site environment), the acquisition unit 201 acquires work data when the skilled worker performs the actual work in the real world. When generating a work posture model and a tool operation model for a virtual work (virtual work) performed in a virtual space, the acquisition unit 201 acquires work data when the skilled worker performs the virtual work in the virtual space.

[0025] Next, in step S302, the learning processing unit 207 of the training support system 100 performs learning processing to generate various models based on the work data of the skilled worker acquired in step S301. The learning processing unit 207 performs machine learning using the work data of the skilled worker acquired in step S301 as input, and generates various models related to the work of the skilled worker.

[0026] For example, as shown in FIG. 4(a), the learning processing unit 207 performs machine learning using posture data 401 obtained by analyzing the analysis unit 205 based on data acquired by the motion sensor 107 during work by a skilled worker as input, and generates a working posture model 402. The posture data 401 is data showing posture during work in a time series. The posture data 401 is a result of three-dimensional measurement of posture, and is composed of data showing the positions (x, y, z) of each measurement point set in advance on the x-axis, y-axis, and z-axis at each time and the rotation angles (φ, θ, ψ) on the x-axis, y-axis, and z-axis, as shown in FIG. 5, for example. Here, examples of the measurement points include joints and tips of the body. The learning processing unit 207 generates a working posture model 402 based on the posture data 401 using the movement variance values ​​of the positions of predetermined measurement points, such as the worker's pelvis, right foot, left foot, right knee, left knee, head, right shoulder, left shoulder, right elbow, left elbow, right hand, and left hand, as explanatory variables. For example, if the work is airbrush painting, the learning processing unit 207 generates a work posture model 402 based on the posture data 401, using the movement variance values ​​of the worker's pelvis, head, right shoulder, left shoulder, right elbow, left elbow, and the position of the hand holding the airbrush as explanatory variables.

[0027] Also, for example, as shown in FIG. 4(b), the learning processing unit 207 performs machine learning using tool operation data 411 obtained by analyzing the analysis unit 205 based on data acquired during work by an expert as input, and generates a tool operation model 412. The tool operation data 411 is data that indicates the motion of the tool during work and the operation amount thereof in a time series. For example, as shown in FIG. 6, the tool operation data 411 is composed of data indicating the position (x, y, z) of the tool on the x-axis, y-axis, and z-axis at each time, the rotation angle (φ, θ, ψ) on the x-axis, y-axis, and z-axis, and the operation amount (mv) of an operation member whose operation amount is controlled, if any. For example, based on the tool operation data 411, the learning processing unit 207 calculates the distance between the work object and the tool, the moving speed of the tool, and the operation amount for the tool, and generates the tool operation model 412 using the respective movement variance values, etc. as explanatory variables. For example, if the work is airbrush painting, the learning processing unit 207 generates a tool operation model 412 based on the tool operation data 411, using the distance between the object to be painted and the airbrush, the airbrush movement speed, the amount of airbrush operation (trigger pull width, trigger pull time, etc.), and the movement variance values ​​of each as explanatory variables.

[0028] 4(c), the learning processing unit 207 receives as input work object data 421 obtained from time-series images captured by the camera 108 during work by an expert, and performs machine learning to generate an object model 422. The learning processing unit 207 generates the object model 422 using, for example, feature amounts and their statistics obtained from the captured images of the work object as explanatory variables.

[0029] Next, in step S303, the learning processing unit 207 stores and saves each model related to the work by an expert generated in step S302 in the storage unit 202. When each model related to the work by an expert is generated and saved in the above manner, the training support system 100 ends the model generation process.

[0030] In the above explanation, a model related to actual work performed in the real world and a model related to virtual work (virtual work) performed in a virtual space are separated, but a single model may be generated without separating the actual work performed in the real world and the virtual work (virtual work) performed in a virtual space. In addition, when there are multiple skilled workers, each model related to the work is generated for each skilled worker.

[0031] Fig. 7 is a flowchart showing an example of processing in the training support system 100 when performing a virtual task in a virtual space and evaluating the task. In the following description, it is assumed that the state of the task (e.g., airbrush painting) is displayed on the HMD 105 using a VR image relating to the task, and the virtual task is performed using a pseudo tool 106. When an instruction to start performing the virtual task in the virtual space is given, the training support system 100 starts the processing shown in Fig. 7.

[0032] In step S701, the image generating unit 208 of the training support system 100 generates VR output data for outputting a state of work in a virtual work (virtual work) so that the worker (trainee) can recognize it. The VR output data includes image data including a VR image showing the state of work to be displayed on the HMD 105 as the display unit 203 in the virtual work, and sound data output from the HMD 105. Here, the work object displayed on the HMD 105 in the virtual work is generated, for example, from a virtual image generated by computer graphics or an image of the object that is actually photographed. The image generating unit 208 generates the VR output data based on setting data related to the virtual work (shape data of the object, etc.) and work data acquired in the virtual work performed by the worker (trainee) up to that point.

[0033] In step S702, the display control unit 209 of the training support system 100 outputs VR output data (VR output) showing the state of the virtual work generated in step S701. The display control unit 209 causes the HMD 105 (display unit 203) to display a display image based on the VR output data, and causes the HMD 105 to output sound based on the VR output data.

[0034] Next, in step S703, the acquisition unit 201 of the training support system 100 acquires various types of work data in the virtual work. The acquisition unit 201 acquires data on the movement and operation of the virtual tool 106 (operation unit 204) by the worker (trainee) in the virtual work, and the posture of the worker, etc.

[0035] In step S704, the training support system 100 determines whether or not to perform real-time evaluation of the virtual work being performed. The training support system 100 determines whether or not to perform real-time evaluation based on, for example, a user setting specifying whether or not to perform real-time evaluation of the work, or the work state in the virtual work (for example, whether or not the work area, work time, etc., satisfy predetermined conditions), etc. If the training support system 100 determines that real-time evaluation is to be performed (YES), it proceeds to step S705, and if it determines that real-time evaluation is not to be performed (NO), it proceeds to step S707.

[0036] In step S705, the evaluation unit 206 of the training support system 100 evaluates the virtual work being performed by the worker (trainee) based on the work data of the virtual work acquired by the acquisition unit 201.

[0037] For example, as shown in FIG. 8(a), the evaluation unit 206 compares posture data 801 of a virtual work obtained by the analysis unit 205 based on data acquired by the motion sensor 107 during a virtual work performed by a worker (trainee) with the working posture model 402 generated as shown in FIG. 4(a), evaluates the posture (movement) of the worker (trainee) during work, and outputs the evaluation result. The evaluation unit 206 compares, for example, all measurement points or a predetermined part of measurement points in the posture data 801 of a virtual work performed by the worker (trainee) obtained by three-dimensional measurement using the motion sensor 107 with the working posture model 402, and evaluates the posture of the worker (trainee) during work. As an example, in airbrush painting, the number of times and the swing width of the arms are swung up and down and left and right when painting the same area are compared with the working posture model 402, and a numerical comparison is made to determine how many times (larger / smaller) the arms are swung.

[0038] Also, for example, as shown in FIG. 8(b), the evaluation unit 206 compares pseudo tool operation data 811 obtained by the analysis unit 205 based on data acquired by the pseudo tool 106 during virtual work by the worker (trainee) with the tool operation model 412 generated as shown in FIG. 4(b), evaluates the motion (speed, position, and angle) and operation amount of the tool during virtual work by the worker (trainee), and outputs the evaluation result. The pseudo tool operation data 811 is configured in the same manner as in FIG. 6. For example, if the work is airbrush painting, the evaluation unit 206 evaluates the distance between the object to be painted and the airbrush, the moving speed of the airbrush, the operation amount of the airbrush (pulling width and pulling time of the trigger, etc.), etc., based on the pseudo tool operation data 811.

[0039] Also, for example, as shown in FIG. 8(c), the evaluation unit 206 compares virtual work object data 821 obtained by analyzing the data acquired by the pseudo tool 106 in the virtual work by the worker (trainee) with the object model 422 generated as shown in FIG. 4(c) to evaluate the object in the virtual work by the worker (trainee) and output the evaluation result. The virtual work object data 821 is data corresponding to changes in the virtual object before and after the work, such as the range of paint on the object to be painted and the degree of gradation. The evaluation unit 206 performs the evaluation by simulating the state of the object based on the virtual work object data 821 and comparing it with the object model 422. For example, in the case of painting a gradation from black to white, it calculates whether or not the color change is expressed smoothly within a predetermined length range based on color information pseudo-applied to the virtual work object.

[0040] Next, in step S706, the evaluation unit 206 records the evaluation result by the evaluation process performed in step S705 in the storage unit 202. The evaluation result recorded in the storage unit 202 may be displayed together with the state of the virtual work on the display unit 203, or may be displayed on another display device.

[0041] In step S707, the training support system 100 determines whether or not to end the virtual work being executed. If the training support system 100 determines that the predetermined end condition is met and the virtual work is to be ended (YES), the system proceeds to step S708. If the training support system 100 determines that the predetermined end condition is not met and the virtual work is not to be ended (NO), the system returns to step S701 and executes the processes from step S701 onward again.

[0042] In step S708, the evaluation unit 206 of the training support system 100 evaluates the performed virtual work based on the work data of the virtual work by the worker (trainee) acquired by the acquisition unit 201, the evaluation result of the real-time evaluation recorded in the storage unit 202 in step S706, etc. The evaluation unit 206 compares the virtual work posture data, pseudo tool operation data, and virtual work object data obtained based on the acquired work data of the virtual work with a work posture model, a tool operation model, and an object model, similar to the evaluation process in step S705, for example, to evaluate the virtual work by the worker (trainee), and outputs the evaluation result.

[0043] Next, in step S709, the evaluation unit 206 records the evaluation result by the evaluation process performed in step S708 in the storage unit 202, and ends the process. Note that, in addition to recording the evaluation result in the storage unit 202, the evaluation result may also be displayed on the display unit 203 or another display device.

[0044] In addition, when there are multiple experts and a model is generated for each of them, for example, a model of an expert having attributes similar to that of the trainee may be selected based on the trainee's attributes (such as the trainee's body shape, period of experience, habits, etc.) to evaluate the virtual work performed by the trainee. Also, the work performed by the trainee may be evaluated using models of multiple experts, and in subsequent evaluations, a model of an expert having a high similarity to the trainee may be selected for evaluation.

[0045] Here, the training support system 100 may output advice such as merits and improvements based on the evaluation results during or after the virtual work is performed. Also, the training support system 100 may display guide information related to the virtual work on the display unit 203 based on the evaluation results and present it to the worker while the virtual work is being performed. Also, the training support system 100 may store and accumulate the evaluation results of the work by the worker (trainee) in, for example, the storage unit 202 so that multiple results by the same trainee can be compared, and, for example, the evaluation results of the current virtual work and the evaluation results of past virtual works by the same trainee can be compared.

[0046] The training support system 100 may display the posture of the skilled worker and the posture of the worker (trainee) while working in a superimposed or juxtaposed manner, so that the difference (deviation) in posture can be recognized. The posture of the same trainee in a current virtual work and the posture of a past virtual work can be displayed in a superimposed or juxtaposed manner, so that the postures can be compared. The posture and working conditions of the worker (trainee) while working may be recorded, so that they can be reproduced and displayed as images. The posture during work may further be displayed as a free viewpoint image.

[0047] The training support system 100 in this embodiment not only evaluates the motion and operation of tools in virtual work (virtual work) performed in a virtual space, but also compares the posture during work with appropriate postures to evaluate it. This makes it possible to evaluate even the appropriate posture during work, making it easier for workers to acquire skills.

[0048] In the above-described embodiment, the virtual work (virtual work) performed in the virtual space is performed by displaying the state of the work on the HMD 105 using VR images, but the present invention is not limited to this, and the work target may be displayed on some display device and the work may be performed using a pseudo tool. The training support system 100 in this embodiment is suitable for use in training for work in which the posture during the work has a large effect on the results of the work, and can be applied to training for work such as polishing, plastering, cutting, connecting, squeezing with a spatula, and laminating FRP, in addition to airbrush painting.

[0049] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0050] 100 Training Support System 201 Acquisition Department 202 Storage section 203 Display section 204 Operation section 205 Analysis Department 206 Evaluation Department 207 Learning Processing Unit 208 Image Generation Unit 209 Display control section 210 Control section

Claims

1. A work training means having a display unit and an operation unit; A measuring means for measuring a posture of a user as three-dimensional data; a learning means for performing machine learning on posture data measured by the measuring means during actual work by a first user to generate a first working posture model; a comparison means for comparing the first working posture model generated by the learning means with posture data measured by the measurement means during a virtual work performed by a second user using the work training means, and outputting a comparison result.

2. the learning means performs machine learning on posture data measured by the measurement means during a virtual work performed by the first user using the work training means to generate a second working posture model; 2. The training support system according to claim 1, wherein the comparison means compares the second working posture model generated by the learning means with posture data measured by the measurement means during a virtual work performed by the second user using the work training means, and outputs a comparison result.

3. A work training means having a display unit and an operation unit; A measuring means for measuring a posture of a user as three-dimensional data; a learning means for performing machine learning of posture data measured by the measuring means during a virtual work performed by a first user using the work training means to generate a first working posture model; a comparison means for comparing the first working posture model generated by the learning means with posture data measured by the measurement means during a virtual work performed by a second user using the work training means, and outputting a comparison result.

4. 4. The training support system according to claim 1, wherein a comparison result by said comparison means is displayed on said display unit of said work training means.

5. 4. The training support system according to claim 1, wherein the comparison means evaluates the work performed by the second user based on a result of the comparison.

6. The training support system according to any one of claims 1 to 3, characterized in that the comparison means compares a plurality of work training results obtained in virtual work performed by the second user using the work training means.

7. The training support system according to any one of claims 1 to 3, further comprising a presentation means for displaying guide information for the second user on the display unit of the work training means based on the comparison result by the comparison means.

8. a result measuring means for measuring an arbitrary item related to operation data of a pseudo tool used in the virtual work or virtual work object data with respect to a work result performed on an object by the second user using the work training means; a value comparison means for comparing the measured value measured by the result measurement means with a preset ideal value, 4. The training support system according to claim 1, wherein a comparison result by said value comparison means is displayed on said display unit of said work training means.

9. The training support system according to any one of claims 1 to 3, characterized in that the work object displayed on the display unit of the work training means is a virtual image created by computer graphics or an image of an object that has actually been photographed.

10. A training support method using a training support system having a work training means having a display unit and an operation unit, A measuring step of measuring a posture of a user as three-dimensional data; a learning process for generating a first working posture model by machine learning the posture data measured in the measuring process during actual work by a first user; a comparison step of comparing the first working posture model generated in the learning step with posture data measured in the measurement step during virtual work performed by a second user using the work training means, and outputting a comparison result.

11. In the learning step, posture data measured in the measuring step during a virtual work performed by the first user using the work training means is machine-learned to generate a second working posture model; 11. The training support method according to claim 10, wherein the comparison step compares the second working posture model generated in the learning step with posture data measured in the measuring step during virtual work performed by the second user using the work training means, and outputs a comparison result.

12. A training support method using a training support system having a work training means having a display unit and an operation unit, A measuring step of measuring a posture of a user as three-dimensional data; a learning process for generating a first working posture model by machine learning posture data measured in the measuring process during a virtual work performed by a first user using the work training means; a comparison step of comparing the first working posture model generated in the learning step with posture data measured in the measurement step during virtual work performed by a second user using the work training means, and outputting a comparison result.

13. A computer of a training support system having a work training means having a display unit and an operation unit, A measuring step of measuring a posture of a user as three-dimensional data; a learning step of generating a first working posture model by machine learning the posture data measured in the measuring step during actual work by a first user; a comparison step of comparing the first working posture model generated in the learning step with posture data measured in the measurement step during virtual work performed by a second user using the work training means, and outputting a comparison result.

14. In the learning step, posture data measured in the measuring step during a virtual work performed by the first user using the work training means is machine-learned to generate a second working posture model; 14. The program according to claim 13, wherein the comparison step compares the second working posture model generated in the learning step with posture data measured in the measuring step during a virtual work performed by the second user using the work training means, and outputs a comparison result.

15. A computer of a training support system having a work training means having a display unit and an operation unit, A measuring step of measuring a posture of a user as three-dimensional data; a learning step of performing machine learning on posture data measured in the measuring step during a virtual work performed by a first user using the work training means to generate a first working posture model; a comparison step of comparing the first working posture model generated in the learning step with posture data measured in the measurement step during virtual work performed by a second user using the work training means, and outputting a comparison result.

16. a result measuring step of measuring any item related to operation data of a pseudo tool used in the virtual work or virtual work object data for a result of the virtual work performed by the second user on an object using the work training means; a value comparison step of comparing the measured value measured in the result measurement step with a preset ideal value; 16. The program according to claim 13, for causing a computer to execute a display step of displaying a comparison result in the value comparison step on the display unit of the work training means.

Citation Information

Patent Citations

  • Work support device and work support method

    JP2020012859A

  • Welding work evaluation device, welding work evaluation method and program

    JP2021189380A

  • Virtual welding system

    JP2017223966A