Posture measurement system, posture measurement method, and program

Abstract

To provide a posture measurement system which enables more accurate collection of posture data used for a virtual welding work device, etc.SOLUTION: A posture measurement system is provided, comprising: virtual welding means having a display unit and an operating unit; measurement means for measuring postures of a user as three-dimensional data; and comparison means configured to compare the first posture data measured by the measurement means during actual welding work done in a site environment by a first user with second posture data measured by the measurement means during virtual welding work done by the first user using the virtual welding means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a posture measurement system, a posture measurement method, and a program for measuring a posture during welding. [Background technology]

[0002] A mechanism for learning the ideal posture when exercising or working has been proposed. Patent Document 1 discloses a technology for comparing an image of a user captured by an imaging device with an image of a virtual model when exercising at home or the like for fitness, and displaying a result screen showing the comparison result. Patent Document 2 discloses a technology for work training at a construction site, in which a worker receiving training wears a display device that displays VR (Virtual Reality) images, holds a simulated work tool, and further wears a drive unit that applies force to the body, and controls the drive unit so that the worker's movements are close to the sample movements of a skilled worker captured by a motion capture device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6804781 [Patent Document 2] JP 2020-173326 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in welding technology, it is known that the accuracy of welding, the beauty of the finish, the work efficiency, or the safety of the work greatly differs depending on the relationship between the distance between the object to be welded and the welding tool, the speed at which the welding tool is moved, and the like, as well as the posture during welding. Therefore, it is conceivable to apply the posture of an expert during welding to the technology disclosed in Patent Document 1 and Patent Document 2 as model data, and to try to make the posture of the welding worker during welding closer to the ideal posture. However, even if an expert is skilled, the posture during welding may change depending on the on-site environment where the welding work is performed and the object to be welded, so there is a problem that it is difficult to identify the ideal posture itself to be used as a model. In addition, since the environment and tools of the VR image and the simulated working tool are different from those of the actual on-site, even an expert cannot work in the same posture as usual, and there is a problem that the ideal posture is not necessarily conveyed to the user working in the virtual space. The object of the present invention is to provide a posture measurement system that makes it possible to more accurately collect posture data used in a virtual welding work device or the like. [Means for solving the problem]

[0005] The posture measurement system according to the present invention includes a virtual welding means having a display unit and an operation unit, a measurement means for measuring a user's posture as three-dimensional data, and a comparison between first posture data measured by the measurement means during actual welding work performed by a first user in a field environment and second posture data measured by the measurement means during virtual welding work performed by the first user using the virtual welding means. and generating correction information regarding the posture of the first user in the virtual welding work based on the comparison result. and comparing means for comparing the detected value with the detected value. Effect of the Invention

[0006] According to the present invention, it is possible to provide a posture measuring system for more accurately collecting posture data to be used in a virtual welding operation device or the like. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of a training support system to which a posture measurement system according to an embodiment of the present invention is applied. [Diagram 2] FIG. 1 is a diagram illustrating an example of a functional configuration of a training support system to which the posture measurement system according to the present embodiment is applied. [Diagram 3] 1A to 1C are diagrams illustrating a virtual welding operation in the present embodiment. [Figure 4] 1A to 1C are diagrams illustrating an example of model creation in the posture measurement system according to the present embodiment. [Diagram 5] 4A and 4B are diagrams illustrating posture data of a user in the present embodiment. [Figure 6] 1A to 1C are diagrams illustrating an example of processing in the posture measurement system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, actual welding performed in an on-site environment is also referred to as "on-site welding", and virtual welding performed in a simulated virtual space is also referred to as "virtual welding".

[0009] 1 is a diagram showing an example of a hardware configuration of a training support system 100 to which a posture measurement system according to one embodiment of the present invention is applied. The training support system 100 is a training support system that provides training in welding techniques based on, for example, the movement of a torch and the posture of a worker during welding work.

[0010] The training support system 100 includes a CPU 101, a main memory device 102, an auxiliary memory device 103, an output device 104, an HMD 105, a controller 106, a motion sensor 107, a camera 108, and a torch motion detection device 109. The CPU 101, the main memory device 102, the auxiliary memory device 103, the output device 104, the HMD 105, the controller 106, the motion sensor 107, the camera 108, and the torch motion detection device 109 are connected to each other so as to be able to communicate with each other.

[0011] The CPU (Central Processing Unit) 101 is a central processing unit that controls the training support system 100. The main memory 102 is a memory device that functions as a work area for the CPU 101 and a temporary storage location for data. The main memory 102 is implemented using, for example, a RAM (Random Access Memory) or the like. The auxiliary memory 103 is a memory device that stores various data, various programs, and the like. The auxiliary memory 103 is implemented using, for example, a ROM (Read Only Memory), a hard disk drive (HDD), a solid state drive (SSD), and the like. The CPU 101 executes processing based on a program stored in, for example, the auxiliary memory 103, thereby realizing various processing operations in the training support system 100. The output device 104 is a device that outputs various information such as an evaluation result of a welding operation, and is used to present various information to a user, etc.

[0012] The HMD (Head Mounted Display) 105 is a head-mounted display device that displays images (VR images) in a virtual space. The HMD 105 may be a non-transparent HMD that prevents the user from checking the outside when worn, or a transparent (see-through) HMD that displays the outside when necessary. The controller 106 is a controller that accompanies the HMD 105 and is used to perform operations in a virtual space displayed on the HMD 105. For example, in virtual welding (virtual welding) performed in a virtual space, a VR image showing the welding state, such as various information and an image during welding, is displayed on the HMD 105, and the virtual welding is performed using the controller 106. At this time, the controller 106 is used to detect, for example, a pseudo torch operation. The HMD 105 is an example of a display unit of a virtual welding means, and the controller 106 is an example of an operation unit of a virtual welding means.

[0013] The motion sensor 107 is a sensor for three-dimensionally measuring (motion capture) the user's posture (movement). The motion sensor 107 is not limited to a type of motion capture as long as it can measure the user's posture, etc. In the following, the motion sensor 107 will be described taking as an example a wearable motion capture suit type, also called smart apparel, incorporating an IMU sensor (inertial measurement unit). The motion sensor 107 may be a wearable suit type motion sensor with a marker that is detected by a tracker, for example. The motion sensor 107 may also be a photographing type (camera type) that detects the distance and position of each part of the subject by capturing the subject, for example.

[0014] Camera 108 is a camera that captures images of welding phenomena during actual welding (on-site welding) performed in an on-site environment. Torch motion detection device 109 detects torch motion during actual welding (on-site welding) performed in an on-site environment. Torch motion detection device 109 detects the movement of the torch, for example, based on a marker attached to the torch. Note that torch motion detection device 109 only needs to be able to detect the motion of the torch during welding, and there are no limitations on the detection method.

[0015] In actual welding (on-site welding) performed in an on-site environment, various data during welding are acquired, for example, using a motion sensor 107, a camera 108, and a torch motion detection device 109. In virtual welding (virtual welding) performed in a virtual space, various data during welding are acquired, for example, using a motion sensor 107 and a controller 106.

[0016] 2 is a diagram showing an example of the functional configuration of a training support system 100 to which the posture measurement system of this embodiment is applied. The training support system 100 includes an acquisition unit 201, a storage unit 202, a display unit 203, an operation unit 204, an analysis unit 205, an evaluation unit 206, a learning processing unit 207, an image generation unit 208, a display control unit 209, and a control unit 210.

[0017] The acquisition unit 201 acquires various data used for training support of welding techniques. For example, the acquisition unit 201 acquires various data related to the operation of a torch in on-site welding, the posture of a user (worker), and welding phenomena such as a molten pool and a weld mark (bead). The various data related to on-site welding are acquired, for example, from the motion sensor 107, the camera 108, and the torch operation detection device 109. Also, for example, the acquisition unit 201 acquires various data related to the operation of a torch and the posture of a user (worker) in virtual welding. The various data related to virtual welding are acquired, for example, from the motion sensor 107 and the controller 106. Also, the molten pool and the weld mark (bead) can be acquired as an image by photographing them with the camera 108.

[0018] The storage unit 202 stores various data acquired by the acquisition unit 201 and various models created by the learning processing unit 207. The storage unit 202 may also store data such as processing results acquired by the analysis unit 205 and the evaluation unit 206. The display unit 203 is a device that outputs various information such as an evaluation result of a welding operation, and displays various information to be presented to a user or the like according to the control of the display control unit 209. For example, the display unit 203 displays a result screen showing the evaluation result of the welding operation by the evaluation unit 206, and displays various information when performing virtual welding and VR images showing the state of welding. The operation unit 204 is used to perform operations in a virtual space. The operation unit 204 is used, for example, when performing various operations in virtual welding. The display unit 203 is an example of a display unit of a virtual welding means, and the operation unit 204 is an example of an operation unit of a virtual welding means.

[0019] The analysis unit 205 analyzes the torch movement during welding, the user (worker) posture, welding phenomena, and the like, based on various data acquired by the acquisition unit 201. For example, the analysis unit 205 analyzes the torch movement (speed, position, and angle) during welding, based on data acquired from the torch movement detection device 109 during on-site welding and data acquired from the controller 106 during virtual welding. The analysis unit 205 also analyzes the posture (movement) of the user (worker) during welding, based on data acquired from the motion sensor 107 during on-site welding and virtual welding.

[0020] The evaluation unit 206 compares the torch movements, working postures, and welding phenomena between the expert and the trainee in the welding work, and evaluates the welding work. For example, the evaluation unit 206 compares the trainee's torch movements with a torch model created in advance regarding the expert's torch movements, and evaluates the trainee's torch movements during welding. The evaluation unit 206 also compares the trainee's working posture based on the three-dimensional measurement results with a welding posture model created in advance regarding the expert's working posture, and evaluates the trainee's posture during welding. The evaluation unit 206 also compares the welding phenomenon simulated based on the trainee's torch movements, etc., with a welding phenomenon model created in advance regarding the welding phenomenon in welding by the expert, and evaluates the welding phenomenon in welding by the trainee.

[0021] The learning processing unit 207 receives various data obtained during welding work by an expert, performs machine learning using the input data as teacher data, and creates a model corresponding to the data. For example, the learning processing unit 207 creates a torch model related to the movement of the torch, a welding posture model related to the posture during welding work, and a welding phenomenon model related to the welding phenomenon (for example, a molten pool model related to the molten pool) based on data obtained during on-site welding or virtual welding work by an expert. The learning processing unit 207 is an example of a comparison means and a learning means.

[0022] The image generation unit 208 generates images to be displayed in the training support system 100. For example, the image generation unit 208 generates VR images to be displayed when performing virtual welding, images showing evaluation results of welding work, etc. The display control unit 209 controls the display on the display unit 203. For example, the display control unit 209 controls the display position, size, etc. of the image generated by the image generation unit 208 on the display unit 203. The control unit 210 controls each functional unit of the training support system 100.

[0023] FIG. 3 is a diagram for explaining virtual welding (virtual welding) performed in a virtual space in this embodiment. FIG. 3(a) shows an example of a VR image 301 displayed on the HMD 105 when virtual welding is performed. In the VR image 301, a torch (welding rod) 302 simulating actual welding (on-site welding) performed in a field environment, a welding mark (bead) 303, and a molten pool 304 formed by a generated arc are displayed as elements related to the welding work itself. In addition, the VR image 301 can also display an evaluation 306 of the welding work (for example, a four-level display, etc.), operation information 307 of the torch (welding rod) 302, and posture information 308 during the welding work. In addition, in virtual welding, it is also possible to output sounds during welding, sounds that beat rhythms, etc. together with the VR image 301. In addition, precautions (reminders) regarding the operation and posture of the torch during virtual welding may be displayed by image or output by sound.

[0024] The display (output) of each element in virtual welding can be set to be displayed (output) or not via a setting screen such as that shown in Fig. 3(b). Fig. 3(b) shows an example in which it is possible to set whether to display the arc light, spatter, fume, and molten pool as elements to be displayed in virtual welding, and whether to display (output) the ruler, metronome (rhythmic sound), molten pool frame, and torch movement angle / operation angle as guides to support the welding work, and whether to display real-time evaluation. Note that the settings shown in Fig. 3(b) are merely examples and are not limited to these.

[0025] The training support system 100 in this embodiment allows the trainee to learn the ideal posture during welding work by comparing the posture during welding work by an expert with the posture during welding work by a trainee. Here, even if the same welding object and the same welding method are used in on-site welding and virtual welding, the posture during welding work may differ even for an expert because the welding environment and the tools used are different. Therefore, in this embodiment, the difference in posture during welding work between on-site welding and virtual welding is obtained, and a correction is made to the welding posture model in virtual welding. In addition, in the virtual welding work by the trainee, display control and operation control are performed based on correction information according to the difference between the posture during welding work by an expert and the posture during welding work by the trainee.

[0026] FIG. 4 is a diagram for explaining a model creation example in the posture measurement system in this embodiment. The learning processing unit 207 receives posture data 401 for on-site welding and posture data 402 for virtual welding obtained by analyzing the data acquired by the motion sensor 107 in on-site welding and virtual welding by an expert, respectively, in the analysis unit 205. Although not shown, data on the torch motion (speed, position, and angle) during welding work obtained from the torch motion detection device 109 is also input. The posture data 401 and 402 are both data showing the posture of an expert during welding work in a time series. The posture data 401 and 402 are the results of three-dimensional measurement of the posture, and are composed of data showing the positions (x, y, z) on the x-axis, y-axis, and z-axis and the rotation angles (φ, θ, ψ) on the x-axis, y-axis, and z-axis of each measurement point set in advance at each time, as shown in FIG. Here, the measurement points include joints and extremities of the body, such as the worker's pelvis, right foot, left foot, head, right shoulder, left shoulder, right elbow, and right hand. The torch motion data is similarly composed of data indicating the coordinates of each marker in chronological order. The same applies to the following explanation, so only the posture data will be described.

[0027] The learning processing unit 207 compares the input on-site welding posture data 401 with the virtual welding posture data 402, and generates correction data (welding posture model correction data) 403 related to the posture information of the expert in virtual welding based on the comparison result. For example, the positional relationship between the welding point and the face, the angle of the arm holding the torch, the angle of the waist, etc. are calculated as differences. The comparison between the on-site welding posture data 401 and the virtual welding posture data 402 by the learning processing unit 207 is performed by extracting the posture data 401 and 402 in an arbitrary range (for example, a range in which postures correspond between the posture data), and using the extracted data groups. The correction data generated by such comparison includes, for example, the positional relationship between the welding point and the face when the expert performs virtual welding is closer by n centimeters than in on-site welding, and is stored, for example, in the storage unit 202 as correction data of the welding posture model in virtual welding of the expert. Similarly, difference data between on-site welding and virtual welding is calculated for the torch model and the welding phenomenon model, and is stored, for example, in the storage unit 202 as correction data of each model. In addition, by creating many welding posture models for on-site welding and virtual welding of a skilled person, rather than creating models for multiple skilled people, the range of models to be selected that are close to the trainee can be expanded in the training of the trainee, which will be described later. Also, by repeating learning multiple times for the same skilled person, the accuracy of the welding posture model of that skilled person can be improved.

[0028] Next, the case where a trainee uses the training support system 100 to conduct welding training will be described with reference to FIG. 6. FIG. 6 is a diagram for explaining an example of processing in the posture measurement system in this embodiment. The evaluation unit 206 receives the skilled person's virtual welding posture data 601 and the trainee's virtual welding posture data 602. The trainee's posture data 602 is analyzed by the analysis unit 205 based on the data acquired by the motion sensor 107 in virtual welding, as with the skilled person, to obtain the virtual welding posture data 602. Although not shown, data on the torch's motion (speed, position, and angle) during welding work obtained from the torch motion detection device 109 is also input. At this time, the above-mentioned correction data 403 is applied to the skilled person's virtual welding posture data 601. For example, as described above, if there is correction data that indicates that the positional relationship between the welding point and the face is closer by n centimeters when a skilled person conducts virtual welding than in on-site welding, then the ideal welding posture at the site should be n centimeters away. Therefore, the positional relationship between the welding point and the face is corrected to a virtual welding posture model of an expert with a distance of n centimeters, and input to the evaluation unit 206. Note that, for ease of explanation, the corrected data has been explained here using expressions such as "n centimeters closer," but, like the posture data explained in Fig. 5, it is stored as numerical values ​​such as three-dimensional coordinates and angle data.

[0029] In the training support system 100, when a trainee performs virtual welding, the evaluation unit 206 compares and evaluates the posture data 601 of the expert and the posture data 602 of the trainee, and if the difference between the two exceeds a certain range as a result of the comparison, the display control unit 209 performs display control processing of the display unit 203 based on the correction information 603, and the control unit 210 performs operation control processing of the operation unit 204 based on the correction information 603. For example, if the worker has difficulty in grasping the sense of distance in virtual welding and the face tends to be too close compared to the posture data 601 of the expert, the display control unit 209 evaluates that display of correction information is necessary, and controls the HMD 105 to display head correction information such as an arrow to move the head away as teaching information based on the correction information 603. Note that, although the display control will be described here, the correction information may be issued not only by display but also by voice. Moreover, if the motion angle or movement angle of the torch is different from the appropriate angle, control is performed so that teaching information (correction information for the motion angle or movement angle by arrow, correction information by voice) is output to the HMD 105 in a similar manner.

[0030] In this way, by generating correction data based on the difference between the posture of a model skilled worker in on-site welding and the posture in virtual welding, and correcting the welding posture model of the skilled worker in virtual welding according to the correction data, ideal posture data of a skilled worker can be collected more accurately and reflected in virtual welding. Therefore, it is possible to convey the ideal posture of a skilled worker to a worker performing virtual welding.

[0031] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0032] 100 Training Support System 201 Acquisition Department 202 Storage section 203 Display section 204 Operation section 205 Analysis Department 206 Evaluation Department 207 Learning Processing Unit 208 Image Generation Unit 209 Display control section 210 Control section

Claims

1. a virtual welding means having a display unit and an operation unit; A measuring means for measuring a posture of a user as three-dimensional data; a comparison means for comparing first posture data measured by the measurement means during actual welding work performed by a first user in an on-site environment with second posture data measured by the measurement means during virtual welding work performed by the first user using the virtual welding means, and generating correction information regarding the posture of the first user in the virtual welding work based on a result of the comparison.

2. 2. The attitude measurement system according to claim 1, wherein the comparison means outputs a difference between the first attitude data and the second attitude data as the correction information.

3. 2. The posture measuring system according to claim 1, wherein the comparison means extracts the first posture data and the second posture data in an arbitrary range and compares the extracted data groups with each other.

4. 2. The posture measurement system according to claim 1, further comprising an evaluation means for evaluating the virtual welding work performed by the second user based on the second posture data and third posture data measured by the measurement means during the virtual welding work performed by the second user using the virtual welding means.

5. 5. The posture measurement system according to claim 4, wherein the evaluation means compares the second posture data corrected based on the correction information with the third posture data to evaluate the virtual welding work by the second user.

6. 6. The posture measurement system according to claim 5, wherein the evaluation means generates correction information regarding the virtual welding work by the second user based on a comparison result between the corrected second posture data and the third posture data.

7. 7. The posture measuring system according to claim 6, wherein the evaluation means generates the correction information when a difference between the corrected second posture data and the third posture data exceeds a certain range.

8. 7. The posture measurement system according to claim 6, wherein at least one of the display unit and the operation unit of the virtual welding means is controlled based on the correction information.

9. a learning means for performing machine learning on at least one of the first posture data and the second posture data to create a welding posture model of the first user; 2. The posture measuring system according to claim 1, wherein the comparison means generates the correction information for the welding posture model of the first user.

10. A posture measurement method using a posture measurement system having a virtual welding means having a display unit and an operation unit, A measuring step of measuring a posture of a user as three-dimensional data; a comparison step of comparing first posture data measured in the measurement step during actual welding work performed by a first user in an on-site environment with second posture data measured in the measurement step during virtual welding work performed by the first user using the virtual welding means, and generating correction information regarding the posture of the first user in the virtual welding work based on a result of the comparison.

11. A computer of a posture measurement system having a virtual welding means having a display unit and an operation unit, A measuring step of measuring a posture of a user as three-dimensional data; a comparison step of comparing first posture data measured in the measurement step during actual welding work performed by a first user in an on-site environment with second posture data measured in the measurement step during virtual welding work performed by the first user using the virtual welding means, and generating correction information regarding the posture of the first user in the virtual welding work based on a result of the comparison.

Images