Work support device, work support method, and program

Abstract

To provide a work support device, a work support method, and a program for improving work efficiency in a site environment.SOLUTION: A work support device includes a generation unit configured to generate object data representing a predetermined object in a site environment where a first person is present, an object display control unit configured to display a virtual object based on the object data on a remote side display device in a remote environment where a second person is present, an acquisition unit configured to acquire a detection result by a sensor configured to detect a motion of a body of the second person, and an avatar display control unit configured to display an avatar operating on the basis of the detection result on a site side display device in the site environment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work assistance device, a work assistance method, and a program. [Background technology]

[0002] Patent Document 1 discloses a remote work support system that makes it easy for a manager to check data collected from a wearable device worn by a field worker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-010101 Summary of the Invention [Problem to be solved by the invention]

[0004] In the related art, the supervisor cannot communicate physical movements to the field workers, which causes a problem of reduced work efficiency for the field workers.

[0005] The present disclosure has been made to solve such problems, and aims to provide a work assistance device, a work assistance method, and a program that improve work efficiency in a field environment. [Means for solving the problem]

[0006] The work assistance device according to the present disclosure comprises: a generation unit that generates object data representing a predetermined object in an on-site environment where the first person is present; an object display control unit that displays a virtual object based on the object data on a remote display device in a remote environment where a second person is present; an acquisition unit that acquires a detection result from a sensor that detects the body movement of the second person; an avatar display control unit that displays an avatar that operates based on the detection result on a site-side display device within the site environment; Equipped with.

[0007] The work assistance method according to the present disclosure includes: generating object data representing a predetermined object within a scene environment in which the first person is present; displaying a virtual object based on the object data on a remote display device in a remote environment where a second person is present; acquiring a detection result from a sensor that detects the body movement of the second person; An avatar that operates based on the detection result is displayed on a site-side display device within the site environment.

[0008] The program according to the present disclosure is On the computer, generating object data representing a predetermined object within a scene environment in which the first person is present; a process of displaying a virtual object based on the object data on a remote display device in a remote environment where a second person is present; A process of acquiring a detection result by a sensor that detects the body movement of the second person; a process of displaying an avatar that operates based on the detection result on a site-side display device within the site environment; Execute the following. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a work assistance device, a work assistance method, and a program that improve work efficiency in a field environment. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a work assistance device according to the present disclosure. [Figure 2] 1 is a flowchart illustrating a work assistance method according to the present disclosure. [Figure 3] 1 is a diagram for explaining an overview of a work assistance device according to the present disclosure; [Figure 4] 1 is a block diagram illustrating a work assistance device according to the present disclosure. [Figure 5] 10 is a flowchart illustrating an example of an operation of the work assistance device according to the present disclosure. [Figure 6] 10 is a flowchart illustrating an example of an operation of the work assistance device according to the present disclosure. [Figure 7] FIG. 1 is a block diagram illustrating a hardware configuration of a work assistance device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Embodiment 1 FIG. 1 is a block diagram showing an example configuration of a work assistance device 100 according to the present disclosure. The work assistance device 100 may be a computer device that operates when a processor executes a program stored in a memory. The work assistance device 100 may be an information processing device, such as a server device. The work assistance device 100 may also be composed of multiple computers. In this case, the components or functions that make up the work assistance device 100 may be distributed across the multiple computers. The multiple computers may be connected via a network or directly via a cable or the like.

[0012] The work assistance device 100 includes a generation unit 110, an object display control unit 120, an acquisition unit 130, and an avatar display control unit 140. The generation unit 110, the object display control unit 120, the acquisition unit 130, and the avatar display control unit 140 may be software or modules whose processes are performed by a processor executing a program stored in a memory. Alternatively, the generation unit 110, the object display control unit 120, the acquisition unit 130, and the avatar display control unit 140 may be hardware such as a circuit or a chip.

[0013] The generation unit 110 generates object data representing a predetermined object (for example, a pipe) in the site environment where the first person is present.

[0014] The object display control unit 120 displays a virtual object based on the object data on a remote display device in a remote environment where the second person is present. The remote display device may be provided in a head-mounted display (e.g., smart glasses) worn by the second person, or may be a normal monitor.

[0015] The acquisition unit 130 acquires the detection result from a sensor that detects the body movement of the second person. The sensor may be a motion sensor worn by the second person, or a sensor that captures an image of the second person.

[0016] The avatar display control unit 140 displays an avatar that operates based on the detection result of the sensor on an on-site display device within the on-site environment. The on-site display device may be provided in a head-mounted display (e.g., smart glasses) worn by the first person, or may be a normal monitor.

[0017] 2 is a flowchart illustrating a work assistance method according to the present disclosure. First, the generation unit 110 generates object data representing a predetermined object (e.g., a pipe) in the work site environment (step S101). Next, the object display control unit 120 causes the remote display device to display a virtual object based on the object data (step S102). Next, the acquisition unit 130 acquires a detection result from a sensor that detects the body movement of the second person (step S103). Finally, the avatar display control unit 140 causes the on-site display device 30 to display an avatar that acts based on the acquired detection result (step S104).

[0018] As described above, the work assistance device 100 displays a virtual object on the remote display device 20 and an avatar that moves based on the body movements of the second person on the on-site display device 30. As a result, the body movements during work can be communicated from the second person to the first person, thereby improving work efficiency.

[0019] Embodiment 2 Embodiment 2 is a specific example of Embodiment 1. In Embodiment 2, the on-site worker and the remote worker wear smart glasses. The on-site worker corresponds to the first user described above. The remote worker corresponds to the second user described above. The smart glasses worn by the on-site worker correspond to the on-site display device described above. The smart glasses worn by the remote worker correspond to the remote display device described above. The on-site worker and the remote worker may wear head-mounted displays other than smart glasses.

[0020] The on-site worker shares the spatial information of the site that he or she sees with the remote worker in a remote location. The remote worker uses the smart glasses he or she wears to refer to the image information from the RGB (Red, Green, Blue) camera and 3D sensor attached to the on-site worker's smart glasses as information on the virtual space. The remote worker immerses himself or herself in the space and acts as an avatar using the motion sensors attached to his or her body. The avatar information of the remote worker is also projected onto the on-site worker's smart glasses. This allows the remote worker to work as a virtual worker, creating the feeling that the remote worker is actually on-site.

[0021] For example, referring to FIG. 3, remote worker W2 can demonstrate how to install pipe P in a 3D virtual space on-site using gestures via the motion sensor 5. On-site worker W1 performs the work while referring to the movements of avatar A of remote worker W2 and the work instructions via the smart glasses 1. Compared to when work instructions are given by voice or on a screen, on-site worker W1 can perform the work with a much higher quality. Furthermore, when on-site worker W1 performs on-site work alone, it is possible to give the on-site worker W1 a sense of security.

[0022] Furthermore, by having both the on-site worker W1 and the remote worker W2 wear the haptic gloves 2 and 6, respectively, it becomes possible to share the sense of force. The on-site worker W1 can perform work taking into consideration the precise amount of force applied.

[0023] 4 is a block diagram illustrating the configuration of a work assistance device 100a according to the present disclosure. A field worker wears smart glasses 1. The field worker also wears a haptic glove 2 as needed. The field worker also holds a field worker terminal 3 for controlling data from the smart glasses 1 and the haptic glove 2. The field worker terminal 3 may be a smartphone, tablet, or PC.

[0024] The field worker terminal 3 may be a computer device that operates when a processor executes a program stored in memory. The field worker terminal 3 includes a data collection unit 31, an information display unit 32, a data processing unit 35, a data transmission unit 36, and a data receiving unit 37. Each component of the field worker terminal 3 may be software or a module that performs processing when the processor executes a program stored in memory. Alternatively, each component of the field worker terminal 3 may be hardware such as a circuit or a chip. The data processing unit 35 in the field worker terminal 3 corresponds to the generation unit 110 in the work support device 100. The information display unit 32 in the field worker terminal 3 corresponds to the avatar display control unit 140 in the work support device 100.

[0025] The smart glasses 1 worn by the field workers are equipped with various sensors 11. The sensors 11 are sensors that can capture images of the field conditions, such as RGB cameras and 3D sensors.

[0026] The data collection unit 31 of the field worker terminal 3 aggregates the photographic data 12 captured by these sensors 11. The data processing unit 35 analyzes the photographic data 12. The results of the analysis are displayed as the field worker's own data 33 in the information display unit 32 as needed, and are displayed on the field worker's smart glasses 1.

[0027] The data on the movement of the remote worker acquired by the motion sensor 5 is shared with the field worker in real time. The data receiving unit 37 of the field worker terminal 3 receives the data on the movement of the remote worker. The data collecting unit 31 acquires the data on the movement of the remote worker. The avatar, which is moved based on the acquired information, is displayed on the smart glasses 1 as remote transmission data 34 of the information display unit 32. This allows the field worker to feel as if the remote worker is actually a virtual worker in the real world seen by the field worker through the smart glasses 1. The data on the remote worker will be explained later.

[0028] The data collection unit 31 of the on-site worker terminal 3 aggregates the photographic data 12 captured by various sensors 11 (e.g., an RGB camera or a 3D sensor) of the on-site worker's smart glasses 1. The data transmission unit 36 ​​transmits the photographic data 12 edited by the data processing unit 35 to the remote worker terminal 7. When 3D point cloud data captured by the 3D sensor attached to the on-site worker's smart glasses 1 is transmitted to the remote worker terminal 7 in real time and the 3D point cloud data is shared with the remote worker, processing delays occur due to the large size of the point cloud data and the processing effects. Therefore, data cannot be shared smoothly. Therefore, the spatial information generation unit 38 of the data processing unit 35 links the RGB data and 3D data and performs data volume reduction processing. The data processing unit 35 transmits the processed photographic data 12 to the remote worker terminal 7. Specifically, the RGB camera and the 3D sensor are aligned in the smart glasses 1, so the positions of the RGB data and the 3D point cloud data are synchronized.

[0029] The spatial information generator 38 uses AI to recognize pipe sections that do not need to be replaced and pipe sections that should be newly installed, for example, as photographed by an RGB camera, in spatial information of the site captured by a 3D sensor. Then, the spatial information generator 38 uses point cloud data corresponding to those sections to recognize the pipes that do not need to be replaced as cylindrical, 6.8 m long and 20 cm in circumference. The spatial information generator 38 then retains only the information for the pipe sections that do not need to be replaced, namely, "cylindrical shape, length 6.8 m, circumference 20 cm, (Xn, Yn, Zn: information on the location of the pipe)," in the RGB data rather than the point cloud data. Similarly, the spatial information generator 38 retains only the information for the pipe sections that should be newly installed, namely, "cylindrical shape, length 2.2 m, circumference 20 cm, (Xn, Yn, Zn: information on the location of the pipe)."

[0030] In this way, the spatial information generator 38 classifies the on-site equipment from the RGB image. Then, the spatial information generator 38 calculates only data representing the shape and size of each piece of equipment from the point cloud information, and stores the calculated information as shape and size information of the equipment. This eliminates the need to transmit point cloud data captured every fps (frames per second) together with RGB data to the remote operator terminal 7 and analyze it at the remote operator terminal 7. This avoids a decrease in real-time performance due to an increase in data volume and processing delays.

[0031] Alternatively, the field worker may use hand gestures or eye tracking to specify only the equipment or materials he or she wants to communicate to the other party from the site view he or she is viewing through the smart glasses 1. The data processing unit 35 edits the data to be transmitted to the other party, with only the specified equipment or materials classified into classes. This also reduces the amount of data.

[0032] Furthermore, the on-site worker terminal 3 may limit the data to be sent to the remote worker terminal 7 to data of an area corresponding to the direction in which the remote worker's virtual worker avatar (also simply referred to as the avatar) is looking in the on-site space, or data of an area corresponding to a destination predicted from the avatar's movements.

[0033] When 3D point cloud data captured by a 3D sensor attached to the on-site worker's smart glasses 1 is transmitted in real time to the remote worker's terminal 7 and shared with the remote worker, processing delays occur due to the large size of the point cloud data and the processing time, preventing smooth data sharing. Therefore, the avatar motion analysis 39 in the data processing unit 35 predicts the direction the remote worker is looking in the on-site space and the movements he or she is about to make. The avatar motion analysis 39 then generates only data with a field of view corresponding to the prediction result in the spatial information generation 38. The avatar motion analysis 39 links the RGB data and the 3D data, reduces the data volume, and transmits the processed data to the remote worker's terminal 7. The effect of this process is explained below. When sharing the captured data 12 captured by the on-site worker with the remote worker, the motion sensor 5 can detect the remote worker's body movements, so the transmitted data can be narrowed down to the range the remote worker is looking at. By transmitting a narrowed range of data from the field worker's side to the remote worker terminal 7, the amount of data to be transmitted is reduced.

[0034] That is, the space information generator 38 excludes three-dimensional data of areas other than the area corresponding to the line of sight of the avatar and the area corresponding to the destination of the avatar from the data of the on-site environment transmitted as data representing the virtual space.

[0035] The remote operator terminal 7 may be a computer device that operates when a processor executes a program stored in memory. The remote operator terminal 7 includes a data collection unit 71, an information display unit 72, a data processing unit 75, a data transmission unit 78, and a data receiving unit 79. Each component of the remote operator terminal 7 may be software or a module that executes processing when a processor executes a program stored in memory. Alternatively, each component of the remote operator terminal 7 may be hardware such as a circuit or a chip. The information display unit 72 and the data processing unit 75 of the remote operator terminal 7 correspond to the object display control unit 120 of the work support device 100. The data collection unit 71 of the remote operator terminal 7 corresponds to the acquisition unit 130 of the work support device 100.

[0036] The data receiving unit 79 of the remote worker terminal 7 receives data from the field worker. The data collecting unit 71 acquires the received data. The acquired data is displayed on the smart glasses 4 as remote transmission data 74 on the information display unit 72. The remote worker can use the smart glasses 4 to share field worker data 41, which represents the status of the field worker's work site, with the field worker in real time.

[0037] If necessary, the data collection unit 71 on the remote worker side holds the real-time situation on the field worker side. The model generation 77 of the data processing unit 75 performs modeling based on the data on the field worker side. The model generation 77 creates a model, for example, "cylindrical shape, length 2.2 m, circumference 20 cm" from the shape and size information of the equipment divided in the RGB image by the field worker terminal 3. Then, the model generation 77 uses (Xn, Yn, Zn: position information of the pipe) to place the model in the actual 3D space.

[0038] The remote worker can simulate the movement of materials and other construction materials at the construction site and share the results with on-site workers. For example, if a new pipe needs to be installed at the work site, photographic data 12 taken at the site is modeled. The remote worker then grasps the new pipe to be installed in the immersive space of the smart glasses 4. The remote worker then moves the modeled pipe to the desired installation position to perform the simulation. A motion sensor 5 is attached to the remote worker's body, and a data collection unit 71 acquires motion data 52 obtained from the sensor 51. An avatar generation unit 76 of a data processing unit 75 then generates a virtual worker avatar for the remote worker based on the acquired data. This avatar is updated in real time in conjunction with the movements detected by the sensor 51.

[0039] When using motion sensors 5, sensors are attached to 6 to 11 points on the head, waist, both hands, both ankles, etc., and data is collected from each point. Even with full-body tracking, there is little concern about an increase in the amount of data.

[0040] By wearing the motion sensor 5, the remote worker can immerse himself as an avatar in the space where he is working on-site. The remote worker can act in the virtual space generated by the smart glasses 4 while feeling as if he is actually present at the site. Therefore, the remote worker can use his own avatar to grab the modeled pipes and move them to the installation position, and then install the pipes at the installation position in the virtual space.

[0041] The information display unit 72 displays the data processed by the data processing unit 75 on the smart glasses 4. The information display unit 72 displays the generated avatar of the remote worker as his / her own data 73. The information display unit 72 displays the work site data and data that models part of the work site data as remote transmission data 74. The remote worker can refer to real-time data that is updated over time on the smart glasses 4.

[0042] The data receiving unit 37 of the on-site worker terminal 3 receives its own data 73 via the data transmitting unit 78 of the remote worker terminal 7. The remote worker can refer to the data on the remote worker side on the smart glasses 1 that he or she wears.

[0043] Even if there is only one field worker at the site, if the field worker wears the smart glasses 1, the remote worker can exist as a virtual worker avatar at the site that the field worker sees. The field worker can get the feeling that he or she is working together with the remote worker at the site.

[0044] When the remote worker works in the remote location, the motion sensor 5 also operates. The remote worker's movement status is linked to the virtual worker avatar. By utilizing the virtual worker avatar, the remote worker can use gestures and hand movements to inform or teach the on-site worker how to perform the work at the site. Furthermore, the remote worker can actually perform the work using data that models the materials on the site, and then inform or teach the on-site worker how to perform the work. The on-site worker may actually perform the work using the materials on the site and inform the remote worker of the work status.

[0045] The virtual worker avatar can not only operate the modeled data, but also accompany the field worker in the scene where he or she is working. The virtual worker avatar can pose as if working together with the field worker, for example, holding a pipe together. This can be used to alleviate the field worker's sense of loneliness and give the field worker a sense of security.

[0046] Next, the haptic glove will be described. The sensor 21 of the haptic glove 2 acquires the amount of force actually used by the field worker when working as haptic data 22. The haptic data 22 is transmitted from the data transmission unit 36 ​​of the field worker terminal 3 to the remote worker terminal 7. The data reception unit 79 of the remote worker terminal 7 receives the haptic data 22. The haptic data 22 is input as haptic data 62 to the haptic glove 6 worn by the remote worker. The sensor 61 enables the remote worker to realistically feel the amount of force actually used by the field worker when working.

[0047] That is, the haptic glove 2 includes a sensor 21 that detects a tactile stimulus on the fingers of the first person. The haptic glove 6 presents the tactile stimulus detected by the sensor 21 when the field worker touches an object to the remote worker.

[0048] Conversely, the force applied by the remote operator using the modeled data is acquired by the sensor 61 of the haptic glove 6. The data transmission unit 78 of the remote operator terminal 7 transmits the acquired haptic data 62 to the on-site operator terminal 3. This makes it possible to share the force as described above.

[0049] In other words, the haptic glove 6 presents a contact stimulus to the remote worker when the remote worker touches a virtual object, and the haptic glove 2 presents the tactile stimulus presented here to the field worker.

[0050] 5 is a flowchart illustrating the operation of the site worker terminal 3. FIG. 6 is a flowchart illustrating the operation of the remote worker terminal 7.

[0051] Referring to FIG. 5, first, the field worker terminal 3 acquires data from the smart glasses 1 and the haptic gloves 2 (step S11). Next, the data receiving unit 37 receives data from the remote worker terminal 7 (step S12). Next, the data collecting unit 31 collects various data (step S13). Next, the data processing unit 35 performs data processing on the data collected in step S13 as needed (step S14). Step S14 includes steps S141 and S142.

[0052] In step S141, the spatial information generator 38 links the RGB data and the 3D data and performs data amount reduction processing.

[0053] In step S142, the avatar movement analysis 39 predicts and analyzes the direction the remote worker's virtual worker avatar is looking in the work site space and the movements he or she is about to make while the remote worker is immersed as a virtual worker avatar in the work site data space on the site worker's side.The avatar movement analysis 39 then performs processing to transmit work site data with a field of view corresponding to the predicted direction and movements to the remote worker.

[0054] Next, the information display unit 32 displays the data processed up to step S14 on the smart glasses 1 (step S15). Next, the data transmission unit 36 ​​transmits the data to be shared with the remote operator side to the remote operator terminal 7 (step S16).

[0055] Referring to FIG. 6, first, the remote operator terminal 7 acquires data from the smart glasses 4, the motion sensor 5, and the haptic glove 6 (step S21). Next, the data receiving unit 79 receives data from the field operator terminal 3 (step S22). Next, the data collecting unit 71 collects various data (step S23). Next, the data processing unit 75 performs data processing on the data collected in step S23 as needed (step S24). Step S24 includes steps S241 and S242.

[0056] In step S241, the model generation 77 generates a model using the data received in step S22.

[0057] In step S242, the avatar generator 76 generates a virtual worker avatar of the remote worker from the motion data 52 acquired in step S21. The avatar generator 76 updates the virtual worker avatar in real time based on the motion data 52.

[0058] Next, the information display unit 72 displays the data processed up to step S24 on the smart glasses 4 (step S25). Next, the data transmission unit 78 transmits the data collected or generated by the remote operator terminal 7 to the field worker terminal 3 (step S26).

[0059] Next, the effects of the second embodiment will be described. First, the first effect will be described. By connecting a field worker and a remote worker with smart glasses and superimposing a virtual avatar generated by motion capturing the movements of the remote worker on the world seen by the field worker, it becomes possible to work on-site with the virtual worker.

[0060] Next, we will explain the second effect. By displaying a virtual space that models the situation at the work site as seen by the on-site worker on the remote worker's side, the remote worker can simulate actually touching, holding, and moving equipment at the work site. This allows the remote worker, as a virtual worker avatar, to demonstrate how to perform the work to the on-site worker.

[0061] Finally, the third effect is that, because smart glasses are often hindered by data volume and processing time, the system performs processing to reduce the amount of data, enabling on-site workers and remote workers to share information in real time.

[0062] Modification of the second embodiment The field worker may wear a motion sensor 5. The avatar of the field worker may be projected onto the world seen by the smart glasses 1 or onto the world seen by the smart glasses 4.

[0063] The relationship between on-site workers and remote workers need not be 1:1, but may be a 1:many or many:many relationship.

[0064] Haptic gloves 2 and 6 may be optional.

[0065] Haptics may be used to provide stimuli from the remote worker to the on-site worker, allowing the remote worker to convey instructions such as putting pressure on the right hand.

[0066] In addition to the RGB camera and 3D sensor that come standard with smart glasses, information from other sensors can also be shared. Furthermore, if necessary, on-site workers can equip themselves with sensors necessary for the survey (such as an infrared camera or hyperspectral camera) and share the information from those sensors with remote workers.

[0067] The on-site worker terminal 3 and the remote worker terminal 7 may not have functions such as data processing, but the functions may be provided to the smart glasses 1 and 4, or a server such as a cloud that handles data between bases.

[0068] When sharing data from the on-site worker's side with the remote worker's side, the movement of the remote worker's virtual worker avatar can be utilized to thin out the shared data, and data from the haptic glove 6 can also be utilized. Even if the remote worker's body is not facing the environment or materials, if the haptic data shows the remote worker holding or touching the materials, the data on the environment or materials does not need to be thinned out. In other words, the 3D data excluded by the spatial information generation 38 does not need to include 3D data for the area corresponding to the virtual object the avatar is touching. The unthinned data is sent from the on-site worker terminal 3 to the remote worker terminal 7 and shared.

[0069] The work assistance device according to the present disclosure can be used not only to assist work at construction sites, but also for remote lessons in sports and other areas.

[0070] FIG. 7 is a block diagram showing an example of the hardware configuration of a work support device 100, a field worker terminal 3, and a remote worker terminal 7 (hereinafter referred to as the work support device 100, etc.). Referring to FIG. 7, the work support device 100, etc. includes a network interface 1001, a processor 1002, and a memory 1003. The network interface 1001 is used to communicate with other network node devices constituting a communication system. The network interface 1001 may be used for wireless communication. For example, the network interface 1001 may be used for wireless LAN communication defined in the IEEE 802.11 series or mobile communication defined in 3GPP (registered trademark) (3rd Generation Partnership Project). Alternatively, the network interface 1001 may include a network interface card (NIC) conforming to the IEEE 802.3 series.

[0071] The processor 1002 reads and executes software (computer programs) from the memory 1003 to perform the processes of steps S101 to S104 in Fig. 2, steps S11 to S16 in Fig. 5, and steps S21 to S26 in Fig. 6. The processor 1002 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1002 may include multiple processors.

[0072] The memory 1003 is configured by a combination of volatile memory and non-volatile memory. The memory 1003 may include storage located remotely from the processor 1002. In this case, the processor 1002 may access the memory 1003 via an I / O (Input / Output) interface (not shown).

[0073] 7, memory 1003 is used to store software modules. Processor 1002 can perform the processes of steps S101 to S104, S11 to S16, and S21 to S26 by reading and executing these software modules from memory 1003.

[0074] As described using FIG. 7, each of the processors included in the work assistance device 100, etc. in the above-described embodiments executes one or more programs including a group of instructions for causing a computer to execute the algorithm described using the drawings.

[0075] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0076] The technical ideas of the present disclosure are not limited to the above-described embodiments, and can be modified as appropriate within the scope of the gist of the present disclosure.

[0077] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0078] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0079] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a generation unit that generates object data representing a predetermined object in an on-site environment where the first person is present; an object display control unit that displays a virtual object based on the object data on a remote display device in a remote environment where a second person is present; an acquisition unit that acquires a detection result from a sensor that detects the body movement of the second person; an avatar display control unit that displays an avatar that operates based on the detection result on a site-side display device within the site environment; A work support device comprising: (Appendix 2) The object data includes data relating to the shape of the predetermined object and data relating to the position of the predetermined object. 2. The work support device according to claim 1. (Appendix 3) a spatial information generation unit that excludes three-dimensional data of an area different from an area corresponding to a line of sight of the avatar and an area corresponding to a destination of the avatar from the three-dimensional data of the on-site environment that is transmitted as data representing a virtual space in which the virtual object is placed; 3. The work support device according to claim 1 or 2. (Appendix 4) The excluded three-dimensional data does not include three-dimensional data of an area corresponding to a virtual object that the avatar is touching. 4. The work support device according to claim 3. (Appendix 5) a first haptic glove worn by the first person; a second haptic glove worn by the second person; and Equipped with The first haptic glove causes the first person to perceive a tactile stimulus that the second haptic glove provides to the second person when the avatar touches the virtual object. 3. The work support device according to claim 1 or 2. (Appendix 6) the first haptic glove includes a tactile sensor that detects a tactile stimulus to a finger of the first person; The second haptic glove causes the second person to perceive a tactile stimulus detected by the tactile sensor when the first person touches the predetermined object. 6. The work support device according to claim 5. (Appendix 7) the predetermined object is a pipe having the cylindrical shape, The predetermined object is attached to a predetermined position within the local environment by the first person. 3. The work support device according to claim 2. (Appendix 8) the on-site display device is smart glasses worn by the first person, The remote display device is a smart glass worn by the second person. 3. The work support device according to claim 1 or 2. (Appendix 9) generating object data representing a predetermined object within a scene environment in which the first person is present; displaying a virtual object based on the object data on a remote display device in a remote environment where a second person is present; acquiring a detection result from a sensor that detects the body movement of the second person; An avatar that operates based on the detection result is displayed on a site-side display device within the site environment. Work support method. (Appendix 10) On the computer, generating object data representing a predetermined object within a scene environment in which the first person is present; a process of displaying a virtual object based on the object data on a remote display device in a remote environment where a second person is present; A process of acquiring a detection result by a sensor that detects the body movement of the second person; a process of displaying an avatar that operates based on the detection result on a site-side display device within the site environment; A program that executes the following.

[0080] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 8 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 9 and 10 in the same dependency relationship as Supplementary Notes 2 to 8. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0081] 100, 100a Work support device 110 Generation part 120 Object display control unit 130 Acquisition Department 140 Avatar display control unit W1 Field worker W2 Remote Worker P piping A avatar 1. Smart Glasses 11, 21, 51, 61 sensors 12 Shooting data 13, 42 Remote worker data 2 Haptic Gloves 22, 62 Haptic data 3. Field worker terminal 31, 71 Data Collection Department 32, 72 Information display section 33, 73 own data 34, 74 Remote transmission data 35, 75 Data processing section 36, 78 Data transmission unit 37, 79 Data receiving section 38 Spatial information generation 39 Avatar movement analysis 4. Smart Glasses 41 Field worker data 5. Motion Sensor 52 Motion Data 6 Haptic Gloves 7 Remote worker terminal 74 Remote Transmission Data 76 Avatar Generation 77 Model Generation

Claims

1. a generation unit that generates object data representing a predetermined object in a scene environment where the first person is present; an object display control unit that displays a virtual object based on the object data on a remote display device in a remote environment where a second person is present; an acquisition unit that acquires a detection result from a sensor that detects a body movement of the second person; an avatar display control unit that displays an avatar that operates based on the detection result on a site-side display device within the site environment; A work support device comprising:

2. The object data includes data relating to the shape of the predetermined object and data relating to the position of the predetermined object. The work support device according to claim 1 .

3. a spatial information generation unit that excludes three-dimensional data of an area different from an area corresponding to a line of sight of the avatar and an area corresponding to a destination of the avatar from the three-dimensional data of the on-site environment that is transmitted as data representing a virtual space in which the virtual object is placed; The work support device according to claim 1 or 2.

4. The excluded three-dimensional data does not include three-dimensional data of an area corresponding to a virtual object that the avatar is touching. The work support device according to claim 3 .

5. a first haptic glove worn by the first person; a second haptic glove worn by the second person; and Equipped with The first haptic glove causes the first person to perceive a tactile stimulus that the second haptic glove provides to the second person when the avatar touches the virtual object. The work support device according to claim 1 or 2.

6. the first haptic glove includes a tactile sensor that detects a tactile stimulus to a finger of the first person; The second haptic glove causes the second person to perceive a tactile stimulus detected by the tactile sensor when the first person touches the predetermined object. The work support device according to claim 5.

7. the predetermined object is a pipe having the cylindrical shape, The predetermined object is attached to a predetermined position within the local environment by the first person. The work support device according to claim 2 .

8. the on-site display device is smart glasses worn by the first person, The remote display device is a smart glass worn by the second person. The work support device according to claim 1 or 2.

9. generating object data representing a predetermined object within a scene environment in which the first person is present; displaying a virtual object based on the object data on a remote display device in a remote environment where a second person is present; acquiring a detection result from a sensor that detects a body movement of the second person; An avatar that operates based on the detection result is displayed on a site-side display device within the site environment. Work support method.

10. On the computer, generating object data representing a predetermined object within a scene environment in which the first person is present; a process of displaying a virtual object based on the object data on a remote display device in a remote environment where a second person is present; A process of acquiring a detection result by a sensor that detects a body movement of the second person; a process of displaying an avatar that operates based on the detection result on a site-side display device within the site environment; A program that executes the following.

Images