Support system, support device, support method, and program

Abstract

To efficiently transmit a work content including a facility operation in a remote support system.SOLUTION: Provided is a support system that includes a support-side device and a work-side device. The support-side device includes: a support-side display unit that displays a virtual model imitating a facility; a detection unit that detects a body motion of a supporter to the virtual model; and a transmission unit that transmits information on the body motion to the virtual model. The work-side device includes a work-side display unit that displays a virtualized image representing the body motion, based on the received body motion information. Transmission of the virtualized body motion of the supporter allows a work content including a facility operation to be efficiently transmitted.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a support system, a support side device, a support method, and a program.

[0002] There is known a support system in which a video image observed by a worker is transmitted to a supporter in a remote location, and the supporter gives instructions to the worker using a slider while observing the video image (see, for example, Patent Document 1). Also, Patent Document 2 is a related technology. Patent Document 1: Japanese Patent Application Laid-Open No. 2006-209664 Patent Document 2: JP 2012-18605 A Summary of the Invention [Problem to be solved by the invention]

[0003] In a support system, it is desirable to be able to communicate the details of work that involves operating equipment. However, it is difficult to communicate the details of work that involves operating equipment simply by sharing a slideshow between the supporter and the worker. [Means for solving the problem]

[0004] In order to solve the above problems, a first aspect of the present invention provides a support system including a support-side device and a work-side device. The support-side device may include a support-side display unit. The support-side display unit may display a virtual model that imitates equipment. The support-side device may include a detection unit. The detection unit may detect a body movement of a supporter relative to the virtual model. The support-side device may include a transmission unit. The transmission unit may transmit information about the body movement relative to the virtual model. The work-side device may include a work-side display unit. The work-side device may display a virtualized image showing the body movement based on the received information about the body movement.

[0005] The body movement detected by the detection unit may include position information about at least a part of the supporter's body in three-dimensional space.

[0006] A reference point marker may be provided at a predetermined position on the equipment. The virtual model may be associated with a reference point provided at a position corresponding to the reference point marker and reference coordinates, which are three-dimensional coordinates based on the reference point. The body movement detected by the detection unit may include position information for at least a part of the supporter's body in the reference coordinates. The work-side device may include a marker detection unit. The marker detection unit may detect a reference point marker on the equipment. The work-side device may include an adjustment unit. The adjustment unit may adjust the position at which the virtualized image is to be superimposed on the image of the equipment based on the detection result of the reference point marker and the position information included in the received body movement information.

[0007] The support side device may further include a selection unit, which may select a virtual model corresponding to the facility from among the plurality of virtual models.

[0008] The selection unit may select a virtual model corresponding to the facility from among the plurality of virtual models based on an instruction from the supporter.

[0009] An identification image for identifying the equipment may be displayed on the equipment. The work-side device may include a reading unit. The reading unit may read the identification image. The work-side device may further include an identification information transmitting unit. The identification information transmitting unit may transmit identification information obtained by reading the identification image with the reading unit to the support-side device. The selecting unit may select a virtual model corresponding to the equipment from among the plurality of virtual models based on the identification information received from the work-side device.

[0010] The work-side device may further include an imaging unit. The imaging unit may capture an image of the equipment to obtain an equipment image. The work-side device may further include an equipment image transmission unit. The equipment image transmission unit may transmit the equipment image to the support-side device. The support-side device may include a model generation unit. The model generation unit may generate a virtual model based on the equipment image received from the work-side device.

[0011] The work-side device may further include an imaging unit. The imaging unit may capture an image of the equipment to obtain an equipment image. The work-side device may further include a correction information generation unit. The correction information generation unit may generate correction information based on the equipment image. The work-side device may further include a correction information transmission unit. The correction information transmission unit may transmit the correction information to the support-side device. The support-side device may include a model correction unit. The model correction unit may correct the virtual model based on the correction information received from the work-side device. The correction information generation unit may generate correction information by comparing the virtual model received from the support-side device with the equipment image.

[0012] In a second aspect of the present invention, there is provided a support method for supporting work using a support-side device and a work-side device capable of communicating with the support-side device. The support method may include a step of displaying a virtual model imitating equipment in the support-side device. The support method may include a step of detecting, in the support-side device, a body movement of a supporter corresponding to the virtual model. The support method may include a step of transmitting, in the support-side device, information on the body movement of the supporter relative to the virtual model. The support method may include a step of receiving, by the work-side device, information on the body movement. The support method may include a step of displaying, in the work-side device, a virtualized image showing the body movement based on the received information on the body movement.

[0013] In a third aspect of the present invention, there is provided a support-side device. The support-side device may be communicably connected via a network to a working-side device at a work site where equipment is installed. The support-side device may include a support-side display unit. The support-side display unit may display a virtual model that imitates the equipment. The support-side device may include a detection unit. The detection unit may detect a body movement of a supporter according to the virtual model. The support-side device may include a transmission unit. The transmission unit may transmit information about the body movement of the supporter relative to the virtual model to the working-side device.

[0014] In a fourth aspect of the present invention, there is provided a program for causing a computer to function as the above-mentioned support side device.

[0015] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a diagram showing an overview of a remote support system 1 according to a first embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of a remote support system 1 according to a first embodiment of the present invention. [Figure 3] FIG. 2 is a schematic diagram showing an example of a virtual model 7 in the support side device. [Figure 4] FIG. 2 is a schematic diagram showing an example of a composite image 10 in the work-side device. [Figure 5] FIG. 10 is a diagram showing an example of posture data representing the motion of a supporter. [Figure 6] FIG. 10 is a diagram showing an example of point cloud data representing the actions of a supporter. [Figure 7] 10 is a flowchart showing an example of processing content of a support-side device in the remote support method of the first embodiment. [Figure 8] 8 is a flowchart showing an example of the processing content in step S50 of FIG. 7. [Figure 9] 6 is a flowchart showing an example of processing content of a work-side apparatus in the remote support method of the first embodiment. [Figure 10] 10 is a flowchart showing an example of the processing content in step S80 of FIG. 9. [Figure 11] FIG. 2 is a diagram showing another example of the remote support system 1 in the first embodiment of the present invention. [Figure 12] FIG. 10 is a schematic diagram showing an example of a virtual model 7 in a support side device in a second embodiment of the present invention. [Figure 13] FIG. 10 is a schematic diagram showing an example of a composite image 80 in the work-side apparatus according to the second embodiment of the present invention. [Figure 14] FIG. 10 is a diagram illustrating an example of a remote support system 1 according to a second embodiment of the present invention. [Figure 15] FIG. 10 is a diagram showing another example of the remote support system 1 according to the second embodiment of the present invention. [Figure 16] 10 is a flowchart showing an example of processing contents of a support-side device in a remote support method according to a second embodiment. [Figure 17] 17 is a flowchart showing an example of the processing content in step S160 of FIG. 16. [Figure 18] 10 is a flowchart showing an example of processing content of a work-side apparatus in a remote support method according to a second embodiment. [Figure 19] 19 is a flowchart showing an example of the processing content in step S310 of FIG. 18. [Figure 20] FIG. 10 is a schematic diagram showing an example of a composite image 90 in the work-side apparatus according to the third embodiment of the present invention. [Figure 21] 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied, in whole or in part. DETAILED DESCRIPTION OF THE INVENTION

[0017] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0018] FIG. 1 is a diagram showing an overview of a remote support system 1 according to a first embodiment of the present invention. The remote support system 1 includes a support side device 2 and a working side device 3. The support side device 2 and the working side device 3 are communicably connected to each other via a communication network 4. In one example, the support side device 2 is a head-mounted display (HMD) type virtual reality wearable computer terminal. The support side device 2 displays a virtual model 7 that simulates a facility 5 to the supporter. The virtual model 7 may be a 3D hologram image.

[0019] The supporter looks at the virtual model 7 and actually moves his / her fingers, hands, etc. to operate it in the virtual space, demonstrating an example to the worker. The support-side device 2 detects the supporter's physical movements relative to the virtual model 7 using motion capture technology. Then, the support-side device 2 transmits information about the physical movements relative to the virtual model 7 to the work-side device 3 via the communication network 4.

[0020] The work-side device 3 displays a body movement image, which is a virtualized image showing the body movement based on the received body movement information. The body movement image may be a video that reproduces the virtualized movements of the supporter's fingers, hands, etc. In one example, the work-side device 3 is an augmented reality wearable computer terminal of a head-mounted display (HMD) type. The worker receives support from the image, etc. displayed on the work-side device 3 and performs operations on the equipment 5.

[0021] A predetermined position P in the facility 5 M0 The reference point marker 6 may be provided at the position P M0 Based on the marker coordinates (X M , Y M , Z M ) may be set on the virtual model 7. M0 A reference point 8 may be set at a position P0 corresponding to the reference point P0. Furthermore, reference coordinates (X0, Y0, Z0) that are three-dimensional coordinates at the reference point P0 may be set in the virtual model 7. The detected body movement may include position information about at least a part of the supporter's body relative to the reference coordinates (X0, Y0, Z0). In this example, in the coordinate system of the support-side device 2, the direction from the supporter toward the virtual model 7 is defined as the Z-axis direction, and a plane perpendicular to the Z-axis direction is defined as the XY plane. In the XY plane, the X-axis direction and the Y-axis direction are defined as directions perpendicular to each other. Similarly, in the coordinate system of the work-side device 3, the direction from the worker toward the equipment is defined as the Z-axis direction, and a plane perpendicular to the Z-axis direction is defined as the XY plane. In the XY plane, the X-axis direction and the Y-axis direction are defined as directions perpendicular to each other.

[0022] FIG. 2 is a diagram showing an example of a remote support system 1 according to the first embodiment of the present invention. The support-side device 2 of this example includes a communication control unit 101, a detection unit 102, and a support-side display unit 103. The support-side display unit 103 displays a virtual model 7 that resembles equipment 5. The support-side display unit 103 may have a holographic lens display. The detection unit 102 detects the body movements of the supporter relative to the virtual model 7. The detection unit 102 acquires motion capture data indicating the supporter's movements from a motion capture device 106. The communication control unit 101 functions as a transmission unit that transmits information about the body movements relative to the virtual model 7 (equipment model). As a result, the support-side device 2 can detect the supporter's relative movements relative to the virtual model 7 that resembles equipment 5 in the virtual space and send the information to the working-side device 3.

[0023] The support-side device 2 in this example may include a virtual model storage unit 104, a selection unit 105, a screen control unit 110, a sensor 112, a microphone 114, a speaker 116, and an audio control unit 118. The virtual model storage unit 104 is a database that stores information about virtual models 7 in advance. The virtual model storage unit 104 may store multiple types of virtual models 7 corresponding to multiple types of equipment 5. However, the virtual model storage unit 104 does not have to be included in the support-side device 2. In this case, the virtual model 7 is downloaded from an external device. The selection unit 105 selects the virtual model 7 corresponding to the equipment 5 from the multiple virtual models. The selection unit 105 may select the virtual model 7 corresponding to the equipment 5 from the multiple virtual models based on an instruction from the supporter. Alternatively, the selection unit 105 may select the virtual model 7 corresponding to the equipment 5 from the multiple virtual models based on identification information received from the work-side device 3.

[0024] The screen control unit 110 may realize virtual reality by integrating the virtual model 7, which is a 3D hologram image, with motion capture data. The screen control unit 110 may combine the hologram image with the physical environment recognized by the imaging unit and the visible light camera for tracking. The screen control unit 110 may receive inertial measurement information from the sensor 112, combine it with data from the imaging unit and the visible light camera for tracking that recognize the environment, and display a hologram image on the support-side display unit 103, which is a holographic lens display with a transparent lens. The microphone 114, speaker 116, and audio control unit 118 enable conversation between the supporter and the worker via the working-side device 3.

[0025] The sensor 112 may include a depth sensor (distance sensor), an imaging unit (camera), an inertial measurement unit, multiple visible light cameras for tracking, and an environment recognition unit. The visible light camera for tracking detects head movement. Two infrared cameras for eye tracking detect eye movement. The depth sensor calculates the distance to the object, its shape, etc. by irradiating an object with a laser from a light source and measuring the reflected light. The inertial measurement unit may include an accelerometer, a gyroscope, and a magnetometer. The inertial measurement unit detects three-dimensional inertial motion (translational motion and rotational motion in three orthogonal axial directions). The accelerometer may detect translational motion, and the gyroscope may detect rotational motion.

[0026] The work-side apparatus 3 includes a communication control unit 201 and a work-side display unit 202 . The work-side apparatus 3 may include an imaging unit 204 , a screen control unit 210 , a sensor 212 , a microphone 214 , a speaker 216 , and an audio control unit 218 .

[0027] The communication control unit 201 functions as a receiving unit that receives information about physical movements from the support-side device 2. The working-side display unit 202 displays a virtualized image (body movement image) showing the physical movement based on the received information about the physical movement. In particular, the working-side display unit 202 may display the body movement image superimposed on an equipment image obtained by photographing the equipment 5. The imaging unit 204 may be a camera. The imaging unit 204 may generate the equipment image by photographing the equipment 5. The equipment image may include not only the equipment 5 but also a background image of the surrounding environment of the equipment 5.

[0028] The sensor 212 may be similar to the sensor 112 described above. The sensor 212 may include a depth sensor (distance sensor), an inertial measurement unit, multiple visible light cameras for tracking, and an environment recognition unit. The visible light camera for tracking detects head movement. Two infrared cameras for eye tracking detect eye movement. The depth sensor calculates the distance to the object, its shape, etc. by irradiating an object with a laser from a light source and measuring the reflected light. The inertial measurement unit may include an accelerometer, a gyroscope, and a magnetometer. The inertial measurement unit detects three-dimensional inertial motion (translational motion and rotational motion in three orthogonal axial directions). The accelerometer may detect translational motion, and the gyroscope may detect rotational motion.

[0029] The microphone 214, speaker 216, and audio control unit 218 enable conversation between the supporter and the worker via the support-side device 2. The screen control unit 210 has various functions such as a marker detection unit 241, an adjustment unit 242, and a reading unit 243. These functions will be described later.

[0030] Fig. 3 is a schematic diagram showing an example of the virtual model 7 in the support-side apparatus 2. Fig. 4 is a schematic diagram showing an example of the composite image 10 in the work-side apparatus 3. Note that Figs. 3 and 4 show coordinates for explanation purposes. The actual virtual model 7 and composite image 10 do not need to show coordinates.

[0031] 4 may be an image obtained by superimposing an equipment image 11 obtained by capturing an image of the equipment 5 using the imaging unit 204 on a body movement image 12, which is a virtual image showing a body movement. As shown in FIG. 4, in the equipment 5, a predetermined position P M0 A reference point marker 6 is provided at the position indicated by the arrow 12. The reference point marker 6 may have an identification image 9 for identifying the equipment 5. In one example, the identification image 9 may be a barcode image. Unlike the case of FIG. 4, the reference point marker 6 and the identification image 9 may be provided separately. In the example shown in FIG. 4, the body movement image 12 is a virtual image of the movement of the supporter's hands and fingers.

[0032] The screen control unit 210 shown in FIG. 2 may function as a marker detection unit 241 that detects the reference point marker 6 on the equipment 5. The screen control unit 210 may also function as an adjustment unit 242. The adjustment unit 242 may adjust the position at which the body movement image 12 is superimposed on the equipment image 11 based on the detection result of the reference point marker 6 and the position information included in the received body movement information. The screen control unit 210 may also function as a reading unit 243 that reads the identification image 9. In this case, the communication control unit 201 also functions as an identification information transmission unit that transmits the identification information obtained by reading the identification image 9 by the reading unit 243 to the support-side device 2. The selection unit 105 in the support-side device 2 selects a virtual model 7 corresponding to the equipment 5 from among multiple virtual models based on the identification information received from the work-side device 3.

[0033] The information on the body motion detected by the detection unit 102 may include position information on at least a part of the supporter's body in three-dimensional space. The position information may include body motion point coordinate values ​​(X Ri , Y Ri , Z Ri ) may include the body motion point coordinate value (X Ri , Y Ri , Z Ri ) are the coordinates (X 0 , Y 0 , Z 0 ) of each body motion point 24 in the body when the reference point (X 0 , Y 0 , Z 0 ) is used as the reference in the reference coordinates (X, Y, Z). i , Y i, Z i ) in Fig. 3 and Fig. 4, the direction perpendicular to the paper surface is the Z-axis direction. Ri , Y Ri , Z Ri ) is X Ri =X i -X0, Y Ri =Y i -Y0, Z Ri =Z i -It is expressed as Z0.

[0034] The screen control unit 210 determines the position where the body movement image 12 is to be superimposed on the equipment image 11 based on the detection result of the reference point marker 6 (marker coordinates X M , Y M , Z M ) and the coordinate value (X Ri , Y Ri , Z Ri ) and the position (X M +X Ri , Y M +Y Ri , Z M +Z Ri ) may be adjusted.

[0035] Fig. 5 is a diagram showing an example of posture data representing the motion of a support worker. Fig. 6 is a diagram showing an example of point cloud data representing the motion of a support worker. Motion capture data obtained by reading the motion of a worker using motion capture technology may be posture data as shown in Fig. 5 or point cloud data as shown in Fig. 6.

[0036] The posture data expresses a movement based on the recognition results of the positions of parts (shown as black circles in the figure) that are basic elements of the movement, such as the joints of the fingers. The posture data may be expressed by the three-dimensional coordinates of the positions of multiple parts that are basic elements. The points that are basic elements of the movement are called body movement points 24. The position information is the body movement coordinate value (X Ri , Y Ri , Z Ri) (i = 1, 2, 3, n).

[0037] The point cloud data shown in FIG. 6 indicates the behavior of a plurality of extracted points (shown as points in the figure). Therefore, each extracted point may be set as a body motion point 24. The point cloud data includes body motion coordinate values ​​(X Ri , Y Ri , Z Ri ) (i=1, 2, 3...N) may be used to represent location information.

[0038] FIG. 7 is a flowchart showing an example of processing details of the support-side device 2 in the remote support method of the first embodiment. In the remote support method, work is supported by the support-side device 2 and the work-side device 3. The selection unit 105 selects a field worker based on instructions from a supporter (step S10). The supporter can instruct the field worker using a pointing device or the like. The voice control unit 118 starts a voice call between the selected field worker and the supporter (step S20). The selection unit 105 selects a virtual model 7 corresponding to the facility 5 to be supported from among multiple virtual models stored in the virtual model storage unit 104 (step S30).

[0039] The selection unit 105 may select a virtual model based on an instruction from a supporter. The selection unit 105 can specify a virtual model based on a signal from a pointing device or the like. The selection unit 105 may select a virtual model 7 corresponding to the equipment from among a plurality of virtual models based on identification information received from the work-side apparatus 3.

[0040] The support-side display unit 103 displays a virtual model 7 that resembles the equipment 5 (step S40). The supporter operates the displayed virtual model 7 in the virtual space. The detection unit 102 detects the supporter's body movement relative to the virtual model 7 and transmits information on the detected body movement to the work-side device 3 (step S50). The voice control unit 118 ends the voice call (step S60). Note that the worker may communicate identification information of the equipment to the supporter via the voice call. However, the voice call processing (steps S20 and S60) and the on-site worker selection processing (step S10) may be omitted.

[0041] FIG. 8 is a flowchart showing an example of the processing content in step S50 of FIG. 7. The communication control unit 101 transmits a start code (step S51). The start code is information indicating the start of processing. Until the operation is completed (step S52: NO), the total number (n) of body motion points 24 is acquired (step S54) each time a transmission period is reached (step S53: YES). The initial value is set to i=n. Next, unless i=0 (step S55: NO), the detection unit 102 uses the motion capture data to acquire one body motion point 24 (X i , Y i , Z i ) in the body motion coordinates (X Ri =X i -X0, Y Ri =Y i -Y0, Z Ri =Z i −Z0) (step S56). The detection unit 102 sets i to i−1 (step S57). i is decremented from the initial value n to n−1. Until i=0 (step S55: NO), the detection unit 102 repeats the processes from step S55 to step S57.

[0042] As a result of steps S54 to S57, the detection unit 102 calculates the body motion coordinates (X Ri =X i -X0, Y Ri =Y i -Y0, ZRi =Z i -Z0) can be acquired. When i=0 (step S55: YES), acquisition of body motion coordinates for all (n) body motion points is completed. The communication control unit 101 acquires information on the total number (n) of body motion points 24 and the body motion coordinates (X Ri =X i -X0, Y Ri =Y i -Y0, Z Ri =Z i -Z0) is transmitted as the location information (step S58).

[0043] When an instruction to end the operation is given (step S52: YES), the communication control unit 101 transmits an end code to the work-side apparatus 3 (step S59), and then the process returns.

[0044] 9 is a flowchart showing an example of processing details of the working-side apparatus 3 in the remote support method of the first embodiment. The voice control unit 218 starts a voice call between the supporter and the worker (step S70). The communication control unit 201 receives body movement information from the support-side apparatus 2, and the working-side display unit 202 displays a body movement image, which is a virtualized image showing the body movement based on the received body movement information (step S80). When the support processing is completed, the voice control unit 218 ends the voice call (step S90).

[0045] 10 is a flowchart showing an example of the processing content in step S80 of FIG. 9. The communication control unit 201 functions as a receiving unit that receives various information from the support side device 2. When the communication control unit 201 receives a start code from the support side device 2 (step S101: YES), it starts the imaging unit 204 (step S102). The imaging unit 204 takes an image of the surrounding environment including the equipment 5 and generates image data. The screen control unit 210 acquires the generated image data as the equipment image 11.

[0046] If the communication control unit 201 receives an end code (step S103: Yes), it determines that the assistance process has ended and stops the image capture unit 204 (step S104). Then, the process returns. Until the communication control unit 201 receives an end code (step S103: NO), when the communication control unit 201 receives information on body movements (step S105: YES), the screen control unit 210 calculates the total number (n) of body movement points 24 and the body movement coordinates (X, Y, Z) of each body movement point 24 (i=1, 2, . . . n). Ri =X i -X0, Y Ri =Y i -Y0, Z Ri =Z i −Z0) is acquired (step S106). The initial value may be set to i=n.

[0047] The screen control unit 210 also functions as a marker detection unit 241 that detects the reference point marker 6. The screen control unit 210 outputs the marker coordinates (X M , Y M , Z M ) (Step S107). Next, unless i=0 (Step S108: NO), the screen control unit 210 draws the body movement image 12 based on the body movement information. The screen control unit 210 generates a composite image 10 by superimposing the equipment image 11 and the body movement image 12. The work-side display unit 202 displays the composite image 12 (Step S109). The screen control unit 210 obtains the marker coordinates (X M , Y M , Z M ) and the body movement coordinates (X Ri =X i -X0, Y Ri =Y i -Y0, Z Ri =Z i -Z0), the position where the body movement image 12 is synthesized with the equipment image 11 is determined as (X M +X Ri , Y M +Y Ri , Z M +Z Ri) to adjust the

[0048] Next, the screen control unit 210 sets i to i-1 (step S110). i is decremented from the initial value n to n-1. Until i becomes 0 (step S108: NO), the detection unit 102 repeats the processes of steps S109 and S110. As a result, the screen control unit 210 can display the virtualized body movement image 12 superimposed on the equipment image 11 for each of the body movement points 24 where i=1, 2, . . . n.

[0049] FIG. 11 is a diagram showing another example of the remote support system 1 according to the first embodiment of the present invention. In the remote support system 1 in FIG. 11, the working-side apparatus 3 includes a correction information generation unit 220. In addition, in the working-side apparatus 3, the communication control unit 201 may function as an equipment image transmission unit that transmits an equipment image 11 obtained by capturing an image of the equipment to the support-side apparatus 2. Alternatively, the communication control unit 201 may function as a correction information transmission unit that transmits correction information generated by the correction information generation unit 220 to the support-side apparatus 2. The correction information generation unit 220 may generate correction information based on the equipment image 11. In one example, the working-side apparatus 3 receives a virtual model 7 from the support-side apparatus 2 via the communication control unit 201. The correction information generation unit 220 may compare the virtual model 7 with the equipment image 11 to create correction information. In addition, the correction information may be an equipment image 11 with a reduced resolution to reduce the amount of data.

[0050] The support-side device 2 may include a model generation unit 120. The model generation unit 120 generates a virtual model 7 based on an equipment image 11 received from the work-side device 3. The model generation unit 120 can detect a dynamic state of the equipment 5 based on an image captured by the work-side device 3. The model generation unit 120 can generate a virtual model that imitates the equipment 5 according to the dynamic state of the equipment 5.

[0051] The support-side device 2 may include a model corrector 122. The model corrector 122 may correct the virtual model 7 based on correction information received from the working-side device 3. In one example, the correction information may be a two-dimensional image of the equipment 5 with a low image quality. In one example, the virtual model 7 has possible states of each part set. The model generator 120 may detect the state of each part of the equipment 5 based on the correction information, and correct the virtual model 7 by changing the state of each part based on the detection result.

[0052] According to the remote support system 1 of the first embodiment of the present invention, the body movements of the supporter are virtualized and used, which makes it possible to intuitively convey the operation content, compared to when virtualizing sliders or the like. This improves the work efficiency of the on-site worker and the work supporter. Furthermore, unlike simple chromakey compositing, there is no need to have the supporter stand and operate against a background of a specific color.

[0053] Fig. 12 is a schematic diagram showing an example of a virtual model 7 in the support-side apparatus 2 in the second embodiment of the present invention. Fig. 13 is a schematic diagram showing an example of a composite image 80 in the work-side apparatus 3 in the second embodiment of the present invention.

[0054] As shown in FIG. 12, in this embodiment, the virtual model 7 may include a door provided with a handle 25. The virtual model 7 includes a plurality of operation parts, namely, an operation part 21 and an operation part 22. The assistant changes the state of the operation part 22 by specifying the operation part 22 in the virtual space. In the example of FIG. 12, the operation part 22 is a door, and the assistant can perform an assistance operation of rotating the operation part 22 to a specific rotation position in the virtual space. Operation information indicating the content of the assistance operation at the operation part 22 of the equipment 5 is transmitted from the assistance-side device 2 to the work-side device 3. The operation information includes information on the position 23 (X1, Y1, Z1) of the operation part 22 and the operation part rotation angle (R X , R Y , R ZThe information about the position 23 (X1, Y1, Z1) of the operation part 22 may include the relative coordinates (X R , Y R , Z R ) can be the relative coordinates of the operation part (X R , Y R , Z R ) is X R =X1-X0, Y R = Y1 - Y0, Z R =Z1-Z0.

[0055] As shown in FIG. 13, the work-side device 3 has a work-side display unit 202 that displays a composite image 80 obtained by combining an equipment image 11 obtained by capturing an image of the equipment 5 using the imaging unit 204 and an operation state image 30 showing the state of the equipment 5 after the assist operation. In this example, the composite image 80 may include an image of a door including a handle 33. The work-side device 3 receives, as operation information, information on the coordinates (X1, Y1, Z1) of the operation part 22 and the operation part rotation angle (R x , R y , R z ) and receives the operation information. The work-side device 3 generates an operation status image 30 based on the operation information. Then, the work-side display unit 202 displays a composite image 80 generated by overlaying the operation status image 30 on the equipment image 11. The operation status image 30 includes images 31 and 32 of multiple operation parts. The image 32 of the operation part can take multiple states. In this example, the image 32 of the operation part indicates the state of opening or closing of the door. The work-side display unit 202 may display a message 35 and a symbol 36 relating to the content of the assistance operation together with the operation status image 30. This makes it easier for the worker to understand the operation status.

[0056] 14 is a diagram showing an example of a remote support system 1 according to the second embodiment of the present invention. The remote support system 1 includes a support side device 2 and a working side device 3. The support side device 2 and the working side device 3 may be communicably connected via a communication network 4. The support side device 2 may be a head-mounted display (HMD) type virtual reality wearable computer terminal.

[0057] The support-side device 2 in this example includes a communication control unit 131, an operation part detection unit 130, and a support-side display unit 103. In this example, the support-side display unit 103 displays a virtual model 7 as shown in FIG. 12. The support-side display unit 103 may have a holographic lens display. The operation part detection unit 130 detects the content of the support operation at the operation part of the equipment 5. In the example shown in FIG. 12, the operation part detection unit 130 detects the content of the support operation, which is to rotate the door at the operation part 22 of the equipment 5 to the operation part rotation angle.

[0058] The operating part detection unit 130 retrieves motion capture data indicating the content of the assisting operation from the motion capture device 106. The operating part detection unit 130 may identify the operating part designated by the assistant by determining a collision between the virtual model 7 and a body region corresponding to the assistant's body, or between the virtual model 7 and an instruction image (for example, a slider, a pointer, etc.) operated by the assistant. The operating part detection unit 130 may detect the body motion of the assistant relative to the virtual model 7, similar to the detection unit 102 in the first embodiment. In this case, the operating part detection unit 130 may detect the content of the assisting operation by detecting the body motion using the body motion detection technology in the first embodiment.

[0059] The communication control unit 131 functions as an operation information transmission unit that transmits operation information indicating the content of the assisting operation. As a result, the support-side device 2 can detect the operation content at the operation part of the virtual model 7 that simulates the facility 5 and transmit it to the work-side device 3. The communication control unit 131 receives, as the operation information, information on the coordinates (X1, Y1, Z1) of the operation part 22 and the rotation angle (R X , R Y , R Z ) can be transmitted to the work-side device 3.

[0060] The support side device 2 of this example may include a virtual model storage unit 104, a selection unit 105, a screen control unit 110, a sensor 112, a microphone 114, a speaker 116, and an audio control unit 118. These components may be similar to the components shown in Fig. 2. Therefore, repeated explanations will be omitted.

[0061] The work-side device 3 includes a communication control unit 231 and a work-side display unit 202. The work-side device 3 may include an imaging unit 204, a screen control unit 210, a sensor 212, a microphone 214, a speaker 216, and an audio control unit 218. The communication control unit 231 functions as a receiving unit that receives operation information. The screen control unit 210 superimposes an operation status image 30 indicating the state of the equipment 5 after the assisting operation on the equipment image 11 to create a composite image. The work-side display unit 202 displays the operation status image 30 indicating the state of the equipment 5 after the assisting operation, superimposed on the image of the equipment 5. Specifically, the work-side display unit 202 displays a composite image 80. The work-side display unit 202 may display a message 35 and a symbol 36 regarding the content of the assisting operation together with the operation status image 30.

[0062] The work-side apparatus 3 may include an image generation unit 230 and a work-side virtual model storage unit 232. The work-side virtual model storage unit 232 stores a work-side virtual model that simulates the equipment 5. The image generation unit 230 generates an operation status image 30 that shows the state of the equipment after the assist operation, according to the assistance content specified for each operation portion 21, 22. The image generation unit 230 may function as a deformation unit that deforms the work-side virtual model based on the operation information. The work-side display unit 202 may display the deformed work-side virtual model as the operation status image 30. The work-side display unit 202 may change the display of the body region or the portion where the instruction image comes into contact with the virtual model 7. In the example shown in FIG. 13 , an image 32 of the operation portion where the assist operation is being performed is displayed with a dotted line.

[0063] Except for these points, the imaging unit 204, screen control unit 210, sensor 212, microphone 214, speaker 216, and audio control unit 218 have the same configuration as shown in Fig. 2. Therefore, repeated description will be omitted.

[0064] Fig. 15 is a diagram showing another example of the remote support system 1 in the second embodiment of the present invention. In the case shown in Fig. 14, the operation part detection unit 130 detects the body motion of the supporter with respect to the virtual model 7. However, the remote support system 1 of this embodiment is not limited to this case.

[0065] In the example shown in Fig. 15, the sensor 112 is omitted. The support side device 2 includes a pointing device 132. The pointing device 132 may be, for example, a mouse. The supporter operates the pointing device 132. The operation part detection unit 130 detects the operation part based on a signal from the pointing device 132. Specifically, the operation part detection unit 130 may detect the operation part by determining whether a cursor image of the pointing device 132 has collided with the virtual model 7. The other structures are similar to those shown in Fig. 14, and therefore repeated explanations will be omitted.

[0066] FIG. 16 is a flowchart showing an example of processing details of the support-side device in the remote support method of the second embodiment. In the remote support method, a work is supported by the support-side device 2 and the working-side device 3. The selection unit 105 selects a field worker based on instructions from a supporter (step S120). The supporter can instruct the field worker using a pointing device or the like. The voice control unit 118 starts a voice call between the selected field worker and the supporter (step S130). The selection unit 105 selects a virtual model 7 corresponding to the facility 5 to be supported from among multiple virtual models stored in the virtual model storage unit 104 (step S140). The selection unit 105 may receive identification information of the facility 5 from the working-side device 3 and select the virtual model 7 based on the identification information. However, the selection unit 105 may also select a virtual model based on instructions from the supporter. The supporter can point to the virtual model 7 using a pointing device or the like.

[0067] The support-side display unit 103 displays a virtual model 7 that simulates the equipment 5 (step S150). The virtual model 7 may be stored in the virtual model storage unit 104 or may be downloaded. The supporter operates the displayed virtual model 7 in the virtual space. The operation part detection unit 130 detects the content of the support operation at the operation part 22 of the equipment. The voice control unit 118 transmits operation information indicating the content of the support operation to the work-side device 3 (step S160) and ends the voice call (step S170). Note that the identification information of the equipment may be communicated from the worker to the supporter via the voice call. However, the voice call processing (steps S130 and S170) and the on-site worker selection processing (step S120) may be omitted.

[0068] Fig. 17 is a flowchart showing an example of the processing content in step S160 of Fig. 16. The communication control unit 101 transmits a start code and a facility ID (step S200). The start code is information indicating the start of processing. The facility ID is identification information for identifying the facility. Until the operation is completed (step S201: NO), the operation part detection unit 130 performs collision determination (step S203) each time the transmission period is reached (step S202: YES).

[0069] In this embodiment, the virtual model 7 is modeled after the facility 5. The virtual model 7 has a plurality of operation parts, namely, operation parts 21 and 22. When the operation parts 21 and 22 are identified by body movements, the operation part detection unit 130 acquires a body area image corresponding to the supporter's body based on data from the motion capture device 106. The body area image may be generated based on the coordinates of the body movement points 24. In one example, the body area image may be posture data shown in FIG. 5 or point cloud data shown in FIG. 6.

[0070] The operation part detection unit 130 executes collision determination between the virtual model 7 and the body region image corresponding to the supporter's body (step S203). When one of the multiple operation parts 21 and 22 collides with the body region image of the supporter, the colliding operation part 22 is selected. When the operation part 21 is selected by the pointing device 132, the operation part detection unit 130 may detect the operation part by determining collision between the cursor image of the pointing device 132 and the virtual model 7. The collision determination (step S203) may be executed for each transmission cycle (step S202: YES).

[0071] If an operation part is present (step S203: YES), the operation part detection unit 130 acquires the identification information (ID) of the operation part 22 (step S204). Then, the operation part detection unit 130 acquires the relative coordinates (X1, Y1, Z1) of the position 23 of the operation part 22 relative to the reference point P0 (X0, Y0, Z0). R , Y R , Z R ) (However, X R =X1-X0, Y R = Y1 - Y0, Z R Furthermore, the operation part detection unit 130 may acquire an operation part rotation angle (R x , R y , R z ) is acquired (step S206). x , R y , R z ) means the rotation angles of the X-axis, Y-axis, and Z-axis.

[0072] The communication control unit 131 transmits the operation information to the work-side apparatus 3. The operation information includes the identification information (ID) of the operation part, the relative coordinates (X R , Y R , Z R ), and the rotation angle of the operating part (R x , R y , R z ) may be included.

[0073] When an instruction to end the operation is given (step S201: NO), the communication control unit 131 transmits an end code to the work-side apparatus 3 (step S208), and then the process returns.

[0074] 18 is a flowchart showing an example of processing details of the working-side apparatus 3 in the remote support method of the second embodiment. The voice control unit 218 starts a voice call between the supporter and the worker (step S300). The communication control unit 201 receives operation information from the support-side apparatus 2. The working-side display unit 202 displays an operation status image 30 indicating the state of the equipment after the support operation, superimposed on the equipment image 11, based on the received operation information (step S310). When the support processing is completed, the voice control unit 218 ends the voice call (step S320).

[0075] Fig. 19 is a flowchart showing an example of the processing content in step S310 of Fig. 18. The communication control unit 201 functions as a receiving unit that receives various information from the support side device 2. When the communication control unit 201 receives a start code from the support side device 2 (step S401: YES), it acquires the virtual model 7 (step S402). The virtual model 7 may be downloaded from an external device, may be received from the support side device 2, or may be acquired from the working side virtual model storage unit 232 in the working side device 3.

[0076] Next, upon receiving the start code from the support-side device 2, the communication control unit 201 activates the imaging unit 204 (step S403). The imaging unit 204 captures an image of the surrounding environment including the equipment 5 and generates image data. The screen control unit 210 acquires the generated image data as the equipment image 11.

[0077] The screen control unit 210 also functions as a marker detection unit 241 that detects the reference point marker 6. The screen control unit 210 outputs the marker coordinates (X M , Y M , Z M) (step S404). The screen control unit 210 may combine the virtual model 7 with the equipment image 11 obtained by capturing an image of the surrounding environment including the equipment 5, and display the combined image on the operation side display unit 202 (step S405).

[0078] Until the communication control unit 201 receives an end code (step S406: NO), when the communication control unit 201 receives operation information (operation part information) (step S407: YES), the image generation unit 230 generates the identification information (ID) of the operation part 22, the relative coordinates of the operation part (X R , Y R , Z R ), and the rotation angle of the operating part (R x , R y , R z ) is acquired (step S408).

[0079] The image generating unit 230 generates the image data including the identification information (ID) of the operation part 22, the relative coordinates (X R , Y R , Z R ), and the rotation angle of the operating part (R x , R y , R z ), the image generator 230 may draw (redraw) the operation state image 30 showing the state of the equipment 5 after the assist operation. In other words, the image generator 230 may draw (redraw) the operation state image 30 by deforming the virtual model 7 synthesized in step S405.

[0080] The screen control unit 210 generates a composite image 80 by superimposing the operation state image 30 on the equipment image 11. The work side display unit 202 displays the composite image 80 (step S409). x , R y , R z ) changes, the operation status image 30 may change accordingly. The equipment image 11, the operation status image 30, and the composite image 80 may be three-dimensional images.

[0081] If the communication control unit 201 receives the end code (step S406: Yes), it determines that the assistance process has ended and stops the image capture unit 204 (step S410), and the process then returns.

[0082] According to the remote support system 1 of the second embodiment of the present invention, virtual objects can be dynamically changed and presented, making it possible to intuitively convey operation details. This improves the work efficiency of on-site workers and work assistants. In particular, it becomes possible to accurately convey to workers complex work details that involve the physical movements of the assistant and the operation of the equipment. Specifically, it becomes possible to accurately convey the status and movement of the equipment to workers. Since an equipment status image can be displayed superimposed on a background that includes the equipment image, it becomes easy to accurately convey the status of each part of the equipment.

[0083] Furthermore, unlike simple chromakey compositing, there is no need to have the supporter stand and move against a background of a specific color.

[0084] FIG. 20 is a schematic diagram showing an example of a composite image 90 on the work-side apparatus 3 in the third embodiment of the present invention. As shown in FIG. 20, the content of the first embodiment and the content of the second embodiment can be integrated to display the composite image 90 on the work-side apparatus 3. Specifically, the work-side display unit 202 may display both the body movement image 12 in the first embodiment and the operation status image 30 in the second embodiment. In this case, the remote support system 1 may have the configurations shown in FIGS. 2 and 11 and the configurations shown in FIGS. 14 and 15. Repetitive explanation will be omitted.

[0085] 21 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform methods or steps of methods according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0086] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0087] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0088] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0089] The ROM 2230 stores therein a boot program or the like that is executed by the computer 2200 upon activation, and / or programs that depend on the hardware of the computer 2200. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0090] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0091] For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into the RAM 2214 and instruct the communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0092] The CPU 2212 may also cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0093] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0094] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0095] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0096] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0097] 1 Remote support system, 2 Support device, 3 Work device, 4 Communication network, 5 Equipment, 6 Reference point marker, 7 Virtual model, 8 Reference point, 9 Identification image, 10 Composite image, 11 Equipment image, 12 Body movement image, 21 Operation part, 22 Operation part, 23 Position, 24 Body movement point, 25 Handle, 30 Operation state image, 31 Image, 32 Image, 33 Handle, 35 Message, 36, Symbol, 80, Composite image, 90, Composite image, 101, Communication control unit, 102, Detection unit, 103, Support side display unit, 104, Virtual model storage unit, 105, Selection unit, 106, Motion capture device, 110, Screen control unit, 112, Sensor, 114, Microphone, 116, Speaker, 118, Audio control unit, 120, Model generation unit, 122, Model correction unit, 130, Operation part detection unit, 13 1 Communication control unit, 132 Pointing device, 201 Communication control unit, 202 Work side display unit, 204 Imaging unit, 210 Screen control unit, 212 Sensor, 214 Microphone, 216 Speaker, 218 Audio control unit, 220 Correction information generation unit, 230 Image generation unit, 231 Communication control unit, 232 Work side virtual model storage unit, 241 Marker detection unit, 242 Adjustment unit, 243 Reading unit, 2200 ···Computer, 2201···DVD-ROM, 2210···Host controller, 2212···CPU, 2214···RAM, 2216···Graphics controller, 2218···Display device, 2220···Input / output controller, 2222···Communication interface, 2224···Hard disk drive, 2226···DVD-ROM drive, 2230···ROM, 2240···Input / output chip, 2242···Keyboard

Claims

1. a support-side device including a support-side display unit that displays a virtual model that imitates a facility, a detection unit that detects a body movement of a supporter relative to the virtual model, and a transmission unit that transmits information about the body movement relative to the virtual model; a work-side device including a work-side display unit that displays a virtual image representing the body movement based on the received information about the body movement, the detection unit detects the body motion, the body motion including position information about at least a part of the body of the support person relative to the virtual model; the transmitting unit transmits the information about the body movement including the position information; the work-side display unit displays a composite image in which the virtualized image is superimposed on the image of the equipment at a position corresponding to the position information, a reference point marker is provided at a predetermined position on the equipment; the virtual model is provided with a reference point provided at a position corresponding to the reference point marker and a reference coordinate that is a three-dimensional coordinate based on the reference point; the body motion detected by the detection unit includes the position information of at least a part of the body of the supporter in the reference coordinate system; The work-side device is a marker detection unit that detects a reference point marker in the facility; and an adjustment unit that adjusts a position at which the virtualized image is to be combined with the image of the equipment based on a detection result of the reference point marker and the position information included in the received information on the body movement. Support system.

2. the body motion detected by the detection unit includes the position information of at least a part of the body of the supporter in a three-dimensional space; The assistance system according to claim 1 .

3. The support-side device further includes a selection unit that selects the virtual model corresponding to the facility from a plurality of virtual models. The assistance system according to claim 1 or 2.

4. the selection unit selects the virtual model corresponding to the facility from the plurality of virtual models based on an instruction from the supporter. The assistance system according to claim 3 .

5. An identification image for identifying the equipment is displayed on the equipment, The work-side device includes a reading unit that reads the identification image; an identification information transmitting unit that transmits identification information obtained by reading the identification image with the reading unit to the support-side device, the selection unit selects a virtual model corresponding to the equipment from among a plurality of virtual models based on the identification information received from the work-side device. The assistance system according to claim 3 .

6. A support-side device having a support-side display unit that displays a virtual model that imitates equipment, a detection unit that detects a body movement of a supporter relative to the virtual model, and a transmission unit that transmits information about the body movement relative to the virtual model; a work-side device including a work-side display unit that displays a virtual image representing the body movement based on the received information about the body movement, the detection unit detects the body motion, the body motion including position information about at least a part of the body of the support person relative to the virtual model; the transmitting unit transmits the information about the body movement including the position information; the work-side display unit displays a composite image in which the virtualized image is superimposed on the image of the equipment at a position corresponding to the position information, The work-side device includes an imaging unit that captures an image of the facility and obtains a facility image; an equipment image transmission unit that transmits the equipment image to the support-side device, the support-side device further includes a model generation unit that generates the virtual model based on the equipment image received from the work-side device. Support system.

7. A support-side device having a support-side display unit that displays a virtual model that imitates equipment, a detection unit that detects a body movement of a supporter relative to the virtual model, and a transmission unit that transmits information about the body movement relative to the virtual model; a work-side device including a work-side display unit that displays a virtual image representing the body movement based on the received information about the body movement, the detection unit detects the body motion, the body motion including position information about at least a part of the body of the support person relative to the virtual model; the transmitting unit transmits the information about the body movement including the position information; the work-side display unit displays a composite image in which the virtualized image is superimposed on the image of the equipment at a position corresponding to the position information, The work-side device includes an imaging unit that captures an image of the facility and obtains a facility image; a correction information generation unit that generates correction information based on the equipment image; a correction information transmitting unit that transmits the correction information to the support-side device, the support-side device further includes a model correction unit that corrects the virtual model based on the correction information received from the work-side device. Support system.

8. the correction information generation unit compares the virtual model received from the support-side device with the facility image to generate the correction information. The assistance system according to claim 7.

Images