Remote support system, terminal device, remote device, guidance image display program, and remote support program

Abstract

To provide a remote support system, a terminal device, a remote device, a guide image display program, and a remote support program capable of guiding a user to a requested photographing position more easily compared with a case where the user is guided using only a photographed image taken from one direction.SOLUTION: A terminal device 30 transmits to a remote device 10 a plurality of pieces of image information obtained by photographing an object from respectively different directions, receives display information for displaying a guide image that guides a user to a requested photographing position, the guide image being derived by the remote device 10 by using the image information, and displays the guide image indicated by the received display information. The remote device 10 receives the plurality of pieces of image information from the terminal device 30, displays images indicated by the received plurality of pieces of image information, accepts selection of images taken from the requested photographing position from the displayed images, derives the display information by using the accepted images, and transmits the derived display information to the terminal device 30.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a remote assistance system, a terminal device, a remote device, a guidance image display program, and a remote assistance program. [Background technology]

[0002] Patent Document 1 discloses a monitoring support system that aims to provide a device that ensures the safety of patrolmen and improves the efficiency of patrol security at low cost by enabling a remote monitoring center to appropriately grasp the situation around the patrolmen. This monitoring support system consists of a remote monitoring means equipped with a three-dimensional spatial information database and a virtual reality display unit that displays the information, and a mobile terminal equipped with a position acquisition means and an orientation direction acquisition means. This monitoring support system transmits line-of-sight information from the mobile terminal carried by the patrolman to the remote monitoring means, allowing the remote monitoring means to appropriately grasp spatial information unfolding in front of the patrolmen at low cost.

[0003] Furthermore, Patent Document 2 discloses an information processing device that aims to provide a mechanism that enables a user's viewpoint to be moved to an appropriate position. This information processing device accepts an instruction to move the viewpoint position in the virtual space of a wearable display device to a previous position of the display device. When the information processing device accepts the instruction to move the viewpoint position, it determines a position a predetermined distance away from the previous position of the display device at which the instruction was accepted as the destination position of the viewpoint position, and changes the viewpoint position in the virtual space of the display device to move to the determined position.

[0004] Furthermore, Patent Document 3 discloses a guidance support method aimed at smoothly guiding a user to a target. This guidance support method detects target information from pre-stored spatial information, and measures the position of a display device in a space defined by the spatial information and the position of a target defined by the target information. This guidance support method also determines the position and movement of an icon to be displayed on the display device based on the measured positions of the display device and the target, and displays the icon at the determined position and with the determined movement on the display device. This guidance support method then performs processing to change the position and movement of the icon displayed on the display device in response to changes in the position of the display device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-257385 [Patent Document 2] Japanese Patent Application Publication No. 2018-063589 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-068689 Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention aims to provide a remote support system, terminal device, remote device, guidance image display program, and remote support program that can more easily guide a user to a required shooting position compared to when guiding using only images taken from one direction. [Means for solving the problem]

[0007] In order to achieve the above object, a remote assistance system according to a first aspect is a remote assistance system including a terminal device and a remote device, wherein the terminal device comprises a first processor, the first processor transmits to the remote device a plurality of pieces of image information obtained by photographing an object from different directions, receives display information derived by the remote device using the image information for displaying a guidance image guiding the user to a requested photographing position, and displays the guidance image indicated by the received display information, and the remote device comprises a second processor, the second processor receives the plurality of pieces of image information from the terminal device, displays images indicated by the received plurality of pieces of image information, accepts selection of an image photographed from the requested photographing position from the displayed images, derives the display information using the accepted image, and transmits the derived display information to the terminal device. A remote support system, wherein the first processor changes the display state of the guidance image according to the amount of deviation between the photographing position taken by the user and the requested photographing position. .

[0008] Furthermore, a remote support system according to a second aspect is the remote support system according to the first aspect, wherein the image information is either image information representing a still image or image information representing a moving image.

[0009] In addition, a remote support system according to a third aspect is a remote support system according to the second aspect, in which, when the image information is image information indicating the moving image, the first processor selects one still image for every predetermined number of images in the moving image and transmits the still image to the remote device.

[0010] A remote support system according to a fourth aspect is the remote support system according to any one of the first to third aspects, wherein the guidance image is a three-dimensional image.

[0011] In addition, the remote support system according to the fifth aspect is a remote support system according to the fourth aspect, in which the three-dimensional image is virtually displayed in the three-dimensional space of the shooting area for the object, and is a three-dimensional image that indicates the required shooting position and shooting direction for the object.

[0012] A remote support system according to a sixth aspect is the remote support system according to the fifth aspect, wherein the three-dimensional image is configured as a polyhedron.

[0013] In addition, a remote support system according to a seventh aspect is a remote support system according to any one of the first to sixth aspects, in which the guidance image is an image that becomes a predetermined state when the terminal device is in the required shooting position and orientation.

[0014] Furthermore, a remote support system according to an eighth aspect is the remote support system according to the seventh aspect, wherein the predetermined state is a state without distortion.

[0015] A remote support system according to a ninth aspect is the remote support system according to the eighth aspect, wherein the distortion-free state is a rectangular state.

[0016] A remote support system according to a tenth aspect is the remote support system according to any one of the first to ninth aspects, wherein the different directions include one direction directly facing the front of the object.

[0017] In addition, the remote support system according to an eleventh aspect is a remote support system according to any one of the first to tenth aspects, wherein the different directions include one direction in which the front, one side, and the top of the object are included in one image.

[0019] Also, 12 A remote assistance system according to an aspect includes: Any one of the first to eleventh aspects In the remote support system according to the above, the display state is at least one of a blinking interval, a display color, and a transmittance.

[0020] Also, 13 The remote assistance system according to the embodiment is 12 In the remote support system according to the aspect, when the first processor changes the transmittance, the transmittance is increased as the amount of deviation decreases.

[0021] Also, 14 The remote support system according to the embodiment includes the first embodiment to the third embodiment. 13 In the remote assistance system according to any one of the aspects, when a target position of the object is photographed, the first processor virtually displays a marker at the target position.

[0022] Also, 15 The remote support system according to the embodiment includes the first embodiment to the third embodiment. 14 In a remote support system relating to any one of the aspects, if the images indicated by the received plurality of image information do not include images taken from the required shooting position, the second processor sends instruction information to the terminal device instructing it to take a new photograph.

[0023] Also, 16 The remote assistance system according to the embodiment is 15 In the remote support system according to the aspect, the instruction information includes specific information that identifies the image information obtained by photographing at a photographing position closest to the requested photographing position.

[0024] In addition, in order to achieve the above objectives, 17 A terminal device according to an aspect includes a processor, and the processor transmits a plurality of pieces of image information obtained by photographing an object from different directions to a remote device, receives display information derived by the remote device using the image information for displaying a guidance image that guides the user to a required photographing position, and displays the guidance image indicated by the received display information. The terminal device is configured such that the processor changes the display state of the guidance image according to the amount of deviation between the photographing position taken by the user and the photographing position requested. .

[0025] In addition, in order to achieve the above objectives, 18A remote device according to an embodiment includes a processor, and the processor receives from a terminal device a plurality of pieces of image information obtained by photographing an object from different directions, displays images represented by the received plurality of pieces of image information, accepts a selection of an image photographed from a requested photographing position from the displayed images, derives display information for displaying a guidance image for guiding the user to the requested photographing position using the selected image, and transmits the derived display information to the terminal device. The processor is a remote device, and when the images indicated by the received plurality of image information do not include images taken from a requested photographing position, the processor transmits instruction information to the terminal device instructing the terminal device to take a photograph again, and the instruction information includes identification information that identifies the image information obtained by photographing at a photographing position closest to the requested photographing position. .

[0026] In addition, in order to achieve the above objectives, 19 The guidance image display program according to the embodiment includes a process of transmitting a plurality of pieces of image information obtained by photographing an object from different directions to a remote device, receiving display information for displaying a guidance image that guides the object to a required photographing position, the display information being derived by the remote device using the image information, and displaying the guidance image indicated by the received display information. and changing the display state of the guidance image according to the amount of deviation between the photographing position taken by the user and the required photographing position. to be executed by the computer.

[0027] Furthermore, in order to achieve the above objectives, 20 A remote assistance program according to an embodiment of the present invention receives, from a terminal device, a plurality of pieces of image information obtained by photographing an object from different directions, displays images indicated by the received plurality of pieces of image information, accepts a selection of an image photographed from a requested photographing position from the displayed images, derives display information for displaying a guidance image for guiding the user to the requested photographing position using the accepted image, and transmits the derived display information to the terminal device. and if the received image information indicates that the image is not taken from a requested photographing position, instruction information instructing the terminal device to take a photograph again is transmitted, the instruction information including specific information for identifying the image information obtained by photographing at a photographing position closest to the requested photographing position. to be executed by the computer. [Effects of the Invention]

[0028] First aspect, 17 Mode ~ No. 20 According to the embodiment, only the photographed image taken from one direction is used. This makes it possible to more easily guide the user to the desired photographing position compared to when guiding the user by using a camera. Furthermore, according to the first and seventeenth to twentieth aspects, the user can be guided to the required shooting position in a shorter time than when the display state of the guidance image is not changed according to the amount of deviation between the user's shooting position and the required shooting position.

[0029] According to the second aspect, when image information representing a still image is applied as the image information, the storage capacity for the image information can be reduced compared to when image information representing a moving image is applied as the image information. Also, according to the second aspect, when image information representing a moving image is applied as the image information, more image information can be obtained compared to when image information representing a still image is applied as the image information.

[0030] According to the third aspect, it is possible to transmit the image information to the remote device in a shorter time than when all of the image information showing the moving image is transmitted to the remote device.

[0031] According to the fourth aspect, compared to when the guidance image is a one-dimensional or two-dimensional image, the user can be guided to the required photographing position more intuitively.

[0032] According to the fifth aspect, it is possible to guide the user to a more accurate shooting state compared to when the target of guidance for the three-dimensional image is only the shooting position.

[0033] According to the sixth aspect, it is possible to more easily derive display information compared to when the three-dimensional image is not a polyhedron.

[0034] According to the seventh aspect, the guidance image can guide the user to a more accurate shooting state compared to when the guidance image is an image that does not change even when the terminal device is in the required shooting position and orientation.

[0035] According to the eighth aspect, it is possible to more intuitively guide the user to the required photographing position, compared to when the predetermined state is set to a distorted state.

[0036] According to the ninth aspect, compared to when the distortion-free state is a state other than a rectangle, it is possible to more intuitively guide the user to the required photographing position.

[0037] According to the tenth aspect, it is possible to obtain image information that is more likely to be selected by a remote device than when the different directions do not include a direction directly facing the front of the object.

[0038] According to the eleventh aspect, compared to when the different directions do not include the front, one side, and one top of the object in one image, the likelihood that the target position of the object will be included in one image is increased, and as a result, the user can be more efficiently guided to the required shooting position.

[0040] No. 12 According to this aspect, the amount of deviation between the user and the required photographing position can be grasped by utilizing the applied change in the display state.

[0041] No. 13 According to this aspect, when the transmittance is changed, the visibility of the object can be improved as the amount of deviation increases, compared to when the transmittance is increased.

[0042] No. 14 According to this aspect, it is possible to allow the user to more easily grasp the target position of the object, compared to when a marker is not displayed at the target position of the object.

[0043] No. 15 According to this aspect, even if the image indicated by the received image information does not include an image taken from the requested shooting position, the user can be guided to the requested shooting position more reliably than when re-shooting is not instructed.

[0044] No. 16 According to this aspect, it is possible to make effective use of image information that has already been obtained, compared to when the instruction information does not include specific information that identifies image information obtained by shooting at the shooting position closest to the required shooting position. [Brief explanation of the drawings]

[0045] [Figure 1]FIG. 2 is a block diagram illustrating an example of a hardware configuration of the remote support system according to the embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of the remote support system according to the embodiment. [Figure 3] FIG. 2 is a schematic diagram showing an example of the configuration of a photography-related information database according to the embodiment. [Figure 4] FIG. 10 is a perspective view illustrating an example of a situation where an image is captured by a terminal device, the view being provided for explaining a remote support process according to the embodiment. [Figure 5] 10 is a flowchart illustrating an example of a guidance image display process according to the first embodiment. [Figure 6] FIG. 10 is a front view showing an example of a re-photographing instruction screen according to the embodiment. [Figure 7] FIG. 10 is a front view showing an example of a guidance image display screen according to the embodiment. [Figure 8] FIG. 10 is a front view showing another example of the guidance image display screen according to the embodiment. [Figure 9] 10 is a flowchart illustrating an example of a remote assistance process according to the embodiment. [Figure 10] FIG. 10 is a front view showing an example of an image selection screen according to the embodiment. [Figure 11] FIG. 10 is a diagram for explaining the remote support process according to the embodiment, and is a perspective view showing another example of the shooting situation by the terminal device. [Figure 12] 10 is a flowchart illustrating an example of a guidance image display process according to the second embodiment. [Figure 13] FIG. 10 is a perspective view showing another example of a guidance image according to the embodiment. [Figure 14] FIG. 10 is a front view showing another example of the guidance image display screen according to the embodiment. [Figure 15] FIG. 10 is a perspective view showing another example of a guidance image according to the embodiment. [Figure 16] FIG. 10 is a perspective view showing another example of a guidance image according to the embodiment. [Figure 17] FIG. 10 is a perspective view showing another example of a guidance image according to the embodiment. [Figure 18]FIG. 10 is a perspective view showing another example of a guidance image according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0046] Hereinafter, with reference to the drawings, an example of an embodiment of the present invention will be described in detail. In this embodiment, the present invention will be described as being applied to a remote support system including a plurality of terminal devices, each of which is a portable terminal used individually by a user, and a remote device installed in a remote location away from each of the terminal devices. In addition, in this embodiment, the users are workers who perform maintenance on image forming devices such as digital multifunction peripherals and printers, and the remote support system remotely supports the workers in performing maintenance on the image forming devices.

[0047] [First embodiment] First, the configuration of a remote support system 90 according to this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a block diagram showing an example of the hardware configuration of the remote support system according to this embodiment. Fig. 2 is a block diagram showing an example of the functional configuration of the remote support system according to this embodiment.

[0048] 1, a remote support system 90 according to this embodiment includes a remote device 10 and multiple terminal devices 30, each of which is capable of accessing a network 80. Examples of the remote device 10 include information processing devices such as a personal computer and a server computer. Examples of the terminal devices 30 include portable terminals such as a smartphone, a tablet terminal, and a PDA (Personal Digital Assistant, mobile information terminal).

[0049] The terminal device 30 according to this embodiment is a device carried by each of multiple users (hereinafter simply referred to as "users") who use the remote support system 90 when performing maintenance on an image forming apparatus. The terminal device 30 includes a CPU (Central Processing Unit) 31, a memory 32 serving as a temporary storage area, a nonvolatile memory unit 33, an input unit 34 such as a touch panel, a display unit 35 such as a liquid crystal display, and a medium read / write device (R / W) 36. The terminal device 30 also includes an image capture unit 38, a microphone 39, a position detection unit 40, and a wireless communication unit 42. The CPU 31, memory 32, memory unit 33, input unit 34, display unit 35, medium read / write device 36, image capture unit 38, microphone 39, position detection unit 40, and wireless communication unit 42 are interconnected via a bus B1. The medium read / write device 36 reads information from and writes information to a recording medium 37.

[0050] The storage unit 33 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. A guidance image display program 33A is stored in the storage unit 33 as a storage medium. The guidance image display program 33A is stored in the storage unit 33 when a recording medium 37 having the guidance image display program 33A written therein is set in the medium reading and writing device 36 and the medium reading and writing device 36 reads the guidance image display program 33A from the recording medium 37. The CPU 31 reads the guidance image display program 33A from the storage unit 33, expands it in the memory 32, and sequentially executes processes included in the guidance image display program 33A.

[0051] Furthermore, a photography-related information database 33B is stored in the storage unit 33. The photography-related information database 33B will be described in detail later.

[0052] The photographing unit 38 according to the present embodiment photographs the image forming apparatus to be maintained and outputs image information obtained by the photographing. The microphone 39 according to the present embodiment collects voices uttered by the user and outputs voice information.

[0053] Furthermore, the position detection unit 40 according to this embodiment detects its own position and outputs position information, and although a GPS (Global Positioning Systems) is used in this embodiment, the present invention is not limited to this. For example, the position detection unit 40 may use position information acquired from a Wi-Fi (registered trademark) router, detect the position using a beacon, or detect the position by image analysis using a captured image.

[0054] On the other hand, the remote device 10 plays a central role in the remote assistance system 90, and is a device that derives a guidance image to be presented to the user, which will be described in detail later. The remote device 10 includes a CPU 11, a memory 12 as a temporary storage area, a non-volatile memory unit 13, an input unit 14 such as a keyboard and a mouse, a display unit 15 such as a liquid crystal display, a medium reading / writing device 16, and a communication interface (I / F) unit 18. The CPU 11, memory 12, memory unit 13, input unit 14, display unit 15, medium reading / writing device 16, and communication I / F unit 18 are connected to one another via a bus B2. The medium reading / writing device 16 reads information written in a recording medium 17 and writes information to the recording medium 17.

[0055] The storage unit 13 is realized by an HDD, an SSD, a flash memory, or the like. A remote assistance program 13A is stored in the storage unit 13 as a storage medium. The remote assistance program 13A is stored in the storage unit 13 when a recording medium 17 on which the remote assistance program 13A is written is set in the medium reading and writing device 16 and the medium reading and writing device 16 reads the remote assistance program 13A from the recording medium 17. The CPU 11 reads the remote assistance program 13A from the storage unit 13, expands it in the memory 12, and sequentially executes the processes of the remote assistance program 13A.

[0056] In this embodiment, the network 80 is a public communication line such as the Internet or a telephone network, but is not limited to this. For example, the network 80 may be an in-company communication line such as a LAN (Local Area Network) or a WAN (Wide Area Network), or may be a combination of these in-company communication lines and public communication lines.

[0057] Next, the functional configuration of the remote device 10 and the terminal device 30 according to this embodiment will be described with reference to FIG.

[0058] 2, the terminal device 30 according to this embodiment includes a first transmitter 31A, a first receiver 31B, and a first display controller 31C. The CPU 31 of the terminal device 30 executes a guidance image display program 33A, thereby functioning as the first transmitter 31A, the first receiver 31B, and the first display controller 31C.

[0059] The first transmission unit 31A according to this embodiment transmits a plurality of pieces of image information (hereinafter simply referred to as "image information") obtained by photographing an object from different directions to the remote device 10. As described above, in this embodiment, the object is an image forming device that is the target of maintenance.

[0060] Furthermore, the first receiving unit 31B according to this embodiment receives display information (hereinafter simply referred to as "display information") for displaying a guidance image for guiding to a requested shooting position, which is derived by the remote device 10 using the image information. Note that, in this embodiment, the first transmitting unit 31A transmits the image information and the first receiving unit 31B receives the display information via the wireless communication unit 42, but this is not limiting.

[0061] Then, the first display control unit 31C according to the present embodiment causes the display unit 35 to display a guidance image indicated by the received display information (hereinafter simply referred to as a "guidance image").

[0062] 2, the remote device 10 according to this embodiment includes a second receiving unit 11A, a second display control unit 11B, a reception unit 11C, a derivation unit 11D, and a second transmission unit 11E. The CPU 11 of the remote device 10 mainly executes the remote assistance program 13A, thereby functioning as the second receiving unit 11A, the second display control unit 11B, the reception unit 11C, the derivation unit 11D, and the second transmission unit 11E.

[0063] The second receiving unit 11A according to this embodiment receives a plurality of pieces of image information from the terminal device 30. The second display control unit 11B according to this embodiment displays, on the display unit 15, images indicated by the received plurality of pieces of image information.

[0064] Furthermore, the reception unit 11C according to this embodiment receives a selection of an image captured from a requested shooting position from among the displayed images. Note that in this embodiment, the reception unit 11C receives the image selection by operation of the remote device 10 by the user via the input unit 14, but this is not limiting. For example, the remote device 10 may be provided with an audio input unit such as a microphone, and the image selection may be received by voice via the audio input unit.

[0065] Here, the derivation unit 11D according to the present embodiment derives the above-mentioned display information using the received image. Then, the second transmission unit 11E according to the present embodiment transmits the derived display information to the terminal device 30.

[0066] In this embodiment, image information indicating a still image is applied as the image information. Therefore, the image capture unit 38 provided in the terminal device 30 is assumed to be capable of capturing still images, but this is not limited thereto. For example, it goes without saying that a capture unit capable of capturing both still images and moving images may be applied as the image capture unit 38. Also, in this embodiment, image information indicating a color image is applied as the image information, and therefore the image capture unit 38 according to this embodiment is assumed to be capable of capturing color images, but this is not limited thereto. For example, image information indicating a monochrome image may be applied as the image information, and the image capture unit 38 may be configured to be capable of capturing monochrome images.

[0067] In addition, in this embodiment, an image that becomes a predetermined state when the terminal device 30 is in a required shooting position and orientation is used as the guidance image, and in particular, in this embodiment, a distortion-free state is used as the predetermined state. Furthermore, in this embodiment, a rectangular state is used as the distortion-free state.

[0068] In this embodiment, the different directions include a direction directly facing the front of the object.

[0069] In this embodiment, the first display control unit 31C changes the display state of the guidance image according to the amount of deviation between the user and the required shooting position. In this embodiment, transmittance is used as the display state, and the smaller the amount of deviation, the greater the transmittance. However, this is not limiting. Instead of the transmittance, either the blinking interval or the display color may be used as the display state, or a combination of two or three of the blinking interval, the display color, and the transmittance may be used as the display state.

[0070] Furthermore, in this embodiment, when a target position of the object is photographed, the first display control unit 31C virtually displays a marker at that position.

[0071] Furthermore, in this embodiment, if the images indicated by the received plurality of pieces of image information do not include images taken from the requested shooting position, the second transmission unit 11E transmits instruction information (hereinafter simply referred to as "instruction information") instructing re-shooting to the terminal device 30. Here, in this embodiment, the instruction information includes specific information that identifies image information obtained by shooting at the shooting position closest to the requested shooting position.

[0072] Next, the photography-related information database 33B according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a schematic diagram showing an example of the configuration of the photography-related information database 33B according to this embodiment.

[0073] The photography-related information database 33B according to this embodiment is a database in which information relating to the above-mentioned plurality of pieces of image information is registered. As shown in Fig. 3, the photography-related information database 33B according to this embodiment stores information on the photography order, photographed images, photography positions, and orientations in association with each other.

[0074] The photographing order is information indicating the order in which the user photographed the object, and the photographed image is image information obtained by the corresponding photograph. Furthermore, the photographing position is information indicating the position of the terminal device 30 when the corresponding photograph was taken, and the orientation is information indicating the orientation of the terminal device 30 when the corresponding photograph was taken.

[0075] In this embodiment, the information indicating the photographing position is obtained by the position detection unit 40 using GPS to obtain information indicating the three-dimensional position of the terminal device 30 in the horizontal and vertical directions, but this is not limiting. For example, as described above, the position may be detected using a beacon or the like.

[0076] In this embodiment, the information indicating the orientation is obtained by a gyro sensor (not shown) built into the terminal device 30, which obtains information indicating the photographing direction of the photographing unit 38 in a state where the target object is positioned at approximately the center in the left-right direction within the photographing angle of view of the photographing unit 38 at the corresponding photographing position. However, the present invention is not limited to this, and the information indicating the orientation may be obtained by, for example, an acceleration sensor or the like.

[0077] That is, in the remote assistance system 90 according to this embodiment, prior to carrying out maintenance, the user uses the terminal device 30 carried by the user to photograph the object 50 to be maintained from a plurality of predetermined directions (four directions in this embodiment: the front, right side, back, and left side of the object 50), as shown in Fig. 4 as an example. Then, the terminal device 30 according to this embodiment transmits to the remote device 10 image information obtained by photographing from the plurality of directions, together with information indicating the photographing order and the photographing position and orientation by the terminal device 30 when photographing.

[0078] The photography-related information database 33B according to this embodiment is for temporarily storing each piece of information in the terminal device 30 itself before transmitting the information to the remote device 10.

[0079] Next, the operation of the remote support system 90 according to this embodiment will be described with reference to Figs. 5 to 10. First, the operation of the terminal device 30 according to this embodiment will be described with reference to Figs. 5 to 8. Fig. 5 is a flowchart showing an example of a guidance image display process according to this embodiment. To avoid confusion, the following description will omit a description of the process for giving instructions regarding the work itself that is the target of remote support by the remote support system 90, and will only describe a process for guiding the user to a required shooting position.

[0080] In the remote support system 90 according to this embodiment, as described above, a user uses the terminal device 30 to photograph the object 50 from multiple directions, and registers the multiple image information thus obtained in the photography-related information database 33B together with information on the photographing order, photographing position, and orientation.

[0081] In this state, when the user inputs an instruction to execute the guidance image display process to the terminal device 30, the CPU 31 of the terminal device 30 executes the guidance image display program 33A, thereby executing the guidance image display process shown in Figure 5.

[0082] In step 100 of FIG. 5, the CPU 31 reads out all information stored in the photography-related information database 33B (hereinafter referred to as “photography-related information”), and in step 102, the CPU 31 transmits the read photography-related information to the remote device 10.

[0083] When receiving the photography-related information, if the multiple photographed images indicated by the image information included in the photography-related information do not include an image photographed at the requested photographing position and orientation (hereinafter referred to as a "requested photographed image"), the remote device 10 transmits re-photographing instruction information (corresponding to the instruction information described above) instructing the accessing terminal device 30 to re-photograph the image. At this time, the remote device 10 transmits to the terminal device 30 the re-photographing instruction information including specification information (in this embodiment, information indicating the photographing order) that specifies image information indicating a photographed image obtained by photographing at the photographing position closest to the requested photographing position.

[0084] Therefore, in step 104, the CPU 11 determines whether or not re-photographing instruction information has been received from the remote device 10, and if the determination is negative, the process proceeds to step 110, whereas if the determination is positive, the process proceeds to step 106.

[0085] In step 106, the CPU 11 controls the display unit 35 to display a re-shooting instruction screen having a predetermined configuration using image information indicated by the specific information received together with the re-shooting instruction information, and then waits until the user has finished re-shooting in step 108. Fig. 6 shows an example of the re-shooting instruction screen according to this embodiment.

[0086] 6, the re-shooting instruction screen according to this embodiment displays a captured image 35A obtained by shooting at a shooting position closest to the requested shooting position. Also, the re-shooting instruction screen according to this embodiment displays a message prompting the user to shoot multiple times again at positions close to the position where the displayed captured image 35A was shot.

[0087] Therefore, the user updates the registered information in the photography-related information database 33B by using the terminal device 30 to re-photograph the object 50 multiple times at positions near the position where the photographed image 35A displayed on the re-photographing instruction screen was photographed. Then, the user designates the end button 35E displayed on the re-photographing instruction screen via the input unit 34. In response to this, a positive determination is made in step 108, and the process returns to step 100, and the process from step 100 is executed again.

[0088] On the other hand, if the requested captured image is included in the multiple captured images indicated by the image information contained in the received capture-related information, the remote device 10 transmits position / orientation information transmission instruction information to the accessing terminal device 30 instructing it to transmit information indicating the capture position and orientation at that time (hereinafter referred to as "position / orientation information").

[0089] Therefore, in step 110, the CPU 31 determines whether or not position / orientation information transmission instruction information has been received from the remote device 10, and if the determination is negative, the process returns to step 100, whereas if the determination is positive, the process proceeds to step 112.

[0090] In step 112, the CPU 11 acquires the position and orientation information in the manner described above, and transmits the acquired position and orientation information to the remote device 10.

[0091] When the remote device 10 receives the position and orientation information from the accessing terminal device 30, the remote device 10 uses the position and orientation information and the imaging-related information about the requested captured image to derive a guidance image for the user to guide the terminal device 30 to the imaging position and orientation when the requested captured image was captured. Then, the remote device 10 derives guidance image display information for virtually displaying the derived guidance image and transmits it to the accessing terminal device 30. At this time, the remote device 10 transmits to the terminal device 30, marker display information for virtually displaying a marker at a position (hereinafter referred to as a "target") that is the target of maintenance of the object 50, included in the guidance image display information.

[0092] Therefore, in step 114, the CPU 31 waits until it receives the guidance image display information and the marker display information from the remote device 10. In step 116, the CPU 31 controls the display unit 35 to display a guidance image display screen having a predetermined configuration using the received guidance image display information, marker display information, and an image currently being captured by the capturing unit 38 (hereinafter referred to as an "actual captured image"). Figure 7 shows an example of a guidance image display screen according to this embodiment.

[0093] As shown in FIG. 7, the guidance image display screen according to this embodiment displays a message prompting the user to set the shooting position and orientation of the terminal device 30 so that the guidance image 35B has a predetermined shape (a rectangle in this embodiment). Furthermore, the guidance image display screen according to this embodiment displays the guidance image 35B superimposed on the actual captured image. Furthermore, the guidance image display screen according to this embodiment sequentially changes the display state (transparency in this embodiment) of the guidance image 35B according to the amount of deviation between the user and the required shooting position (in this embodiment, the transmittance increases as the deviation decreases). Furthermore, the guidance image display screen according to this embodiment displays a shooting direction image 35C (an image showing an arrow in this embodiment) indicating the required shooting direction (orientation) combined with the guidance image 35B.

[0094] That is, the guidance image 35B in this embodiment is a three-dimensional image that takes on the above-mentioned predetermined shape (rectangle in this embodiment) and size when the shooting position and orientation by the terminal device 30 become the required shooting position and orientation.

[0095] In the terminal device 30 according to this embodiment, when the shooting position and orientation of the terminal device 30 are different from the requested shooting position and orientation, the guidance image 35B that appears in the predetermined shape and size when the shooting position and orientation are the requested one is displayed in that state.

[0096] 7 shows an example of a guidance image display screen in the case where the object 50 is currently being photographed from the front and the required photographing position is the right side of the object 50. In this case, the guidance image 35B is a parallelogram that is a distorted rectangle and is close to a straight line, and its size becomes smaller as the distance from the terminal device 30 to the object 50 increases. Therefore, by referring to the guidance image display screen, the user can intuitively grasp the required photographing position and orientation for the object 50 from the display position, shape, and size of the guidance image 35B.

[0097] The example shown in Fig. 8 is an example of a guidance image display screen when the terminal device 30 is in the required shooting direction and orientation in the example shown in Fig. 7. In this case, the object 50 is displayed as if it were shot from the right side, and the guidance image 35B is rectangular. In addition, in this case, since the target is included in the shooting angle of view, the above-mentioned marker 35D is displayed. Therefore, the user can easily grasp the position of the target.

[0098] In this embodiment, the remote device 10 sequentially generates information indicating the guidance image 35B having different shapes, sizes, and transmittances depending on the shooting position and orientation of the terminal device 30. However, the present invention is not limited to this. For example, the remote device 10 may generate only a single original form of the guidance image 35B and transmit it to the accessing terminal device 30, and the terminal device 30 may generate the guidance image 35B according to its own shooting position and orientation.

[0099] In step 118, the CPU 31 determines whether the terminal device 30 has reached the required shooting position and orientation and has completed guiding the user to the destination; if the determination is negative, the process returns to step 112; if the determination is positive, the CPU 31 terminates this guidance image display process.

[0100] By repeating the process of steps 112 to 118, the display unit 35 of the terminal device 30 displays the object 50 being photographed by the terminal device 30, and also displays a superimposed guidance image 35B in a state corresponding to the photographing position and orientation by the terminal device 30. Therefore, by referring to the guidance image display screen in this state, the user can intuitively set the terminal device 30 to the required photographing position and orientation.

[0101] When the guidance image display process is completed, a remote support process for performing necessary maintenance on the object 50 is executed using a conventionally known technique.

[0102] Next, the operation of the remote device 10 according to this embodiment will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is a flowchart showing an example of the remote assistance process according to this embodiment. Note that the case where the user of the remote device 10 is a person who gives remote instructions to the user (hereinafter referred to as "remote instructor") will be described.

[0103] In the remote support system 90 according to this embodiment, when the above-described shooting-related information is received from any of the terminal devices 30, the CPU 11 of the remote device 10 executes the remote support program 13A, thereby executing the remote support processing shown in FIG. 9.

[0104] 9, the CPU 11 controls the display unit 15 to display an image selection screen having a predetermined configuration using the received photography-related information, and then waits until the predetermined information is input in step 202. An example of the image selection screen according to this embodiment is shown in FIG.

[0105] 10, the image selection screen according to this embodiment displays the captured images taken at the guide destination, a message prompting the user to specify the position of the target as described above, and a message prompting the user to specify the captured image closest to the guide destination when there is no captured image at the guide destination. Also, the image selection screen according to this embodiment displays the captured images indicated by the plurality of image information included in the received photography-related information (in this embodiment, four captured images 15a to 15d taken from four directions, i.e., the front, right side, back, and left side of the object 50).

[0106] Therefore, by referring to the image selection screen, the remote instructor can grasp all captured images taken by the accessing user (hereinafter simply referred to as the "user") by the terminal device 30 prior to the execution of the remote assistance process. If a captured image (the above-mentioned requested captured image) taken at a requested shooting position and orientation is present among the multiple captured images displayed, the remote instructor specifies the requested captured image and the position of the target in the requested captured image via the input unit 14. Then, after completing these specifications, the remote instructor specifies the requested captured image specification button 15A via the input unit 14.

[0107] In this embodiment, the object of remote assistance provided by the remote assistance system 90 is maintenance of the object 50, and therefore the requested photographing position is a position facing the object 50 at the time the maintenance is started. For example, when performing maintenance on the inside of a digital multifunction peripheral, which is the object 50, it may be necessary to open the front door of the digital multifunction peripheral at the beginning of the maintenance. In this case, the remote instructor specifies, as the requested photographed image, an image obtained by photographing from a photographing position in front of the digital multifunction peripheral.

[0108] On the other hand, if the requested photographed image does not exist among the multiple photographed images being displayed, the remote instructor specifies a photographed image (corresponding to the photographed image 35A described above, hereinafter referred to as the "nearest position photographed image") taken from a photographing position closest to the photographing position of the requested photographed image via the input unit 14. Then, when the remote instructor has finished specifying the nearest position photographed image, he or she specifies nearest position photographed image specification button 15B via the input unit 14.

[0109] When the remote instructor designates the requested photographed image designation button 15A or the nearest photographed image designation button 15B, the determination in step 202 is affirmative, and the process proceeds to step 204.

[0110] In step 204, the CPU 11 determines whether or not a requested captured image has been designated by the remote instructor, and if the determination is negative, the process proceeds to step 206. In step 206, the CPU 11 generates the above-mentioned re-photographing instruction information and transmits it to the accessing terminal device 30, and then ends this remote support process.

[0111] On the other hand, if the determination in step 204 is affirmative, the process proceeds to step 208, where the CPU 11 transmits the above-mentioned position and orientation information transmission instruction information to the accessing terminal device 30. In response to this, the accessing terminal device 30 transmits position and orientation information, and therefore in step 210 the CPU 11 waits until the position and orientation information from the accessing terminal device 30 is received.

[0112] In step 212, the CPU 11 derives the above-mentioned guidance image display information and marker display information using the received position and orientation information, the imaging-related information corresponding to the specified requested captured image, and the position of the target. In step 214, the CPU 11 transmits the derived guidance image display information and marker display information to the accessing terminal device 30.

[0113] In step 216, the CPU 11 determines whether the timing to end the remote support process has arrived. If the determination is negative, the process returns to step 210, whereas if the determination is positive, the remote support process is terminated. In this embodiment, the timing at which the remote instructor inputs an instruction to end the remote support process is used as the timing at which the end of the remote support process is initiated, but this is not limited to this. For example, the timing at which it is detected that the shooting position and orientation indicated by the received position and orientation information have become the requested shooting position and orientation, or the timing at which information indicating the end of the work is received from the user may be used as the timing at which the end of the remote support process is initiated.

[0114] In the above embodiment, as an example of multiple photographs taken prior to the execution of the remote assistance process, a case has been described in which four photographs of the front, right side, back, and left side of the object 50 are taken as shown in Fig. 4, but the present invention is not limited to this. The number of photographs taken is not limited to four, and any number of photographs may be taken as long as they are taken multiple times. As for the photographing position and orientation, as an example, any photographing position and orientation may be applied, such as an upper front right or a lower rear left, as shown in Fig. 11.

[0115] In this case, it is preferable to include the front, one side, and one top of the object 50 in one direction in one image, as this allows more sides to be captured in one image and allows for more efficient guidance of the user.

[0116] In the above embodiment, the case where transmittance is applied as the display state of the guidance image that changes depending on the amount of deviation between the user and the required shooting position has been described, but this is not limited thereto. For example, as described above, at least one of the blinking interval and the display color may be applied as the display state, or a combination of multiple types of three display states, which are the blinking interval, the display color, and the transmittance, may be applied as the display state. For example, when the blinking interval is applied as the display state, an example is that the blinking interval is shortened as the amount of deviation decreases, and when the display color is applied as the display state, an example is that the color approaches red as the amount of deviation decreases.

[0117] [Second embodiment] In the first embodiment, an example was given of a case where a still image is captured by the terminal device 30 prior to the execution of the remote support processing, but in this embodiment, an example of a case where a moving image is captured by the terminal device 30 prior to the execution of the remote support processing will be described.

[0118] The configuration of the remote support system 90 according to this embodiment is the same as that of the remote support system 90 according to the first embodiment, except for the first transmission unit 31A and the photography-related information database 33B. Therefore, only the first transmission unit 31A and the photography-related information database 33B according to this embodiment will be described here.

[0119] That is, in this embodiment, when the photographing of the object performed prior to the execution of the remote support processing is video photography, the first transmission unit 31A selects one still image for every predetermined number of frames in the moving images obtained by the video photography and transmits it to the remote device 10.

[0120] Although not shown in the drawings, in this embodiment, image information indicating a moving image is registered as information indicating a captured image in place of information indicating a still image in the photography-related information database 33B. Furthermore, as information on the photography position and orientation in the photography-related information database 33B, information on the photography position and orientation in the corresponding moving image is registered for each predetermined photography period (0.3 seconds in this embodiment).

[0121] The operation of the remote support system 90 according to this embodiment will be described below. Note that the operation according to this embodiment is the same as that of the remote support system 90 according to the first embodiment except for the guidance image display process. Therefore, only the operation of the terminal device 30 according to this embodiment when executing the guidance image display process will be described here with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the guidance image display process according to this embodiment. Steps that perform the same processes as those in Fig. 5 are assigned the same step numbers as those in Fig. 5, and their description will be omitted.

[0122] As shown in FIG. 12, the guidance image display process according to this embodiment differs from the guidance image display process according to the first embodiment only in that the processes of steps 101A and 101B are applied instead of the process of step 100.

[0123] That is, in step 101A, CPU 31 reads out all information (hereinafter referred to as "moving image information") stored in imaging-related information database 33B. In step 101B, CPU 11 extracts image information in which information indicating the imaging position and orientation is stored from the read-out moving image information, and selects one piece of image information for every predetermined number of images (10 images in this embodiment) from the extracted image information.

[0124] Each of the multiple pieces of image information selected by the above processing is information indicating a still image, and in step 102, CPU 11 associates the image information with information indicating the shooting position and orientation corresponding to the image information, and transmits the associated image information to remote device 10. After this, terminal device 30 and remote device 10 operate in the same manner as in the first embodiment.

[0125] In the above-described embodiments, the rectangular guidance image 35B shown in Fig. 7 and Fig. 8 is used as an example of the guidance image, but the present invention is not limited to this. For example, as shown in Fig. 13, a polyhedron having an imaging direction indication surface 60A that is a surface perpendicular to the required imaging direction (orientation) may be used as the guidance image 60. Fig. 13 is a perspective view showing an example of the guidance image 60 according to this embodiment.

[0126] As shown in FIG. 13, this guidance image 60 has the above-mentioned imaging direction indication surface 60A and a regular hexahedron (cube) shape having eight vertices 60a to 60h.

[0127] When this guidance image 60 is applied, the CPU 11 of the remote device 10 displays, on the display unit 15, an enlarged version of the requested captured image designated by the remote instructor, following the image selection screen shown in Fig. 10 as an example. Then, the CPU 11 of the remote device 10 superimposes on the requested captured image an image excluding the shooting direction indication surface 60A in the guidance image 60, i.e., an image consisting only of the edges of a regular hexahedron (hereinafter referred to as "original guidance image"), on the requested captured image.

[0128] When the original guidance image is displayed on the display unit 15, the remote instructor performs an operation to move the displayed original guidance image to a requested shooting position, and also specifies a vertex at which the shooting direction indication plane 60A becomes a plane perpendicular to the shooting direction according to the requested orientation, or a plane whose inclination angle is closest to the plane. When specifying the shooting direction indication plane 60A in the example shown in FIG. 13, the remote instructor specifies three vertices, vertex 60a, vertex 60c, and vertex 60f. When the guidance image 60 is obtained through the specification by the remote instructor, the CPU 11 of the remote device 10 transmits guidance image display information indicating the guidance image 60 to the access source terminal device 30.

[0129] When receiving the guidance image display information, the CPU 31 of the accessing terminal device 30 uses the received guidance image display information to display, on the display unit 35, a guidance image display screen as shown in FIG. 14, for example.

[0130] In this case, since the original guidance image is a regular hexahedron, the shooting direction indication surface 60A becomes an equilateral triangle without distortion when viewed from the front, and is oriented diagonally downward to the right with an inclination angle of 45 degrees. Therefore, in this case, the user refers to the guidance image display screen and adjusts the shooting position and orientation of the terminal device 30 so that the shooting direction indication surface 60A becomes an equilateral triangle.

[0131] Note that, when a regular hexahedron is applied as the original guidance image, and when a vertical surface is designated as the shooting direction indication surface 60A and a shooting direction diagonally forward to the right is designated, four vertices, namely, vertices 60a, 60c, 60e, and 60g, are designated as shown in FIG. 15 . Furthermore, when the outer peripheral surface of the original guidance image is designated as the shooting direction indication surface 60A, for example, vertices 60b, 60c, 60f, and 60g are designated. In this case, instead of designating four vertices, a surface to be used as the shooting direction indication surface 60A may be directly designated. In this case, the shooting direction indication surface 60A can be designated with only one designation operation, which makes it easier to designate the shooting direction indication surface 60A than when the shooting direction indication surface 60A is designated by designating vertices.

[0132] The polyhedron is not limited to a regular hexahedron, and may be a guidance image 64 in the form of a regular tetrahedron as the original guidance image, as shown in Fig. 16, for example, or may be a hexahedron or tetrahedron with different side lengths. Furthermore, the guidance image using a polyhedron is not limited to having only the shooting direction indication surface 60A as described above, and may be a guidance image 66 including the shooting direction indication surface 60A and a columnar image 66A leading to the target, as shown in Fig. 17, for example. Note that, to avoid confusion, the shooting direction indication surface 60A and the target are not shown in Fig. 17.

[0133] 18, in addition to the guidance image 60, a configuration may be adopted in which a plurality of polyhedrons 60B, which have the same shape and dimensions as the polyhedron of the guidance image 60 but do not have a shooting direction indicating surface 60A, are displayed side by side on the display unit 35 of the terminal device 30 so as to encompass the area including the object 50. According to this configuration, even if the object 50 is located relatively far away, it becomes easier to grasp the required shooting position and orientation designated by the remote instructor.

[0134] Although the embodiments have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications and improvements can be made to the above embodiments without departing from the gist of the invention, and such modifications and improvements are also included in the technical scope of the present invention.

[0135] Furthermore, the above-described embodiments do not limit the inventions described in the claims, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. The above-described embodiments include inventions at various stages, and various inventions can be extracted by combining the multiple disclosed constituent elements. Even if some constituent elements are deleted from all of the constituent elements shown in the embodiments, as long as the effect is still obtained, the configuration from which these constituent elements are deleted can be extracted as an invention.

[0136] Furthermore, in the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0137] Furthermore, in the above embodiment, the guidance image display process and the remote support process are described as being implemented by a software configuration using a computer by executing a program, but the present invention is not limited to this. For example, the guidance image display process and the remote support process may be implemented by a hardware configuration or a combination of a hardware configuration and a software configuration.

[0138] Furthermore, the configurations of the remote device 10 and the terminal device 30 described in the above embodiment are merely examples, and it goes without saying that unnecessary parts may be deleted or new parts may be added within the scope of the present invention.

[0139] Furthermore, the flow of the guidance image display processing and remote support processing described in the above embodiment is also an example, and it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope of the gist of the present invention. [Explanation of symbols]

[0140] 10 Remote Device 11 CPU 11A Second receiving unit 11B Second display control unit 11C Reception 11D Derivation part 11E Second Transmission Unit 12 Memory 13 Storage section 13A Remote Support Program 14 Input section 15 Display section 15A Requested image selection button 15B Recent image selection button 15a~15d Images 16 Media reading and writing device 17 Recording Media 18 Communication I / F section 30 Terminal Equipment 31 CPU 31A First Transmitter 31B First receiving unit 31C First display control unit 32 memory 33 Storage section 33A Guidance image display program 33B Filming-related information database 34 Input section 35 Display section 35A Photographed Image 35B Guidance Image 35C Shooting direction image 35D marker 35E Exit button 36 Media reading and writing device 37 Recording Media 38 Photography Department 39. Mike 40 Position detection unit 42 Radio Communication Department 50 Objects 60 Guided Images 60A Shooting direction indicator 60B Polyhedron 60a~60h apex 64 Guided Image 66 Guided Image 66A Column Image 80 Network 90 Remote Support System

Claims

1. A remote support system including a terminal device and a remote device, the terminal device comprises a first processor; The first processor transmitting a plurality of pieces of image information obtained by photographing the object from different directions to the remote device; receiving display information derived by the remote device using the image information for displaying a guidance image that guides the user to a requested photographing position; Displaying a guidance image indicated by the received display information; the remote device comprises a second processor; The second processor receiving the plurality of pieces of image information from the terminal device; Displaying images indicated by the received plurality of pieces of image information; Accepting a selection of an image taken from a requested position from the displayed images; deriving the display information using the received image; transmitting the derived display information to the terminal device; It is a remote support system, the first processor changes the display state of the guidance image according to a deviation between the photographing position taken by the user and a required photographing position. Remote support system.

2. The image information is either image information representing a still image or image information representing a moving image. The remote assistance system according to claim 1 .

3. When the image information is image information representing the moving image, the first processor selects one still image for every predetermined number of images in the moving image and transmits the selected still image to the remote device. The remote assistance system according to claim 2 .

4. The guidance image is a three-dimensional image. The remote support system according to any one of claims 1 to 3.

5. the three-dimensional image is virtually displayed in a three-dimensional space of a shooting area for the object, and is a three-dimensional image indicating a required shooting position and shooting direction for the object; The remote assistance system according to claim 4 .

6. The three-dimensional image is configured as a polyhedron. The remote assistance system according to claim 5 .

7. The guidance image is an image that becomes a predetermined state when the terminal device is in a required photographing position and orientation. The remote support system according to any one of claims 1 to 6.

8. the predetermined state is a distortion-free state; The remote assistance system according to claim 7.

9. The undistorted state is a rectangular state. The remote assistance system according to claim 8.

10. The different direction includes a direction directly facing the front of the object. The remote support system according to any one of claims 1 to 9.

11. The different directions include a direction in which the front, one side, and one top of the object are included in one image. The remote support system according to any one of claims 1 to 10.

12. The display state is at least one of a blinking interval, a display color, and a transmittance. The remote support system according to any one of claims 1 to 11.

13. When changing the transmittance, the first processor increases the transmittance as the amount of deviation decreases. The remote assistance system according to claim 12.

14. when a target position of the object is photographed, the first processor virtually displays a marker at the target position; The remote support system according to any one of claims 1 to 13.

15. the second processor, when the images indicated by the received plurality of image information do not include images taken from a required photographing position, transmits instruction information to the terminal device instructing the terminal device to take a photograph again; The remote support system according to any one of claims 1 to 14.

16. the instruction information includes identification information for identifying the image information obtained by photographing at a photographing position closest to the requested photographing position; The remote assistance system according to claim 15.

17. a processor; The processor: A plurality of pieces of image information obtained by photographing the object from different directions are transmitted to a remote device; receiving display information derived by the remote device using the image information for displaying a guidance image that guides the user to a requested photographing position; displaying a guidance image indicated by the received display information; a terminal device, the processor changes the display state of the guidance image according to the amount of deviation between the photographing position taken by the user and the required photographing position. Terminal device.

18. a processor; The processor: receiving a plurality of pieces of image information obtained by photographing an object from different directions from a terminal device; Displaying images indicated by the received plurality of pieces of image information; Accepting a selection of an image taken from a requested position from the displayed images; Using the received image, derive display information for displaying a guidance image that guides the user to the required photographing position; transmitting the derived display information to the terminal device; a remote device, the processor, when the images indicated by the received plurality of image information do not include images taken from a required photographing position, transmits instruction information to the terminal device instructing the terminal device to take a photograph again; the instruction information includes identification information for identifying the image information obtained by photographing at a photographing position closest to the requested photographing position; Remote device.

19. A plurality of pieces of image information obtained by photographing the object from different directions are transmitted to a remote device; receiving display information derived by the remote device using the image information for displaying a guidance image that guides the user to a requested photographing position; displaying a guidance image indicated by the received display information; Processing, changing the display state of the guidance image according to the amount of deviation between the photographing position taken by the user and the required photographing position; A guidance image display program for causing a computer to execute processing.

20. receiving a plurality of pieces of image information obtained by photographing an object from different directions from a terminal device; Displaying images indicated by the received plurality of pieces of image information; Accepting a selection of an image taken from a requested position from the displayed images; Using the received image, derive display information for displaying a guidance image that guides the user to the required photographing position; transmitting the derived display information to the terminal device; Processing, If the images indicated by the received plurality of pieces of image information do not include images taken from the required photographing position, instruction information instructing the terminal device to take a photograph again is transmitted; the instruction information includes identification information for identifying the image information obtained by photographing at a photographing position closest to the requested photographing position; A remote support program that allows a computer to execute processing.

Images