Information processing apparatus, information processing method, and program

Abstract

To shorten the time from when a client requests an image after an electric pole is made non-conductive to when the client acquires the image.SOLUTION: The information processing device includes a detector that detects a parallel plane parallel to a display surface of the display from an image displayed by the display, an information acquirer that acquires rising protective tube installation position information indicating an installation position of a rising protective tube that protects a rising portion of a power lead-in wire from the ground, and a synthesizer that generates synthesized image information indicating a synthesized image obtained by synthesizing an image of the rising protective tube with the image displayed by the display based on the parallel plane detected by the detector and the rising protective tube installation position information acquired by the information acquirer.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] The so-called utility pole removal is being implemented, which involves burying aboveground distribution lines underground and eliminating utility poles. As a result of this, power lines to homes will be pulled underground, making it necessary to install rising protection pipes on the home side. When negotiating the installation of rising protection pipes, negotiation materials are used. The materials for negotiations were prepared by taking photographs of the location where the riser protection pipes would be installed on-site, and then using spreadsheet software, presentation software, etc., in the office, drawing an image of the area before the riser protection pipes were installed onto an image of the area after the riser protection pipes had been installed.

[0003] Regarding image images of ground equipment, there is known a technique for shortening the time from when a requester requests an image image of the ground equipment after it has been installed until when the requester acquires the image image (see, for example, Patent Document 1). According to this technique, an information processing device includes a display unit, an information acquisition unit that acquires installation position information that is information indicating the installation position of the ground equipment, a derivation unit that derives the size of the ground equipment to be displayed on the display unit, and a composition unit that composites and displays the ground equipment on the display unit based on the installation position information and the size of the ground equipment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-191044 Summary of the Invention [Problem to be solved by the invention]

[0005] Since it is not possible to create an image of what the rising protection pipe will look like after it has been installed on-site, it is necessary to take photographs of the location where the rising protection pipe will be installed, then return to the office to edit the photographs and create the materials. If you want to create an image of what the rising protection pipe will look like after it has been installed, photographed from multiple directions, you will need to take photographs of the location where the rising protection pipe will be installed on site from multiple directions. The above applies not only to rising protection pipes, but also to buried pipes that pull power lines from underground, the excavation range when pulling power lines from underground, and image images created in conjunction with the removal of utility poles for above-ground equipment, etc. The present invention has been made in consideration of the above-mentioned points, and its purpose is to provide an information processing device, an information processing method, and a program that can shorten the time from when a client requests an image of what the area will look like after utility poles are removed until the client obtains that image. [Means for solving the problem]

[0006] One aspect of the present invention is an information processing device that includes a detection unit that detects parallel planes that are parallel to the display surface of the display unit from an image displayed by the display unit; an information acquisition unit that acquires rising protection pipe installation location information, which is information indicating the installation location of a rising protection pipe that protects the rising portion of a power feed line from underground; and a synthesis unit that creates synthetic image information that shows a synthetic image in which an image of the rising protection pipe is synthesized with the image displayed by the display unit based on the parallel planes detected by the detection unit and the rising protection pipe installation location information acquired by the information acquisition unit. In one embodiment of the information processing device of the present invention, the detection unit detects a vertical plane perpendicular to the display surface of the display unit from the image displayed by the display unit, the information acquisition unit acquires buried pipe installation location information, which is information indicating the installation location of a buried pipe that pulls the power supply line from underground, and the synthesis unit creates synthetic image information showing a synthetic image in which an image of a protective pipe is synthesized with the image displayed by the display unit based on the vertical plane detected by the detection unit and the buried pipe installation location information acquired by the information acquisition unit. In one embodiment of the information processing device of the present invention, the information processing device further includes a modification unit that moves the position of the image of the buried pipe included in the composite image, and the composition unit creates composite image information by combining the image of the buried pipe whose position has been moved by the modification unit with the image displayed by the display unit. In one embodiment of the information processing device of the present invention, the information processing device further includes a modification unit that modifies either or both of the size and the depth to which the image of the buried pipe included in the composite image, and the modification unit creates composite image information by combining the image of the buried pipe, whose size and either or both of the depth to which the image is buried, modified by the modification unit with the image displayed by the display unit. In one embodiment of the information processing device of the present invention, the information processing device further includes a modification unit that scrolls the image of the buried pipe included in the composite image, and the composition unit creates composite image information by combining the image of the buried pipe that the modification unit has scrolled with the image displayed by the display unit. In one embodiment of the information processing device of the present invention, the information acquisition unit acquires excavation range information, which is information indicating the excavation range when the power line is pulled in from underground, and the synthesis unit creates synthetic image information by synthesizing an image of the excavation range with the image displayed by the display unit based on the vertical plane detected by the detection unit and the excavation range information acquired by the information acquisition unit. In one embodiment of the information processing device of the present invention, the information processing device further includes a modification unit that moves the position of the image of the excavation range included in the composite image, and the composition unit creates composite image information by combining the image of the excavation range whose position has been moved by the modification unit with the image displayed by the display unit. In one embodiment of the information processing device of the present invention, the information processing device further includes a modification unit that changes the size of the image of the excavation range included in the composite image, and the modification unit creates composite image information by combining the image displayed by the display unit with the image of the excavation range whose size has been changed by the modification unit. In one embodiment of the information processing device of the present invention, the information processing device further includes a modification unit that scrolls the image of the excavation range included in the composite image, and the composition unit creates composite image information by combining the image of the excavation range scrolled by the modification unit with the image displayed by the display unit. In one embodiment of the information processing device of the present invention, the information acquisition unit acquires ground equipment installation location information, which is information indicating the installation location of ground equipment, and the information processing device further includes a derivation unit that derives the size of the ground equipment to be displayed on the display unit, and the synthesis unit creates synthetic image information by synthesizing an image of the ground equipment with the image displayed by the display unit based on the size of the ground equipment derived by the derivation unit, the vertical surface detected by the detection unit, and the ground equipment installation location information acquired by the information acquisition unit. One aspect of the present invention is an information processing method executed by a computer, which detects parallel planes parallel to the display surface of a display unit from an image displayed by the display unit, obtains rising protection pipe installation position information which is information indicating the installation position of a rising protection pipe that protects the rising portion of a power feed line from underground, and creates composite image information which shows a composite image in which an image of the rising protection pipe is combined with the image displayed by the display unit based on the parallel planes and the rising protection pipe installation position information. One aspect of the present invention is a program that causes a computer to detect parallel planes that are parallel to the display surface of a display unit from an image displayed by the display unit, obtain rising protection pipe installation location information that indicates the installation location of a rising protection pipe that protects the rising portion of a power feed line from underground, and create composite image information that shows a composite image in which an image of the rising protection pipe is combined with the image displayed by the display unit based on the parallel planes and the rising protection pipe installation location information. [Effects of the Invention]

[0007] According to an embodiment of the present invention, it is possible to reduce the time from when a client requests an image of the area after utility pole removal until the client obtains the image. [Brief explanation of the drawings]

[0008] [Figure 1A] 10 is a diagram showing an example of an image displayed by the information processing device of this embodiment after the installation of a rising protection pipe and a buried pipe. FIG. [Figure 1B] FIG. 10 is a diagram showing an example of an image displayed by the information processing device of this embodiment after ground facilities have been installed. [Figure 2] FIG. 1 is a block diagram illustrating an example of an information processing apparatus according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of processing performed by the information processing device according to the present embodiment. [Figure 5] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 6A] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6E] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6F] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6G] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 6H] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 7] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 8A]FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8E] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8F] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8G] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8H] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8I] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8J] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 8K] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 9] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 10A] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 10B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 10C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 10D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 10E] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 10F]FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 11] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 12A] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12E] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12F] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12G] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 12H] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 13] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 14A] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 14B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 14C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 14D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 14E] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 15] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 16A]FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 16B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 16C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 16D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 16E] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 16F] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 17] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. [Figure 18A] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 18B] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 18C] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 18D] FIG. 10 is a diagram illustrating an example of a screen displayed by the information processing device according to the embodiment. [Figure 19] 10 is a flowchart illustrating an example of an operation of the information processing device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Next, an information processing apparatus, an information processing method, and a program according to the present embodiment will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiments to which the present invention is applied are not limited to the following embodiments. In all the drawings for explaining the embodiments, the same reference numerals are used for components having the same functions, and repeated explanations will be omitted. Furthermore, in this application, "based on XX" means "based on at least XX," and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been calculated or processed. "XX" is any element (for example, any information).

[0010] (Embodiment) (Information processing device) FIG. 1A is a diagram showing an example of an image displayed by the information processing device of this embodiment after the installation of a rising protection pipe and a buried pipe. As power lines to houses are now laid underground, rising protection pipes are installed in the walls of houses to protect the rising portions of the power line from underground, and buried pipes are buried underground by excavating the ground. Figure 1A displays an image of a house (hereinafter referred to as "house image H"), and also displays an image of the rising protection pipe (hereinafter referred to as "rising protection pipe image 10") alongside an image of the wall of house image H (hereinafter referred to as "wall image WL"), and an image of the buried pipe buried underground (hereinafter referred to as "buried pipe image 20"). In other words, rising protection pipe image 10 of the installed rising protection pipe and buried pipe image 20 of the buried pipe are displayed on house image H before the rising protection pipe is installed and buried.

[0011] FIG. 1B is a diagram showing an example of an image displayed by the information processing device of this embodiment after ground equipment has been installed. Ground equipment is installed on a road. In FIG. 1B, an image of the newly installed ground equipment (hereinafter referred to as "ground equipment image 30") is displayed on an image of the road before the ground equipment was installed (hereinafter referred to as "road image RO"). In other words, ground equipment image 30 of the installed ground equipment is displayed on an image before the ground equipment was installed on the road.

[0012] The ground equipment will now be described. Ground equipment is classified into, for example, multi-circuit switchgear and ground transformers. Multi-circuit switchgear is a branching equipment for high-voltage wiring and is equipment for switching during maintenance work. Multi-circuit switchgear is stored in a switchgear tower. The front of the switchgear tower has a switching section. For example, the switching section has a double-door structure. By opening the switching section, maintenance and inspection of the multi-circuit switchgear are possible.

[0013] Ground transformers are equipment that step down high voltage electricity to low voltage electricity for general household use. Ground transformers are installed in transformer towers. Ground transformers are housed in transformer towers. The front of the transformer tower has a switchgear. For example, the switchgear is a double-door type. By opening the switchgear, maintenance and inspection of the ground transformer is possible. The size of the switchgear tower and the size of the transformer tower are specified.

[0014] When displaying the ground equipment image 30 after the ground equipment has been newly installed, the information processing device 100 displays either or both of the ground equipment image 30 after the switch tower has been newly installed and the ground equipment image 30 after the transformer tower has been newly installed. The user can set the information processing device 100 to display the ground equipment image 30 after the switch tower has been newly installed or the ground equipment image 30 after the transformer tower has been newly installed.

[0015] A case will be described where a rising protection pipe image 10 of the installed rising protection pipe and a buried pipe image 20 of the buried buried pipe are displayed on a house image H before the installation of the rising protection pipe and the buried pipe are buried (Figure 1A). The information processing device 100 uses augmented reality (AR) technology to display a vertical protection pipe image 10 and a buried pipe image 20. For example, when negotiating with a resident of a house about installing a vertical protection pipe, the information processing device 100 presents the resident with an image of the vertical protection pipe installed on the wall of the house and an image of the buried pipe buried there. In this embodiment, as an example, marker-type vision-based and markerless vision-based augmented reality technologies are used. Marker-type vision-based augmented reality technology refers to presenting information by recognizing pre-registered figures using image recognition technology, spatial recognition technology, etc. Markerless vision-based augmented reality technology refers to presenting information by recognizing information read from a camera, such as space, scenery, or objects, rather than pre-registered figures, using image recognition technology, spatial recognition technology, etc.

[0016] Examples of the information processing device 100 include a personal computer, a server, a smartphone, a tablet computer, an industrial computer, etc. To display an image of the state after the erection protection pipe has been installed and an image of the state after the buried pipe has been buried on the information processing device 100, an image display application is installed in advance.

[0017] The user operates the information processing device 100 to cause the information processing device 100 to display an image H of the house before the installation of the upright protection pipe and the buried pipe. In this state, a predetermined operation is performed on the information processing device 100 to cause the information processing device 100 to display an upright protection pipe image 10 of the upright protection pipe and an underground pipe image 20 of the buried pipe. According to FIG. 1A, a rising protection pipe image 10 of a rising protection pipe installed along a wall image WL of a house image H and a buried pipe image 20 of a buried pipe buried so as to be connected to the rising protection pipe image 10 are displayed.

[0018] A case where a newly installed ground equipment image 30 is displayed on a road image RO before the ground equipment is installed (FIG. 1B) will be described. The information processing device 100 uses augmented reality technology to display the ground equipment image 30. For example, if a requester such as a road administrator or a resident requests an image of what the ground equipment will look like after it has been installed before the equipment is installed, the information processing device 100 presents the image to the requester. In this embodiment, as an example, marker-type vision-based and markerless vision-based augmented reality technologies are used. To display an image of the state after the ground equipment has been newly installed on the information processing device 100, an image display application is installed in advance. The user operates the information processing device 100 to display a road image RO before the ground equipment is newly installed on the information processing device 100. In this state, a ground equipment image 30 is displayed by performing a predetermined operation on the information processing device 100. As shown in FIG. 1B, the ground equipment image 30 is displayed on the road image RO.

[0019] (Information processing device) FIG. 2 is a block diagram illustrating an example of an information processing apparatus according to the embodiment. The information processing device 100 includes, for example, a communication unit 102, a memory unit 103, an information processing unit 130, a display unit 140, an operation unit 106, an imaging unit 160, a GNSS receiving unit 170, an acceleration sensor 180, and a gyro sensor 190. The communication unit 102 is realized by a communication module. The communication unit 102 communicates with other terminal devices via a network. For example, the communication unit 102 transmits image information including an image H of the installation of the vertical protection pipe and the house before the buried pipe is laid, a vertical protection pipe image 10, and a buried pipe image 20 to the terminal device of the requester. Also, for example, the communication unit 102 transmits image information including a road image RO and a ground equipment image 30 to the terminal device of the requester.

[0020] The storage unit 103 is realized by, for example, a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a flash memory, or a hybrid storage device that combines two or more of these. Instead of being provided as part of the information processing device 100, part or all of the storage unit 103 may be realized by an external device that the processor of the information processing device 100 can access via a network, such as a network attached storage (NAS) or an external storage server. The storage unit 103 stores a program 104 executed by the information processing unit 130 and an application 105 such as an image display application.

[0021] The display unit 140 is realized by, for example, a liquid crystal display or an organic EL (Electroluminescence) display, and displays various information under the control of the CPU. The operation unit 106 is configured by, for example, a touch panel, detects a touch operation on a menu button or the like displayed on the display unit 140, and outputs the detection result of the touch operation to the information processing unit 130.

[0022] The imaging unit 160 is configured with a CCD (Charge Coupled Devices) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, or the like, and is fixed to the housing of the information processing device 100. The imaging unit 160 outputs image data obtained by capturing an image directly or using a half mirror, or the like, to the information processing unit 130.

[0023] The GNSS receiver 170 is realized by a global navigation satellite system (GNSS) or the like. In response to a positioning command from the information processing unit 130, the GNSS receiver 170 receives navigation signals transmitted by navigation satellites such as GPS, GLONASS, Galileo, and Quasi-Zenith Satellite System (QZSS), and demodulates the navigation signals. Then, based on the demodulated navigation signals, the GNSS receiver 170 performs a predetermined positioning calculation to perform positioning of the information processing device 100, thereby obtaining position information. The GNSS receiver 170 outputs the position information to the information processing unit 130.

[0024] Acceleration sensor 180 is configured by, for example, a triaxial acceleration sensor, and measures the magnitude of acceleration in three mutually orthogonal axial directions. Acceleration sensor 180 outputs information indicating the magnitude of acceleration in the three axial directions (hereinafter referred to as "acceleration information") to information processing unit 130. The gyro sensor 190 is configured by, for example, a three-axis gyro sensor, and measures the magnitude of the earth's axis in each of three mutually perpendicular axial directions. The gyro sensor 190 inputs information indicating the magnitude of the earth's axis in each of the three axial directions (hereinafter referred to as "gyro information") to the information processing unit 130.

[0025] All or part of the information processing unit 130 is a functional unit (hereinafter referred to as a software functional unit) realized by a processor such as a CPU executing a program 104 or an application 105 stored in the storage unit 103. All or part of the information processing unit 130 may be realized by hardware such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), or may be realized by a combination of a software function unit and hardware. The information processing unit 130 includes, for example, a detection unit 132, an information acquisition unit 134, a modification unit 135, a derivation unit 136, and a synthesis unit 138.

[0026] When the user operates the operation unit 106 to start the image display application, the detection unit 132 detects, from the image acquired by the imaging unit 160, either or both of a vertical plane that is perpendicular to the display surface of the display unit 140 and a parallel plane that is parallel to the display surface of the display unit 140. An example of a vertical plane is the ground, and an example of a parallel plane is a wall surface of a building such as a house. The synthesis unit 138 outputs to the display unit 140 an image obtained by synthesizing an image showing the operation buttons and the detected plane (hereinafter referred to as the "detected plane image DP") with the image acquired by the imaging unit 160. The detected plane is the ground or wall surface automatically detected by the detection unit 132. For example, the detection unit 132 detects a vertical plane such as the ground in the ground equipment display mode, which is a mode that displays the ground equipment image 30, and a parallel plane such as a wall surface in the rising protection pipe display mode, which is a mode that displays the rising protection pipe image 10.

[0027] 3 is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 3, the screen includes, in an image acquired by the imaging unit 160, a menu button 110, an object switching button 111, a major mode button 112, information indicating the detected ground surface (hereinafter referred to as a "detected ground surface image DG"), which is the ground surface detected as a detected plane image DP, and a shutter button SH. In the information processing device 100, the directions parallel to the display surface of the display unit 140 are defined as the X-axis and Y-axis, the X-axis is defined as the direction parallel to the horizontal plane when the information processing device 100 is held, and the direction perpendicular to the X-axis is defined as the Y-axis. The direction perpendicular to the X-axis and Y-axis is defined as the Z-axis.

[0028] When pressed, the menu button 110 displays a menu for making various adjustments to objects such as rising protection pipes and ground equipment. When pressed, the object switching button 111 switches between ground equipment display mode, rising protection pipe display mode, and buried display mode. For example, the initial state may be set to ground equipment display mode. Below, the explanation will continue assuming that the initial state is set to ground equipment display mode. The measure mode button 112 starts measure mode, allowing measures to be displayed. Returning to Figure 2, the explanation will continue.

[0029] When a user operates the operation unit 106 to launch an image display application and shakes the information processing device 100, the information acquisition unit 134 acquires information indicating the amount of deviation of feature points (FP) contained in an image acquired by the imaging unit 160. Furthermore, the information acquisition unit 134 controls the acceleration sensor 180 to acquire acceleration information generated when the information processing device 100 is shaken. Furthermore, the information acquisition unit 134 acquires gyro information generated by shaking the information processing device 100 by controlling the gyro sensor 190. The information acquisition unit 134 outputs the acquired information indicating the amount of deviation of the feature points, the acceleration information, and the gyro information to the derivation unit 136.

[0030] For example, the information acquisition unit 134 recognizes feature points (FP) included in the image acquired by the imaging unit 160. Here, a feature point refers to a detectable point on an image. Specifically, a feature point is an isolated point such as a point of maximum or minimum intensity, the end point of a curve, or a point on a curve with a maximum curvature. The information acquisition unit 134 acquires the amount of shift of the feature points caused by shaking, and analyzes the acquired amount of shift of the feature points together with information on acceleration and angular velocity caused in the information processing device 100 by shaking. Based on the analysis results, the information acquisition unit 134 recognizes a plane such as the ground and determines the size of an object formed by a collection of feature points. Here, an example of an object is an object of known size, such as a utility pole.

[0031] When a user taps on the display screen of the display unit 140, the detection unit 132 detects that a tap has been made. When the detection unit 132 detects that a tap has been made on the display screen of the display unit 140, the information acquisition unit 134 acquires information indicating the position where the tap was made (hereinafter referred to as "tap position information"). The information acquisition unit 134 outputs the acquired tap position information to the derivation unit 136. For example, the user pokes (tap) the position on the display screen of the display unit 140 where the rising protection pipe is to be installed with the tip of a pen or the tip of a finger. Here, the position where the tap is made may be the base of the end of the rising protection pipe to be installed.

[0032] Furthermore, the information acquisition unit 134 acquires information indicating that the shutter has been pressed, output by the operation unit 106, and based on the acquired information indicating that the shutter has been pressed, outputs a positioning command to the GNSS receiving unit 170. The information acquisition unit 134 acquires the position information (longitude information, latitude information) of the information processing device 100 output by the GNSS receiving unit 170, and acquires address information based on the acquired position information (longitude information, latitude information). Specifically, the information acquisition unit 134 accesses a server (not shown) connected to a network (not shown) such as the Internet from the communication unit 102. Here, the server acquires address information corresponding to the location information (longitude information, latitude information) from the location information (longitude information, latitude information). The information acquisition unit 134 transmits the location information (longitude information, latitude information) from the communication unit 102 to the server, and the server acquires address information corresponding to the transmitted location information (longitude information, latitude information) from the communication unit 102. The information acquisition unit 134 outputs the acquired address information to the synthesis unit 138.

[0033] The information acquisition unit 134 acquires information indicating the type of rise-up protection pipe that is output when the user operates the operation unit 106. Here, the information indicating the type of rise-up protection pipe includes information indicating the size of the rise-up protection pipe. In this embodiment, the explanation will continue with a case where information indicating the diameter of the rise-up protection pipe is applied as an example of information indicating the size of the rise-up protection pipe. The information acquisition unit 134 outputs the acquired information indicating the diameter of the rise-up protection pipe to the derivation unit 136. The information acquisition unit 134 acquires information indicating the type of ground equipment that is output by the user operating the operation unit 106 to display the ground equipment image 30. Here, the information indicating the type of ground equipment includes information indicating the size of the ground equipment. In this embodiment, the explanation will continue with a case where information indicating the width, height, and depth of the ground equipment is applied as an example of information indicating the size of the ground equipment. Furthermore, the information acquisition unit 134 acquires information indicating the front face of the ground equipment that is output when the user operates the operation unit 106. The information acquisition unit 134 outputs the acquired information indicating the width, height, and depth of the ground equipment and the information indicating the front face of the ground equipment to the derivation unit 136.

[0034] The derivation unit 136 acquires the information indicating the amount of deviation of the feature points, the acceleration information, and the gyro information output by the information acquisition unit 134. Based on the acquired information indicating the amount of deviation of the feature points, the acceleration information, and the gyro information, the derivation unit 136 recognizes flat surfaces such as the ground and wall surfaces, and also recognizes the size of an object formed by a collection of feature points. Fig. 4 is a diagram for explaining an example of processing of the information processing device according to this embodiment. Fig. 4 shows processing in which the derivation unit 136 derives the actual distance d between the information processing device 100 and an object formed by a set of feature points included in an image acquired by the imaging unit 160 when the information processing device 100 is shaken left and right.

[0035] It is assumed that the information processing device 100 moves between position A and position B by being shaken left and right. Here, the length between position A and position B is defined as distance L. A feature point FP is displayed on the display unit 140 of the information processing device 100 at positions A and B. At position A, an angle α is defined as an angle formed between a line connecting the information processing device 100 and the actual position of the feature point (FP) included in an image acquired by the imaging unit 160 of the information processing device 100 and a line connecting position A and position B. At position B, an angle β is defined as an angle formed between a line connecting the information processing device 100 and the actual position of the feature point (FP) included in an image acquired by the imaging unit 160 of the information processing device 100 and a line connecting position A and position B.

[0036] The derivation unit 136 derives the angle α and the angle β using the gyro information. The derivation unit 136 derives the distance d between the information processing device 100 and the actual position of the feature point FP from equation (1) using the principle of triangulation based on the derived angles α, β, and distance L. d=sinα×sinβ×L / sin(α+β) (1) 4 shows a case where the distance d between the information processing device 100 and the actual position of one feature point FP is derived, but the derivation unit 136 derives the distance d for all feature points included in an image acquired by the imaging unit 160 of the information processing device 100. In this way, by deriving the distance d between the information processing device 100 and the actual position of the feature point FP, the actual size (dimension) of an object present in an image acquired by the imaging unit 160 can be grasped.

[0037] When the derivation unit 136 acquires the information indicating the diameter of the rising protection pipe output by the information acquisition unit 134, the derivation unit 136 performs the following process. The derivation unit 136 derives the actual size of an object of known size based on the distance d between the information processing device 100 and each of a plurality of feature points FP that form an object of known size, such as a utility pole. The derivation unit 136 derives the size (diameter) of the rising protecting pipe image 10 based on the derived size (diameter) of the object of known size. Specifically, the derivation unit 136 derives the size of the rising protection pipe image 10 based on the actual size (diameter) of the rising protection pipe whose derived size is known and the information indicating the type of rising protection pipe output by the information acquisition unit 134.

[0038] The derivation unit 136 calculates a ratio, such as the ratio between the actual diameter of the rising protection pipe to be installed output by the information acquisition unit 134 and the size of the known size displayed on the display unit 140, and based on the calculated ratio, derives the diameter of the rising protection pipe image 10 when displayed on the display unit 140. The derivation unit 136 outputs information indicating the derived diameter of the rising protection pipe image 10 and the tap position information output by the information acquisition unit 134 to the synthesis unit 138.

[0039] When the derivation unit 136 acquires the information indicating the width, height, and depth of the ground equipment output by the information acquisition unit 134 and the information indicating the front face of the ground equipment, the derivation unit 136 performs the following process. The derivation unit 136 calculates a ratio, such as a ratio between the actual size of the acquired ground equipment to be newly installed and the size of a known object such as a utility pole displayed on the display unit 140, and derives the width, height, and depth of the ground equipment image 30 when it is displayed on the display unit 140 based on the calculated ratio. The derivation unit 136 outputs the derived information indicating the width, height, and depth of the ground equipment image 30, the information indicating the front face of the ground equipment, and the tap position information output by the information acquisition unit 134 to the synthesis unit 138.

[0040] When the synthesis unit 138 acquires the information indicating the diameter of the rising protection pipe image 10 and the tap position information output by the derivation unit 136, it performs the following process: The synthesis unit 138 synthesizes, onto the image displayed by the display unit 140 before the rising protection pipe was installed, a rising protection pipe image 10 with a diameter corresponding to the information indicating the diameter of the rising protection pipe image 10 and a length corresponding to the information indicating the preset length of the rising protection pipe image 10, starting from the position of the wall surface indicated by the tap position information. Furthermore, when the synthesis unit 138 acquires the address information output by the information acquisition unit 134, it synthesizes the rising protection pipe image 10 with the image before the rising protection pipe was installed, which is displayed by the display unit 140, and also synthesizes the address information. The synthesis unit 138 outputs the synthesized image obtained by the synthesis to the display unit 140. The display unit 140 displays the image output by the synthesis unit 138.

[0041] When the user operates the operation unit 106 to change the state of the rising protecting pipe image 10, the change unit 135 acquires change information that is output by the user operating the operation unit 106. The change unit 135 outputs the acquired change information to the synthesis unit 138. Changes to the state include changes in movement and inclination. When the synthesis unit 138 acquires the change information output by the change unit 135, the synthesis unit 138 outputs an image in which the state of the displayed rising protecting pipe image 10 has been changed based on the acquired change information to the display unit 140. The display unit 140 displays the image output by the synthesis unit 138.

[0042] The synthesis unit 138 outputs the synthesized image obtained by the synthesis to the display unit 140. Furthermore, when the synthesis unit 138 acquires information indicating the respective lengths of the width, height, and depth of the ground equipment image 30 output by the derivation unit 136, information indicating the front of the ground equipment image 30, and tap position information, it synthesizes the ground equipment image 30, starting from the position of the ground surface (ground) indicated by the tap position information, onto the road image RO before the new ground equipment was installed that is displayed by the display unit 140, with the ground equipment image 30 having the direction corresponding to the information indicating the respective lengths of the width, height, and depth of the ground equipment image 30 and corresponding to the information indicating the front of the ground equipment image 30 as the front. Furthermore, when the synthesis unit 138 acquires the address information output by the information acquisition unit 134, it synthesizes the ground equipment image 30 and the address information onto the road image RO before the ground equipment was newly installed, which is displayed by the display unit 140. The synthesis unit 138 outputs the synthesized image obtained by the synthesis to the display unit 140. The display unit 140 displays the image output by the synthesis unit 138.

[0043] When the user operates the operation unit 106 to change the state of the ground equipment image 30, the change unit 135 acquires change information that is output by the user operating the operation unit 106. The change unit 135 outputs the acquired change information to the composition unit 138. Changes to the state include changes in color, transparency, duplication, and deletion. Furthermore, when the composition unit 138 acquires the change information output by the change unit 135, the composition unit 138 outputs a composite image in which the state of the displayed ground equipment image 30 has been changed based on the acquired change information to the display unit 140. The display unit 140 displays the image output by the composition unit 138.

[0044] (Operation of information processing device) FIG. 5 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-1) The information processing device 100 is shaken left and right. The information acquisition unit 134 acquires acceleration information by controlling the acceleration sensor 180. The information acquisition unit 134 acquires angular velocity information by controlling the gyro sensor 190. (Step S2-1) The detection unit 132 detects and recognizes a plane based on the image acquired by the imaging unit 160.

[0045] (Step S3-1) The synthesis unit 138 acquires the plane recognition result from the detection unit 132, and creates synthesized image information that indicates a synthesized screen by synthesizing the acquired plane recognition result and multiple operation buttons with the image acquired by the imaging unit 160 and displayed by the display unit 140. The synthesis unit 138 outputs the created synthesized screen information to the display unit 140. The display unit 140 displays the synthesized screen based on the synthesized screen information output by the synthesis unit 138. 6A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 6A, the screen includes a menu button 110, an object switching button 111, a major mode button 112, a detected ground image DG, and a shutter button SH. Returning to FIG. 5, the description will continue.

[0046] (Step S4-1) The detection unit 132 determines whether or not the menu button 110 has been pressed. The case where the menu button 110 has been pressed will be described later. (Step S5-1) If the menu button 110 is not pressed, the detection unit 132 determines whether or not the object switching button 111 is pressed. The case where the object switching button 111 is pressed will be described later. (Step S6-1) The detection unit 132 detects the position tapped by the user when the object switching button 111 is not pressed. The information acquisition unit 134 acquires tap position information indicating the position tapped by the user detected by the detection unit 132.

[0047] (Step S7-1) The synthesis unit 138 acquires tap position information from the information acquisition unit 134. Based on the acquired tap position information, the synthesis unit 138 determines to display the ground device image 30. This is because the initial state is set to the ground device display mode. (Step S8-1) The composition unit 138 creates a composite image by combining the ground equipment image 30 and the plurality of adjustment button images with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 6B is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 6B, the display unit 140 displays a ground equipment image 30 on the detected ground image DG (in the positive Y-axis direction of the detected ground image DG) on the screen shown in Fig. 6A. The adjustment button image includes a move button 113, a rotate button 114, a tilt change button 115, and a confirm button 116. Returning to Fig. 5, the explanation will be continued.

[0048] (Step S9-1) The detection unit 132 determines whether or not the major mode button 112 has been pressed. The case where the major mode button 112 has been pressed will be described later. (Step S10-1) If the major mode button 112 is not pressed, the detection unit 132 determines whether the move button 113 is pressed.

[0049] (Step S11-1) When detection unit 132 determines that movement button 113 has been pressed, composition unit 138 creates a composite image by further combining the position movement designation diagram PMSD with the image displayed by display unit 140. Composition unit 138 outputs composite image information indicating the created composite image to display unit 140. Display unit 140 displays the composite image based on the composite image information output by composition unit 138. FIG. 6C is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 6C, the screen shown in FIG. 6B is further superimposed with a position movement designation diagram PMSD. For example, the position movement designation diagram PMSD includes two arrows, one parallel to the X-axis and the other parallel to the Z-axis. The user specifies the position to which the ground equipment image 30 is to be moved by swiping the ground equipment image 30 in the direction to which the user wishes to move it. The detection unit 132 determines whether the confirm button 116 has been pressed. If the confirm button 116 has not been pressed, the detection unit 132 proceeds to step S11-1.

[0050] (Step S12-1) When the detection unit 132 detects that the confirmation button 116 has been pressed, the change unit 135 moves the ground device image 30 to a position designated by the user to which the ground device image 30 is to be moved. Fig. 6D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 6D, the screen shown in Fig. 6C shows the ground equipment image 30 moved to a position specified by the user and displayed, and the display of the detected ground image DG is stopped. Returning to Fig. 5, the explanation will continue. (Step S13-1) If the movement button 113 is not pressed in step S10-1, the detection unit 132 determines whether the rotation button 114 is pressed.

[0051] (Step S14-1) When detection unit 132 determines that rotation button 114 has been pressed, composition unit 138 creates a composite image by further combining the rotation movement specification diagram RMSD with the image being displayed. Composition unit 138 outputs composite image information indicating the created composite image to display unit 140. Display unit 140 displays the composite image based on the composite image information output by composition unit 138. FIG. 6E is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 6E, the screen shown in FIG. 6B is further combined with a rotation movement designation diagram RMSD and displayed. For example, the rotation movement designation diagram RMSD includes an arrow designating rotation of the ground equipment image 30 around a straight line perpendicular to a plane formed by the X-axis and Z-axis. The user designates the direction in which to rotate the ground equipment image 30 by swiping the ground equipment image 30 in the desired direction. The detection unit 132 determines whether the confirm button 116 has been pressed. If the confirm button 116 has not been pressed, the detection unit 132 proceeds to step S14-1.

[0052] (Step S15-1) When the detection unit 132 detects that the confirmation button 116 has been pressed, the change unit 135 rotates the ground device image 30 in the direction in which the ground device image 30 is to be rotated, which is specified by the user. Fig. 6F is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 6F, the screen shows the ground equipment image 30 rotated in the direction specified by the user on the screen shown in Fig. 6E. Returning to Fig. 5, the explanation will be continued. (Step S16-1) If the rotation button 114 is not pressed in step S13-1, the detection unit 132 determines whether the tilt change button 115 is pressed.

[0053] (Step S17-1) When the detection unit 132 determines that the tilt change button 115 has been pressed, the synthesis unit 138 creates a composite image by further synthesizing the tilt change designation diagram TCSD with the image being displayed. The synthesis unit 138 outputs synthesized image information indicating the created synthesized image to the display unit 140. The synthesis unit 138 outputs synthesized image information indicating the created synthesized image to the display unit 140. The display unit 140 displays the synthesized image based on the synthesized image information output by the synthesis unit 138. FIG. 6G is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 6G, the screen shown in FIG. 6B is displayed with a tilt change designation diagram TCSD further superimposed thereon. For example, the tilt change designation diagram TCSD includes an arrow designating rotation around a line parallel to the Z axis. The user designates the tilt of the ground equipment image 30 by tilting the information processing device 100 in the direction in which the tilt is to be changed. The detection unit 132 determines whether the confirm button 116 has been pressed. If the confirm button 116 has not been pressed, the detection unit 132 proceeds to step S17-1.

[0054] (Step S18-1) When the detection unit 132 detects that the confirmation button 116 has been pressed, the change unit 135 tilts the ground equipment image 30 in a direction that changes the tilt of the ground equipment image 30 specified by the user, using gyro information measured by the gyro sensor 190. For example, when the rotation direction (tilt direction) of the information processing device 100 is positive clockwise, the change unit 135 tilts the ground equipment image 30 around the lower right corner on the front side of the ground equipment, and when the rotation direction is negative clockwise, the change unit 135 tilts the ground equipment image 30 around the lower left corner on the front side of the ground equipment. Fig. 6H is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 6H, the screen is displayed with the tilt changed to the direction in which the ground equipment image 30 is tilted by the user on the screen shown in Fig. 6G. The description will continue with reference to Fig. 7.

[0055] FIG. 7 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-2) 5, when the detection unit 132 determines that the menu button 110 has been pressed, the composition unit 138 creates a composite image by combining the menu bar acquired by the imaging unit 160 with the image displayed by the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 8A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8A, the screen includes a menu button 110, a menu bar MB, an object switching button 111, a major mode button 112, a detected ground image DG, and a shutter button SH. The menu bar MB includes an add object button 117, a change color button 118, a change transparency button 119, a duplicate button 120, and a delete object button 121. Returning to Fig. 7, the description will continue.

[0056] (Step S2-2) The detection unit 132 determines whether or not the add object button 117 has been pressed. If the add object button 117 has not been pressed, the process returns to step S2-2. (Step S3-2) When the object addition button is pressed, the detection unit 132 detects and recognizes a plane based on the image acquired by the imaging unit 160. (Step S4-2) The synthesis unit 138 acquires the plane recognition result from the detection unit 132, and synthesizes the acquired plane recognition result and the menu bar with the image acquired by the imaging unit 160 and displayed by the display unit 140 to create synthesized screen information. The synthesis unit 138 outputs the created synthesized screen information to the display unit 140. The display unit 140 displays the synthesized screen based on the synthesized screen information output by the synthesis unit 138. Fig. 8B is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8B, the screen shown in Fig. 8A displays a ground equipment image 30 on the detected ground image DG (in the positive Y-axis direction of the detected ground image DG), and the display of the detected ground image DG is stopped. Returning to Fig. 7, the explanation will be continued.

[0057] (Step S5-2) The detection unit 132 determines whether the color change button 118 has been pressed. (Step S6-2) When detection unit 132 determines that the color change button has been pressed, composition unit 138 further composites the color palette with the image displayed on display unit 140 to create composite screen information. Composition unit 138 outputs the created composite screen information to display unit 140. Display unit 140 displays a composite screen based on the composite screen information output by composition unit 138. 8C is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 8C, a color palette CP is further displayed on the display unit 140 in addition to the screen shown in FIG. 8B. For example, the color palette CP includes a plurality of color buttons shown in different colors. In FIG. 8C, each of the plurality of color buttons is shown as a circle. The user specifies the color of the ground equipment image 30 by pressing (touching) one of the plurality of color buttons. Returning to FIG. 7, the explanation will be continued.

[0058] (Step S7-2) The detection unit 132 determines whether or not any of the colored buttons has been pressed. If none of the colored buttons has been pressed, the process returns to step S7-2. (Step S8-2) When the detection unit 132 determines that one of the multiple color buttons has been pressed, the change unit 135 changes the color of the ground equipment image 30 to the color corresponding to the pressed color button. The composition unit 138 creates a composite image by combining the ground equipment image 30, the color of which has been changed by the change unit 135, with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 8D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8D, on the screen shown in Fig. 8C, the display of the color palette CP on the display unit 140 is stopped, and the color of the ground equipment image 30 is changed to the color specified by the color button. Returning to Fig. 7, the explanation will be continued.

[0059] (Step S9-2) If the color change button 118 has not been pressed in step S5-2, the detection unit 132 determines whether the transparency change button 119 has been pressed. (Step S10-2) When detection unit 132 determines that transparency change button 119 has been pressed, composition unit 138 further composites the transparency palette with the image displayed on display unit 140 to create composite screen information. Composition unit 138 outputs the created composite screen information to display unit 140. Display unit 140 displays the composite screen based on the composite screen information output by composition unit 138. FIG. 8E is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 8E, a transparency palette TP is further displayed on the display unit 140 in addition to the screen shown in FIG. 8B. For example, the transparency palette TP includes a plurality of transparency buttons on which numerical values ​​indicating different transparency levels are displayed. In FIG. 8E, each of the plurality of transparency buttons is represented by a circle. The user specifies the transparency of the ground equipment image 30 by pressing one of the plurality of transparency buttons. Returning to FIG. 7, the explanation will be continued.

[0060] (Step S11-2) The detection unit 132 determines whether or not any of the plurality of transparency buttons has been pressed. If none of the plurality of transparency buttons has been pressed, the process returns to step S11-2. (Step S12-2) When the detection unit 132 determines that one of the plurality of transparency buttons has been pressed, the change unit 135 changes the transparency of the ground equipment image 30 to the transparency corresponding to the pressed transparency button. The composition unit 138 creates a composite image by combining the ground equipment image 30, the transparency of which has been changed by the change unit 135, with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 8F is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8F, on the screen shown in Fig. 8E, the display of the transparency palette TP is stopped on the display unit 140, and the transparency of the ground equipment image 30 is changed to the transparency specified by the transparency button. Returning to Fig. 7, the explanation will be continued.

[0061] (Step S13-2) If the transparency change button 119 has not been pressed in step S9-2, the detection unit 132 determines whether the copy button 120 has been pressed. (Step S14-2) When the detection unit 132 determines that the duplicate button 120 has been pressed, the composition unit 138 further composites a slider onto the image being displayed to create composite screen information. The composition unit 138 outputs the created composite screen information to the display unit 140. The display unit 140 displays the composite screen based on the composite screen information output by the composition unit 138.

[0062] 8G is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 8G, the display unit 140 displays the screen shown in FIG. 8B, with a slider image S and a confirmation button CB further superimposed thereon. For example, the slider image S is used to specify the distance between the displayed ground device image 30 and the ground device image to be duplicated, and the distance can be specified by sliding the circle button on the slider image S.

[0063] Fig. 8H is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8H, the display unit 140 displays the ground device image 30R to be duplicated on the screen shown in Fig. 8G. The user can change the distance between the displayed ground device image 30 and the ground device image 30R to be duplicated by sliding the round button on the slider image S. When the user slides the round button on the slider image S, the ground device image 30R to be duplicated moves in the X-axis direction relative to the displayed ground device image 30. Returning to Fig. 7, the explanation will be continued.

[0064] (Step S15-2) The detection unit 132 determines whether or not the confirmation button CB has been pressed. If the confirmation button CB has not been pressed, the process returns to step S15-2. (Step S16-2) When the detection unit 132 determines that the confirm button CB has been pressed, the synthesis unit 138 duplicates the displayed ground device image 30R. The synthesis unit 138 creates a composite image by further compositing the duplicated ground device image 30R onto the image currently being displayed so that the distance between the duplicated ground device image 30R and the image is the distance specified by the slider image S. The synthesis unit 138 outputs synthetic image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the synthetic image information output by the synthesis unit 138. Fig. 8I is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8I, the screen is displayed by combining the ground device image 30R with the screen shown in Fig. 8B so that the distance between the ground device image 30 and the ground device image 30R is 0.5 m, as specified by the slider image S. Furthermore, the screen displays a shutter button SH in place of the slider image S and the confirm button. Returning to Fig. 7, the explanation will be continued.

[0065] (Step S17-2) If the duplicate button 120 has not been pressed in step S13-2, the detection unit 132 determines whether the object delete button 121 has been pressed. (Step S18-2) When the detection unit 132 determines that the object deletion button 121 has been pressed, the composition unit 138 further composites the selected area with the image being displayed to create composite screen information. The composition unit 138 outputs the created composite screen information to the display unit 140. The display unit 140 displays the composite screen based on the composite screen information output by the composition unit 138. Fig. 8J is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8J, the display unit 140 displays the selection area SR and the confirmation button CB on the screen shown in Fig. 8B. For example, the selection area SR is used to specify an object to be deleted, and can be selected by sliding the selection area SR so that it overlaps with the object to be deleted. Returning to Fig. 7, the explanation will be continued.

[0066] (Step S19-2) The detection unit 132 determines whether or not the confirmation button CB has been pressed. If the confirmation button CB has not been pressed, the process returns to step S19-2. (Step S20-2) When the detection unit 132 determines that the confirm button CB has been pressed, the synthesis unit 138 deletes the object selected by the selection area SR. The synthesis unit 138 creates a composite image in which the object selected by the selection area SR has been deleted from the image being displayed. The synthesis unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the synthesis unit 138. Fig. 8K is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 8K, the screen shown in Fig. 8J has the ground equipment image 30 deleted. The description will continue with reference to Fig. 9.

[0067] FIG. 9 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-3) When the detection unit 132 determines in step S5-1 of FIG. 5 that the object switching button has been pressed, the detection unit 132 detects and recognizes a wall surface based on the image acquired by the imaging unit 160.

[0068] (Step S2-3) The synthesis unit 138 acquires the wall surface recognition result from the detection unit 132, and creates a composite image by synthesizing the acquired plane recognition result and multiple operation buttons with the image displayed by the display unit 140. The synthesis unit 138 outputs synthetic image information indicating the created synthetic image to the display unit 140. The display unit 140 displays the synthetic image based on the synthetic image information output by the synthesis unit 138. Fig. 10A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 10A, the screen includes a menu button 110, an object switching button 111, a major mode button 112, a detected wall image DW, and a shutter button SH. Returning to Fig. 9, the description will continue.

[0069] (Step S3-3) The detection unit 132 detects the position tapped by the user. The information acquisition unit 134 acquires tap position information indicating the position tapped by the user detected by the detection unit 132. (Step S4-3) The synthesis unit 138 acquires tap position information from the information acquisition unit 134. The synthesis unit 138 determines to display the rising protecting pipe image 10 based on the acquired tap position information.

[0070] (Step S5-3) The composition unit 138 creates a composite image by combining the rising protecting pipe image 10 and a plurality of adjustment button images with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 10B is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 10B, the display unit 140 displays the rising protecting pipe image 10 on the detected wall surface image DW (in the negative Z-axis direction of the detected wall surface image DW) on the screen shown in Fig. 10A. The adjustment button image includes a move button 122, an inclination change button 123, and a confirm button 124. Returning to Fig. 9, the explanation will continue.

[0071] (Step S6-3) The detection unit 132 determines whether or not the menu button 110 has been pressed. The case where the menu button 110 has been pressed will be described later. (Step S7-3) If the menu button 110 is not pressed, the detection unit 132 determines whether the movement button 122 is pressed.

[0072] (Step S8-3) When detection unit 132 determines that movement button 122 has been pressed, composition unit 138 creates a composite image by further combining the position movement designation diagram PMSD with the image displayed by display unit 140. Composition unit 138 outputs composite image information indicating the created composite image to display unit 140. Display unit 140 displays the composite image based on the composite image information output by composition unit 138. FIG. 10C is a diagram illustrating an example of a screen displayed by the information processing device according to this embodiment. As illustrated in FIG. 10C, the screen further includes a position movement designation diagram PMSD superimposed on the screen illustrated in FIG. 10B. For example, the position movement designation diagram PMSD includes two arrows, one parallel to the X-axis and the other parallel to the Z-axis. The user specifies the position to which the rising protection pipe image 10 is to be moved by swiping the rising protection pipe image 10 in the desired direction. The detection unit 132 determines whether the confirm button 124 has been pressed. If the confirm button 124 has not been pressed, the detection unit 132 proceeds to step S8-3.

[0073] (Step S9-3) When the detection unit 132 detects that the confirmation button 124 has been pressed, the change unit 135 moves the rising protection pipe image 10 to the position to which the rising protection pipe image 10 is to be moved, as specified by the user. Fig. 10D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 10D, the screen shows the ground-raised protecting pipe image 10 moved to a position specified by the user on the screen shown in Fig. 10C. Returning to Fig. 9, the explanation will be continued. (Step S10-3) If the movement button 122 is not pressed in step S7-3, the detection unit 132 determines whether or not the tilt change button 123 is pressed. If the tilt change button 123 is not pressed, the process proceeds to step S10-3.

[0074] (Step S11-3) When the detection unit 132 determines that the tilt change button 123 has been pressed, the synthesis unit 138 creates a composite image by further synthesizing the tilt change designation diagram TCSD with the image being displayed. The synthesis unit 138 outputs synthesized image information indicating the created synthesized image to the display unit 140. The display unit 140 displays the synthesized image based on the synthesized image information output by the synthesis unit 138. FIG. 10E is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 10E, the screen shown in FIG. 10B is displayed with a tilt change designation diagram TCSD further superimposed thereon. For example, the tilt change designation diagram TCSD includes an arrow indicating a tilt change along a line perpendicular to a plane formed by the X-axis and Y-axis. The user specifies the tilt of the rising protecting pipe image 10 by tilting the information processing device 100 in the direction in which the tilt is to be changed. The detection unit 132 determines whether the confirm button 124 has been pressed. If the confirm button 124 has not been pressed, the detection unit 132 proceeds to step S11-3.

[0075] (Step S12-3) When the detection unit 132 detects that the confirmation button 124 has been pressed, the change unit 135 tilts the rising protection pipe image 10 in a direction that changes the tilt of the rising protection pipe image 10 specified by the user. Fig. 10F is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 10F, the screen displays the rising protecting pipe image 10 on the screen shown in Fig. 10E with its tilt changed to match the direction in which it has been tilted by the user. The description will continue with reference to Fig. 11.

[0076] FIG. 11 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-4) 9, when the detection unit 132 determines that the menu button 110 has been pressed, the composition unit 138 creates a composite image by combining the menu bar acquired by the imaging unit 160 with the image displayed by the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 12A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 12A, the screen includes a menu button 110, a menu bar MB, an object switching button 111, a major mode button 112, and a shutter button SH. The menu bar MB includes an object addition button 125, a type switching button 126, a transparency change button 127, and an object deletion button 128. Returning to Fig. 11, the explanation will continue.

[0077] (Step S2-4) The detection unit 132 determines whether or not the add object button 117 has been pressed. If the add object button 117 has not been pressed, the process returns to step S2-4. (Step S3-4) When the object addition button is pressed, the detection unit 132 detects and recognizes a wall surface based on the image acquired by the imaging unit 160.

[0078] (Step S4-4) The synthesis unit 138 acquires the wall surface recognition result from the detection unit 132, and creates synthesized screen information that shows a synthesized screen by further synthesizing the acquired wall surface recognition result and the rising protecting pipe image 10 with the image displayed by the display unit 140. The synthesis unit 138 outputs the created synthesized screen information to the display unit 140. The display unit 140 displays the synthesized screen based on the synthesized screen information output by the synthesis unit 138. Fig. 12B is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 12B, the screen shown in Fig. 12A displays the detected wall surface image DW and the rising protecting pipe image 10. Returning to Fig. 11, the explanation will be continued.

[0079] (Step S5-4) The detection unit 132 determines whether the type switching button 126 has been pressed. (Step S6-4) When detection unit 132 determines that type switching button 126 has been pressed, composition unit 138 creates composite screen information that shows a composite screen by further combining the object type palette with the image displayed on display unit 140. Composition unit 138 outputs the created composite screen information to display unit 140. Display unit 140 displays the composite screen based on the composite screen information output by composition unit 138. FIG. 12C is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 12C, the screen shown in FIG. 12B is further combined with an object type palette OTP to display the screen. For example, the object type palette OTP includes a plurality of object type buttons each showing a different color and diameter. In FIG. 12C, each of the plurality of object type buttons is represented by a circle. The user specifies the type of the rising protecting pipe image 10 by pressing one of the plurality of object type buttons. Returning to FIG. 11, the explanation will continue.

[0080] (Step S7-4) The detection unit 132 determines whether or not any of the plurality of object type buttons has been pressed. If none of the plurality of object type buttons has been pressed, the process returns to step S7-4. (Step S8-4) When the detection unit 132 determines that one of the multiple object type buttons has been pressed, the change unit 135 changes the type of the rising protecting pipe image 10 to the type corresponding to the pressed object type button. The synthesis unit 138 creates a composite image by combining the ground equipment image 30, the type of which has been changed by the change unit 135, with the image displayed on the display unit 140. The synthesis unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the synthesis unit 138. Fig. 12D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 12D, on the screen shown in Fig. 12C, the display of the object type palette OTP on the display unit 140 has stopped, and the type of the rising protecting pipe image 10 has been changed to the type specified by the object type button. Returning to Fig. 11, the explanation will continue.

[0081] (Step S9-4) If the type switching button 126 has not been pressed in step S5-4, the detection unit 132 determines whether the transparency change button 127 has been pressed. (Step S10-4) When detection unit 132 determines that transparency change button 127 has been pressed, composition unit 138 creates composite screen information indicating a composite screen by further combining a transparency palette with the image displayed on display unit 140. Composition unit 138 outputs the created composite screen information to display unit 140. Display unit 140 displays the composite screen based on the composite screen information output by composition unit 138. FIG. 12E is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 12E, the display unit 140 stops displaying the detected wall surface image DW on the screen shown in FIG. 12B, and further displays a transparency palette TP. For example, the transparency palette TP includes a plurality of transparency buttons, each of which displays a numerical value indicating a different transparency. In FIG. 12E, each of the plurality of transparency buttons is represented by a circle. The user specifies the transparency of the rising protecting pipe image 10 by pressing one of the plurality of transparency buttons. Returning to FIG. 11, the explanation will be continued.

[0082] (Step S11-4) The detection unit 132 determines whether or not any of the plurality of transparency buttons has been pressed. If none of the plurality of transparency buttons has been pressed, the process returns to step S11-4. (Step S12-4) When the detection unit 132 determines that one of the plurality of transparency buttons has been pressed, the change unit 135 changes the transparency of the rising protecting pipe image 10 to the transparency corresponding to the pressed transparency button. The synthesis unit 138 creates a composite image by combining the ground equipment image 30, the transparency of which has been changed by the change unit 135, with the image displayed on the display unit 140. The synthesis unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the synthesis unit 138. Fig. 12F is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 12F, on the screen shown in Fig. 12E, the display of the transparency palette TP on the display unit 140 has been stopped, and the transparency of the rising protecting pipe image 10 has been changed to the transparency specified by the transparency button. Returning to Fig. 11, the explanation will continue.

[0083] (Step S13-4) If the transparency change button 127 has not been pressed in step S9-4, the detection unit 132 determines whether or not the object deletion button 128 has been pressed. If the object deletion button 128 has not been pressed, the process proceeds to step S5-4. (Step S14-4) When the detection unit 132 determines that the object deletion button 128 has been pressed, the composition unit 138 further composites the selected region onto the image being displayed, and outputs the composite screen information to the display unit 140. The composition unit 138 outputs the created composite screen information to the display unit 140. The display unit 140 displays the composite screen based on the composite screen information output by the composition unit 138. FIG. 12G is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 12G, the display unit 140 displays a message for confirming deletion, a selection area SR, and a confirm button CB on the screen shown in FIG. 12B. For example, the selection area SR is used to specify an object to be deleted, and can be selected by sliding the selection area SR so that it overlaps with the object to be deleted. The detection unit 132 determines whether the confirm button CB has been pressed. If the confirm button CB has not been pressed, the process returns to step S14-4. The description will continue, returning to FIG. 11.

[0084] (Step S15-4) When the detection unit 132 determines that the confirm button CB has been pressed, the synthesis unit 138 deletes the object selected by the selection area SR. The synthesis unit 138 creates a composite image in which the object selected by the selection area SR has been deleted from the image being displayed. The synthesis unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the synthesis unit 138. Fig. 12H is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 12H, the rising protection pipe image 10, the message for confirming deletion, the selection area SR, and the confirmation button CB have been deleted from the screen shown in Fig. 12G. The description will continue with reference to Fig. 13.

[0085] FIG. 13 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-5) When it is determined in step S9-1 of FIG. 5 that the major mode button 112 has been pressed, the detection unit 132 detects and recognizes the ground and wall surfaces based on the image acquired by the imaging unit 160. (Step S2-5) The synthesis unit 138 acquires the recognition results of the ground and wall surfaces from the detection unit 132, and creates a composite image by synthesizing the acquired recognition results of the ground and wall surfaces and the operation buttons with the image displayed by the display unit 140. The synthesis unit 138 outputs synthetic image information indicating the created synthetic image to the display unit 140. The display unit 140 displays the synthetic image based on the synthetic image information output by the synthesis unit 138. Fig. 14A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 14A, the screen shown in Fig. 6B displays the detected ground image DG, the detected wall image DW, and a delete all measures button 129, and displays a ground equipment image 30 on the detected ground image DG (in the positive Y-axis direction of the detected ground image DG). Returning to Fig. 13, the explanation will be continued.

[0086] (Step S3-5) The detection unit 132 detects the position tapped by the user. The information acquisition unit 134 acquires tap position information indicating the position tapped by the user detected by the detection unit 132. (Step S4-5) The synthesis unit 138 acquires the tap position information from the information acquisition unit 134, and determines to display the measure based on the acquired tap position information.

[0087] (Step S5-5) The composition unit 138 creates a composite image by combining the measure, the slider, and the selected measure deletion button with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. 14B and 14C are diagrams showing examples of screens displayed by the information processing device according to this embodiment. As shown in FIGS. 14B and 14C, the display unit 140 further displays a measure image ME-1, a slider image S, and a delete selected measure button 141 in addition to the screen shown in FIG. 14A. The measure image ME includes a pair of square handles and an arrow indicating the length. As shown in FIG. 14C, the user can adjust the length of the measure image ME-1 by dragging the handle of the measure image ME-1. The slider image S is used to fine-tune the displayed length of the measure image ME-1, and the user can fine-tune the displayed length by sliding the round button on the slider image S. The user may fine-tune the displayed length by sliding the round button included on the slider image S as necessary. Returning to FIG. 13, the description will continue.

[0088] (Step S6-5) The detection unit 132 determines whether or not a tap has been made by the user. If the detection unit 132 determines that a tap has been made, the process proceeds to step S3-5. When a tap is made by the user, a plurality of measure images ME can be displayed. Fig. 14D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 14D, the display unit 140 further displays a measuring image ME-2 on the screen shown in Fig. 14C. The measuring image ME-2 includes a pair of handles indicated by squares and an arrow indicating the length. Returning to Fig. 13, the explanation will be continued. (Step S7-5) If the detection unit 132 does not detect a tap in step S6-5, it determines whether the major mode button 112 has been pressed.

[0089] (Step S8-5) The synthesis unit 138 finalizes the measure image ME when the detection unit 132 determines that the measure mode button 112 has been pressed. The synthesis unit 138 creates a composite image by synthesizing the measure image ME image with the image displayed on the display unit 140. The synthesis unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the synthesis unit 138. Fig. 14E is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 14E, the display unit 140 displays the measure image ME-1 and measure image ME-2 on the screen shown in Fig. 14D by pressing the measure mode button 112. Furthermore, the detected ground image DG, detected wall image DW, delete all measures button 129, slider, and delete selected measure button 141 are no longer displayed on the display unit 140. Thereafter, the process proceeds to step S9-1 in Fig. 5. Returning to Fig. 13, the explanation will continue.

[0090] (Step S9-5) If it is determined in step S7-5 that the measure mode button 112 has not been pressed, the detection unit 132 determines whether the selected measure deletion button 141 has been pressed. (Step S10-5) When the Delete Selected Measure button 141 is pressed, the detection unit 132 deletes the selected measure image ME. For example, a selection area SR and a confirmation button CB for specifying an object to be deleted are displayed on the display unit 140, and the selection area SR is selected by sliding it so that it overlaps with the object to be deleted. When the Delete Selected Measure button 141 is pressed, the selected object is deleted. Thereafter, the process proceeds to step S9-1 in FIG. 5. Returning to FIG. 13, the explanation will be continued.

[0091] (Step S11-5) If it is determined in step S9-5 that the delete selected measure button 141 has not been pressed, the detection unit 132 determines whether the delete all measures button 129 has been pressed. (Step S12-5) When the delete all measure button 129 is pressed, the detection unit 132 deletes all measure images ME, and then the process proceeds to step S9-1 in FIG. If the detection section 132 determines in step S11-5 that the delete all measures button 129 has not been pressed, the process proceeds to step S6-5. The description will continue with reference to FIG.

[0092] FIG. 15 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-6) The detection unit 132 determines whether or not the excavation and burying mode has been selected by pressing the object switching button 111 on the screen shown in Fig. 6A. If the excavation and burying mode has not been selected, the process returns to step S1-6. (Step S2-6) When the detection unit 132 determines that the excavation and burying mode has been selected by pressing the object switching button 111, the synthesis unit 138 creates a composite image by synthesizing the menu bar with the image displayed on the display unit 140. The synthesis unit 138 outputs synthetic image information indicating the created synthetic image to the display unit 140. The display unit 140 displays the synthetic image based on the synthetic image information output by the synthesis unit 138. Fig. 16A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 16A, a menu bar MB is displayed on the screen shown in Fig. 6A on the display unit 140. The menu bar MB includes an excavation range visualization button 142, a buried pipe visualization button 143, and an object deletion button 144. Returning to Fig. 15, the explanation will be continued.

[0093] (Step S3-6) The detection unit 132 detects and recognizes the ground based on the image acquired by the imaging unit 160. (Step S4-6) The detection unit 132 determines whether or not the excavation range visualization button 142 has been pressed. The case where the excavation range visualization button 142 has not been pressed will be described later. (Step S5-6) The detection unit 132 detects the position tapped by the user when the excavation range visualization button 142 is pressed. The information acquisition unit 134 acquires tap position information indicating the position tapped by the user detected by the detection unit 132. (Step S6-6) The synthesis unit 138 acquires the tap position information from the information acquisition unit 134, and determines to display the excavation range based on the acquired tap position information.

[0094] (Step S7-6) The composition unit 138 creates a composite image by combining an image of the excavation range (hereinafter referred to as "excavation range image 40") and a plurality of edit buttons with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 16B is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 16B, the display unit 140 further displays an excavation range image 40 and an edit button on the screen shown in Fig. 16A. The edit button includes a move button 145, a resize button 146, a scroll button 147, and a confirm button 148. Returning to Fig. 15, the explanation will be continued. (Step S8-6) The detection unit 132 determines whether the movement button 145 has been pressed.

[0095] (Step S9-6) When detection unit 132 determines that movement button 145 has been pressed, composition unit 138 creates a composite image by further combining the position movement designation diagram PMSD with the image displayed by display unit 140. Composition unit 138 outputs composite image information indicating the created composite image to display unit 140. Display unit 140 displays the composite image based on the composite image information output by composition unit 138. FIG. 16C is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 16C, the screen further includes a position movement designation diagram PMSD superimposed on the screen shown in FIG. 16B. For example, the position movement designation diagram PMSD includes two arrows, one parallel to the X-axis and the other parallel to the Z-axis. The user swipes the excavation range image 40 in the direction in which the user wants to move the excavation range image 40, thereby designating the position to which the excavation range image 40 is to be moved. The detection unit 132 determines whether the confirm button 148 has been pressed. If the confirm button 148 has not been pressed, the detection unit 132 proceeds to step S9-6. Returning to FIG. 15, the description will be continued.

[0096] (Step S10-6) When the detection unit 132 detects that the confirmation button 148 has been pressed, the change unit 135 moves the excavation range image 40 to a position designated by the user to which the excavation range image 40 is to be moved. Fig. 16D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 16D, the screen shows the excavation range image 40 moved to a position designated by the user on the screen shown in Fig. 16C and displayed. Returning to Fig. 15, the explanation will be continued. (Step S11-6) If the move button 145 has not been pressed in step S8-6, the detection unit 132 determines whether the size change button 146 has been pressed.

[0097] (Step S12-6) When the detection unit 132 determines that the resize button 146 has been pressed, the composition unit 138 creates a composite image by further combining the resize designation diagram SCSD with the image being displayed. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. FIG. 16E is a diagram illustrating an example of a screen displayed by the information processing device according to this embodiment. As illustrated in FIG. 16E, the screen shown in FIG. 16B is displayed with a resize designation diagram SCSD superimposed thereon instead of the edit button. For example, the resize designation diagram SCSD includes three buttons—a first button, a second button, and a third button—for selecting a direction to move in the first, second, and third directions, and a slider. For example, the first direction is the X-axis direction, the second direction is the Z-axis direction, and the third direction is the Y-axis direction. In FIG. 16E, the first, second, and third directions are indicated by “(1),” “(2),” and “(3),” respectively. The user presses any of the first to third buttons to specify the length of the corresponding direction using the slider. The detection unit 132 determines whether the confirm button 148 has been pressed. If the confirm button 148 has not been pressed, the detection unit 132 proceeds to step S12-6. Returning to FIG. 15, the description continues.

[0098] (Step S13-6) When the detection unit 132 detects that the confirmation button 148 has been pressed, the change unit 135 changes the size of the excavation range image 40 to the size designated by the user. (Step S14-6) If it is determined in step S11-6 that the size change button 146 has not been pressed, the detection unit 132 determines whether or not the scroll button 147 has been pressed. If it is determined by the detection unit 132 that the scroll button 147 has not been pressed, the process proceeds to step S8-6.

[0099] (Step S15-6) When the detection unit 132 determines that the scroll button 147 has been pressed, the synthesis unit 138 creates a synthetic image by further synthesizing the scroll designation diagram SSD with the image being displayed. The synthesis unit 138 outputs synthetic image information indicating the created synthetic image to the display unit 140. The display unit 140 displays the synthetic image based on the synthetic image information output by the synthesis unit 138. FIG. 16F is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 16F, the screen shown in FIG. 6B is further superimposed with a scroll designation image SSD and displayed. For example, the scroll button 147 includes an arrow specifying scrolling of the excavation range image 40 around a straight line perpendicular to a plane formed by the X-axis and Z-axis. The user specifies the direction in which to scroll the excavation range image 40 by swiping in the desired direction of scrolling the excavation range image 40. The detection unit 132 determines whether the confirm button 148 has been pressed. If the confirm button 148 has not been pressed, the detection unit 132 proceeds to step S15-6. Returning to FIG. 15, the description will continue. (Step S16-6) When the detection unit 132 detects that the confirmation button 148 has been pressed, the change unit 135 scrolls the excavation range image 40 in a direction in which the excavation range image 40 specified by the user is scrolled. The description will continue with reference to Fig. 17 .

[0100] FIG. 17 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-7) The detection unit 132 determines whether or not the buried pipe visualization button 143 has been pressed when the excavation range visualization button has not been pressed in step S4-6 of Fig. 15. The case where the buried pipe visualization button 143 has not been pressed will be described later. (Step S2-7) The detection unit 132 detects the position tapped by the user when the excavation range visualization button 142 is pressed. The information acquisition unit 134 acquires tap position information indicating the position tapped by the user detected by the detection unit 132. (Step S3-7) The synthesis unit 138 acquires the tap position information from the information acquisition unit 134, and determines to display the buried pipe based on the acquired tap position information.

[0101] (Step S4-7) The composition unit 138 creates a composite image by combining an image of a buried pipe (hereinafter referred to as "buried pipe image 20") and a plurality of edit buttons with the image displayed on the display unit 140. The composition unit 138 outputs composite image information indicating the created composite image to the display unit 140. The display unit 140 displays the composite image based on the composite image information output by the composition unit 138. Fig. 18A is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in Fig. 18A, the display unit 140 further displays the buried pipe image 20 and an edit button in addition to the screen shown in Fig. 16A. The edit button includes a move button 149, a size / depth change button 150, a scroll button 151, and a confirm button 152. Returning to Fig. 17, the explanation will be continued.

[0102] (Step S5-7) The detection unit 132 determines whether the movement button 149 has been pressed. (Step S6-7) When detection unit 132 determines that movement button 149 has been pressed, composition unit 138 creates a composite image by further combining the position movement designation diagram PMSD with the image displayed by display unit 140. Composition unit 138 outputs composite image information indicating the created composite image to display unit 140. Display unit 140 displays the composite image based on the composite image information output by composition unit 138. FIG. 18B is a diagram illustrating an example of a screen displayed by the information processing device according to this embodiment. As illustrated in FIG. 18B, the screen further includes a position movement designation diagram PMSD superimposed on the screen illustrated in FIG. 18A. For example, the position movement designation diagram PMSD includes two arrows, one parallel to the X-axis and the other parallel to the Z-axis. The user specifies the position to which the buried pipe image 20 is to be moved by swiping the buried pipe image 20 in the desired direction. The detection unit 132 determines whether the confirm button 152 has been pressed. If the confirm button 152 has not been pressed, the detection unit 132 proceeds to step S6-7 in FIG. 17.

[0103] (Step S7-7) When the detection unit 132 detects that the confirmation button 152 has been pressed, the change unit 135 moves the buried pipe image 20 to the position designated by the user to which the buried pipe image 20 is to be moved. (Step S8-7) If the move button 149 is not pressed in step S5-7, the detection unit 132 determines whether the size / depth change button 150 is pressed.

[0104] (Step S9-7) When the detection unit 132 determines that the size and depth change button 150 has been pressed, the synthesis unit 138 creates a composite image by further synthesizing the size and depth change designation diagram SDCSD with the image being displayed. The synthesis unit 138 outputs synthesized image information indicating the created synthesized image to the display unit 140. The display unit 140 displays the synthesized image based on the synthesized image information output by the synthesis unit 138. FIG. 18C is a diagram illustrating an example of a screen displayed by the information processing device according to this embodiment. As illustrated in FIG. 18C, the screen shown in FIG. 18A is displayed with a size / depth change designation diagram SDCSD added instead of the edit button. For example, the size / depth change designation diagram SDCSD includes two buttons, a first button and a second button, and a slider for selecting when moving in a first direction and a second direction. The first direction is, for example, the length direction of the buried pipe, and the second direction is, for example, the depth direction of the buried pipe. In FIG. 18C, the first direction and the second direction are indicated by "(1)" and "(2)," respectively. The user presses either the first button, the second button, or the second button to specify the length of the corresponding direction using the slider. The detection unit 132 determines whether the confirm button 152 has been pressed. If the confirm button 152 has not been pressed, the detection unit 132 proceeds to step S9-7. Returning to FIG. 17, the description will continue.

[0105] (Step S10-7) When the detection unit 132 detects that the confirmation button 152 has been pressed, the change unit 135 changes the buried pipe image 20 based on at least one of the size and depth specified by the user. (Step S11-7) If the size / depth change button 150 is not pressed in step S8-7, the detection unit 132 determines whether or not the scroll button 151 is pressed. If the scroll button 151 is not pressed, the process proceeds to step S5-7.

[0106] (Step S12-7) When the detection unit 132 determines that the scroll button 151 has been pressed, the synthesis unit 138 creates a synthetic image by further synthesizing the scroll designation diagram SSD with the image being displayed. The synthesis unit 138 outputs synthetic image information indicating the created synthetic image to the display unit 140. The display unit 140 displays the synthetic image based on the synthetic image information output by the synthesis unit 138. FIG. 18D is a diagram showing an example of a screen displayed by the information processing device according to this embodiment. As shown in FIG. 18D, the screen shown in FIG. 18A is further superimposed with a scroll designation image SSD. For example, the scroll designation image SSD includes an arrow designating scrolling of the buried pipe image 20 around a line perpendicular to a plane formed by the X-axis and Z-axis. The user designates the direction in which to scroll the buried pipe image 20 by swiping in the desired direction. The detection unit 132 determines whether the confirm button 152 has been pressed. If the confirm button 152 has not been pressed, the detection unit 132 proceeds to step S12-7 of FIG. 17. (Step S13-7) When the detection unit 132 detects that the confirmation button 152 has been pressed, the change unit 135 scrolls the buried pipe image 20 in the direction specified by the user to scroll the buried pipe image 20. The description will continue with reference to FIG.

[0107] FIG. 19 is a flowchart showing an example of the operation of the information processing device according to this embodiment. (Step S1-8) 17, if the buried pipe visualization button has not been pressed, the detection unit 132 determines whether or not the object deletion button 144 has been pressed. If the object deletion button 144 has not been pressed, the process proceeds to step S4-6 in FIG. (Step S2-8) When detection unit 132 determines that object deletion button 144 has been pressed, composition unit 138 deletes the object from the image being displayed and creates screen information. Composition unit 138 outputs the created screen information to display unit 140. Display unit 140 displays a screen based on the screen information output by composition unit 138.

[0108] In the above-described embodiment, the process of calculating the actual distance d between the information processing device 100 and an object formed by a set of feature points included in an image acquired by the imaging unit 160 when the information processing device 100 is shaken left and right has been described, but the present invention is not limited to this example. For example, the process can also be applied to a case where the information processing device 100 is shaken up and down.

[0109] According to the information processing device 100 of the embodiment, the information processing device 100 includes a detection unit 132 that detects parallel planes parallel to the display surface of the display unit 140 from the image displayed by the display unit 140, an information acquisition unit 134 that acquires buried pipe installation location information, which is information indicating the installation location of a rising protection pipe that protects the rising portion of the power feed line from underground, and a synthesis unit 138 that creates synthetic image information showing a synthetic image in which an image of the rising protection pipe is synthesized with the image displayed by the display unit 140 based on the parallel planes detected by the detection unit 132 and the rising protection pipe installation location information acquired by the information acquisition unit 134. With this configuration, the information processing device 100 can create composite image information that combines an image of the rising protection pipe with the image displayed by the display unit 140 based on the size of the rising protection pipe, the parallel planes parallel to the display surface of the display unit 140, and the rising protection pipe installation position information, thereby using augmented reality technology to create an image at the site where the rising protection pipe is to be installed that combines an image after the rising protection pipe has been installed with an image before the rising protection pipe has been installed. This reduces the time from when a client requests an image of the situation after utility pole removal until the client obtains that image.

[0110] In the information processing device 100, the detection unit 132 detects a vertical plane perpendicular to the display surface of the display unit 140 from the image displayed by the display unit 140, the information acquisition unit 134 acquires buried pipe installation location information, which is information indicating the installation location of a buried pipe that pulls in a power line from underground, and the synthesis unit 138 creates synthetic image information showing a synthetic image in which an image of the protective pipe is synthesized with the image displayed by the display unit 140 based on the vertical plane detected by the detection unit 132 and the buried pipe installation location information acquired by the information acquisition unit 134. In this way, by synthesizing an image of the buried pipe on the displayed screen based on information indicating the installation location of the buried pipe, it is possible to use augmented reality technology to create an image at the site where the buried pipe is to be installed that combines an image of the buried pipe before it is installed with an image of the buried pipe after it has been installed.

[0111] Information processing device 100 further includes modification unit 135 that moves the position of the image of the buried pipe included in the composite image. Combining unit 138 creates composite image information by combining the image of the buried pipe, the position of which has been moved by modification unit 135, with the image displayed on display unit 140. With this configuration, it is possible to move the image of the buried pipe in the image after the buried pipe has been installed.

[0112] Information processing device 100 further includes modification unit 135, which modifies either or both of the size and the depth to which the buried pipe is buried, of the image of the buried pipe included in the composite image. Combining unit 138 creates composite image information by combining the image of the buried pipe, the size and / or the depth to which the buried pipe is buried, modified by modification unit 135, with the image displayed on display unit 140. With this configuration, either or both of the size and the depth to which the buried pipe is buried can be modified in the image after the buried pipe has been installed.

[0113] Information processing device 100 further includes change unit 135 that scrolls the image of the buried pipe included in the composite image. Combining unit 138 creates composite image information by combining the image of the buried pipe scrolled by change unit 135 with the image displayed on display unit 140. With this configuration, the buried pipe can be scrolled in the image after it has been installed.

[0114] In the information processing device 100, the information acquisition unit 134 acquires excavation range information, which is information indicating the excavation range when pulling in a power service line from underground. The synthesis unit 138 creates composite image information indicating a composite image obtained by combining the image of the excavation range with the image displayed by the display unit 140, based on the vertical plane detected by the detection unit 132 and the excavation range information acquired by the information acquisition unit 134. In this way, by synthesizing an image of the excavation area on the displayed screen based on information indicating the excavation area when pulling in power service lines from underground, it is possible to use augmented reality technology to create an image that combines the image of the excavation area at the site where the buried pipe is to be installed. This reduces the time from when a client requests an image of the area after utility pole removal until the client obtains that image.

[0115] The information processing device 100 further includes a change unit 135 that moves the position of the image image of the excavation range included in a composite image obtained by combining the image image of the excavation range. The combination unit 138 creates composite image information by combining the image displayed on the display unit 140 with the image image of the excavation range whose position has been moved by the change unit 135. With this configuration, it is possible to move the excavation range in the image image combined with the excavation range. Therefore, it is possible to respond to a request from a client to move the excavation range.

[0116] The information processing device 100 further includes a change unit 135 that changes the size of the image of the excavation range included in the composite image. The composition unit 138 creates composite image information by combining the image displayed on the display unit 140 with the image of the excavation range whose size has been changed by the change unit 135. With this configuration, the size of the excavation range can be changed in the image combined with the excavation range. Therefore, it is possible to respond to a request from a client to change the size of the excavation range.

[0117] The information processing device 100 further includes a change unit 135 that scrolls the image of the excavation range included in the composite image. The composition unit 138 creates composite image information by combining the image displayed on the display unit 140 with the image of the excavation range scrolled by the change unit 135. With this configuration, the excavation range can be scrolled in the image combined with the excavation range. Therefore, it is possible to respond to a request from a client to scroll the excavation range.

[0118] In the information processing device 100, the information acquisition unit 134 acquires ground equipment installation position information, which is information indicating the installation position of the ground equipment. The information processing device 100 further includes a derivation unit 136 that derives the size of the ground equipment to be displayed on the display unit 140. The synthesis unit 138 creates synthesized image information by synthesizing an image of the ground equipment with the image displayed on the display unit 140, based on the size of the ground equipment derived by the derivation unit 136, the vertical plane detected by the detection unit 132, and the ground equipment installation position information acquired by the information acquisition unit 134. With this configuration, the information processing device 100 can create composite image information by combining an image of the ground equipment with the image displayed by the display unit 140 based on the size of the ground equipment, a vertical plane parallel to the display surface of the display unit 140, and ground equipment installation position information, thereby using augmented reality technology to create an image at the site where the ground equipment is to be installed, by combining an image after the ground equipment has been installed with an image before the ground equipment has been installed. This reduces the time from when a client requests an image of the area after utility pole removal until the client obtains that image.

[0119] Although the embodiments have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. These embodiments are included within the scope and spirit of the invention, as well as within the scope of the inventions described in the claims and their equivalents.

[0120] The information processing device 100 described above may be implemented by a computer. In this case, a program for implementing the functions of each functional block is recorded on a computer-readable recording medium. The program recorded on this recording medium may be loaded into a computer system and executed by a CPU. The term "computer system" as used herein includes hardware such as an OS (Operating System) and peripheral devices. In addition, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, etc. "Computer-readable recording media" also includes storage devices such as hard disks built into computer systems.

[0121] Furthermore, the term "computer-readable recording medium" may include a medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. Furthermore, the term "computer-readable recording medium" may also include a storage medium that stores a program for a certain period of time, such as a volatile memory within a computer system that is a server or client. The program may also be for implementing part of the functions described above. The program may also be capable of implementing the functions described above in combination with a program already stored in the computer system. The program may also be implemented using a programmable logic device. An example of a programmable logic device is an FPGA (Field Programmable Gate Array).

[0122] The information processing device 100 includes a computer therein. The steps of each process of the information processing device are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes the program to perform the above processes. Here, computer-readable recording media refers to magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, etc. Also, this computer program may be distributed to a computer via a communication line, and the computer that receives this distribution may execute the program. The program may also be for realizing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program). [Explanation of symbols]

[0123] 100...information processing device, 102...communication unit, 103...storage unit, 104...program, 105...application, 130...information processing unit, 132...detection unit, 134...information acquisition unit, 135...change unit, 136...derivation unit, 138...synthesis unit, 140...display unit, 106...operation unit, 160...imaging unit, 170...GNSS receiving unit, 180...acceleration sensor, 190...gyro sensor

Claims

1. a detection unit that detects a plane parallel to a display surface of the display unit from an image displayed on the display unit; an information acquisition unit that acquires rising protection pipe installation position information, which is information indicating the installation position of a rising protection pipe that protects the rising portion of the power feed line from underground; a synthesis unit that creates synthetic image information showing a synthetic image obtained by synthesizing an image of the rising protection pipe with the image displayed by the display unit, based on the parallel plane detected by the detection unit and the rising protection pipe installation position information acquired by the information acquisition unit; and An information processing device comprising:

2. the detection unit detects a vertical plane perpendicular to a display surface of the display unit from the image displayed on the display unit; the information acquisition unit acquires buried pipe installation position information that indicates an installation position of a buried pipe that draws the power lead-in line from underground; The information processing device described in claim 1, wherein the synthesis unit creates synthetic image information showing a synthetic image in which an image of a protective pipe is synthesized with the image displayed by the display unit based on the vertical surface detected by the detection unit and the buried pipe installation location information acquired by the information acquisition unit.

3. A change unit that moves the position of the image of the buried pipe included in the composite image. Furthermore, The information processing device according to claim 2 , wherein the combining unit creates combined image information by combining the image of the buried pipe whose position has been moved by the changing unit with the image displayed by the display unit.

4. A change unit that changes either or both of the size of the image of the buried pipe included in the composite image and the depth at which the image is buried. Furthermore, The information processing device according to claim 2, wherein the synthesis unit creates synthetic image information by synthesizing the image of a buried pipe whose size and / or buried depth have been changed by the modification unit with the image displayed by the display unit.

5. a change section for scrolling the image of the buried pipe included in the composite image; Furthermore, The information processing device according to claim 2 , wherein the combining unit creates combined image information by combining the image of the buried pipe scrolled by the changing unit with the image displayed by the display unit.

6. the information acquisition unit acquires excavation range information that is information indicating an excavation range when the power service line is drawn from underground; The information processing device described in claim 2, wherein the synthesis unit creates synthetic image information showing a synthetic image obtained by synthesizing an image of the excavation range with the image displayed by the display unit based on the vertical surface detected by the detection unit and the excavation range information acquired by the information acquisition unit.

7. A change unit that moves a position of the image image of the excavation range included in the composite image obtained by combining the image image of the excavation range, The information processing device according to claim 6 , wherein the combining unit creates combined image information by combining the image of the excavation range whose position has been moved by the changing unit with the image displayed by the display unit.

8. A change unit that changes the size of the image of the excavation range included in the composite image, The information processing device according to claim 6 , wherein the combining unit creates combined image information by combining the image of the excavation range whose size has been changed by the changing unit with the image displayed by the display unit.

9. A change unit that scrolls the image of the excavation range included in the composite image is further provided, The information processing device according to claim 6 , wherein the combining unit creates combined image information by combining the image of the excavation range scrolled by the changing unit with the image displayed by the display unit.

10. the information acquisition unit acquires ground equipment installation position information that is information indicating an installation position of the ground equipment, The information processing device includes: a derivation unit that derives the size of the ground equipment to be displayed on the display unit, The information processing device described in claim 2, wherein the synthesis unit creates synthetic image information showing a synthetic image obtained by synthesizing an image of the ground equipment with the image displayed by the display unit based on the size of the ground equipment derived by the derivation unit, the vertical surface detected by the detection unit, and the ground equipment installation position information acquired by the information acquisition unit.

11. A computer-implemented information processing method, comprising: Detecting a parallel plane parallel to a display surface of the display unit from the image displayed on the display unit; Acquire rising protection pipe installation position information, which is information indicating the installation position of a rising protection pipe that protects the rising portion of the power feed line from underground; An information processing method that creates composite image information showing a composite image in which an image of the rising protection pipe is combined with an image displayed by the display unit based on the parallel plane and the rising protection pipe installation position information.

12. On the computer, detecting a plane parallel to a display surface of the display unit from an image displayed on the display unit; acquiring rising protection pipe installation position information, which is information indicating the installation position of a rising protection pipe that protects the rising portion of the power feed line from underground; A program that creates composite image information showing a composite image obtained by combining an image of the rising protection pipe with the image displayed on the display unit based on the parallel plane and the rising protection pipe installation position information.

Images