Utility pole installation image display system and utility pole installation image display method

Abstract

To provide a power pole installation image display system capable of eliminating a troublesome labor and easily displaying a power pole object in an image as AR content.SOLUTION: In a power pole installation image display system 1, shooting means 20 shoots a place in which a power pole is installed. Display means 30 displays an image shot by the shooting means 20. GNSS reception means 60 acquires location information from a GNSS (Global Navigation Satellite System). Information processing means 40 acquires installation location information at a power pole location position in the image shot by the shooting means 20 on the basis of the location information acquired by the GNSS reception means, and controls display of a 3D power pole 3 as AR content of the power pole stored in storage means 70 in a position corresponding to the installation position information of the acquired power pole in the image of the display means 30 shot by the shooting means 20.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a utility pole installation image display system and a utility pole installation image display method that are capable of displaying an image of the utility pole after it has been reconstructed in response to the installation, relocation, or removal of the utility pole. [Background technology]

[0002] For example, when a new house is built or an existing house is cleared, utility poles may need to be installed, relocated, or removed. In such cases, there are requests from neighboring residents to have an idea of ​​what kind of utility poles will be installed, where and how, and what the condition and situation will be after installation.

[0003] In the past, we would give verbal explanations on-site, or use editing software to draw an image of the utility pole on photos taken on-site, and then have residents check the edited photos and floor plans.

[0004] This resulted in problems such as the time-consuming task of preparing materials to explain the state and situation after the utility poles were installed, and a discrepancy in residents' understanding before and after the poles were installed.

[0005] Therefore, when relocating a utility pole, a technique is known in which AR (Augmented Reality) technology is used to photograph the location where the existing utility pole is installed along with the utility pole, an AR marker is placed at the location where the utility pole will be installed after relocation, the photographed image of the utility pole is stored as AR content (or object), and the utility pole object as the AR content is displayed on the AR marker in the photographed screen (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-95890 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, the utility pole relocation image display device described in Patent Document 1 relocates an existing utility pole to a new location, and requires that an image of the existing utility pole be photographed by a photographing means and registered as AR content, which is a cumbersome process.

[0008] In addition, it was necessary to place an AR marker at the location where the utility pole was to be relocated, and there was also the problem that it took time to take the photo and display the AR content on the screen.

[0009] Furthermore, if a utility pole different from the existing one is to be installed at the relocation site, multiple AR markers would have to be prepared, which would increase the amount of management work required.

[0010] Therefore, an object of the present invention is to provide a utility pole installation image display system and a utility pole installation image display method that can be applied even when installing a new utility pole, and that can easily display a utility pole object as AR content together with an image of the surrounding area of ​​the installation location, without the need to place an AR marker at the utility pole installation location. [Means for solving the problem]

[0011] In order to solve the above problem, the invention described in claim 1 includes an imaging means for imaging a location where a utility pole is to be installed, a display means for displaying an image captured by the imaging means, a GNSS receiving means for acquiring position information from a global navigation satellite system (GNSS), and an installation position information acquiring means for acquiring installation position information of an installation location of a utility pole in the image captured by the imaging means based on the position information acquired by the GNSS receiving means. A mobile terminal device comprising: A storage means for storing a utility pole object as an AR content of a utility pole, The installation location information acquisition means identifies the installation location of the utility pole by performing image analysis on the image or by specifying the installation location of the utility pole in the image displayed on the display means, determines a relative positional relationship between the identified installation location of the utility pole and the mobile terminal device, and acquires installation location information of the installation location of the utility pole based on position information of the mobile terminal device acquired by the GNSS receiving means, and the display meansThis is a utility pole installation image display system, characterized in that the utility pole object stored in the memory means is displayed in a location in an image captured by the photographing means that corresponds to the utility pole installation location information acquired by the installation location information acquisition means.

[0012] The invention described in claim 2 is characterized in that, in the utility pole installation image display system described in claim 1, a plurality of utility pole objects are stored in the memory means, and a desired utility pole object is selected and displayed at a location corresponding to the installation position information in the image captured by the photographing means.

[0013] The invention described in claim 3 is a utility pole installation image display method using a portable terminal device including: an imaging means for imaging a location where a utility pole is to be installed; a display means for displaying an image captured by the imaging means; a GNSS receiving means for acquiring position information from a global navigation satellite system (GNSS); and an installation position information acquiring means for acquiring installation position information of a utility pole installation location in the image captured by the imaging means based on the position information acquired by the GNSS receiving means, the method comprising: holding the portable terminal device over the location where the utility pole is to be installed; The location of the utility pole is specified by performing image analysis on the image or by specifying the location of the utility pole in the image displayed on the display means, and a relative positional relationship between the specified location of the utility pole and the portable terminal device is obtained, and based on the position information of the portable terminal device acquired by the GNSS receiving means, The location of the utility pole Installation This is a method for displaying an image of a utility pole installation, characterized in that location information is acquired and a utility pole object as AR content of a utility pole stored in a built-in storage means or an external storage means is displayed at a location in the image captured by the imaging means that corresponds to the installation location information. Effect of the Invention

[0014] According to the inventions described in claims 1 and 3, it is possible to display a utility pole object as AR content in an image captured by a photographing means at a location corresponding to the installation location information of the utility pole acquired by the installation location information acquiring means, without placing an AR marker in advance at the installation location of the utility pole, so that it can be applied when a new utility pole is to be installed, and since it is not necessary to photograph the utility pole and register it as AR content, it is possible to easily display the utility pole object in the image. Therefore, it is possible to significantly reduce the effort required to create materials to explain the state and situation after the installation of the utility pole, and it is also possible to avoid the problem of a discrepancy in residents' understanding before and after the installation of the utility pole.

[0015] According to the invention described in claim 2, a plurality of utility pole objects as AR content are stored in a storage means in advance, and by appropriate selection, a desired utility pole object can be displayed at a location corresponding to the utility pole installation location information. This makes it possible to accurately display a utility pole that is the same as or similar to an actually installed utility pole, and also makes it possible to reduce the time required to display the utility pole object. [Brief description of the drawings]

[0016] [Figure 1] 1 is a schematic configuration diagram of a utility pole installation image display system 1 according to a first embodiment. [Diagram 2] FIG. 13 is a diagram showing a display screen showing the location where a utility pole is to be installed. [Diagram 3] FIG. 2 is a perspective view showing a state in which a portable terminal device is held up over a utility pole installation location. [Figure 4] FIG. 11 is a plan view of a portable terminal device held over a utility pole installation location. [Diagram 5] FIG. 13 is a diagram showing a state in which a portable terminal device is held up; [Figure 6] FIG. 1 is a schematic diagram of a 3D utility pole as AR content. [Figure 7] FIG. 13 is a perspective view showing an AR marker seal. [Figure 8]FIG. 2 is a perspective view showing an AR marker post. [Figure 9] FIG. 11 is a configuration diagram of a utility pole installation image display system 1A according to a second embodiment. [Figure 10] This is a diagram showing a 3D utility pole displayed near an existing utility pole. [Figure 11] This is a photograph of an existing utility pole and a 3D utility pole. [Figure 12] FIG. 13 is a diagram showing how an existing utility pole is deleted. [Figure 13] This is a diagram showing the existing utility poles after erasing them and displaying 3D utility poles. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, the present invention will be described based on the illustrated embodiment. (Embodiment 1) FIG. 1 is a schematic configuration diagram showing a utility pole installation image display system 1 according to the first embodiment. The first embodiment illustrates a case in which a new utility pole is to be installed at a utility pole installation location 2. As shown in FIG.

[0018] This utility pole installation image display system 1 uses a portable terminal device 10 such as a smartphone or tablet PC, and utilizes AR (Augmented Reality) technology to synthesize and display a 3D utility pole 3 as AR content onto an image taken by the portable terminal device 10 of the location where the utility pole is to be installed.

[0019] The mobile terminal device 10 includes an imaging means 20, a display means 30, an information processing means 40, an operation means 50, a global navigation satellite system (GNSS) receiving means 60, a storage means 70, and an inertial sensor 80.

[0020] The photographing means 20 is a camera mounted on the mobile terminal device 10, photographs the real space to obtain image information of the vicinity of the utility pole installation location 2, and outputs this image information to the information processing means .

[0021] The display means 30 is a monitor mounted on the mobile terminal device 10, and is configured, for example, by a liquid crystal display or an organic EL (Electroluminescence) display, and displays various information under the control of the CPU. It also displays images captured by the imaging means 20.

[0022] The information processing means 40 has programs such as a program (installation location information acquisition means) that acquires installation location information of the utility pole installation location 2 based on information from the GNSS receiving means 60, an utility pole installation image display application (AR application), and an AR engine, as well as an image synthesis program that combines a captured image with AR content, an image editing program that edits an image, and a selection program that selects multiple AR contents.

[0023] The operation means 50 is configured, for example, by a touch panel, detects a touch operation on an icon or the like displayed on the display means 30, and outputs the detection result of the touch operation to the information processing means 40.

[0024] The Global Navigation Satellite System (GNSS) receiving means 60 receives navigation signals transmitted by navigation satellites such as GPS, GLONASS, Galileo, and Quasi-Zenith Satellite System (QZSS), and demodulates the navigation signals. Then, based on the demodulated navigation signals, a predetermined positioning calculation is performed to determine the position of the mobile terminal device 10, and absolute position information is obtained.

[0025] The storage means 70 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), a flash memory, or the like, which is implemented / built-in to the mobile terminal device 10. Note that the storage means 70 is not limited to being implemented / built-in to the mobile terminal device 10, and may be realized by an external device that the mobile terminal device 10 can access via a network, such as an external storage server.

[0026] The storage means 70 stores information about the 3D utility pole 3 as AR content, in addition to programs such as the program executed by the information processing means 40, a utility pole installation image display application (AR application), and an AR engine. Note that the information about the 3D utility pole 3 includes a plurality of AR contents with different types of utility pole (bare pole, utility pole with cross arm, utility pole with guy wire), as well as different heights, diameters, etc., which can be appropriately selected.

[0027] The inertial sensor 80 is an acceleration sensor, an angular velocity sensor (gyro sensor), an electronic compass, or the like, and is implemented in the mobile terminal device 10. The inertial sensor 80 identifies the attitude (direction, direction, inclination) of the mobile terminal device 10 itself.

[0028] When the portable terminal device 10 can be positioned at the exact location (or in the vicinity) of the utility pole installation location 2, the absolute position information of the portable terminal device 10 is directly acquired as the position information of the utility pole installation location 2. However, when the utility pole installation location 2 and the portable terminal device 10 are separated from each other, the absolute position information of the utility pole installation location 2 can be acquired by photographing a location including the utility pole installation location 2 using the photographing means 20 of the portable terminal device 10.

[0029] Specifically, by utilizing the fact that the attitude of the mobile terminal device 10 itself is specified by the inertial sensor 80, the image of the location including the utility pole installation location 2 photographed by the imaging means 20 is analyzed, or the utility pole installation location 2 is specified (specifically, for example, by touch operation) in the image of the location including the utility pole installation location 2 photographed by the imaging means 20 and displayed on the display means 30, and the relative positional relationship between the specified utility pole installation location 2 and the mobile terminal device 10 is calculated, thereby obtaining the relative positional relationship between the utility pole installation location 2 and the mobile terminal device 10. Then, since the absolute positional information of the mobile terminal device 10 can be obtained as described above, the absolute positional information of the utility pole installation location 2 can be obtained.

[0030] This absolute position information of the utility pole installation location 2 makes it possible to display a 3D utility pole 3 as AR content at the utility pole installation location 2 on the shooting screen without placing an AR marker at the utility pole installation location 2. Moreover, any portable terminal device 10 having a program for displaying the 3D utility pole 3 as AR content, a utility pole installation image display application (AR application), an AR engine, etc. installed therein can display the 3D utility pole 3 in the image on the display screen.

[0031] Next, we will explain the procedure for displaying a desired 3D utility pole 3 at a utility pole installation location 2 in an image of real space captured by the imaging means 20 on the display means 30 of the mobile terminal device 10 using the utility pole installation image display system 1 of embodiment 1.

[0032] (Step S1) Check the pole installation location 2 of the newly installed utility pole on site.

[0033] (Step S2) The mobile terminal device 10 is held toward the confirmed utility pole installation location 2, and the plane (horizontal) of the mobile terminal device 10 is detected (see FIG. 3). Whether the mobile terminal device 10 has detected a plane can be confirmed by the inertial sensor 80. The mobile terminal device 10 may be tilted, and the depth, etc. can be obtained by moving the mobile terminal device 10.

[0034] (Step S3) When the flat surface of the mobile terminal device 10 is detected, the operation means (touch panel) 50 is tapped (see FIG. 4). This makes it possible to obtain absolute position information of the utility pole installation location 2, and the absolute position information of the utility pole installation location 2 is stored in the storage means 70. By storing the absolute position information of the utility pole installation location 2, it is possible to register it as an AR marker and have it function as an AR marker, thereby eliminating the need to acquire the position information of the utility pole installation location 2 when, for example, conducting a field survey on a different day.

[0035] In addition, the utility pole installation location 2 may be specified by specifying the utility pole installation location 2 (specifically, for example, by a touch operation) in an image of a location including the utility pole installation location 2 displayed on the display means 30 (see FIG. 2), and absolute position information of the utility pole installation location 2 may be obtained by calculating the positional relationship between the specified utility pole installation location 2 and the mobile terminal device 10.

[0036] (Step S4) When the operation means (touch panel) 50 is tapped, the 3D utility pole 3 stored in the storage means 70 as AR content is read, and the 3D utility pole 3 is displayed at the utility pole installation location 2 on the display screen (see FIG. 5).

[0037] In Figure 5, the utility pole 3 with arm is represented as a 3D utility pole. If it is desired to install a 3D utility pole 3a with a guy wire or a bare utility pole 3b (see Figure 6), the 3D utility pole 3a with a guy wire or the bare utility pole 3b can be stored in advance in the storage means 70, and the desired 3D utility pole 3 can be displayed by selecting the utility pole 3 with arm, the 3D utility pole 3a with a guy wire, or the bare utility pole 3b at a stage prior to step S4.

[0038] (Step S5) By checking the 3D utility pole 3 displayed on the display means 30, it is possible to get a concrete image of how the utility pole will be installed at the utility pole installation location 2. In addition, by long-tapping (touching) the 3D utility pole 3 on the display screen with a fingertip and moving the fingertip, it is possible to move it to a desired position or rotate it to change its orientation.

[0039] (Step S6) The 3D utility pole 3 displayed on the display means 30 is linked to absolute position information in the real space and stored in the storage means 70. This makes it possible to display the 3D utility pole 3 as AR content at the corresponding absolute position on the display screen at any time.

[0040] (Step S7) The length of the 3D utility pole 3 displayed on the display means 30 can be edited from the editing screen, and in addition to the standard utility pole, a longer or shorter utility pole can be displayed, providing a concrete image of what it would look like if different utility poles were installed.

[0041] (Step S8) The photographing means 20 has a photographing function and can take still images as well as moving images, and can take a photograph showing a desired 3D utility pole 3 at the utility pole installation site (in other words, store an image including the 3D utility pole 3). This allows the photograph to be provided to multiple residents.

[0042] (Step S9) It is also possible to drive an AR marker sticker 4a or an AR marker stake 4b into the utility pole installation location 2. Even in the case of a mobile terminal device 10 in which the position information of the utility pole installation location 2 is not stored in the storage means 70, by linking the AR marker 4 with the 3D utility pole 3 stored in the storage means 70, it is possible to display the 3D utility pole 3 at the utility pole installation location 2 on the display screen of the display means 30.

[0043] When the position information of the utility pole installation location 2 stored in the memory means 70 is in planar coordinates (latitude information, longitude information), it becomes unclear where in the height direction (Z-axis direction) the center point of the 3D utility pole 3 should be aligned. However, by installing the AR marker 4, the position of the 3D utility pole 3 in the height (Z-axis) direction can be determined by aligning the center point X of the AR marker sticker 4a with the center point X of the 3D utility pole 3 as the AR content, and the 3D utility pole 3 can be displayed in the desired position on the screen of the display means 30.

[0044] The center point X of the AR marker seal 4a is the center on the surface of the seal 4a (see FIG. 7), while the center point X of the AR marker pile 4b is the center of the pile 4b, which is at approximately the same height as the ground surface when buried, because the lower part of the pile 4b is buried in the ground (see FIG. 8).

[0045] By placing the AR marker 4 in advance, it is possible to eliminate the need to correct the position in the height direction (Z axis) after display, even if the AR marker 4 is displayed with its position in the height direction (Z axis) shifted. The center point X of the utility pole is on the center line of the utility pole, and is a position that coincides with the ground surface when the utility pole is buried. The center point X of the AR marker pile 4b is a position on the center line of the pile 4b that coincides with the ground surface.

[0046] As described above, according to the utility pole installation image display system 1 of embodiment 1, the 3D utility pole 3 is stored in advance as AR content, and by holding the mobile terminal device 10 over the utility pole installation location 2, absolute position information of the utility pole installation location 2 based on the GNSS receiving means 60 can be obtained. Even without installing an AR marker 4 in advance, the 3D utility pole 3 can be displayed on the capture screen captured by the capturing means 20. Therefore, the 3D utility pole 3 can be displayed as a newly installed utility pole at the utility pole installation location 2 where no utility pole has been erected. (Embodiment 2) FIG. 9 is a schematic configuration diagram showing a utility pole installation image display system 1A according to the second embodiment. In the second embodiment, an existing utility pole 5 is relocated to a utility pole installation location 2. As shown in FIG.

[0047] In this embodiment 2, as in the above-mentioned embodiment 1, after the 3D utility pole 3 is newly installed, an additional operation is performed in which the image of the existing utility pole 5 is erased from the display screen of the display means 30. Since the utility pole is to be relocated, the 3D utility pole as the AR content is selected to be the same as the existing utility pole 5.

[0048] The information processing means 40 of the utility pole installation image display system 1A according to the second embodiment is provided with a program having an inpainting function.

[0049] Then, using the procedure described in the first embodiment, the 3D utility pole 3 is displayed at the utility pole installation location 2 on the display screen, and the process proceeds to step S10 below.

[0050] (Step S10) With an image including an existing utility pole 5 photographed by the photographing means 20 being displayed on the display screen, a 3D utility pole 3 is displayed on the screen of the display means 30 in the same manner as in steps S1 to S9 above (see FIG. 10).

[0051] (Step S11) After the 3D utility pole 3 is displayed on the screen of the display means 30, a photograph is taken by the photographing means 20 using the photographing function of the mobile terminal device 10A (see FIG. 11). As a result, a still image showing the existing utility pole 5 and the 3D utility pole 3 is displayed on the screen of the display means 30.

[0052] (Step S12) Next, in order to erase the existing utility pole 5, an inpainting function provided in the information processing means 40 is used. Specifically, the image of the existing utility pole 5 on the display screen of the display means 30 is slid with a fingertip or a touch pen or the like to surround it, thereby selecting the range to be erased (see FIG. 12).

[0053] (Step S13) By activating the inpainting function of the information processing means 40, the existing utility pole 5 is erased (see FIG. 13).

[0054] As a result, the image captured by the imaging means 20 on the display screen of the mobile terminal device 10 is displayed in an image in which the image of the existing utility pole 5 is removed and the relocated 3D utility pole 3 is displayed.

[0055] As described above, according to the utility pole installation image display system 1A of embodiment 2, by erasing the image of the existing utility pole 5, it is possible to produce an image in which only the relocated 3D utility pole 3 is displayed as if it were a utility pole, thereby making it possible to concretely express an image of the state in which the utility pole has been relocated.

[0056] Although the embodiment of the present invention has been described above, the specific configuration is not limited to the above embodiment, and even if there are design changes and the like within the scope of the gist of the present invention, they are included in the present invention.

[0057] For example, in the above embodiment, various programs and 3D utility poles as AR content are stored in the storage means 70, but by providing the mobile terminal device 10 with a communication function, various programs and AR content may be stored in a cloud service or an external device. [Explanation of symbols]

[0058] 1. Utility pole installation image display system 2. Electric pole installation locations 3 3D utility pole with bracket (utility pole object) 3a 3D utility pole with guy wire (utility pole object) 3b Plain 3D utility pole (utility pole object) 10. Portable terminal devices 20. Shooting Method 30 Display means 60 GNSS Receiving Means 70 Memory means 1A Utility pole installation image display system 10A Portable terminal equipment

Claims

1. a portable terminal device including: an imaging means for imaging a location where a utility pole is to be installed; a display means for displaying an image captured by said imaging means; a GNSS receiving means for acquiring position information from a global navigation satellite system (GNSS); and an installation position information acquiring means for acquiring installation position information of an installation location of a utility pole in the image captured by said imaging means based on the position information acquired by said GNSS receiving means; A storage means for storing a utility pole object as an AR content of a utility pole, the installation position information acquisition means identifies the installation position of the utility pole by performing image analysis on the image or by specifying the installation position of the utility pole in the image displayed on the display means, determines a relative positional relationship between the identified installation position of the utility pole and the mobile terminal device, and acquires installation position information of the installation position of the utility pole based on position information of the mobile terminal device acquired by the GNSS receiving means; The display means displays the utility pole object stored in the storage means at a location in the image captured by the imaging means that corresponds to the installation position information of the utility pole acquired by the installation position information acquisition means. A utility pole installation image display system.

2. The utility pole installation image display system according to claim 1, characterized in that a plurality of utility pole objects are stored in the memory means, and a desired utility pole object is selected and displayed at a location corresponding to the installation position information in the image captured by the photographing means.

3. A method for displaying an image of a utility pole installation using a portable terminal device, the method comprising: an imaging means for imaging a location where a utility pole is to be installed; a display means for displaying an image captured by the imaging means; a global navigation satellite system (GNSS) receiving means for acquiring position information from a global navigation satellite system (GNSS); and an installation position information acquiring means for acquiring installation position information of an installation location of a utility pole in the image captured by the imaging means based on the position information acquired by the GNSS receiving means, The portable terminal device is held over the installation location of the utility pole, and the installation location of the utility pole is identified by performing image analysis of the image, or by specifying the installation location of the utility pole in the image displayed on the display means, a relative positional relationship between the identified installation location of the utility pole and the portable terminal device is obtained, and installation location information of the installation location of the utility pole is obtained based on location information of the portable terminal device acquired by the GNSS receiving means, a utility pole object as an AR content of a utility pole stored in a storage means built in the portable terminal device or an external storage means is displayed at a location in the image captured by the imaging means that corresponds to the installation position information; A method for displaying an image of a utility pole installation.

Images