Display device, program, display method, and display system

Abstract

To facilitate checking of the installation status of electricity poles and electricity wires after an electricity pole construction.SOLUTION: An embodiment of the present disclosure is a display device comprising an image capturing unit for capturing an image of the surroundings, and a display unit configured to display electricity pole objects and electricity wire objects superimposed on the image captured by the image capturing unit, where the display unit responds to receipt of an instruction regarding electricity pole construction by displaying electricity pole objects and the electricity wire objects using height information of the electricity wires connected to the electricity poles.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a display device, a program, a display method, and a display system.

Background Art

[0002] Conventionally, there is known a technique for securing a certain distance between an electric wire and surrounding buildings or the like when installing electric poles and electric wires. For example, the distance measuring system described in Patent Document 1 generates a linear AR (Augmented Reality) object connecting a first point and a second point in a photographed area in space, and calculates the distance between a third point in space and the linear AR object, thereby calculating the distance between a virtual linear object and a point in the real space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique described in Patent Document 1 mentioned above accepts a user operation for adjusting the sag and position of an electric wire, and arranges an AR object at a specified location in an image captured by a camera. Therefore, in the technique described in Patent Document 1, the user operation burden for accurately setting the position of the AR object may increase. Further, in the technique described in Patent Document 1, depending on the user's proficiency, the position of the AR object may not be accurately set.

[0005] In view of such circumstances, the present disclosure has been made, and an object thereof is to provide a display device, a program, a display method, and a display system that can easily confirm the installation status of electric poles and electric wires after pole construction. [Means for solving the problem]

[0006] This disclosure has been made to solve the above-mentioned problems, and one aspect of this disclosure is a display device comprising an imaging unit that images the surroundings, and a display unit that superimposes and displays utility pole objects and power line objects on an image captured by the imaging unit, wherein the display unit displays the utility pole objects and power line objects using height information of power lines connected to the utility pole in response to receiving instructions regarding utility pole construction.

[0007] Another aspect of this disclosure is a program that causes the computer of a display device to perform the steps of: causing an imaging unit to image the surroundings; superimposing utility pole objects and power line objects onto the image captured by the imaging unit and displaying them on the display unit; and updating the display state of the utility pole objects and power line objects using height information of the power lines connected to the utility pole in response to receiving instructions regarding utility pole construction.

[0008] Another aspect of the present disclosure is a display method that includes the steps of: having an imaging unit capture an image of the surroundings; superimposing a utility pole object and a power line object onto the image captured by the imaging unit and displaying them on a display unit; and, in response to receiving instructions regarding utility pole construction, displaying the utility pole object and the power line object using height information of the power lines connected to the utility pole.

[0009] Another aspect of the present disclosure is a display system comprising: a data management device that provides geographic information; an imaging unit that images the surroundings; a utility pole and power line recognition unit that recognizes utility pole objects and power line objects included in the imaged images; a geographic information acquisition unit that acquires the geographic information based on the position of the device; an association unit that associates the utility pole objects and power line objects recognized by the utility pole and power line recognition unit with utility poles and power lines included in the geographic information; and a display unit that superimposes the utility pole objects and power line objects associated by the association unit onto the imaged images captured by the imaging unit, wherein the display unit displays the utility pole objects and power line objects using height information of power lines connected to the utility poles in response to receiving instructions regarding utility pole construction. [Effects of the Invention]

[0010] According to one aspect of the present invention, the installation status of utility poles and power lines after utility pole construction can be easily confirmed. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing an example of a display system in an embodiment. [Figure 2] This figure shows an example of GIS data in the embodiment. [Figure 3] This is a block diagram showing an example of a display device in an embodiment. [Figure 4] A flowchart illustrating an example of the operation procedure of the display device in the embodiment. [Figure 5] This is an example of an image captured in the embodiment. [Figure 6] This figure shows an example of a utility pole object, a power line object, and the height of the power line recognized in the captured image. [Figure 7] This figure shows an example of how utility pole objects corresponding to utility poles and power line objects corresponding to power lines are removed from an captured image. [Figure 8]This is a diagram showing an example of a background object superimposed on a captured image. [Figure 9] This is a diagram showing an example of a captured image in which objects corresponding to utility poles and electric wires are deleted in response to a request to relocate a utility pole in an embodiment. [Figure 10] This is a diagram showing an example of a captured image in which objects corresponding to the utility pole after relocation of the original utility pole and the electric wires connected to the utility pole at the relocation destination are drawn. [Figure 11] This is a sequence diagram showing an example of the operation procedure of a display system in an embodiment. [Figure 12] This is a diagram showing an example of a utility pole update table included in GIS data in an embodiment. [Figure 13] This is an example of a captured image including facility objects before utility pole relocation. [Figure 14] This is an example of a captured image including facility objects after utility pole relocation. [[ID=2,0]] [Figure 15] This is an example of a captured image including topographical objects before utility pole relocation. [Figure 16] This is an example of a captured image after deletion of topographical objects. [Figure 17] This is an example of a captured image after utility pole relocation.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, a display device, a program, a display method, and a display system to which the present invention is applied will be described with reference to the drawings.

[0013] (Configuration of Display System) FIG. 1 is a block diagram showing an example of a display system according to an embodiment. The display system includes, for example, a display device 100, a data management device 200, and a data storage device 300. The display device 100 is a terminal device operated by an operator involved in work related to the construction of newly installing, relocating, or removing utility poles. The display device 100, the data management device 200, and the data storage device 300 have a communication interface such as a NIC (Network Interface Card) or a wireless communication module for connecting to a network NW such as the Internet. The network NW may include, for example, a general-purpose network such as the Internet, and a private network such as local 5G or Wifi (registered trademark).

[0014] The data management device 200 is, for example, a server device that manages GIS (Geographic Information System) data. The data management device 200 includes, for example, a data providing unit 210 and a data updating unit 220. The data providing unit 210 receives a request from the display device 100 and provides the display device 100 with GIS data corresponding to the request. The data updating unit 220 receives a request from the display device 100 and updates the GIS data based on the request. The data storage device 300 is a server device including a storage device that stores GIS data. The data storage device 300 may be integrated with the data management device 200 and may be connected to the data management device 200 via the network NW. The data storage device 300 provides the data management device 200 with GIS data according to a request from the data management device 200. The data storage device 300 updates the GIS data according to a request from the data management device 200. In the embodiment, the geographic information is GIS data, but it is not limited to this, and any data that registers information such as utility poles, electric wires, facilities, and features in association with map information may be used.

[0015] Figure 2 shows an example of GIS data in the embodiment. GIS data includes, for example, map information, feature information, and utility pole information. Map information includes, for example, a map information table that registers information contained in a map, such as location information expressed in latitude and longitude, type information indicating the type of road, etc., shape information indicating the shape of road, etc., and height information expressed in altitude. Feature information includes a feature information table that registers information to identify each feature, such as location information, type information, shape information, and height information. Feature examples include trees and buildings standing on the ground, such as houses. Utility pole information includes a utility pole information table that registers information to identify each utility pole, such as location information expressed in latitude and longitude, shape information indicating the shape of the utility pole, utility pole connection information, power line height information, and equipment information. Utility pole connection information is, for example, information that identifies other utility poles connected via power lines. Power line height information is information indicating the height at which power lines are installed on the utility pole. Power line height information is information indicating the height from the base of the utility pole. Equipment information is used to identify equipment installed on utility poles. Examples of equipment installed on utility poles include transformers, switches, and IEDs (Intelligent Electronic Devices).

[0016] Figure 3 is a block diagram showing an example of a display device 100 in an embodiment. The display device 100 is, for example, a tablet terminal. The display device 100 includes, for example, an imaging unit 102, a display unit 104, an operation unit 106, an operation reception unit 108, a GPS processing unit 110, a position acquisition unit 112, a magnetic sensor unit 114, a direction acquisition unit 116, a communication unit 118, a geographic information acquisition unit 120, and a display control unit 130. The imaging unit 102 is a camera device mounted on the display device 100 and generates an image. The display unit 104 is, for example, a display device such as an LCD (Liquid Crystal Display). The operation reception unit 108 is a touch panel built into the display unit 104. The imaging unit 102 captures the surroundings. The display unit 104 displays utility pole objects and power line objects superimposed on the image captured by the imaging unit 102. The display unit 104, upon receiving instructions regarding utility pole construction, displays utility pole objects and power line objects using height information of the power lines connected to the utility pole. Instructions regarding utility pole construction include, for example, instructions to delete, relocate, or install a new utility pole, but are not limited to these; any instruction that changes the position of a utility pole or power line, equipment, etc., as a result of utility pole construction is acceptable.

[0017] The GPS (Global Positioning System) processing unit 110 receives signals from GPS satellites. The position acquisition unit 112 determines the position of the display device 100 based on the GPS signals. The magnetic sensor unit 114 measures the magnetic field around the display device 100. The direction acquisition unit 116 determines the direction the display device 100 is facing based on the detection results of the magnetic sensor unit 114. The communication unit 118 is a NIC that communicates with the data management device 200, etc., via the network NW. The geographic information acquisition unit 120 acquires geographic information based on the position of its own device. For example, the geographic information acquisition unit 120 provides the position information acquired by the position acquisition unit 112 to the data management device 200 and acquires geographic information about the area around the position indicated by the position information.

[0018] The display control unit 130 includes, for example, a recognition unit 132, a mapping unit 134, a self-position correction unit 136, a wire height calculation unit 138, and an image processing unit 140. Functional units such as the recognition unit 132, the mapping unit 134, the self-position correction unit 136, the wire height calculation unit 138, and the image processing unit 140 are realized by a processor, such as a CPU (Central Processing Unit), executing a program stored in program memory. Some or all of these functional units may be realized by hardware such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), or they may be realized by the cooperation of software and hardware.

[0019] The recognition unit 132 includes, for example, a utility pole / power line recognition unit 150, an equipment recognition unit 152, and a geographical feature recognition unit 154.

[0020] The utility pole / power line recognition unit 150 recognizes utility pole objects and power line objects based on the captured image or geographic information. For example, the utility pole / power line recognition unit 150 recognizes the region containing utility poles and the region containing power lines in the captured image by performing pattern recognition processing on the captured image. If the geographic information acquired by the geographic information acquisition unit 120 includes utility pole information, the utility pole / power line recognition unit 150 may recognize utility pole objects and power line objects in the captured image based on the utility pole information included in the geographic information.

[0021] The equipment recognition unit 152 recognizes equipment objects installed on utility poles based on the captured image or geographic information. For example, the equipment recognition unit 152 recognizes the area containing equipment in the captured image by performing pattern recognition processing on the captured image. If the geographic information acquired by the geographic information acquisition unit 120 includes equipment information, the equipment recognition unit 152 may recognize equipment objects in the captured image based on the equipment information included in the geographic information.

[0022] The feature recognition unit 154 recognizes feature objects around the utility poles and power lines based on the captured image or geographic information. The feature recognition unit 154 recognizes areas in the captured image that contain features by, for example, performing pattern recognition processing on the captured image. If the geographic information acquired by the geographic information acquisition unit 120 includes feature information, the feature recognition unit 154 may recognize feature objects in the captured image based on the feature information included in the geographic information. In this embodiment, objects in the captured image recognized by the recognition unit 132, areas in the captured image, and objects included in the geographic information are referred to as "objects."

[0023] The mapping unit 134 maps the utility pole objects and power line objects recognized by the utility pole / power line recognition unit 150 to the utility pole information included in the geographic information. For example, the mapping unit 134 estimates the latitude and longitude of the utility pole recognized by the utility pole / power line recognition unit 150 based on its own position and the position of the utility pole object in the captured image, and searches for a utility pole included in the geographic information that is close to the estimated latitude and longitude. The mapping unit 134 compares the shape of the utility pole included in the searched geographic information with the shape of the utility pole recognized by the utility pole / power line recognition unit 150, and determines that the utility pole object recognized by the utility pole / power line recognition unit 150 corresponds to the utility pole included in the geographic information if the degree of match exceeds a threshold. The mapping unit 134 maps the power line objects connecting the mapped utility poles to the power lines included in the utility pole information.

[0024] The self-position correction unit 136 corrects the position of its own device based on the difference between the position information of the utility pole object, which has been associated by the matching unit 134 and recognized by the utility pole / power line recognition unit 150, and the position information of the utility pole included in the geographic information. The self-position correction unit 136 calculates the difference between the position of the utility pole object recognized by the utility pole / power line recognition unit 150 and the position of the utility pole object in the geographic information associated by the matching unit 134. The self-position correction unit 136 corrects the position of the display device 100 acquired by the position acquisition unit 112 by the amount of the calculated difference.

[0025] The self-position correction unit 136 may also correct the direction of the display device 100 acquired by the direction acquisition unit 116. For example, the self-position correction unit 136 corrects the direction of the display device 100 acquired by the direction acquisition unit 116 by the difference between the direction of the utility pole object recognized by the utility pole / power line recognition unit 150 and the direction in the geographic information of the utility pole object associated by the association unit 134.

[0026] The power line height derivation unit 138 calculates the in-image height from the base position of the utility pole object to the power line installation position of the utility pole object, and converts the in-image height of the power line to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole. The power line height derivation unit 138 identifies the base position of the utility pole object in the captured image by pattern recognition of the base position in the captured image. For example, the power line height derivation unit 138 may identify the base position of the utility pole object by information indicating the latitude, longitude, and height of the utility pole in geographic information. The power line height derivation unit 138 identifies the power line installation position of the utility pole object by pattern recognition of the base position in the captured image.

[0027] The image processing unit 140, upon receiving a change request related to utility pole construction from the display device 100, performs image processing to display utility pole objects and power line objects using height information of power lines connected to the utility pole. The image processing unit 140 includes, for example, an object deletion unit 160 and an object drawing unit 162. The object deletion unit 160 performs image processing to delete objects included in the captured image. The object drawing unit 162 performs image processing to draw objects on the captured image. In this embodiment, objects are not only utility poles and power lines, but also landmarks, equipment, trees, buildings, and other objects related to utility pole construction such as new construction, relocation, and deletion of utility poles. In this embodiment, objects may be marks indicating the height position of power lines identified on the utility pole recognized by the power line height derivation unit 138.

[0028] Figure 4 is a flowchart showing an example of the operation procedure of the display device 100 in the embodiment. First, the display device 100 starts a program (step S100) and starts imaging by activating the imaging unit 102 according to the operator's operation (step S102). Figure 5 is an example of an image captured in the embodiment. The image captured includes, for example, utility poles A, B, and C, and power lines D and E.

[0029] The utility pole / wire recognition unit 150 of the display control unit 130 recognizes utility poles A, B, C and wires D, E included in the image captured by the imaging unit 102 (step S104). Furthermore, when the wire height derivation unit 138 recognizes utility poles, wires, and the in-image height of wires, it may highlight the recognized objects to indicate to the operator that they have been recognized. Figure 6 shows an example of utility pole objects, wire objects, and wire heights recognized in an image captured. The utility pole / wire recognition unit 150 can recognize, for example, the in-image heights of utility poles A, wires D, E, and wires on utility poles B, C as objects. Note that the connection position of wires on utility poles may not be recognized in the object recognition process in step S104, but may be recognized in the subsequent process of deriving the wire heights.

[0030] The geographic information acquisition unit 120 acquires geographic information about the area around the display device 100 (step S106). Next, the mapping unit 134 maps the utility poles A, B, and C recognized by the utility pole / power line recognition unit 150 to the utility poles included in the geographic information (step S108). Next, the self-position correction unit 136 corrects the position of the display device 100 based on the difference in position between the utility pole objects (A, B, C) recognized by the utility pole / power line recognition unit 150 and the utility pole objects included in the geographic information, which have been mapped by the mapping unit 134 (step S110). Next, the power line height derivation unit 138 derives the heights of the power lines D and E connected to the utility pole objects (A, B, C) for the utility pole objects (A, B, C) that have been mapped by the mapping unit 134 (step S112). As shown in Figure 6, the wire height derivation unit 138 can derive the height of the connection position of wire D on utility pole B, and the height of the connection position of wire E on utility pole C.

[0031] The display control unit 130 performs the processing from steps S104 to S112 for each of the utility pole objects (A, B, C) in the captured image generated in step S102. Furthermore, even if the captured image generated by the operator is changed due to the imaging direction, the display control unit 130 continues to perform the processing from steps S104 to S112 for the utility pole objects included in the captured image.

[0032] The display control unit 130 determines whether or not it has received a request to delete a utility pole as an instruction related to utility pole construction based on the operation received by the operation reception unit 108 (step S114). If the object deletion unit 160 has received a request to delete a utility pole as an instruction related to utility pole construction (step S114: YES), it deletes the areas corresponding to the utility pole object (A) and power line objects (D, E) included in the captured image (step S116). Figure 7 shows an example of how the utility pole object (A) corresponding to utility pole A, the power line object (D) corresponding to power line D, and the power line object (E) corresponding to power line E are deleted from the captured image.

[0033] Next, the object drawing unit 162 overlays a background created using the surrounding image onto the deleted area. Specifically, the object drawing unit 162 generates a background to be overlaid on the deleted utility pole object (A) and power line objects (D, E) using the surrounding images of the deleted utility pole object (A) and power line objects (D, E) (step S118). The object drawing unit 162 overlays the generated background onto the area of ​​the deleted utility pole object (A) and power line objects (D, E) and displays it on the display unit 104 (step S120). Figure 8 is a diagram showing an example of how the background is overlaid on the captured image. As shown in Figure 7, the object drawing unit 162 creates the background by interpolating the image of the area occupied by the deleted utility pole object (A) and power line objects (D, E) from the surrounding images of the deleted utility pole object (A) and power line objects (D, E). As a result, the display unit 104 can display an image showing the work performed to remove utility pole A and power lines D and E, as shown in Figure 8.

[0034] The image processing unit 140 determines whether it has received a request to relocate a utility pole as an instruction regarding utility pole construction based on the operation received by the operation reception unit 108 (step S122). The operation received by the operation reception unit 108 includes, for example, an operation to select a utility pole object (A) and move the utility pole object (A) from its current position to the relocation destination. The display control unit 130 determines that it has received a relocation request when it receives such an operation from the operation reception unit 108. If the object deletion unit 160 has received a request to relocate the utility pole object (A) (step S114: YES), it deletes the areas corresponding to the utility pole object (A) and power line objects (D, E) included in the captured image (step S124, Figure 7).

[0035] Next, the object drawing unit 162 generates a background to be superimposed on the deleted utility pole object (A) and power line objects (D, E) using images of the surrounding area (step S126). The object drawing unit 162 superimposes and displays the background created using images of the surrounding area onto the deleted area (step S128, Figure 9). Next, the object drawing unit 162 displays the utility pole object (F) at the relocation site and, using the height information of the power lines, displays the power line objects (G, H) connected to the utility pole object (F) displayed at the relocation site (step S130, Figure 10). As a result, the object drawing unit 162 can superimpose and display the utility pole object (F) and power line objects (G, H) on the captured image.

[0036] Figure 9 shows an example of an image captured in an embodiment where objects corresponding to utility pole A and power lines D and E have been deleted in response to a request to relocate utility pole A. Figure 10 shows an example of an image captured where objects corresponding to utility pole F, to which utility pole A has been relocated, and power lines G and H connected to utility pole F have been drawn. Suppose the display control unit 130 receives a request to relocate utility pole A, shown in Figure 5, to utility pole F, shown in Figure 10. When the relocation request is received, the power line height derivation unit 138 derives the power line heights at utility poles B and C, to which power lines D and E, which were connected to utility pole A, were connected, as shown in Figure 9. The object drawing unit 162 places a utility pole object corresponding to utility pole F at the relocation destination in the image captured, as shown in Figure 10, and draws a power line object for power line G that connects the power line connection position of the utility pole object corresponding to utility pole F to the power line height position of utility pole B. Similarly, the object drawing unit 162 draws a wire object for wire H that connects the wire connection position of the utility pole object corresponding to utility pole F to the wire height position of utility pole C. The object drawing unit 162 may also highlight the relocated objects (utility poles, wires, height, etc.) to indicate to the operator that the drawing process has been completed.

[0037] Next, the image processing unit 140 determines whether or not it has received a request to install a new utility pole as an instruction regarding utility pole construction based on the operation received by the operation reception unit 108 (step S132). The operation received by the operation reception unit 108 includes, for example, an operation to specify the location where the new utility pole will be installed, and the display control unit 130 determines that it has received a request to install a new utility pole when it receives such an operation from the operation reception unit 108. If the object drawing unit 162 has received a request to install a new utility pole object (step S132: YES), it displays the utility pole object and the power line object connected to the utility pole object at the received location (step S134). This allows the object drawing unit 162 to display the newly installed utility pole object and power line object superimposed on the captured image. If the request to install a new utility pole object has not been received (step S132: NO), the display control unit 130 determines whether or not the program has been terminated (step S136). If the program has not been terminated (step S136: NO), the display control unit 130 repeats the processing from step S102 onwards. If the program has been terminated (step S136: YES), it terminates the processing in this flowchart.

[0038] Figure 11 is a sequence diagram showing an example of the operation procedure of the display system in the embodiment. When the program is started, the display device 100 provides the data management device 200 with the location information (self-position) of the display device 100 acquired by the GPS processing unit 110 and the position acquisition unit 112. The data provision unit 210 of the data management device 200 requests geographic information of the area around the location of the display device 100 from the data storage device 300, and the data storage device 300 provides the geographic information of the area around the location of the display device 100 to the data provision unit 210. The data provision unit 210 provides the geographic information of the area around the location of the display device 100 to the display device 100.

[0039] The display unit 104 displays the captured image generated by the imaging unit 102, and also displays the utility pole object and power line object that have been associated by the mapping unit 134 based on geographic information around the location of the display device 100 (step S200). The display control unit 130 receives a request to change a utility pole as an instruction regarding utility pole construction from the operation unit 106 and the operation reception unit 108 (step S202). The change request may include, for example, utility pole construction such as deletion, relocation, or new construction of a utility pole. Accordingly, the display control unit 130 deletes the object using the object deletion unit 160 and draws the object using the object drawing unit 162. The display device 100 can display the results of object deletion and drawing performed in response to the change request on the display unit 104. The above is the object modification stage.

[0040] Next, the display system performs the update process for the GIS data, which stores the object changes in the modification phase in the data storage device 300. When the operation unit 106 and the operation reception unit 108 receive an operation to save the changes to the GIS data, they provide the data update unit 220 with change data indicating the changes to the objects. The change data is, for example, information indicating deleted utility pole objects and power line objects from the geographic information. The change data may also include information indicating relocated utility pole objects from the geographic information, information indicating the location of the relocated power line objects, information indicating power line objects connected to the relocated utility pole objects, information indicating the utility poles to which the power line objects are connected, and information indicating the power line height of the utility poles to which the power line objects are connected.

[0041] The data update unit 220 receives the change data from the display device 100 and sends a request to the data storage device 300 to save the change data. The data storage device 300 updates the GIS data based on the change data provided via the data update unit 220 (step S206).

[0042] Figure 12 shows an example of a utility pole update table included in the GIS data in the embodiment. The data storage device 300 generates a utility pole update table based on the change data provided by the data update unit 220. The utility pole update table includes, for example, the location of the utility pole, the shape of the utility pole, utility pole connection information, power line height information, and equipment information updated according to instructions regarding utility pole construction. The data storage device 300 transmits completion report information to the data update unit 220 indicating that the creation of the utility pole update table is complete, and the data update unit 220 provides completion report information to the display device 100 indicating that the saving of the change data is complete. The above is the GIS data update stage.

[0043] For example, when explaining the details of utility pole construction to residents near the construction site, the operation unit 106 and operation reception unit 108 receive a request to view the change data based on the worker's operation (step S208), and the display device 100 requests the data provision unit 210 to provide the change data. In response to receiving the request for change data from the display device 100, the data provision unit 210 requests the data storage device 300 to provide the change data. In response to the request for change data, the data storage device 300 provides the data provision unit 210 with the GIS data included in the utility pole update table. This allows the data provision unit 210 to provide the change data (GIS data) requested by the display device 100. The display device 100 can then use the requested change data to display an image including the utility pole object and power line object after the utility pole construction has been completed (step S210). This completes the change confirmation stage for utility pole construction.

[0044] Figure 13 is an example of an image captured including equipment objects before the relocation of the utility pole, and Figure 14 is an example of an image captured including equipment objects after the relocation of the utility pole. As shown in Figure 13, the equipment recognition unit 152 recognizes the equipment object corresponding to equipment I when equipment I is installed on utility pole A. When the display device 100 receives a request to delete utility pole A, it deletes the area corresponding to the equipment object recognized by the equipment recognition unit 152 and superimposes a background object created using the image of the area surrounding the deleted area.

[0045] When the object drawing unit 162 receives a request to relocate utility pole A, on which equipment object I is installed, as an instruction regarding utility pole construction, it draws a utility pole object corresponding to utility pole F at the relocation site, as shown in Figure 14, and draws equipment object J installed on the relocated utility pole F. Furthermore, the object drawing unit 162 draws a wire object corresponding to the wire G connected to the utility pole object corresponding to utility pole F, connecting it to the wire height position on utility pole B. Furthermore, the object drawing unit 162 draws a wire object corresponding to the wire H connected to the utility pole object corresponding to utility pole F, connecting it to the wire height position on utility pole C. As a result, according to the display device 100, even when equipment is attached to a utility pole, equipment objects corresponding to the equipment can be drawn in accordance with the change in the utility pole. Consequently, according to the display device 100, changes in the installation status of equipment can be easily confirmed in addition to the installation status of utility poles and wires.

[0046] Figure 15 is an example of an image captured including a feature object before the relocation of the utility pole, Figure 16 is an example of an image captured after the removal of the feature object, and Figure 17 is an example of an image captured after the relocation of the utility pole. As shown in Figure 15, the feature recognition unit 154 recognizes a feature object corresponding to a tree K if a tree K exists around the utility pole A. When the display device 100 receives a request to remove the tree K, as shown in Figure 16, it deletes the area corresponding to the feature object recognized by the feature recognition unit 154 and superimposes a background object created using the image of the area surrounding the deleted area. As a result, the display device 100 makes it easy to check the situation after construction even when a feature is deleted during utility pole construction.

[0047] When the display device 100 receives a request to relocate utility pole A to the location where tree K exists, it deletes the area corresponding to the feature object recognized by the feature recognition unit 154 and superimposes a background object created using the surrounding image onto the deleted area. Furthermore, as shown in Figure 17, the object drawing unit 162 draws a utility pole object corresponding to utility pole L at the relocation site. Furthermore, the object drawing unit 162 draws a power line object corresponding to the power line M connected to the utility pole object corresponding to utility pole L, connecting it to the power line height position of utility pole B. Furthermore, the object drawing unit 162 draws a power line object corresponding to the power line N connected to the utility pole object corresponding to utility pole L, connecting it to the power line height position of utility pole C. As a result, the display device 100 can delete the feature object and draw the utility pole object even if a feature exists at the relocation site of the utility pole. Consequently, the display device 100 makes it easy to check the situation after the work is completed, such as deleting the feature and relocating the utility pole.

[0048] As described above, the display device 100 of this embodiment includes an imaging unit 102 that images the surroundings, and a display unit 104 that superimposes and displays utility pole objects and power line objects on the image captured by the imaging unit 102. The display unit 104 can display utility pole objects and power line objects using height information of power lines connected to utility poles in response to receiving instructions regarding utility pole construction. According to the display device 100, since it displays utility pole objects and power line objects in response to receiving instructions regarding utility pole construction, it is possible to easily check the installation status of utility poles and power lines after utility pole construction, such as deletion, relocation, or new installation.

[0049] According to this embodiment, a program can be implemented that causes the computer of the display device 100 to perform the following steps: to have the imaging unit 102 capture images of the surroundings; to superimpose utility pole objects and power line objects onto the captured images taken by the imaging unit 102 and display them on the display unit 104; and to update the display state of the utility pole objects and power line objects using height information of the power lines connected to the utility pole in response to receiving instructions regarding utility pole construction. According to the program, since the utility pole objects and power line objects are displayed in response to receiving instructions regarding utility pole construction, the installation status of utility poles and power lines after utility pole construction, such as deletion, relocation, or new installation, can be easily confirmed.

[0050] According to this embodiment, a display method can be realized that includes the steps of: having the imaging unit 102 capture images of the surroundings; superimposing utility pole objects and power line objects onto the captured images taken by the imaging unit 102 and displaying them on the display unit 104; and displaying utility pole objects and power line objects using height information of power lines connected to the utility pole in response to receiving instructions regarding utility pole construction. With this display method, since utility pole objects and power line objects are displayed in response to receiving instructions regarding utility pole construction, the installation status of utility poles and power lines after utility pole construction, such as deletion, relocation, or new installation, can be easily confirmed.

[0051] According to this embodiment, a display system can be realized that includes a data management device 200 that provides geographic information, an imaging unit 102 that images the surroundings, a utility pole / power line recognition unit 150 that recognizes utility pole objects and power line objects included in the captured image, a geographic information acquisition unit 120 that acquires geographic information based on the position of the device, a mapping unit 134 that maps the utility pole objects and power line objects recognized by the utility pole / power line recognition unit 150 to utility poles and power lines included in the geographic information, and a display unit 104 that superimposes and displays the utility pole objects and power line objects mapped by the mapping unit 134 onto the captured image taken by the imaging unit 102, and a display device 100 that displays utility pole objects and power line objects using height information of power lines connected to utility poles in response to receiving instructions regarding utility pole construction via the display unit 104. According to the display system, utility pole objects and power line objects are displayed in response to instructions received regarding utility pole construction, making it easy to check the installation status of utility poles and power lines after construction work, such as deletion, relocation, and new installation.

[0052] Furthermore, according to the embodiment, the system includes a self-position correction unit 136 that corrects the position of the device based on the difference between the position information of the utility pole object and the utility wire object recognized by the utility pole / power line recognition unit 150, which are associated by the matching unit 134, and the position information of the utility pole included in the geographic information. According to the embodiment, the utility pole object and the power line object can be displayed based on the position of the high-precision display device 100. As a result, according to the embodiment, the utility pole object and the power line object can be displayed at a more accurate position in the captured image.

[0053] Furthermore, according to the embodiment, the system includes a height derivation unit that calculates the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object, and converts the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole. According to the embodiment, since the height information of the power lines on the utility pole object can be derived, the work of setting the height of the power lines by the operator can be eliminated. In addition, according to the embodiment, the work of setting the start and end points of the power line object by the operator can be eliminated.

[0054] Although various embodiments and modifications have been described, these are merely examples and are not limited to these. For example, one embodiment or modification, or a part of one embodiment or modification, may be combined with one or more other embodiments or modifications to realize one aspect of the present invention. [Explanation of Symbols]

[0055] 100…Display device, 100a…Captured image, 102…Imaging unit, 104…Display unit, 106…Operation unit, 108…Operation reception unit, 110…GPS processing unit, 112…Position acquisition unit, 114…Magnetic sensor unit, 116…Direction acquisition unit, 118…Communication unit, 120…Geographic information acquisition unit, 130…Display control unit, 132…Recognition unit, 134…Mapping unit, 136…Self-position correction unit, 138…Power line height derivation unit, 140…Image processing unit, 150…Utility pole / power line recognition unit, 152…Equipment recognition unit, 154…Feature recognition unit, 160…Object deletion unit, 162…Object drawing unit, 200…Data management device, 210…Data provision unit, 220…Data update unit, 300…Data storage device

Claims

1. An imaging unit that captures images of the surroundings, The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit, The aforementioned display unit is In response to receiving instructions regarding utility pole construction, the system displays the utility pole object and the power line object using the height information of the power lines connected to the utility pole. When a request to delete the utility pole is received as an instruction regarding the aforementioned utility pole construction, the areas corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the image surrounding the deleted area is superimposed and displayed on the deleted area. Display device.

2. An imaging unit that captures images of the surroundings, The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit, The aforementioned display unit is In response to receiving instructions regarding utility pole construction, the system displays the utility pole object and the power line object using the height information of the power lines connected to the utility pole. If a request to relocate the aforementioned utility pole is received as an instruction regarding the aforementioned utility pole construction, The regions corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the surrounding image of the region is superimposed and displayed on the deleted region. The utility pole object is displayed at the relocation site, and the power line object connected to the utility pole object displayed at the relocation site is displayed using the power line height information. Display device.

3. An imaging unit that captures images of the surroundings, The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit, In response to receiving instructions regarding utility pole construction, the display unit displays the utility pole object and the power line object using height information of the power lines connected to the utility pole. A geographic information acquisition unit that acquires geographic information based on the position of the device, A utility pole and power line recognition unit that recognizes the utility pole object and the power line object based on the captured image or the geographical information, The system includes a matching unit that associates the utility pole object and the power line object recognized by the utility pole and power line recognition unit with the utility poles included in the geographical information. Display device.

4. The display device according to claim 3, further comprising a correction unit that corrects the position of the device based on the difference between the position information of the utility pole object, which has been associated by the association unit and recognized by the utility pole and power line recognition unit, and the position information of the utility pole included in the geographical information.

5. An imaging unit that captures images of the surroundings, The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit, In response to receiving instructions regarding utility pole construction, the display unit displays the utility pole object and the power line object using height information of the power lines connected to the utility pole. A display device comprising a height derivation unit that calculates the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object, and converts the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole.

6. The system includes an equipment recognition unit that recognizes equipment objects installed on the utility pole based on the captured image or the geographical information, The aforementioned display unit is When a request to delete the aforementioned utility pole is received, the area corresponding to the equipment object recognized by the equipment recognition unit is deleted, and a background object created using the image surrounding the area is superimposed on the deleted area. When a request is received to relocate the aforementioned utility pole, the equipment object will be displayed at the relocation site. The display device according to claim 3.

7. The system includes a feature recognition unit that recognizes feature objects around the utility pole and the power line based on the captured image or the geographic information, When the display unit receives a request to delete a feature, it deletes the area corresponding to the feature recognized by the feature recognition unit, and overlays and displays a background object created using the image surrounding the area onto the deleted area. The display device according to claim 3.

8. In the computer of the display device, The steps include: having the imaging unit capture images of the surroundings, The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit and displaying them on the display unit; In response to receiving instructions regarding utility pole construction, the system performs the step of updating the display state of the utility pole object and the power line object using the height information of the power lines connected to the utility pole. A program that, upon receiving a request to delete the utility pole as an instruction related to the aforementioned utility pole construction, deletes the area corresponding to the utility pole object and the power line object included in the captured image, and overlays and displays a background object created using the image surrounding the deleted area.

9. In the computer of the display device, The steps include: having the imaging unit capture images of the surroundings, The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit and displaying them on the display unit; In response to receiving instructions regarding utility pole construction, the system performs the step of updating the display state of the utility pole object and the power line object using the height information of the power lines connected to the utility pole. If a request to relocate the aforementioned utility pole is received as an instruction regarding the aforementioned utility pole construction, The regions corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the surrounding image of the region is superimposed and displayed on the deleted region. The utility pole object is displayed at the relocation site, and the power line object connected to the utility pole object displayed at the relocation site is displayed using the power line height information. program.

10. In the computer of the display device, The steps include: having the imaging unit capture images of the surroundings, The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit and displaying them on the display unit; In response to receiving instructions regarding utility pole construction, the display state of the utility pole object and the power line object is updated using height information of the power lines connected to the utility pole. A step of acquiring geographic information based on the location of the device, The steps include recognizing the utility pole object and the power line object based on the captured image or the geographical information, The steps include: associating the recognized utility pole object and the power line object with the utility poles included in the geographic information; A program that executes something.

11. In the computer of the display device, The steps include: having the imaging unit capture images of the surroundings, The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit and displaying them on the display unit; In response to receiving instructions regarding utility pole construction, the display state of the utility pole object and the power line object is updated using height information of the power lines connected to the utility pole. The steps include: calculating the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object; and converting the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole; A program that executes something.

12. A display device comprising the steps of causing the surrounding area to be imaged by the imaging unit, The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit, The display device, upon receiving instructions regarding utility pole construction, displays the utility pole object and the power line object using height information of the power lines connected to the utility pole; When the display device receives a request to delete the utility pole as an instruction regarding the utility pole construction, it deletes the area corresponding to the utility pole object and the power line object included in the captured image, and overlays and displays a background object created using the image surrounding the deleted area. Display methods, including those mentioned above.

13. A display device comprising the steps of causing the surrounding area to be imaged by the imaging unit, The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit, The display device, upon receiving instructions regarding utility pole construction, includes the step of displaying the utility pole object and the power line object using height information of the power lines connected to the utility pole, The aforementioned display device If a request to relocate the aforementioned utility pole is received as an instruction regarding the aforementioned utility pole construction, The regions corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the surrounding image of the region is superimposed and displayed on the deleted region. The utility pole object is displayed at the relocation site, and the power line object connected to the utility pole object displayed at the relocation site is displayed using the power line height information. Display method.

14. A display device comprising the steps of causing the surrounding area to be imaged by the imaging unit, The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit, The display device, upon receiving instructions regarding utility pole construction, displays the utility pole object and the power line object using height information of the power lines connected to the utility pole; The display device acquires geographic information based on its own location. The display device includes the steps of recognizing the utility pole object and the power line object based on the captured image or the geographic information, The display device performs the steps of associating the recognized utility pole object and the power line object with the utility pole included in the geographic information, Display methods, including those mentioned above.

15. A display device comprising the steps of causing the surrounding area to be imaged by the imaging unit, The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit, The display device, upon receiving instructions regarding utility pole construction, displays the utility pole object and the power line object using height information of the power lines connected to the utility pole; The display device calculates the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object, and converts the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole. Display methods, including those mentioned above.

16. A data management device that provides geographic information, The device comprises: an imaging unit that images the surroundings; a utility pole and power line recognition unit that recognizes utility pole and power line objects included in the captured image; a geographic information acquisition unit that acquires geographic information based on the position of the device; a mapping unit that maps the utility pole and power line objects recognized by the utility pole and power line recognition unit to the utility poles and power lines included in the geographic information; and a display unit that superimposes the utility pole and power line objects mapped by the mapping unit onto the captured image taken by the imaging unit, wherein the display unit, in response to receiving instructions regarding utility pole construction, displays the utility pole and power line objects using height information of the power lines connected to the utility pole. A display system equipped with the following features.

17. The display system according to claim 16, further comprising a correction unit that corrects the position of the device based on the difference between the position information of the utility pole object, which has been associated by the association unit and recognized by the utility pole and power line recognition unit, and the position information of the utility pole included in the geographical information.

18. The system includes an equipment recognition unit that recognizes equipment objects installed on the utility pole based on the captured image or the geographical information, The aforementioned display unit is When a request to delete the aforementioned utility pole is received, the area corresponding to the equipment object recognized by the equipment recognition unit is deleted, and a background object created using the image surrounding the area is superimposed on the deleted area. When a request is received to relocate the aforementioned utility pole, the equipment object will be displayed at the relocation site. The display system according to claim 16.

19. The system includes a feature recognition unit that recognizes feature objects around the utility pole and the power line based on the captured image or the geographic information, When the display unit receives a request to delete a feature, it deletes the area corresponding to the feature recognized by the feature recognition unit, and overlays and displays a background object created using the image surrounding the area onto the deleted area. The display system according to claim 16.

Images