Terminal device and program

Abstract

To provide a terminal device and the like that execute AR navigation that does not acquire images that may cause suspicion of sneak photography.SOLUTION: A terminal device comprises: a route information acquisition unit that acquires route information indicating a route to a destination; a current position acquisition unit that acquires a current position of a user; an imaging unit that generates a real-world image obtained by imaging the real world at the current position; an orientation detection unit that detects a shooting orientation of the imaging unit; an angle detection unit that detects an elevation / depression angle of the shooting direction of the imaging unit; an AR image generation unit that generates an AR image indicating a traveling direction according to the route on the basis of the route information, the current position, the shooting orientation, and the elevation / depression angle; a storage unit that stores shooting stop conditions; a control unit that stops generation of the real-world image by the imaging unit when any of the current position, the shooting orientation, or the elevation / depression angle satisfies the shooting stop conditions; and a display unit that displays the AR image in a manner of being superimposed on the real-world image or the real world.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an AR navigation that provides route guidance to a destination using augmented reality (AR) technology.

Background Art

[0002] For example, Patent Document 1 discloses an AR navigation device that generates an AR image in which a virtual character is superimposed on a real-world image captured by a camera in a terminal equipped with a display unit such as a liquid crystal screen and a camera, and displays the AR image on the display unit to provide route guidance. Patent Document 1 discloses detecting the degree of inclination of a terminal held by a user with respect to the horizontal direction, and performing a display switch between displaying the AR image on the screen or displaying a map image according to the degree of inclination.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when an AR image is displayed on a terminal, since the user makes the same gesture as when taking a picture of the surrounding scenery, it may seem to people around that the user is taking a picture, and thus there is a possibility of being suspected.

[0005] For example, although the user himself / herself who uses AR navigation has no malicious intent at all, if the user makes a gesture of displaying an AR image near the dance floor of a staircase to check the route, there is a possibility of being suspected of voyeurism.

[0006] In one aspect, the present specification discloses a terminal device or the like that executes AR navigation without acquiring an image that may be suspected of voyeurism.

Means for Solving the Problem

[0007] (1) To solve the above problems, a terminal device according to an aspect of the present disclosure includes a route information acquisition unit that acquires route information indicating a route to a destination, a current position acquisition unit that acquires the current position of a user, a photographing unit that generates a real image of the real world at the current position, an orientation detection unit that detects the photographing orientation of the photographing unit, an angle detection unit that detects the elevation angle of the photographing direction of the photographing unit, an AR image generation unit that generates an AR image indicating a traveling direction along the route based on the route information, the current position, the photographing orientation, and the elevation angle, a storage unit that stores a photographing stop condition, and a control unit that stops the generation of the real image by the photographing unit when any of the current position, the photographing orientation, or the elevation angle satisfies the photographing stop condition, and a display unit that superimposes and displays the AR image on the real image or the real world.

[0008] (2) In the terminal device described in (1), even when at least one of the current position, the photographing orientation, and the elevation angle satisfies the photographing stop condition, the AR image generation unit generates an AR image, and the AR image includes position information indicating a position in a display area for superimposing and displaying on the real image or the real world. The display unit may display the AR image based on the position information. Thereby, although the generation of the real image stops, the guidance by the AR image can continue.

[0009] (3) In the terminal device described in (1), when at least one of the current position, the photographing orientation, and the elevation angle satisfies the photographing stop condition, the control unit may further cause the display unit to display that the generation of the real image by the photographing unit has stopped. Thereby, the user can know that the terminal device is at a position, or an orientation, or an angle where the photographing stops, and can take appropriate measures.

[0010] (4) A program according to an aspect of the present disclosure includes a route information acquisition unit that acquires route information indicating a route to a destination, a current position acquisition unit that acquires a current position, a photographing unit that generates a real image obtained by photographing the real world at the current position, an orientation detection unit that detects the photographing orientation of the photographing unit, an angle detection unit that detects the elevation angle of the photographing direction of the photographing unit, an AR image generation unit that generates an AR image indicating a traveling direction along the route based on the route information, the current position, the photographing orientation, and the elevation angle, a storage unit that stores a photographing stop condition, and a display unit that overlays and displays the AR image on the real image or the real world, and is a program for causing a computer that executes AR navigation to perform a photographing control process, where the photographing control process includes a step of determining whether any one of the current position, the photographing orientation, or the elevation angle satisfies the photographing stop condition during the execution of AR navigation, and a step of stopping the generation of the real image by the photographing unit when it is determined that the photographing stop condition is satisfied.

Effect of the Invention

[0011] The terminal device to which the present invention is applied does not acquire an image that may be suspected of being secretly photographed during the execution of AR navigation, so that the user can dispel the suspicion of secret photography.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0013] [Overview of Navigation System] Hereinafter, a navigation system according to an embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the navigation system includes a terminal device 1 and a server 3. A navigation app, which is a program for performing route guidance (navigation) to a set destination, is installed in the terminal device 1.

[0014] When the user operates the terminal device 1 to start the navigation app and sets a destination, the terminal device 1 communicates with the server 3 via the network 2, acquires route information to the set destination from the server 3, and performs route guidance to the destination.

[0015] Although not shown, there are a plurality of terminal devices 1. And the server 3 can provide a navigation service to a plurality of terminal devices 1 via the network 2. The network 2 is, for example, a communication network composed of a mobile phone base station, the Internet, or the like.

[0016] The navigation app has two display modes. One is the map mode. In the map mode, the current position and the destination are shown on the map by icons or the like, and the route from the current position to the destination is highlighted in a color different from other routes.

[0017] The other is the AR mode. In the AR mode, AR navigation is realized by superimposing an AR image, which is an indicator such as an arrow indicating the direction of travel to the destination, on the real world or a real image generated by photographing the real world.

[0018] Here, AR navigation refers to a function of easily teaching the user the direction of travel from the current position to the destination by superimposing an AR image on a real image generated by photographing the real world or the real world visible to the user. For simplicity, it is also simply called "AR nav". An example of a specific AR image will be described later.

[0019] The user can operate the terminal device 1 to select either of these two modes, and by causing the terminal device 1 to display the direction of travel and the route from the current location to the destination, the user can move to the destination while confirming the direction of travel and the route.

[0020] [Configuration of Terminal Device 1] The terminal device 1 is a portable communication information terminal device commonly called a smartphone or a tablet. The terminal device 1 is a computer equipped with a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory).

[0021] As shown in FIG. 1, the terminal device 1 includes a communication unit 11, a direction detection unit 12, an angle detection unit 13, a photographing unit 14, a current position acquisition unit 15, a route information acquisition unit 16, an AR image generation unit 17, a display unit 18, a control unit 19, and a terminal storage unit 20. These functional units function when the CPU executes various control programs stored in the terminal storage unit 20.

[0022] The communication unit 11 has a function of communicating with the server 3 via the network 2. The communication unit 11 has a function of performing wireless communication and includes an antenna (not shown) for transmitting and receiving radio waves for calls and communication. The communication unit 11 converts a signal for calls and communication into a wireless signal, transmits the converted wireless signal to the network 2 via the antenna, and converts the wireless signal received from the network 2 via the antenna and outputs it to the CPU.

[0023] The direction detection unit 12 has a function of detecting a photographing direction indicating the north-south-east-west direction in which the lens of the photographing unit 14 provided on the back of the terminal device 1 is directed. The direction detection unit 12 can be realized, for example, by using a geomagnetic sensor. Information regarding the detected photographing direction is input to the control unit 19.

[0024] The angle detection unit 13 has a function of detecting the elevation angle formed between the vertical direction (shooting direction) and the horizontal direction with respect to the display unit 17 provided on the surface of the terminal device 1. The angle detection unit 13 can be realized, for example, by using a gyro sensor and an acceleration sensor. Information regarding the detected elevation angle is input to the control unit 19.

[0025] The imaging unit 14 has a function of capturing the real world, which is an image of light incident through the lens provided on the back of the terminal device 1, and generating a real image. As a hardware configuration, the imaging unit 14 includes a lens and an imaging element such as a CCD or a CMOS sensor that captures an image of the light incident from the lens. The imaging unit 14 digitizes the imaging signal output from the imaging element, performs various corrections and video compression processing to generate a real image. The generated real image is stored in the terminal storage unit 20.

[0026] The current position acquisition unit 15 has a function of receiving radio waves transmitted from a plurality of navigation satellites in the sky toward the ground and calculating and acquiring the current position. The current position includes information such as latitude, longitude, altitude, and positioning accuracy. Information regarding the acquired current position is input to the control unit 19 and stored in the terminal storage unit 20 as a movement history.

[0027] The route information acquisition unit 16 has a function of acquiring route information for guiding a user to a destination from the server 3 via the network 2 and storing the acquired route information in the terminal storage unit 20.

[0028] Specifically, the terminal device 1 receives an input of route search conditions, which are information necessary for route search such as the departure place or the current position and the destination, based on the operation of the terminal device 1 by the user. By transmitting the route search conditions to the server 3, the route information acquisition unit 16 can acquire route information indicating the route to the destination from the server 3. The route information includes, for example, information on links and nodes constituting the route, map information for route guidance, voice information, and the like.

[0029] The AR image generation unit 17 has a function of generating an AR image indicating the traveling direction along the route based on at least the acquired route information, the current position, and the detected shooting direction and elevation angle. The generated AR image includes position information indicating the position in the display area for overlapping and displaying with the real image generated by the shooting unit 14. The generated AR image is stored in the terminal storage unit 20.

[0030] The display unit 18 is a touch panel display. The display unit includes a screen having a predetermined display area, outputs information to the display area, and has a function of receiving a predetermined instruction input for the information when the user performs a touch operation on the information output to the display area.

[0031] The display unit 18 has a function of displaying various images such as a GUI or content based on the control of the control unit 19. Specifically, when the user activates the navigation application of the terminal device 1 and further selects the map mode, the display unit 18 displays a map and a route. When the user selects the AR mode, the display unit 18 overlays and displays the AR image generated by the AR image generation unit 17 on the real image.

[0032] The control unit 19 has a CPU and has a function of controlling each function provided in the terminal device 1. For example, when the AR mode is selected by the user, if any of the shooting direction detected by the azimuth detection unit 12, the elevation angle detected by the angle detection unit 13, or the current position acquired by the current position acquisition unit 15 satisfies the shooting stop condition stored in the terminal storage unit 20, the control unit 19 has a function of stopping the generation of the real image by the shooting unit 14. In this embodiment, the shooting stop condition will be described on the condition that the elevation angle is the elevation angle.

[0033] The terminal storage unit 20 is composed of a non-volatile semiconductor memory such as a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), or a hard disk drive.

[0034] The terminal storage unit 20 stores programs and data used by the control unit 19 to perform various processes, and data generated or acquired by the control unit 19 by performing various processes. Specifically, the terminal storage unit 20 stores data of a real image generated by the imaging unit 14, route information acquired by the route information acquisition unit 16, data of an AR image generated by the AR image generation unit 17, data of a shooting stop condition, and the like.

[0035] [Configuration of Server 3] The server 3 is a so-called computer, and includes a CPU, a storage device, and a RAM as hardware. The server 3 realizes various functional units by the CPU executing a control program stored in the storage device. The server 3 may be a computer group configured by connecting a plurality of computers. The server 3 has a function of communicating with other communication devices via the network 2.

[0036] As shown in FIG. 1, the server 3 includes a server communication unit 31, a server control unit 32, and a server storage unit 34. The server control unit 32 has a route search unit 33. The server storage unit 34 stores route network information 35, map information 36, and POI information 37.

[0037] The server communication unit 31 has a function of communicating with other communication devices via the network 2. Specifically, the server communication unit 31 receives a route search condition from the terminal device 1 and transmits route information obtained by searching based on the route search condition to the terminal device 1.

[0038] The server control unit 32 has a CPU and has a function of controlling each function of the server 3. The route search unit 33 has a function of searching for a route from the current position or the departure position of the terminal device 1 to the destination based on the route search condition received from the terminal device 1 and the route network information 35, map information 36, and POI information 37 stored in the server storage unit 34. The route search unit 33 outputs the route information as a search result, and the server communication unit 31 transmits the route information to the terminal device 1 that transmitted the route search condition.

[0039] The server storage unit 34 is the same as the terminal device 1 in terms of hardware, but it is desirable that the storage capacity is larger than that of the terminal device 1 and is large-capacity. The server storage unit 34 stores programs and data used by the server control unit 32 to perform various processes, and data generated or acquired by the server control unit 32 by performing various processes.

[0040] The route network information 35 is information regarding nodes and links that make up the roads on which the vehicle travels. Specifically, the route network information 35 includes position information indicating the latitude and longitude of each node, etc., and links indicating the roads connecting the nodes. Information regarding the links includes the number of lanes, length, node IDs specifying the start and end nodes, etc.

[0041] Specifically, the map information 36 includes, for example, illustration images showing roads, buildings, facilities, etc., or planar image information composed of aerial photographs, satellite photographs.

[0042] In addition to the position information of various facilities, etc., the POI information 37 includes information indicating the characteristics of the facilities, etc. For example, in the case of a store, it includes what genre of store it is, the presence or absence of toilets and parking lots, review information, etc. Information indicating the characteristics includes the exterior image of the facility, name, etc.

[0043] [Operation of Terminal Device 3] Using FIGS. 2 to 6, the shooting control in the AR navigation of the terminal device 3 will be described. FIG. 2 shows an example of an operation flow including shooting control after the navigation app is started. First, the terminal device 3 receives the setting of the destination from the user (S1).

[0044] FIG. 3 shows an example of the destination setting screen displayed on the display unit 18 of the terminal device 1. A map is displayed on the display unit 18, and on the map, the names of various facilities based on the POI information are displayed together with icons.

[0045] When the user long - presses the display position of any one of the plurality of facility icons on the map displayed by the display unit 18 with a finger or the like, the display unit 18 detects the long - press and displays a destination icon 41 indicating that the setting with the facility as the destination has been made.

[0046] In addition, the display unit 18 displays a box 42 for displaying facility information such as the location of the set facility and the distance from the current position at the lower part of the screen. Note that the user can also select and set a departure position different from the current position from the map.

[0047] When the terminal device 1 receives a destination setting from the user, it transmits to the server 3 the identification information of the terminal device 1, the set destination information, and the route search conditions including the current position or the departure position. The server 3 searches for a route based on the received route search conditions, transmits the route information which is the result of the search to the terminal device 1, and the terminal device 1 acquires the route information (S2).

[0048] Subsequently, when the terminal device 1 receives a selection of the AR mode from the user (S3), it starts shooting by the shooting unit 14, and generates an AR image for indicating the traveling direction along the route based on the acquired route information, the current position of the terminal device 1, the shooting azimuth, and the pitch angle (S4).

[0049] Subsequently, the control unit 19 determines whether any of the shooting azimuth input from the azimuth detection unit 12, the pitch angle input from the angle detection unit 13, and the current position input from the current position acquisition unit 15 satisfies the shooting stop condition stored in the terminal storage unit 20 (S5).

[0050] As described above, in this embodiment, the shooting stop condition is that the pitch angle is the elevation angle. As shown in FIG. 4, when the shooting direction of the shooting unit 14 of the terminal device 1 is upward with respect to the horizontal direction = 0°, the angle detection unit 13 detects that the shooting direction is the elevation angle and outputs it to the control unit 19.

[0051] When the control unit 19 determines from the input information that the shooting stop condition is satisfied (S5: Yes), the control unit 19 stops the shooting by the shooting unit 14 (S6). When the control unit 19 determines that the shooting stop condition is not satisfied (S5: No), for example, when the shooting direction is a depression angle that is downward with respect to the horizontal direction = 0°, the shooting unit 19 continues shooting and generates a real image (S7).

[0052] When a real image is generated, the display unit 18 overlays and displays the AR image on the real image based on the position information indicating the position in the display area for overlaying and displaying with the real image included in the AR image (S8). Note that even when a real image is not generated, if the position information of the AR image is included in the display area, only the AR image may be displayed.

[0053] FIG. 5 is a diagram showing an example in which a destination icon 41, which is an AR image, and an arrow 43 indicating the traveling direction are overlaid and displayed on a real image. FIG. 6 is a diagram showing an example in which, since the shooting stop condition is satisfied, shooting is stopped and only the destination icon 41 and the arrow 43 are displayed without a real image.

[0054] Note that a window 44 for notifying that shooting has stopped may be displayed on the display unit 18. The window 44 is marked with "This is the shooting prohibited angle. Shooting will be stopped." By displaying the window 44 on the display unit 18, the user can notice that the terminal device 1 is directed in a shooting direction that is not appropriate as the AR mode.

[0055] When the current position of the terminal device 1 matches the position of the destination and the user arrives at the destination (S9: Yes), the control unit 19 stops the shooting by the shooting unit 14 (S11). Even when the user has not arrived at the destination (S9: No), if the user selects the map mode (S10: Yes), the control unit 19 also stops the shooting by the shooting unit 14 (S11).

[0056] If the user does not select the map mode and the display mode remains in the AR mode (S10: No), the control unit 19 returns to the process of S5. After the process of S11, the control unit 19 ends the shooting control process.

[0057] As described above, the embodiments of the present invention have been described. However, the present invention is not limited to the configuration of the terminal device 1 and the shooting control process of the navigation application described above. For example, in the above embodiment, the AR navigation superimposed the AR image on the real image generated by shooting and displayed it on the display unit 18. However, the present invention can also be applied to devices such as AR glasses, HUD (Head-Up Display), and HMD (Head Mounted Display) that allow the user to visually recognize the real world through the display unit and display the AR image according to the position of the real world.

[0058] In the above embodiment, the shooting stop condition was that the pitch angle was the elevation angle. However, for example, it is also possible to set a range such as an elevation angle of 5° or more, or an elevation angle of 10° or more and 90° or less as the shooting stop condition. Further, in addition to the pitch angle, a predetermined area and a predetermined shooting direction may be set independently as the shooting stop conditions, or a combination of these conditions may be set as the shooting stop conditions.

[0059] The navigation system of the above embodiment was composed of the terminal device 1 and the server 3. However, by providing the terminal device 1 with a route search function, it is also possible to realize shooting control in the AR mode with the terminal device 1 alone.

[0060] The present invention can be embodied by modifying the components without departing from the gist of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of components disclosed in the above embodiments.

Industrial Applicability

[0061] The present invention can be applied to a terminal device including a shooting unit and a display and executing AR navigation.

Explanation of Signs

[0062] 1 Terminal device 2 Network 3 Server 11 Communication unit 12 Azimuth detection unit 13 Angle detection unit 14 Imaging unit 15 Current position acquisition unit 16 Route information acquisition unit 17 AR image generation unit 18 Display unit 19 Control unit 20 Terminal memory unit 31 Server communication unit 32 Server control unit 33 Route search unit 34 Server memory unit

Claims

1. A route information acquisition unit that acquires route information indicating a route to a destination; A current position acquisition unit that acquires the current position of the user; A photographing unit that generates a real image obtained by photographing the real world at the current position; An azimuth detection unit that detects the photographing azimuth of the photographing unit; An angle detection unit that detects the elevation angle of the photographing direction of the photographing unit; An AR image generation unit that generates an AR image indicating a traveling direction along the route based on the route information, the current position, the photographing azimuth, and the elevation angle; A storage unit that stores a photographing stop condition; A control unit that stops the generation of the real image by the photographing unit when any one of the current position, the photographing azimuth, or the elevation angle satisfies the photographing stop condition; A display unit that overlays and displays the AR image on the real image or the real world; A terminal device comprising the above.

2. The AR image generation unit generates the AR image even when at least one of the current position, the photographing azimuth, and the elevation angle satisfies the photographing stop condition, The AR image includes position information indicating a position in a display area for being overlaid and displayed on the real image or the real world, The display unit displays the AR image based on the position information, The terminal device according to Claim 1.

3. When at least one of the current position, the photographing azimuth, and the elevation angle satisfies the photographing stop condition, the control unit further causes the display unit to display a message indicating that the generation of the real image by the photographing unit is stopped, The terminal device according to Claim 1.

4. A program that causes a computer that executes AR navigation and includes a route information acquisition unit that acquires route information indicating a route to a destination, a current position acquisition unit that acquires a current position, a photographing unit that generates a real image obtained by photographing the real world at the current position, an azimuth detection unit that detects the photographing azimuth of the photographing unit, an angle detection unit that detects the elevation angle of the photographing direction of the photographing unit, an AR image generation unit that generates an AR image indicating a traveling direction along the route based on the route information, the current position, the photographing azimuth, and the elevation angle, a storage unit that stores a photographing stop condition, and a display unit that overlays and displays the AR image on the real image or the real world, to perform a photographing control process, The photographing control process includes: A step of determining whether any one of the current position, the photographing azimuth, or the elevation angle satisfies the photographing stop condition during the execution of the AR navigation; When it is determined that the shooting stop condition is satisfied, a step of stopping the generation of a real image by the shooting unit, and a program including the same.

Images