Mobile device and vehicle

Abstract

The present invention relates to a mobile device and a vehicle, the mobile device including: an input device; a position receiver for obtaining a current position; an image obtaining device; a display device provided to display a surrounding image obtained from the image obtaining device in response to execution of an augmented reality mode; and a controller provided to search for route information from the current position to a destination, destination information, and map information obtained by the position receiver, identify a movement situation according to the searched route information corresponding to the route, obtain position information of a turning point at which the movement situation changes according to information on the identified movement situation, and control the display device to display an object image of the turning point on the surrounding image displayed on the display device based on the position information of the turning point and current position information.

Description

Technical Field

[0001] This invention relates to mobile devices and vehicles with navigation functions that guide routes to destinations. Background Technology

[0002] With the recent development of digital technology, various types of mobile devices (such as mobile communication terminals, smartphones, tablet PCs, laptops, personal digital assistants (PDAs), wearable devices, digital cameras, and personal computers) are widely used.

[0003] Recently, research and development of navigation functions, in addition to calling functions, multimedia playback functions (e.g., music playback and video playback) and Internet functions, have been increasing among the functions that can be performed on mobile devices.

[0004] The navigation function here is also a function performed in the vehicle, and when the navigation function is performed, the mobile device or vehicle typically uses the Global Positioning System (GPS) to identify the current location.

[0005] This Global Positioning System (GPS) may not accurately identify the current location due to communication failures with satellites indoors, underground, or in areas with many tall buildings, and may have a very large margin of error between the actual current location and the identified current location.

[0006] Furthermore, when a mobile device or vehicle is performing navigation, it may not be able to accurately guide the user to the destination until the user reaches the destination because the navigation function terminates at a location near the destination.

[0007] The information included in the background section of this invention is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art. Summary of the Invention

[0008] Various aspects of the present invention relate to providing mobile devices and vehicles configured to guide a route to a destination while controlling the display of an object image in an augmented reality function during the execution of a navigation mode and an augmented reality mode, and to display congested and accident-prone road sections and suggest alternative routes during the route guidance to the destination.

[0009] Various aspects of the present invention relate to providing mobile devices and vehicles configured to use object images to highlight points of interest (POIs) such as destinations, or to display points of interest (POIs) with transparency adjustment.

[0010] Additional aspects of the invention will be set forth in part in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention.

[0011] According to one aspect of the invention, a mobile device includes: an input device configured to receive user input; a location receiver configured to receive location information for obtaining a current location; an image acquisition device configured to acquire an image of the surrounding environment; a display device configured to display the surrounding image acquired from the image acquisition device in response to the execution of an augmented reality mode; and a controller configured to, when receiving destination information from user input during the execution of the augmented reality mode, search for a route from the current location to a destination, identify a movement based on route information corresponding to the searched route, obtain location information of an inflection point where the movement changes based on information about the identified movement, and control the display device to display an image of the object at the inflection point on the surrounding image displayed on the display device based on the location information of the inflection point and the current location information.

[0012] The controller of the mobile device can obtain distance information from the current position to the inflection point based on the current position information and the inflection point position information, and change the display information of the object image at the inflection point based on the obtained distance information. The display information of the object image at the inflection point may include at least one of shape information and color information.

[0013] When the controller of the mobile device determines that the distance to the inflection point is greater than or equal to the first reference distance, the controller can identify the direction of the destination based on the current location information, destination information and map information, and control the display device to display an image of an object oriented from the current location to the destination direction on the surrounding image displayed on the display device.

[0014] When the controller of the mobile device determines that the distance to the inflection point is less than a first reference distance and greater than or equal to a second reference distance, the controller can control the display device to display an image of the long-distance inflection point on the surrounding image based on the location of the inflection point and map information. The first reference distance may be longer than the second reference distance.

[0015] When the controller of the mobile device determines that there are multiple inflection points within a preset distance based on the route information, the controller can change the second reference distance to a third reference distance.

[0016] The controller of the mobile device can, based on information about the identified movement situation, control the display device to display the object image of the long-distance inflection point in a first color when the movement situation is smooth; control the display device to display the object image of the long-distance inflection point in a second color when the controller determines that the movement situation is congested or an accident; and control the display device to display the object image of the long-distance inflection point in a third color when the controller determines that the movement situation is a route change.

[0017] When the mobile device's controller receives traffic information, it can determine whether a route change is necessary based on the received traffic information, destination information, current location information, route information, and map information.

[0018] When the controller of the mobile device determines that the distance to the inflection point is less than the second reference distance, the controller can control the display device to display an image of the object at the short-distance inflection point on the surrounding image based on the location of the inflection point and map information.

[0019] The controller of the mobile device can, based on information about the identified movement situation, control the display device to display an image of the object at a short-distance inflection point in a first color when the controller determines that the movement situation is a route holding situation; control the display device to display an image of the object at a short-distance inflection point in a second color when the controller determines that the movement situation is a congestion situation or an accident situation; and control the display device to display an image of the object at a short-distance inflection point in a third color when the controller determines that the movement situation is a route change situation.

[0020] When the controller of the mobile device determines that the display position of the directional object image is consistent with the display position of the object image at a short-distance inflection point but inconsistent with the display position of the object image at a long-distance inflection point, the controller can control the display device to display traffic information.

[0021] When the controller of the mobile device determines that the display position of the directional object image is inconsistent with the display position of the object image at the short-distance inflection point and inconsistent with the display position of the object image at the long-distance inflection point, the controller can control the display device to display traffic information and alternative route information.

[0022] The controller of the mobile device can determine whether the distance to the destination is less than or equal to a predetermined distance based on destination information, current location information, route information, and map information. When the controller determines that the distance to the destination is less than or equal to the predetermined distance, it identifies the destination image corresponding to the destination information from the surrounding images and controls the display device to overlay and display the emphasized object image on the identified destination image.

[0023] The controller of the mobile device can determine whether the distance to the destination is less than or equal to a predetermined distance based on destination information, current location information, route information, and map information. When the controller determines that the distance to the destination is less than or equal to the predetermined distance, it identifies the destination image corresponding to the destination information from the surrounding images and controls the transparency of the identified destination image and the remaining images to be different from each other.

[0024] The controller of the mobile device can identify ground images adjacent to the destination image from surrounding images, control the display device to display grid-like object images on the identified ground images, and control the display device to overlay and display anchor mark images on the destination image.

[0025] According to another aspect of the invention, a vehicle includes: an input device configured to receive user input; a position receiver configured to receive position information for obtaining a current position; a speed detector configured to detect a driving speed; an image acquisition device configured to acquire an image of the surrounding environment; a display device configured to display the surrounding image acquired from the image acquisition device in response to the execution of an augmented reality mode; and a controller configured to, when receiving destination information from user input during the interaction of augmented reality mode and navigation mode, search for a route from the current position to the destination, identify a movement based on route information corresponding to the searched route and driving speed information detected by the speed detector, obtain position information of an inflection point where the movement changes based on information about the identified movement, and control the display device to display an image of the object at the inflection point on the surrounding image displayed on the display device based on the position information of the inflection point and the current position information.

[0026] The vehicle controller can obtain distance information from the current position to the inflection point based on the current position information and the inflection point position information, and change the display information of the object image at the inflection point according to the obtained distance information. The display information of the object image at the inflection point may include at least one of shape information and color information.

[0027] When the vehicle's controller determines that the distance to the inflection point is greater than or equal to a first reference distance, the controller can identify the direction of the destination based on the current location information, destination information, and map information, and control the display device to display an image of an object oriented from the current location to the destination direction on the surrounding image displayed on the display device. When the controller determines that the distance to the inflection point is less than the first reference distance but greater than or equal to a second reference distance, it controls the display device to display an image of a long-distance inflection point on the surrounding image based on the location of the inflection point and map information. And when the controller determines that the distance to the inflection point is less than the second reference distance, it controls the display device to display an image of a short-distance inflection point on the surrounding image based on the location of the inflection point and map information. The first reference distance can be longer than the second reference distance.

[0028] When the vehicle controller determines that the display position of the orientation object image is consistent with the display position of the object image at a short-distance inflection point but inconsistent with the display position of the object image at a long-distance inflection point, the controller can control the display device to display traffic information; and when the controller determines that the display position of the orientation object image is inconsistent with the display position of the object image at a short-distance inflection point and inconsistent with the display position of the object image at a long-distance inflection point, the controller controls the display device to display traffic information and alternative route information.

[0029] When the vehicle's controller receives traffic information, it can determine whether a route change is necessary based on the received traffic information, destination information, current location information, route information, and map information. When the controller determines that a route change is necessary, it identifies the point where the route is changed as an inflection point.

[0030] The vehicle controller can determine whether the distance to the destination is less than or equal to a predetermined distance based on destination information, current location information, route information, driving speed information, and map information. When the controller determines that the distance to the destination is less than or equal to the predetermined distance, it identifies the destination image corresponding to the destination information from the surrounding images and controls the display device to overlay and display the emphasized object image on the identified destination image.

[0031] When searching for a route from the current location to the destination, the vehicle's controller can search for a route from the current location to the destination based on the current location information, destination information, and map information obtained from the location receiver; and when controlling the display device to display an object image of an inflection point on the surrounding image displayed on the display device, the controller can control the display device to display the object image of the inflection point on the surrounding image displayed on the display device based on the location information of the inflection point, map information, and current location information.

[0032] The methods and apparatus of the present invention have other features and advantages, which will be apparent or set forth in more detail from the accompanying drawings and the following detailed description, which together serve to explain certain principles of the invention. Attached Figure Description

[0033] Figure 1 This is a control schematic diagram of a mobile device according to various exemplary embodiments of the present invention;

[0034] Figure 2 and Figure 3 These are exemplary display diagrams of display devices for mobile devices according to various exemplary embodiments of the present invention;

[0035] Figure 4A , Figure 4B , Figure 5 , Figure 6 , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 and Figure 11 This is an exemplary display diagram of object images of the augmented reality function of a mobile device according to various exemplary embodiments of the present invention;

[0036] Figure 12A and Figure 12B This is a control flowchart of a mobile device according to various exemplary embodiments of the present invention; and

[0037] Figure 13 This is a schematic diagram of vehicle control according to another exemplary embodiment of the present invention.

[0038] It is understood that the accompanying drawings are not necessarily drawn to scale and present slightly simplified representations of various features illustrating the basic principles of the invention. Specific design features of the invention as included herein (e.g., specific dimensions, orientations, positions, and shapes) will be determined in part by the specific intended application and environment of use.

[0039] In the accompanying drawings, reference numerals refer to the same or equivalent portions of the invention throughout several figures. Detailed Implementation

[0040] Reference will now be made in detail to various embodiments of the invention, examples of which are illustrated in the accompanying drawings and described below. While the invention will be described in conjunction with exemplary embodiments thereof, it will be understood that this description is not intended to limit the invention to those exemplary embodiments. On the other hand, the invention is intended to cover not only the exemplary embodiments thereof, but also various alternatives, modifications, equivalents and other embodiments that may be included within the spirit and scope of the invention as defined in the appended claims.

[0041] Throughout this specification, the same reference numerals refer to the same elements. This specification does not describe all elements of the embodiments, and general descriptions of the technical aspects of the invention or repetitions between embodiments are omitted. The terms "component," "module," "building block," and "block" used in exemplary embodiments may embody software or hardware, and according to exemplary embodiments, multiple "units," "modules," "building blocks," and "blocks" may also embody a single component, or a single "unit," "module," "building block," and "block" may comprise multiple components.

[0042] Throughout the manual, when a component is referred to as being “connected” to another component, it includes not only direct connections but also indirect connections, and indirect connections include connections via wireless networks.

[0043] Furthermore, when a component is described as "including" an element, unless otherwise specified, it means that the element may further include other elements, and does not exclude other elements.

[0044] The terms “first”, “second”, etc., are used to distinguish one element from another, and the element is not limited by the terms mentioned above.

[0045] Unless the context clearly indicates otherwise, the singular forms “a”, “an”, and “the” include plural references.

[0046] In each step, an identifier is used for ease of explanation. The identifier does not describe the order of the steps, and each step may be performed in a different order than specified unless the context clearly indicates a particular order.

[0047] The invention will now be described in detail with reference to the accompanying drawings.

[0048] Figure 1 This is a control block diagram of a mobile device according to various exemplary embodiments of the present invention, which will be referenced. Figure 2 , Figure 3 , Figure 4A , Figure 4B , Figure 5 , Figure 6 , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 and Figure 11 To describe the mobile device.

[0049] Mobile device 1 can be a terminal that the user can carry.

[0050] Mobile device 1 can be a terminal that can be detachably installed in a vehicle, or a terminal that can be embedded in the vehicle's dashboard. Mobile device 1 can be an audio-visual navigation (AVN) terminal that performs audio, video, and navigation functions in the vehicle.

[0051] Mobile device 1 can be implemented as a computer or portable terminal configured to access the vehicle via a network.

[0052] Here, computers may include, for example, laptops, desktops, folding computers, tablet PCs, slate PCs, etc., on which a web browser is installed, as well as portable terminals, which are wireless communication devices that ensure portability and mobility. These may include, for example, all kinds of handheld wireless communication devices, such as personal communication systems (PCS), Global System for Mobile Communications (GSM), personal digital cell (PDC), personal handheld telephone systems (PHS), personal digital assistants (PDAs), International Mobile Telecommunications (IMT)-2000, Code Division Multiple Access (CDMA)-2000, W-Code Division Multiple Access (W-CDMA), WiBro wireless broadband internet terminals and smartphones, as well as wearable devices, such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, and head-mounted devices (HMDs).

[0053] The mobile device 1 includes a user interface 110, a sound output device 120, a location receiver 130, an image acquisition device 140, a communication device 150, a controller 160, and a storage device 161.

[0054] User interface 110 receives user input and outputs various information so that the user can recognize it. User interface 110 may include input device 111 and display device 112.

[0055] Input device 111 receives user input.

[0056] Input device 111 can receive lock commands, lock release commands, power-on commands and power-off commands from mobile device 1, and can also receive image display commands from display device 112.

[0057] The input device 111 can receive operation commands for various functions that can be performed by the mobile device 1, and can also receive setting values ​​for various functions.

[0058] When the mobile device 1 is installed in the vehicle, the input device 111 can receive operation commands for various functions to be performed in the vehicle.

[0059] For example, the functions to be performed in mobile device 1 may include calling function, text function, audio function, video function, navigation function, map function, broadcast playback function, radio function, content playback function, Internet search function, and augmented reality function, and may include the function of performing at least one application installed in mobile device 1.

[0060] Augmented reality (AR) is a technology that displays information by combining virtual-related information (e.g., text, images, etc.) with real-world objects (e.g., the real environment). Unlike virtual reality (VR), which only targets virtual spaces and objects, AR provides users with additional information that is difficult to obtain in the real environment by providing virtual-related objects as objects in the real environment.

[0061] At least one application installed on mobile device 1 may be an application for providing at least one service to a user. Here, the service may be providing information for the user's safety, convenience, and enjoyment.

[0062] Input device 111 can receive execution commands from a navigation application (NAVI API) for performing navigation functions, can receive execution commands from an augmented reality application (AR API) for performing augmented reality functions, and can receive execution commands from a map application (MAP API) for performing map functions.

[0063] Input device 111 can receive destination information in response to the execution of navigation function, and can also receive route selection information for selecting any one of multiple routes.

[0064] Input device 111 can receive destination information during the execution of augmented reality or map functions.

[0065] Input device 111 can receive Point of Interest (POI) information about points of interest (POIs) during the execution of augmented reality functions.

[0066] Input device 111 may be implemented as a micro-motion dial or touchpad for inputting cursor movement commands and icon or button selection commands displayed on display device 112.

[0067] Input device 111 may include hardware devices such as various buttons or switches, pedals, keyboards, mice, trackballs, various joysticks, handles, and joysticks.

[0068] In addition, input device 111 may include a graphical user interface (GUI), such as a touch panel, i.e., a software device. The touch panel may be implemented as a touch screen panel (TSP) to form a layer structure with display device 112.

[0069] Display device 112 can display execution information about at least one function performed in mobile device 1 as an image in response to control commands from controller 160, and can also display information corresponding to user input received by input device 112 as an image.

[0070] Display device 112 can display icons for applications that can perform functions on mobile device 1. For example, such as Figure 2 As shown, the display device 112 can display the icon 110a of the augmented reality application, the icon 110b of the navigation application, and the icon 110c of the map display application.

[0071] Display device 112 can display map information, destination information, and route guidance information during navigation, and can also display current location information about the current location. That is, when the navigation function is executed, display device 112 can display a navigation image in which the route guidance image to the destination in the map image is matched with the current location image indicating the current location.

[0072] like Figure 3 As shown, display device 112 can display a search window 110d for searching POIs in response to user input during the execution of augmented reality functions, and can display icons of pre-stored points of interest (POIs). Display device 112 can also display information about previously searched or stored POIs in the form of buttons.

[0073] Display device 112 can display images acquired by image acquisition device 140 while performing augmented reality functions, and simultaneously display images of objects at inflection points where changes in motion may occur.

[0074] The object image of an inflection point can include short-distance inflection point images and long-distance inflection point images. Furthermore, the object image of an inflection point can include dashed line images, dot images, and arrow images.

[0075] For example, when the distance to the inflection point is long, the display device 112 can display an object image pointing towards the destination based on the current position, allowing the user to easily identify the direction of the destination. Here, the object image pointing towards the destination can be a dotted straight line (hereinafter referred to as a dashed line) image that serves as the orientation object image.

[0076] The display device 112 can display an object image by overlaying it onto a destination image corresponding to the destination. The object image overlaid on the destination image, which serves as an emphasis image for highlighting the destination image, can be the object image being emphasized.

[0077] For example, such as Figure 4A As shown, the display device 112 can enable the user to easily identify the destination by displaying an image AR1 of an object that indicates the destination on a destination image (e.g., a building image) corresponding to the destination.

[0078] Display device 112 can display an emphasized object image AR1 so as to show the user that it is moving by repeatedly displaying the emphasized object image (AR1) indicating the destination at regular intervals or by repeatedly displaying and not displaying the emphasized object image AR1 indicating the destination at regular time intervals.

[0079] Display device 112 can display an image M1 other than the destination image, which is blurrier than the destination image.

[0080] like Figure 4B As shown, the display device 112 can accurately and precisely display the destination by displaying a first emphasized object image AR1 on the destination image corresponding to the destination, a second emphasized object image AR2 guiding the destination on the ground image adjacent to the destination image, and a third emphasized object image AR3 emphasizing that it is the destination, during the execution of the augmented reality function.

[0081] The first emphasized object image AR1, indicating the destination, can be a polygonal image formed by dashed lines.

[0082] The second emphasized object image AR2 used to guide the destination can be a grid image obtained through plane detection. Here, plane detection is the detection of planes when it is necessary to display the augmented reality (AR) object image on a vertical plane.

[0083] As an anchor mark image, AR3 emphasizes the third object image of the destination, allowing users to pinpoint the precise location of the anchor value.

[0084] The display device 112 can emphasize the destination image corresponding to the destination, and can use dimming processing to display the remaining image M1 other than the destination image as more blurred than the destination image.

[0085] like Figure 5 As shown, the display device 112 can simultaneously display navigation images and augmented reality images, and display navigation images and augmented reality images in different display areas of the display device 112.

[0086] That is, the display device 112 can display an augmented reality image in the first display area a1 and a navigation image in the second display area a2.

[0087] Display device 112 may include, but is not limited to, cathode ray tube (CRT), digital light processing (DLP) panel, plasma display panel, liquid crystal display (LCD) panel, electroluminescent (EL) panel, electrophoretic display (EPD) panel, electrochromic display (ECD) panel, light-emitting diode (LED) panel, or organic light-emitting diode (OLED) panel.

[0088] The mobile device 1 may further include a sound receiver for receiving the user's voice. In this case, the controller 160 can perform a voice recognition function and recognize user input through the voice recognition function.

[0089] The sound receiver may include a microphone that converts sound waves into electrical signals. One, two, or more microphones may be provided, and one of the microphones may be directional.

[0090] The two or more microphones can be implemented as a microphone array.

[0091] The sound output device 120 can output sound for functions performed by the mobile device 1. The sound output device 120 may include at least one or more speakers.

[0092] For example, the sound output device 120 can output route guidance information as sound when performing navigation functions.

[0093] The speaker converts the amplified low-frequency audio signal into the original sound wave, generates wavelets in the air, replicates the sound wave, and outputs the audio data as the sound that the user can hear.

[0094] The location receiver 130 receives a signal for obtaining current location information about the current location of the mobile device 1.

[0095] The location receiver 130 may be a Global Positioning System (GPS) receiver that communicates with multiple GPS satellites. The GPS receiver includes: an antenna module for receiving signals from the multiple GPS satellites, and may include: software for obtaining the current location using distance and time information corresponding to the location signals from the multiple GPS satellites; and an output device configured to output the obtained vehicle location information.

[0096] The location receiver 130 may include a visual positioning service (VPS).

[0097] Image acquisition device 140 acquires images of the surroundings of mobile device 1 and sends image information about the acquired images to controller 160. The image information may be image data.

[0098] The image acquisition device 140 has a front view of the mobile device 1.

[0099] Image acquisition device 140 may include at least one or more cameras for acquiring external images of the mobile device 1 in the forward and backward directions.

[0100] Assuming the display surface of the display device of the mobile device is the front surface of the main body of the mobile device, at least one camera can be arranged on the front surface of the main body of the mobile device, and other cameras can be arranged on the rear surface of the main body of the mobile device. The rear surface can be a surface oriented in a direction opposite to that of the front surface relative to the main body.

[0101] Image acquisition device 140 may include a CCD or CMOS image sensor as a camera, and may also include a three-dimensional spatial recognition sensor such as an RGB-D sensor (KINECT), a structured light sensor (TOF), and a stereo camera.

[0102] The communication device 150 can receive at least one application from the external server 2, and can receive update information about the installed application.

[0103] The communication device 150 can receive the number of times a user accesses at least one of the multipurpose facilities from the external server 2.

[0104] The communication device 150 may include one or more components that enable communication between internal components, and may include, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module.

[0105] Short-range communication modules can include various short-range communication modules that use wireless communication networks to send and receive signals over short distances, such as Bluetooth modules, infrared communication modules, radio frequency identification (RFID) communication modules, near field communication (NFC) modules, and Zigbee communication modules.

[0106] Wired communication modules can include a variety of wired communication modules, such as Controller Area Network (CAN) communication modules, Local Area Network (LAN) modules, Wide Area Network (WAN) modules and Value-Added Network (VAN) modules, as well as various wired communication modules such as Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Digital Visual Interface (DVI), Recommended Standard 232 (RS-232), Power Line Communication and Common Old Telephone Service (POTS).

[0107] The wired communication module may also include a local interconnect network (LIN).

[0108] In addition to Wi-Fi modules and wireless broadband modules, wireless communication modules may also include wireless communication modules that support various wireless communication methods, such as Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Universal Mobile Telecommunications System (UMTS), Time Division Multiple Access (TDMA), Long Term Evolution (LTE), and Ultra Wideband (UWB) modules.

[0109] The controller 160 controls the image display of the display device 112 based on at least one of a lock release command, a power-on command, and an image display command received from at least one of the input device 111, the image acquisition device 140, and the sound receiver.

[0110] The controller 160 can control the display of the execution image of the augmented reality function when it receives the execution command of the augmented reality application from the input device 111, can control the execution of the navigation application to activate the navigation application, and can also control the execution of the map application to activate the map application.

[0111] When the controller 160 receives a selection signal from the execution button of the augmented reality application, it can determine that it has received the execution command from the augmented reality application. The execution button of the augmented reality application can be a physical button.

[0112] When the touch position information received by the input device 111 corresponds to the display position information of the icon of the augmented reality application, the controller 160 can determine that the execution command of the augmented reality application has been received.

[0113] The controller 160 can control the activation of the image acquisition device 140 when it receives an execution command from the input device 111 for the augmented reality application, and can also control the activation of the position receiver 130.

[0114] During the execution of an augmented reality application, when the image acquisition device 140 is activated, the controller 160 can perform image processing on the image acquired by the image acquisition device 140 and control the display of the image-processed image. When the location receiver 130 is activated, the controller 160 can obtain the current location information of the mobile device 1 based on the location information output from the location receiver 130.

[0115] When the input device 111 receives the execution command of the navigation application, the controller 160 can obtain the map information stored in the storage device and control the activation of the location receiver 130.

[0116] When multiple navigation applications are set up, the controller 160 can perform interactions with the preset navigation application or the navigation application selected by the user during the execution of augmented reality functions.

[0117] When multiple navigation applications are set, the controller 160 can receive a selection signal for a navigation application used for driving a vehicle or a selection signal for a navigation application used for walking by a user.

[0118] The controller 160 can combine navigation functions with the augmented reality function to send POI information, destination information, current location information and information about multiple routes stored in the navigation application to the augmented reality application.

[0119] The controller 160 can recognize destination information input by the input device 111 during navigation, recognize current location information received in the location receiver 130, search for a route from the current location to the destination based on the recognized current location information and destination information, obtain route guidance information about the searched route, control the display device 112 to display a navigation image matching the map information, current location information, destination information, and route information, and control at least one of the display device 112 and the sound output device 120 to output route guidance information based on the current location information. The current location information can change in real time in response to the movement of the mobile device 1.

[0120] The controller 160 can also set the information of the point of interest received through the input device 111 as destination information.

[0121] When multiple routes are found, the controller 160 can control the display device 112 to display route information for the multiple routes.

[0122] The controller 160 can control the display device 112 to display detailed information about each of the multiple routes. This detailed information may include arrival time, travel distance, traffic information, etc.

[0123] When any of the multiple routes is selected by the input device 111, the controller 160 can control the display of route information about the selected route.

[0124] When a route guidance command is received during the interaction of navigation and augmented reality functions, the controller 160 can control the operation of at least one of the display device 112 and the sound output device 120 to output route information and route guidance information while displaying navigation images.

[0125] The following describes the display control configuration of the controller 160 during the interaction of augmented reality and navigation functions.

[0126] The controller 160 can control the display device 112 to simultaneously display navigation images and augmented reality images during the interaction of augmented reality and navigation functions. It can also divide the area of ​​the display device 112 into at least two areas and control the display device 112 to display navigation images and augmented reality images in the divided areas respectively.

[0127] The controller 160 identifies the movement of the mobile device 1 based on at least one of route information from the current location to the destination, movement speed information, and traffic information, and determines whether the identified movement has changed. In this case, the controller 160 can determine whether the change in movement is due to a change in traffic conditions, and if it is determined to be a change in traffic conditions, it can further determine whether the route information to the destination has changed.

[0128] The movement speed information can be driving speed information. Changes in the movement of mobile device 1 can include changes in traffic conditions and changes in route.

[0129] The controller 160 can obtain the location information of the inflection point where the movement of the mobile device 1 is to change based on at least one of route information, movement speed information, and traffic information; obtain the distance information from the current location to the inflection point based on the obtained location information of the inflection point, map information, route information, and current location information; obtain information about the object image of the inflection point to be displayed on the surrounding image based on the obtained distance information; and control the display device 112 to display the object image of the inflection point on the surrounding image based on the obtained information about the object image of the inflection point.

[0130] The object image to be displayed as an inflection point in augmented reality is an image that allows users to easily identify the location of the change in motion and the reason for the change in motion. Depending on the distance to the location where the change in motion occurs and the reason for the change in motion, the object image can have different shapes and colors.

[0131] The reasons for the change in the movement of mobile device 1 may include at least one of traffic congestion and traffic accidents, and may include a change in route corresponding to a change in the arrival time to the destination.

[0132] The controller 160 can change the display information of the object image at the inflection point based on the current position information and the distance information to the inflection point. In this case, the controller 160 can change at least one of the shape and color of the object image at the inflection point.

[0133] like Figure 6 As shown, when the distance to the inflection point is determined to be greater than or equal to the first reference distance based on the current location information and the distance information to the inflection point, the controller 160 can determine the direction of the destination based on the current location information and the location information of the destination, and based on the display position of the current location in the augmented reality image. It can also control the display device 112 to display an object image of the inflection point pointing from the display position of the current location to the destination. The object image of the inflection point pointing to the destination (which is a directional object image) can be a dashed line image AR4.

[0134] When a dashed line image pointing in the direction of the destination is displayed, the controller 160 can control the dashed line image not to be displayed based on a display off command input by the input device 111, and control the display of the dashed line image based on a display on command input by the input device 111.

[0135] like Figure 7A As shown, when the distance to the inflection point is determined to be greater than or equal to a second reference distance and less than a first reference distance based on the distance information from the current location to the inflection point, the controller 160 can identify the display location information of the inflection point. Based on the current location information, map information, and the location information of the inflection point, the controller displays the inflection point in the augmented reality image. The controller also controls the display device 112 to display the object image AR5 of the inflection point based on the identified display location information, and controls the display device 112 to display the distance information from the current location to the inflection point as a text image AR6 surrounding the object image of the inflection point. The object image AR5 of the inflection point, as a point image, can be a long-distance inflection point image indicating a long-distance inflection point.

[0136] like Figure 7B As shown, when the object image of the control inflection point is displayed as a dot image, when the reason for the change in the movement of the mobile device 1 is smooth driving, the controller 160 can control the display device 112 to display the object image of the inflection point as a dot image of the first color. When the reason for the change in the movement is traffic congestion or a traffic accident, the controller 112 can control the display device 112 to display the object image of the inflection point as a dot image of the second color. And when the reason for the change in the movement is a change to an alternative route, the controller 112 can control the display device 112 to display the object image of the inflection point as a dot image of the third color.

[0137] The first, second, and third colors can be different colors. For example, the first color can be green, the second color can be gray, and the third color can be red.

[0138] The controller 160 can determine whether the movement of the mobile device 1 has changed based on route information and traffic information from the current location to the destination; when it is determined that the movement of the mobile device 1 has changed, it can determine whether the arrival time to the destination has changed; and when it is determined that the arrival time has changed, it can control the display device 112 to search for secondary routes that may arrive earlier than the arrival time when moving to the main route, and display the searched secondary routes as alternative routes.

[0139] When guiding a route based on the route information of the changed alternative route, the controller 160 can identify the location information of the inflection point of the route change based on the primary route information and the secondary route information, identify the display location information of the inflection point to display the inflection point in the augmented reality image according to the current location information and the location information of the inflection point, and control the display device 112 to display the inflection point in the identified display location information as a dot image of a third color.

[0140] like Figure 8 As shown, when the distance to the inflection point is determined to be less than the second reference distance based on the distance information from the current location to the inflection point, the controller 160 can identify the display location information of the inflection point, and display the inflection point in the augmented reality image based on the map information, the current location information, and the location information of the inflection point. The controller also controls the display device 112 to display the object image of the inflection point in the identified display location information as an arrow image. The arrow image can be a short-distance inflection point image indicating a short-distance inflection point.

[0141] When the object image of the control inflection point is displayed as an arrow image, while maintaining the movement of the moving device 1, the controller 160 can control the display device 112 to display the object image of the inflection point as an arrow image of a first color. When the reason for the change in movement is traffic congestion or a traffic accident, the controller 160 can control the display device 112 to display the object image of the inflection point as a dot image of a second color. And when the reason for the change in movement is a change in the route, the controller 112 can control the display device 112 to display the object image of the inflection point as an arrow image of a third color.

[0142] Maintaining mobility can include keeping the route to the destination as the primary route.

[0143] The first, second, and third colors of the arrow image can be different colors. For example, the first color can be green, the second color can be gray, and the third color can be red.

[0144] When multiple adjacent inflection points are determined based on route information, the controller 160 can change the second reference distance to a third reference distance. The third reference distance can be longer than the second reference distance and shorter than the first reference distance. Adjacent inflection points can be inflection points that exist within a preset distance.

[0145] Adjacent turning points can exist at locations near intersection changes (IC) or junction changes (JC). A third reference distance can be set by the user.

[0146] The controller 160 identifies whether the positions of the object images at corresponding inflection points in the augmented reality image (i.e., the display positions of the dashed line image, the arrow image, and the dot image) are consistent with each other.

[0147] like Figure 9As shown, when it is determined that the display position of the dashed line image AR4 is consistent with the display position of the arrow image AR7, and the display position of the dashed line image AR4 is consistent with the display position of the dot image AR5, the controller 160 can control the display of information about the remaining time and distance to the destination, control the display device 112 to display road name information about the road name to be moved to the destination, as well as left and right turn information, control the color change of the images of short-distance turning points and long-distance turning points based on traffic information, and control route guidance until the destination is reached when it is determined that the current position is close to the destination.

[0148] When it is determined that the current location is close to the destination, the controller 160 can control the display device 112 to delete the navigation image and display an augmented reality image on the entire area of ​​the display device 112, and control the display device 112 to display arrow images and straight line images on the augmented reality image to guide the route to the destination.

[0149] like Figure 10 As shown, when it is determined that the display position of the dashed line image AR4 is consistent with the display position of the arrow image AR7, but the display position of the dashed line image AR4 is inconsistent with the display position of the dot image AR5, the controller 160 can also control the display device 112 to additionally display the traffic information as a text image. In this case, the controller 160 can change the color of the image of the long-distance inflection point based on the traffic information, and control the display of the traffic information as a text image only within a specific time period.

[0150] like Figure 11 As shown, when it is determined that the display position of the dashed line image AR4 is inconsistent with the display position of the arrow image AR7 and the display position of the dashed line image AR4 is consistent with the display position of the dot image AR5, the controller 160 can control the display device 112 to display traffic information and alternative route information AR8.

[0151] Traffic information can include pictographic images indicating traffic congestion or traffic accidents.

[0152] The controller 160 can control the display device 112 to display a navigation image, and determine whether the current location is close to the destination based on at least one of the expected arrival time information of the destination, current time information, reference distance information, movement speed information, current location information and destination information, and when it is determined that the current location is close to the destination, control the display device 112 to display an augmented reality image.

[0153] When it is determined that the current location is close to the destination, the controller 160 can identify the image of the object in the image obtained by the image acquisition device 140 that corresponds to the destination information based on map information, image information obtained by the image acquisition device 140 and destination information, and display the object image for highlighting the destination based on the display location information of the identified object image.

[0154] The map information may be information stored in storage device 161 and may be received from server 2. The map information may include building information and multi-purpose facility information, and may also include address information and parking location information for each building or multi-purpose facility.

[0155] The emphasized object image used to highlight the destination may include a highlighted image or a polygonal image used to visually identify the object image corresponding to the destination information, and may include a polygonal image formed by dashed lines.

[0156] For example, controller 160 can identify a region of an object image corresponding to destination information, identify the edge portion of the identified region, and match and display a polygon image formed by dashed lines with the identified edge portion. The polygon image formed by dashed lines can be the same as or similar in shape to the shape forming the identified edge portion.

[0157] The controller 160 can identify the region of the object image corresponding to the destination information, and overlay and display a polygonal image of a preset color in the identified region, thereby displaying the destination image as a highlighted image.

[0158] The controller 160 can identify an object image corresponding to the destination information, identify a vertical plane based on the position information of the identified object image, control the display device 112 to display a grid image based on the position information of the identified vertical plane, and display an anchor mark image based on the position information of the grid image and the position information of the object image.

[0159] The controller 160 can identify an object image corresponding to the destination information and adjust the transparency of the remaining object images (i.e., the second object image) other than the identified object image (i.e., the first object image) to a different transparency than the first object image.

[0160] For example, controller 160 can maintain the transparency of the first object image and adjust the transparency of the second object image to be higher than that of the first object image.

[0161] The controller 160 can adjust the transparency of the first object image to a first transparency level and adjust the transparency of the second object image to a second transparency level. The second transparency level can be higher than the first transparency level.

[0162] The controller 160 can overlap and display polygonal shape images with preset colors in the area of ​​the first object image.

[0163] The controller 160 can determine whether the moving speed is less than a reference speed based on the moving speed information; when it is determined that the moving speed is less than the reference speed, it determines whether the distance from the current position to the destination is less than or equal to a set distance based on the current position information and the destination position information; when it is determined that the distance from the current position to the destination is less than or equal to the set distance, it identifies an object image corresponding to the destination information; and controls the display device 112 to display an image of an object indicating the destination on the image of the identified object.

[0164] When it is determined that the distance from the current location to the destination is less than or equal to the set distance, the controller 160 can obtain information about the remaining time until the destination is reached and information about the remaining distance, control the display of the obtained remaining time information and remaining distance information, and control the display device to display road name information about the road to the destination, as well as left turn and right turn information.

[0165] Storage device 161 stores map information.

[0166] Storage device 161 stores information about the first reference distance, the second reference distance, and the third reference distance, stores information about the set distance, and stores color information of the point image corresponding to the long-distance inflection point and color information of the arrow image corresponding to the short-distance inflection point.

[0167] Storage device 161 can store information about object images used for guiding roads and information about object images indicating destinations.

[0168] Storage device 161 can store location information about the point of interest. The point of interest can be a point selected by the user.

[0169] Storage device 161 may be implemented as at least one of, but is not limited to, non-volatile storage devices (such as cache, read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), and flash memory), volatile storage devices (such as random access memory (RAM)), and storage media (such as hard disk drive (HDD) and CD-ROM). Storage device 161 may be a memory chip implemented as separate from the processor described above with respect to controller 160, or may be implemented as a single chip having a processor.

[0170] according to Figure 1The performance of the components of the mobile device 1 shown can be improved by adding or removing at least one component. Those skilled in the art will readily understand that the relative positions of the components can be changed depending on the system's performance or structure.

[0171] Figure 1 Each component shown represents a software and / or hardware component such as a field-programmable gate array (FPGA) and an application-specific integrated circuit (ASIC).

[0172] Figure 12A and Figure 12B This is a control flowchart of a mobile device according to various exemplary embodiments of the present invention.

[0173] When the input device 111 receives an execution command for the augmented reality application, the mobile device 1 can perform augmented reality functions by executing the augmented reality application. In this case, the mobile device 1 can acquire an image through the image acquisition device 140 and display the acquired image (171) through the display device 112.

[0174] When the destination information is received by the input device 111 while the augmented reality function is being executed (172), the mobile device 1 can execute the navigation application and send the destination information to the navigation application.

[0175] The mobile device 1 can control the activation of the location receiver 130 in response to the execution of a navigation application.

[0176] Mobile device 1 executes a navigation application to search for a route from its current location to its destination based on its current location information and destination information (173), and displays a navigation image (174) in which the route information and route guidance information for the searched route are matched with map information. In this case, mobile device 1 may display an augmented reality image in one area of ​​the display device 112 and a navigation image in another area.

[0177] Mobile device 1 periodically identifies its current location information while displaying a navigation image. Mobile device 1 can display information about the total time required to reach the destination, and can also display information about the arrival time at the final destination.

[0178] Mobile device 1 identifies its movement status (175) based on at least one of route information from its current location to its destination, movement speed information, and traffic information, and determines whether the identified movement status has changed (176). In this case, mobile device 1 can determine whether the change in movement status is a change in traffic conditions, and if it is determined to be a change in traffic conditions, it can further determine whether the route information to the destination has changed.

[0179] The movement speed information can be driving speed information. Changes in the movement of mobile device 1 can include changes in traffic conditions and changes in route.

[0180] Mobile device 1 can obtain the location information of the inflection point where the movement of mobile device 1 is to change based on at least one of route information, movement speed information and traffic information (177), obtain the distance information from the current position to the inflection point based on the obtained location information of the inflection point and the current position information (178), obtain information about the object image to be displayed in the augmented reality image based on the obtained distance information, and display the object image in the augmented reality image based on the obtained information about the object image (179).

[0181] The object image to be displayed in augmented reality is an image that allows users to easily identify the location of the change in motion and the reason for the change in motion. Depending on the distance to the location where the change in motion occurred and the reason for the change in motion, the object image can have different shapes and colors.

[0182] The reasons for the change in the movement of mobile device 1 may include at least one of traffic congestion and traffic accidents, and may include a change in route corresponding to a change in the arrival time to the destination.

[0183] When the distance to the inflection point is determined to be greater than or equal to a first reference distance based on the current location information and the distance information to the inflection point, the mobile device 1 can identify the direction of the destination based on the current location information, route information, map information, and destination location information, and based on the displayed position of the current location in the augmented reality image. It can also display an object image of the inflection point pointing from the displayed position of the current location to the destination. The object image of the inflection point pointing to the destination can be a dashed line image AR4.

[0184] When the distance to the inflection point is determined to be greater than or equal to the second reference distance and less than the first reference distance based on the distance information from the current position to the inflection point, the mobile device 1 can display an object image of a long-distance inflection point, and when the distance to the inflection point is determined to be less than the second reference distance, it can display an object image of a short-distance inflection point.

[0185] The object image of a long-distance inflection point can be displayed as a point image, and the distance information from the current position to the inflection points surrounding the point image can be displayed as a text image.

[0186] When displaying an object image at a long inflection point, the object image can be displayed as a dot image of the first color when the traffic conditions are stable, as a dot image of the second color when there is traffic congestion or a traffic accident, and as a dot image of the third color when the route changes to an alternative route.

[0187] An object image with a short-distance inflection point can be displayed as an arrow image.

[0188] When displaying an object image at a short-distance inflection point, the object image can be displayed as an arrow image of the first color when maintaining the main route, as an arrow image of the second color when traffic congestion or a traffic accident occurs, and as an arrow image of the third color when the route changes to an alternative route.

[0189] After displaying the object image of the inflection point in the augmented reality image, the mobile device 1 identifies whether the display positions of the dashed line image, the object image of the short-distance inflection point (i.e., the arrow image), and the object image of the long-distance inflection point (i.e., the dot image) are consistent with each other, and additionally displays route guidance information (180) in response to the identification result.

[0190] When it is determined that the display position of the dashed line image in the augmented reality image is consistent with the display position of the arrow image, and the display position of the dashed line image is consistent with the display position of the dot image, the mobile device 1 can display information about the remaining time and distance to the destination, display road name information about the road name to be moved to the destination, as well as left and right turn information, change the color of the images of short-distance turning points and long-distance turning points based on traffic information, and control route guidance until the destination is reached when it is determined that the current position is close to the destination.

[0191] The mobile device 1 can determine whether the moving speed is less than a reference speed based on the moving speed information; when it is determined that the moving speed is less than the reference speed, it can determine whether the distance from the current location to the destination is less than or equal to a set distance based on the current location information and the location information of the destination; when it is determined that the distance from the current location to the destination is less than or equal to the set distance, it can identify an object image corresponding to the destination information; and control the display device 112 to display an object image indicating the destination on the image of the identified object.

[0192] The mobile device 1 can display navigation images and augmented reality images simultaneously while moving from the departure time point, and can display augmented reality images by switching image display modes when it is determined that the current time point corresponds to information about the arrival time.

[0193] When a navigation image and an augmented reality image are displayed together during movement from the departure point, the mobile device 1 may display only the object image from the augmented reality image on the navigation image.

[0194] When the current time corresponds to information about arrival time, mobile device 1 can display an augmented reality image and a navigation image, respectively, while simultaneously displaying an augmented reality image larger than the navigation image. In this case, mobile device 1 can display an object image on a surrounding image obtained while displaying the augmented reality image. When the destination changes, mobile device 1 can display information about the possible arrival time of the changed destination.

[0195] When it is determined that the current location is close to the destination, the mobile device 1 can delete the navigation image and display an augmented reality image on the entire area of ​​the display device 112, and display arrow images and straight lines on the augmented reality image to guide the route to the destination.

[0196] When it is determined that the display position of the dashed line image is consistent with the display position of the arrow image, but the display position of the dashed line image is inconsistent with the display position of the dot image, the mobile device 1 additionally displays the traffic information as a text image. In this case, the mobile device 1 can change the color of the image of long-distance inflection points based on the traffic information, and display the traffic information as a text image only within a specific time period.

[0197] When it is determined that the display position of the dashed line image is inconsistent with the display position of the arrow image, and the display position of the dashed line image is inconsistent with the display position of the dot image, the mobile device 1 displays traffic information and alternative route information.

[0198] Traffic information can include pictographic images indicating traffic congestion or traffic accidents.

[0199] The mobile device 1 can determine whether the current location is close to the destination based on the current location information and the destination information during the execution of the navigation function (181), and can switch to an augmented reality image when it is determined that the current location is close to the destination.

[0200] Determining whether the current location is close to the destination may include: obtaining distance information between the current location and the destination based on the current location information and the destination information; and determining that the current location is close to the destination when the distance between the current location and the destination is less than or equal to the predetermined distance based on the obtained distance information and the predetermined distance information.

[0201] Mobile device 1 can determine whether its current location is near the destination based on the expected arrival time of the destination and the current time. That is, mobile device 1 can obtain the remaining time until it reaches the destination based on the expected arrival time of the destination and the current time, and determine that the current location is near the destination when the obtained remaining time is less than or equal to a predetermined time period.

[0202] When it is determined that the current location is close to the destination, the mobile device 1 can identify the object image in the image obtained by the image obtaining device 140 that corresponds to the destination information based on map information, image information obtained by the image obtaining device 140 and destination information (182), and display the object image for emphasizing the destination based on the display location information of the identified object image.

[0203] The emphasized object image used to highlight the destination may include a highlighted image or a polygonal image used to visually identify the object image corresponding to the destination information, and may include a polygonal image formed by dashed lines.

[0204] The emphasized object image used to highlight the destination can include a polygonal image of a preset color that overlaps with a region of the object image.

[0205] When it is determined based on the moving speed information that the moving speed is less than the reference speed and the distance to the destination is less than the predetermined distance, the moving device 1 can determine whether plane detection is possible (183). When it is determined that plane detection is possible, an anchor mark image is displayed at a position adjacent to the object image (184). When it is determined that plane detection is impossible, a dashed line graphic image is displayed on the object image (185).

[0206] The display of anchor mark images may include: identifying an object image corresponding to destination information; identifying a vertical plane based on the location information of the identified object image; displaying a grid image based on the location information of the identified vertical plane; and displaying anchor mark images based on the location information of the grid image and the location information of the object image.

[0207] The mobile device 1 can identify an object image corresponding to the destination information and adjust the transparency of the remaining object images (i.e., the second object image) other than the identified object image (i.e., the first object image) to be different from the transparency of the first object image.

[0208] When the user receives a destination arrival command, the mobile device 1 can terminate the augmented reality application and the navigation application.

[0209] Figure 13 This is a schematic diagram of vehicle control according to another exemplary embodiment of the present invention.

[0210] Vehicle 3 includes a main body with an exterior and an interior, and a chassis in which the mechanical equipment required for driving is installed as the rest of the vehicle, excluding the main body.

[0211] The chassis of vehicle 3 (which is a frame for supporting the main body) is provided with wheels respectively arranged in the front, rear, left and right, a power device configured to provide driving force to the front, rear, left and right wheels, a steering device, a braking device configured to provide braking force to the front, rear, left and right wheels, and a suspension device configured to adjust the suspension of vehicle 3.

[0212] The exterior of the main body includes the front panel, engine hood, roof panel, rear panel, front door, rear door, left door, right door, and windows respectively installed on the front door, rear door, left door, and right door for opening and closing.

[0213] The exterior of the main body further includes antennas for receiving signals from GPS satellites, broadcasting stations, etc., and for performing wireless vehicle networking (vehicle-to-everything (V2X)), such as communication with other vehicles (V2V) and communication with infrastructure (V2I).

[0214] The interior of the main body includes the seats for the occupants, the instrument panel, the instrument panel (i.e., the instrument cluster) arranged on the instrument panel and including the tachometer, speedometer, coolant temperature gauge, fuel gauge, turn indicators, high beam indicator, warning lights, seat belt warning lights, trip meter, odometer, gear lever indicator, door open warning light, engine oil warning light and low fuel warning light, the central panel with air conditioning vents and control panel, and the host unit located on the central panel to receive operating commands for audio equipment and air conditioning.

[0215] For user convenience, vehicle 3 includes vehicle terminal 310. Vehicle terminal 310 can be installed on the dashboard in a recessed or mounted manner.

[0216] The vehicle terminal 310 can receive user input and display information about the various functions performed in the vehicle 3 as images.

[0217] Various functions may include audio functions, video functions, navigation functions, broadcasting functions, radio functions, content playback functions, and Internet search functions, which are functions of at least one application installed by the user.

[0218] The vehicle terminal 310 may include a display panel as a display device, and may further include a touch panel as an input device. The vehicle terminal 310 may include only the display panel or a touch screen in which the touch panel is integrated with the display panel.

[0219] When the vehicle terminal 310 is implemented using only the display panel, the input device provided on the central panel can be used to select the buttons displayed on the display panel.

[0220] The vehicle terminal 310 may include an input device and a display device. The input device and display device of the vehicle 3 are the same as those of the mobile device, and therefore their description is omitted.

[0221] According to various exemplary embodiments of the present invention, the vehicle terminal 310 can perform various control functions performed by the controller of the mobile device. The control configurations for the navigation function and the augmented reality function performed by the vehicle terminal 310 are the same as the control configurations for the functions performed by the controller of the mobile device according to various exemplary embodiments of the present invention, and therefore their descriptions will be omitted.

[0222] The vehicle terminal 310 may further include a storage device configured to store map information and location information of the POI.

[0223] The sound output device 320 outputs audio data corresponding to the function currently being performed in the vehicle 3 as sound.

[0224] The functions currently being implemented may include radio functions, audio functions corresponding to content reproduction and music reproduction, and navigation functions.

[0225] The sound output device 320 may include a speaker. One or more speakers may be provided.

[0226] The speaker can be installed in the vehicle terminal 310.

[0227] The location receiver 330 includes a Global Positioning System (GPS) receiver and a signal processing device configured to process GPS signals obtained from the GPS receiver.

[0228] Vehicle 3 may further include an image acquisition device 340 for acquiring images of the surroundings. The image acquisition device 340 may be an image acquisition device disposed in a black box, an image acquisition device configured for autonomous driving control equipment, or an image acquisition device configured for obstacle detection.

[0229] The image acquisition device 340 can be mounted on the windshield, or on the interior window of the vehicle 3, or on the interior rearview mirror of the vehicle 3, or on the roof panel to be exposed to the outside.

[0230] The image acquisition device 340 may further include at least one of a left camera and a right camera for acquiring images of the left and right sides of the vehicle 3, a rear camera for acquiring images of the rear of the vehicle 3, and a front camera for acquiring images of the front of the vehicle 3.

[0231] Image acquisition device 340 may include a CCD or CMOS image sensor as a camera, and may include a three-dimensional spatial recognition sensor such as an RGB-D sensor (KINECT), a structured light sensor (TOF), and a stereo camera.

[0232] Vehicle 3 may further include communication equipment 350 for communicating between various internal electronic devices, communicating with user terminals, and communicating with servers.

[0233] Communication device 350 can communicate with external devices via an antenna.

[0234] External devices may include at least one of servers, user terminals, other vehicles, and infrastructure.

[0235] Communication methods using antennas can include: second-generation (2G) communication methods, such as Time Division Multiple Access (TDMA) and Code Division Multiple Access (CDMA); third-generation (3G) communication methods, such as Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access 2000 (CDMA2000), Wireless Broadband Internet (WiBro), and Global Microwave Access Interoperability (WiMAX); fourth-generation (4G) communication methods, such as Long Term Evolution (LTE) and Wireless Broadband Evolution (WBE); or fifth-generation (5G) communication methods.

[0236] The controller 360 controls the communication between the vehicle terminal 310 and the image acquisition device 340, the position receiver 330, and the sound output device 320.

[0237] The controller 360 can send image information from the image acquisition device 340 to the vehicle terminal 310, send location information from the location receiver 330 to the vehicle terminal 310, and send audio information from the vehicle terminal 310 to the audio output device 320.

[0238] Vehicle 3 may further include a speed detector 370 for detecting travel speed (i.e., the speed at which the vehicle is moving).

[0239] The speed detector 370 can be a wheel speed sensor located on each of the multiple wheels, or it can be an acceleration sensor.

[0240] The controller 360 can obtain the vehicle speed based on the wheel speed detected by multiple wheel speed sensors, the vehicle speed based on the acceleration detected by the acceleration sensor, or the vehicle speed based on the wheel speed detected by multiple wheel speed sensors and the acceleration detected by the acceleration sensor.

[0241] The controller 360 can send the obtained driving speed to the vehicle terminal 310 to obtain the expected arrival time at the destination or the remaining time until the destination.

[0242] According to various exemplary embodiments of the present invention, the controller 360 can also perform all control operations performed by the controller of the mobile device.

[0243] according to Figure 13 The performance of the components of the vehicle 3 shown can be improved by adding or removing at least one component. Those skilled in the art will readily understand that the relative positions of the components can be changed depending on the system's performance or structure.

[0244] Figure 13 Each component shown represents a software and / or hardware component such as a field-programmable gate array (FPGA) and an application-specific integrated circuit (ASIC).

[0245] The disclosed exemplary embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored as program code, and when executed by a processor, a program module may be created to perform the operations of the disclosed exemplary embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0246] Computer-readable recording media include various recording media in which instructions that can be decrypted by a computer are stored. For example, they can be read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, and optical data storage devices.

[0247] As is evident from the above, the present invention uses object images to display the location of the destination through planar detection technology and cloud anchoring technology, which allows users to drive to their destination comfortably and easily, maintaining the practicality of commercial services.

[0248] Furthermore, this invention allows users to accurately identify the location of a destination by using augmented reality object images to emphasize or adjust the transparency of points of interest (POIs) such as the destination.

[0249] Furthermore, this invention uses augmented reality to display the direction of the destination and the location of turning points through object images, as well as display congested and accident-prone road sections, and suggests alternative routes in real time, thereby allowing users to understand the entire route guidance situation.

[0250] Therefore, the present invention can improve the quality and marketability of mobile devices and vehicles, increase user satisfaction, improve user convenience, reliability and vehicle safety, and ensure product competitiveness.

[0251] For ease of interpretation and accurate definition of the appended claims, the terms “upper part,” “lower part,” “inside,” “outside,” “above,” “below,” “upward,” “downward,” “front,” “back,” “rear,” “inside,” “outside,” “inward,” “outer,” “inner,” “outer,” “forward,” and “backward” are used to describe features of exemplary embodiments with reference to the location of features as shown in the accompanying drawings. It will also be understood that the term “connection” or its derivatives refer to both direct and indirect connections.

[0252] For purposes of illustration and description, the foregoing description of specific exemplary embodiments of the invention has been presented. These descriptions are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and it will be apparent that many modifications and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain certain principles of the invention and its practical application, enabling others skilled in the art to make and utilize various exemplary embodiments of the invention, as well as various alternatives and modifications thereof. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A mobile device, comprising: The input device is configured to receive user input. A location receiver is configured to receive current location information used to obtain the current location; An image acquisition device, configured to acquire images of the surrounding environment; The display device is configured to display an ambient image obtained from the image acquisition device in response to the execution of the augmented reality mode; as well as The controller is configured to, when receiving destination information from user input during the execution of the augmented reality mode, search for a route from the current location to the destination, identify movement based on route information corresponding to the searched route, obtain location information of an inflection point where the movement changes based on information about the identified movement, obtain a distance from the current location to the inflection point based on the current location information and the location information of the inflection point, determine an object image of the inflection point based on the obtained distance, a first reference distance, and a second reference distance, and control the display device to display the object image of the inflection point on the surrounding image displayed on the display device based on the location information of the inflection point and the current location information. The controller is configured to change the second reference distance to a third reference distance when it determines, based on the route information, that multiple inflection points exist within a preset distance. Wherein, the first reference distance is longer than the second reference distance, and the third reference distance is longer than the second reference distance but shorter than the first reference distance.

2. The mobile device according to claim 1, in, The controller is configured to change the display information of the object image at the inflection point based on the obtained distance information, and The display information of the object image at the inflection point includes at least one of the shape information and color information of the object image.

3. The mobile device according to claim 2, wherein, When the controller determines that the distance to the inflection point is greater than or equal to the first reference distance, the controller is configured to identify the direction of the destination based on the current location information, the destination information, and the map information, and control the display device to display an image of the object oriented from the current location to the destination on the surrounding image displayed on the display device.

4. The mobile device according to claim 2, in, When the controller determines that the distance to the inflection point is less than the first reference distance and greater than or equal to the second reference distance, the controller is configured to control the display device to display an object image of the long-distance inflection point on the surrounding image based on the location of the inflection point and map information.

5. The mobile device according to claim 4, in, Based on information regarding the identified movement, the controller is configured to control the display device: When the controller determines that the movement is stable, it displays the object image at the long-distance inflection point in a first color. When the controller determines that the movement is a congestion or accident, it displays the object image at the long-distance inflection point in a second color. When the controller determines that the movement is a route change, the object image of the long-distance inflection point is displayed in a third color.

6. The mobile device according to claim 5, in, When the controller receives traffic information, the controller is configured to determine whether a route change is necessary based on the received traffic information, the destination information, the current location information, the route information, and the map information.

7. The mobile device according to claim 4, in, When the controller determines that the distance to the inflection point is less than the second reference distance, the controller is configured to control the display device to display an object image of the short-distance inflection point on the surrounding image based on the location of the inflection point and the map information.

8. The mobile device according to claim 7, wherein, Based on information regarding the identified movement, the controller is configured to control the display device: When the controller determines that the movement is a route-keeping situation, it displays the object image at the short-distance inflection point in a first color. When the controller determines that the movement is a congestion or accident, it displays the object image at the short-distance inflection point in a second color. When the controller determines that the movement is a route change, the object image of the short-distance inflection point is displayed in a third color.

9. The mobile device according to claim 7, in, The controller is configured to control the display device to display traffic information when the controller determines that the display position of the directional object image is consistent with the display position of the object image at the short-distance inflection point but inconsistent with the display position of the object image at the long-distance inflection point.

10. The mobile device according to claim 7, in, The controller is configured to control the display device to display traffic information and alternative route information when the controller determines that the display position of the directional object image is inconsistent with the display position of the object image at the short-distance inflection point and inconsistent with the display position of the object image at the long-distance inflection point.

11. The mobile device according to claim 1, in, The controller is configured to determine whether the distance to the destination is less than or equal to a predetermined distance based on the destination information, the current location information, the route information, and the map information. The controller is configured to, when it determines that the distance to the destination is less than or equal to the predetermined distance, identify a destination image corresponding to the destination information from the surrounding images, and The controller is configured to control the display device to overlay and display an emphasized object image on the identified destination image.

12. The mobile device according to claim 11, in, The controller is configured to determine whether the distance to the destination is less than or equal to a predetermined distance based on the destination information, the current location information, the route information, and the map information. The controller is configured to, when determining that the distance to the destination is less than or equal to the predetermined distance, identify a destination image corresponding to the destination information from the surrounding images, and The controller is configured to control the transparency of the identified destination image and the remaining image to be different from each other.

13. The mobile device according to claim 11, wherein, The controller is configured to: Identify ground images adjacent to the destination image from the surrounding images. Control the display device to display a grid-like object image on the identified ground image, and The display device is controlled to overlay and display the anchor mark image on the destination image.

14. A vehicle comprising: The input device is configured to receive user input. A location receiver is configured to receive current location information used to obtain the current location; The speed detector is configured to detect driving speed; An image acquisition device, configured to acquire images of the surrounding environment; The display device is configured to display an ambient image obtained from the image acquisition device in response to the execution of the augmented reality mode; as well as The controller is configured to, when receiving destination information from user input during interaction between the augmented reality mode and the navigation mode, search for a route from the current location to the destination; identify movement based on route information corresponding to the searched route and driving speed information detected by the speed detector; obtain location information of an inflection point where the movement changes based on information about the identified movement; obtain a distance from the current location to the inflection point based on the current location information and the location information of the inflection point; determine an object image of the inflection point based on the obtained distance, a first reference distance, and a second reference distance; and control the display device to display the object image of the inflection point on the surrounding image displayed on the display device based on the location information of the inflection point and the current location information. The controller is configured to change the second reference distance to a third reference distance when it determines, based on the route information, that multiple inflection points exist within a preset distance. Wherein, the first reference distance is longer than the second reference distance, and the third reference distance is longer than the second reference distance but shorter than the first reference distance.

15. The vehicle according to claim 14, in, The controller is configured to change the display information of the object image at the inflection point based on the obtained distance information, and The display information of the object image at the inflection point includes at least one of the shape information and color information of the object image.

16. The vehicle according to claim 15, wherein, When the controller determines that the distance to the inflection point is greater than or equal to the first reference distance, the controller is configured to identify the direction of the destination based on the current location information, the destination information, and map information, and control the display device to display an image of the object oriented from the current location to the destination on the surrounding image displayed on the display device. When the controller determines that the distance to the inflection point is less than the first reference distance and greater than or equal to the second reference distance, the controller is configured to control the display device to display an object image of the long-distance inflection point on the surrounding image based on the location of the inflection point and the map information. When the controller determines that the distance to the inflection point is less than the second reference distance, the controller is configured to control the display device to display an image of the object at the short-distance inflection point on the surrounding image based on the location of the inflection point and the map information.

17. The vehicle according to claim 16, in, The controller is configured to control the display device to display traffic information when it determines that the display position of the directional object image is consistent with the display position of the object image at the short-distance inflection point but inconsistent with the display position of the object image at the long-distance inflection point. The controller is configured to control the display device to display the traffic information and alternative route information when it determines that the display position of the directional object image is inconsistent with the display position of the object image at the short-distance inflection point and inconsistent with the display position of the object image at the long-distance inflection point.

18. The vehicle according to claim 14, in, When the controller receives traffic information, the controller is configured to determine whether a route change is necessary based on the received traffic information, the destination information, the current location information, the route information, and the map information, and to identify the point where the route change is necessary as an inflection point when the controller determines that the route change is necessary.

19. The vehicle according to claim 14, wherein, The controller is configured to: Based on the destination information, the current location information, the route information, the driving speed information, and the map information, determine whether the distance to the destination is less than or equal to a predetermined distance. When the controller determines that the distance to the destination is less than or equal to the predetermined distance, it identifies a destination image corresponding to the destination information from the surrounding images, and The display device is controlled to overlay and display an emphasized object image on the identified destination image.

20. The vehicle according to claim 14, wherein, When searching for a route from the current location to the destination, the controller is configured to search for the route from the current location to the destination based on the current location information, the destination information, and map information obtained by the location receiver. When the controller controls the display device to display the object image of the inflection point on the surrounding image displayed on the display device, the controller is configured to control the display device to display the object image of the inflection point on the surrounding image displayed on the display device based on the location information of the inflection point, the map information, and the current location information.

Images