Electronic device and control method thereof, computer readable recording medium

Abstract

Disclosed are an electronic device, a control method thereof, and a computer-readable recording medium. The control method of the electronic device includes determining the types and colors of lane lines on both sides of a lane on which a vehicle is traveling from driving-related image data of the vehicle, comparing the types and colors of the lane lines on both sides based on the determination with a stored lane determination table to determine lane position information of the vehicle, and comparing the determined lane position information with a direction of a path at a turning point when the path ahead of the vehicle by a predetermined distance has the turning point, to determine whether the vehicle needs to change lanes. The stored lane determination table is a table that defines the types and colors of the lane lines on both sides of the lane.

Description

[0001] This application is a divisional application of the invention patent application filed on June 17, 2015, with application number 201510336818.5 and entitled "Electronic Device and Control Method Thereof". Technical Field

[0002] This invention relates to electronic devices and control methods thereof, computer-readable recording media, and more specifically, to electronic devices and control methods thereof capable of identifying the location of a roadway and performing driving-related guidance based on the identified roadway, as well as computer-readable recording media. Background Technology

[0003] When a vehicle is in motion, the most important thing is safe driving and prevention of traffic accidents. To this end, vehicles are equipped with various auxiliary devices that perform functions such as controlling the vehicle's posture and controlling the vehicle's structure, as well as safety devices such as seat belts and airbags.

[0004] Furthermore, recently, devices such as black boxes installed in vehicles store vehicle driving images and data transmitted from various sensors, thus creating a trend towards installing devices in vehicles to determine the cause of accidents when a traffic accident occurs. Since black boxes or navigation applications can also be installed on portable terminal devices such as smartphones and tablets, they are being used as vehicle devices as described above.

[0005] However, in reality, the utilization rate of driving images in such vehicle devices is currently very low. More specifically, even when driving images of a vehicle are obtained through visual sensors such as cameras currently installed in the vehicle, the vehicle's electronic devices are limited to simply displaying and transmitting these images or generating simple peripheral warning information such as whether the vehicle has deviated from its lane markings.

[0006] Furthermore, as a new type of vehicle electronic device that is currently attracting attention, head-up displays (HUDs) or augmented reality interfaces have been proposed. However, in these devices, the utilization of the vehicle's driving images is limited to simply displaying or generating simple prompts. Summary of the Invention

[0007] The present invention is proposed to solve the above-mentioned problems. The purpose of the present invention is to provide an electronic device and control method thereof that can use driving-related image data of a vehicle to generate the position information of the vehicle in the roadway and perform driving-related guidance based thereon.

[0008] Furthermore, another object of the present invention is to provide an electronic device and control method thereof capable of effectively performing driving-related guidance based on augmented reality.

[0009] A control method for an electronic device according to an embodiment of the present invention for achieving the above-mentioned objective includes: a step of identifying a lane line region portion from driving-related image data of a vehicle; a step of generating lane line information corresponding to the roadway where the vehicle is located from the identified lane line region portion image data; a step of generating roadway position information where the vehicle is located using at least one of the generated lane line information and roadway information of the road where the vehicle is located; and a step of performing driving-related guidance for the vehicle using the generated roadway position information.

[0010] Furthermore, the lane line information may include lane line type information and lane line color information corresponding to each lane line located on both sides of the carriageway based on the carriageway where the vehicle is located.

[0011] Furthermore, the steps for generating the aforementioned lane location information may include: obtaining lane information of the road where the vehicle is located from map data; determining whether the vehicle is located in the first or last lane of the road using the generated lane line information; and generating lane location information of the vehicle based on the lane information of the road when the vehicle is located in the first or last lane.

[0012] Furthermore, the step of generating the above-mentioned lane location information may also include the following steps: if the lane where the vehicle is located changes to a lane between the first lane and the last lane as the vehicle changes lanes, then the generated lane location information is updated to the changed lane location information.

[0013] Furthermore, the present invention may also include the step of regenerating the lane location information of the vehicle based on the lane information of the road where the vehicle is located if the lane where the vehicle is located changes from the lane between the first lane and the last lane to the first lane or the last lane as the vehicle changes lanes.

[0014] Furthermore, the steps for implementing driving-related guidance for the aforementioned vehicles may include outputting lane change guidance using the vehicle's navigation path and the lane location information.

[0015] Furthermore, the steps for performing driving-related guidance for the aforementioned vehicle may include using the aforementioned lane location information to output lane guidance for the vehicle's location.

[0016] Furthermore, the present invention also includes a step of selecting and outputting appropriate lane departure guidance based on the types of lane lines on both sides of the roadway where the vehicle is located, as identified based on the lane line information described above.

[0017] Furthermore, the above output steps may include: generating an indicator for performing the above driving-related guidance; and outputting the generated indicator via augmented reality.

[0018] On the other hand, an electronic device according to an embodiment of the present invention for achieving the above-mentioned objective includes: a lane line information generation unit that identifies a lane line area portion from driving-related image data of a vehicle and generates lane line information corresponding to the roadway where the vehicle is located from the image data of the identified lane line area portion; a roadway position information generation unit that generates roadway position information where the vehicle is located using at least one of the generated lane line information and roadway information of the road where the vehicle is located; and a control unit that performs driving-related guidance for the vehicle using the generated roadway position information.

[0019] Furthermore, the lane line information may include lane line type information and lane line color information corresponding to each lane line located on both sides of the carriageway based on the carriageway where the vehicle is located.

[0020] Furthermore, the aforementioned lane location information generation unit can obtain the lane information of the road where the vehicle is located from the map data, and use the generated lane line information to determine whether the vehicle is located in the first lane or the last lane of the road. If the vehicle is located in the first lane or the last lane, the unit generates the lane location information of the vehicle based on the lane information of the road.

[0021] Furthermore, if the lane in which the vehicle is located changes to a lane between the first lane and the last lane as the vehicle changes lanes, the lane position information generation unit can update the generated lane position information to the changed lane position information.

[0022] Furthermore, if the lane in which the vehicle is located changes from the lane between the first lane and the last lane to the first lane or the last lane as the vehicle changes lanes, the lane position information generation unit can regenerate the lane position information of the vehicle by reflecting the lane information of the road in which the vehicle is located.

[0023] Furthermore, the aforementioned control unit can control the output unit to output lane change guidance using the navigation path of the aforementioned vehicle and the lane location information.

[0024] Furthermore, the aforementioned control unit can control the output unit to output lane guidance for the vehicle using the aforementioned lane position information.

[0025] Furthermore, the aforementioned control unit can control the output unit to select and output appropriate lane departure guidance based on the types of lane lines on both sides of the roadway where the vehicle is located, as identified based on the aforementioned lane line information.

[0026] Furthermore, the aforementioned control unit can control the output unit to generate indicators for performing the aforementioned driving-related guidance, and output the generated indicators through augmented reality.

[0027] On the other hand, a control method for an electronic device according to an embodiment of the present invention for achieving the above-mentioned objective includes: receiving input from a user requesting route guidance; generating a route guidance line based on destination information corresponding to the request; correcting the generated route guidance line by reflecting the radius of the actual driving trajectory of the vehicle; performing variable stereoscopic rendering by differentiating the height of the corrected route guidance line based on the distance to the vehicle; mapping texture to the stereoscopic data generated based on variable stereoscopic rendering to generate a route guidance indicator; and outputting the route guidance indicator on a screen using augmented reality.

[0028] Furthermore, the steps for correcting the aforementioned path guide lines may include: adding vertices to keep the forward path guide lines of the vehicle straight; adding vertices to the curved sections of the path guide lines to achieve curvature of the curved sections; and using the added vertices to generate a path guide line with a radius that reflects the actual driving trajectory of the vehicle.

[0029] Furthermore, the steps of performing the above-mentioned variable stereoscopic transformation may include: generating virtual path guide lines on both sides of the corrected path guide line; calculating the height value in such a way that the height value of the vertices included in the corrected path guide line increases proportionally to the distance of the vehicle; and performing stereoscopic transformation on the vertices of the path guide line and the vertices included in the virtual path guide line, respectively, by using polygons.

[0030] Furthermore, in the step of generating the aforementioned path guidance indicator, a texture with displacement based on the vehicle's speed can be generated from the aforementioned 3D data mapping to generate the path guidance indicator.

[0031] Furthermore, the present invention may also include: performing a step of calibration to estimate camera parameters corresponding to the camera from the captured images obtained from the camera; and a step of generating a virtual three-dimensional space from the captured images based on the camera parameters, wherein in the above output step, the generated three-dimensional space may be output in conjunction with the generated path guidance indicator.

[0032] On the other hand, an electronic device according to an embodiment of the present invention for achieving the above-mentioned objective may include: an input unit for receiving input from a user requesting route guidance; a route guidance line generation unit for generating a route guidance line based on destination information corresponding to the request; a route guidance indicator generation unit for generating a route guidance indicator for route guidance in augmented reality using the generated route guidance line; and a display unit for outputting the generated route guidance indicator on a screen via augmented reality. The route guidance indicator generation unit may include: a route guidance line processing unit for correcting the generated route guidance line based on the radius of the actual driving trajectory of the vehicle; a route guidance line stereoscopic unit for varying the height of the corrected route guidance line based on the distance to the vehicle and performing variable stereoscopic transformation; and a texture mapping unit for mapping textures onto the stereoscopic data generated based on variable stereoscopic transformation, thereby generating the route guidance indicator.

[0033] Furthermore, the path guide processing unit can add vertices to maintain the straightness of the forward path guide line of the vehicle, add vertices to the curved section of the path guide line to achieve curvature of the curved section, and use the added vertices to generate a path guide line with a radius reflecting the actual driving trajectory of the vehicle. Moreover, the path guide 3Dization unit generates virtual path guide lines on both sides of the corrected path guide line, calculates the height value of the vertices included in the corrected path guide line in a manner that increases proportionally to the distance to the vehicle, and performs 3Dization on the vertices of the path guide line with the calculated height value and the vertices included in the virtual path guide lines using polygons.

[0034] Furthermore, the aforementioned texture mapping unit can generate a path guidance indicator by mapping a texture with displacement based on the vehicle's speed in the aforementioned 3D data mapping.

[0035] Furthermore, the present invention may also include: a calibration unit that performs calibration for estimating camera parameters corresponding to the camera from the captured images obtained from the camera; and a three-dimensional space generation unit that generates a virtual three-dimensional space from the captured images based on the camera parameters, wherein the display unit may output the generated three-dimensional space in conjunction with the generated path guidance indicator.

[0036] On the other hand, a recording medium of one embodiment of the present invention for achieving the above-mentioned objectives can record program code for executing the control method of the above-mentioned electronic device on a computer.

[0037] According to the various embodiments of the present invention described above, lane line information corresponding to the roadway where the vehicle is located can be generated from image data of the lane line area, and necessary processing can be performed. Thus, various information processing operations, including outputting a lane line interface using the lane line information and generating an augmented reality interface, can be performed.

[0038] Furthermore, according to various embodiments of the present invention, when there are multiple central carriageways in a road (for example, when the number of carriageways is four or more), the carriageway in which a vehicle is located can be accurately determined.

[0039] Furthermore, according to various embodiments of the present invention, the location of the vehicle in the roadway can be determined to guide the driver, thereby enabling the function of assisting the driver.

[0040] Furthermore, according to various embodiments of the present invention, the navigator's route information and the vehicle's lane are used to accurately execute lane change prompts, thereby providing convenience to the user.

[0041] Furthermore, according to various embodiments of the present invention, lane line information can be used to selectively execute appropriate lane change prompts based on the types of lane lines on both sides of the roadway that is currently in motion, thereby improving the performance of lane change prompts and providing richer information.

[0042] Furthermore, according to various embodiments of the present invention, by processing the path guidance lines in three dimensions, a path guidance indicator suitable for augmented reality can be constructed in real time, thereby enabling the realistic and effective display of a three-dimensional path guidance indicator on a two-dimensional camera image. That is, the path guidance lines can be displayed as if they were located on an actual road, rather than displaying the simple path guidance lines of conventional augmented reality navigators. Attached Figure Description

[0043] Figure 1 A block diagram illustrating an embodiment of the electronic device of the present invention.

[0044] Figure 2 This is a diagram illustrating a network used to explain a system connected to an electronic device according to an embodiment of the present invention.

[0045] Figure 3 This is a flowchart illustrating a method for generating lane line information using an electronic device according to an embodiment of the present invention.

[0046] Figure 4 This is a flowchart illustrating a method for generating lane line information using an electronic device according to an embodiment of the present invention.

[0047] Figure 5 This diagram illustrates the grayscale image conversion and lane line region detection process according to an embodiment of the present invention.

[0048] Figure 6 This is a diagram illustrating the region of interest for different lane line types in a grayscale image according to an embodiment of the present invention.

[0049] Figure 7 This diagram illustrates the binary representation and one-dimensional mapping of lane line type interest areas in a grayscale image according to an embodiment of the present invention.

[0050] Figure 8 The flowchart illustrates a method for generating lane location information according to an embodiment of the present invention.

[0051] Figure 9 A diagram illustrating a lane determination table according to an embodiment of the present invention.

[0052] Figure 10 A diagram illustrating a roadway determination table according to another embodiment of the present invention.

[0053] Figure 11 This is a flowchart illustrating a control method for an electronic device according to an embodiment of the present invention.

[0054] Figure 12 This is a block diagram illustrating an embodiment of the augmented reality control unit of the present invention.

[0055] Figure 13 This is a diagram comparing the path guide lines before and after processing.

[0056] Figure 14 This diagram illustrates the three-dimensionalization process of a path guide line according to an embodiment of the present invention.

[0057] Figure 15 This is a flowchart illustrating an embodiment of the augmented reality path guidance method of the present invention.

[0058] Figure 16 This is a diagram illustrating a path guidance screen according to an embodiment of the present invention.

[0059] Figure 17 This diagram illustrates the embodiment of the present invention where the camera and electronic device are separate.

[0060] Figure 18 This diagram illustrates the embodiment of the present invention in which the camera and electronic device are integrated into one unit.

[0061] Figure 19 The figure illustrates the embodiment of a head-up display and electronic device utilizing an embodiment of the present invention. Detailed Implementation

[0062] The following description is merely illustrative of the principles of the invention. Therefore, those skilled in the art can invent various devices that, while not explicitly described or illustrated in this specification, embody the principles of the invention and are included within its concept and scope. Furthermore, all conditional terms and embodiments listed herein are intended to be used in principle to understand the concept of the invention, and should be understood as not limiting the embodiments and states specifically listed as described above.

[0063] Furthermore, all detailed descriptions used to illustrate the principles, viewpoints, and embodiments of the present invention, and to list specific embodiments, should be understood as encompassing structural and functional equivalents of such technical solutions. Moreover, such equivalents should be understood to include not only currently disclosed equivalents but also equivalents to be developed in the future, i.e., all elements invented in a manner that performs the same function regardless of structure.

[0064] Therefore, for example, the block diagrams in this specification should be understood as illustrating the conceptual viewpoint of an exemplary circuit that embodies the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudocode, etc., should be understood as representing various programs executed by means of a computer or processor, whether or not they are substantially displayed on a computer-readable medium, and whether or not a computer or processor is explicitly shown.

[0065] The functionality of a processor, or the various elements shown in the accompanying drawings that include functional blocks illustrated in a similar concept, can be provided not only as dedicated hardware but also as hardware capable of running software, used in conjunction with appropriate software. When the aforementioned functionality is provided by a processor, it can be provided by means of a single dedicated processor, a single shared processor, or multiple individual processors, some of which may be shared.

[0066] Furthermore, the explicit use of terms such as processing, controlling, or similar concepts should not be interpreted in a way that excludes hardware capable of running software, but should be understood in an unrestricted manner to implicitly include digital signal processor (DSP) hardware, read-only memory (ROM), random access memory (RAM), and non-volatile memory for storing software. Other well-known conventional hardware may also be included.

[0067] Within the scope of the invention claims in this specification, structural elements of a mechanism for performing the functions described in the detailed description include combinations of circuit elements performing the aforementioned functions or all methods for performing functions containing software of all forms, such as firmware / microcode, and in combination with appropriate circuits for running the aforementioned software to perform the aforementioned functions. The invention as defined by such claims combines functions provided by methods listed in various ways with those claimed by the invention claims; therefore, any method capable of providing the aforementioned functions should also be understood to be equivalent to the methods mastered from this specification.

[0068] The above-described objectives, features, and advantages will become clearer from the following detailed description in conjunction with the accompanying drawings, thereby enabling those skilled in the art to readily implement the technical concept of the present invention. Furthermore, in the process of describing the present invention, if it is determined that a detailed description of well-known techniques might obscure the main points of the invention, such detailed description will be omitted.

[0069] Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0070] Figure 1 A block diagram illustrating an embodiment of an electronic device according to the present invention. (Refer to...) Figure 1 The electronic device 100 includes all or part of the following: storage unit 110, input unit 120, output unit 130, lane line information generation unit 140, roadway position information generation unit 150, augmented reality supply unit 160, control unit 170, communication unit 180, and detection unit 190.

[0071] Here, electronic device 100 can be embodied in various devices such as smartphones, tablets, laptops, personal digital assistants (PDAs), portable multimedia players (PMPs), smart glasses, augmented reality glasses, navigators, and black boxes that can provide driving-related guidance to vehicle drivers.

[0072] Here, the vehicle's driving status can include various states in which the driver is driving the vehicle, such as the vehicle being parked, the vehicle being driven, and the vehicle being parked.

[0073] Driving-related guidance can include various types of guidance used to assist drivers in driving vehicles, such as route guidance, lane departure guidance, guidance on departure of vehicles ahead, signal change guidance, guidance to prevent collisions with vehicles ahead, guidance on changing lanes, and guidance on the roadway.

[0074] Here, route guidance can include: augmented reality route guidance, which combines the user's location, direction, and other information from an image taken of the front of a moving vehicle to provide route guidance; and two-dimensional (2D) or three-dimensional (3D) navigation, which combines the user's location, direction, and other information from two-dimensional or three-dimensional map data to provide navigation. In this context, route guidance can be interpreted as including not only navigation when the user is driving a vehicle, but also navigation when the user is walking or running.

[0075] Furthermore, lane departure guidance can guide vehicles in motion to determine whether they have deviated from their lane lines.

[0076] Furthermore, the vehicle departure guidance function can guide vehicles in front of a parked vehicle on whether to depart.

[0077] Furthermore, traffic light change guidance can instruct vehicles in front of them whether the traffic light in front of them needs to be changed. For example, guidance can be provided if the light changes from a red light (indicating a stop signal) to a green light (indicating a departure signal).

[0078] Furthermore, guidance to prevent collisions with vehicles ahead can be provided if the distance between a parked or moving vehicle and the vehicle ahead is within a specified distance, in order to prevent a collision with the vehicle ahead.

[0079] Furthermore, lane change guidance can guide vehicles to change lanes from their current lane to another lane in order to guide them to their destination.

[0080] Furthermore, lane guidance can be used to guide vehicles to the lane they are currently in.

[0081] This type of driving-related imagery, capable of providing various guidance options, can be captured in real-time by a camera positioned facing forward of the vehicle. Here, the camera can be integrated with an electronic device 100 placed within the vehicle and capture images of the front of the vehicle. In this case, the camera can be integrated with a smartphone, navigation system, or black box, and the electronic device 100 can receive the images captured by the integrated camera.

[0082] As another example, the camera can be a separate camera placed in the vehicle from the electronic device 100, and it captures images of the front of the vehicle. In this case, the camera can be a separate black box placed facing the front of the vehicle, and the electronic device 100 can receive the images captured by the separately placed black box via wired / wireless communication, or if a storage medium for storing the images captured by the black box is inserted into the electronic device 100, the electronic device 100 can receive the images captured by the black box.

[0083] Based on the above, an electronic device 100 according to an embodiment of the present invention will be described in more detail below.

[0084] The storage unit 110 performs the functions of storing various data and applications required for the operation of the electronic device 100. Specifically, the storage unit 110 can store data required for the operation of the electronic device 100, such as the operating system (OS), pathfinding applications, and map data. Furthermore, the storage unit 110 can store data generated by the operation of the electronic device 100, such as explored path data and received images. Additionally, the storage unit 110 can store information on the positional relationships of multiple signals included in traffic lights, lane determination tables, etc.

[0085] Here, the storage unit 110 can be embodied not only as built-in storage elements such as random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), registers, hard disks, removable disks, memory cards, and universal subscriber identity modules (USIM), but also as detachable storage elements such as universal serial bus memory.

[0086] The input unit 120 performs the function of converting physical inputs from external inputs of the electronic device 100 into specific electrical signals. Here, the input unit 120 may include all or part of the user input unit 121 and the microphone unit 123.

[0087] The user input unit 121 can receive user input such as touch and swipe gestures. Here, the user input unit 121 can be embodied by at least one of various button shapes, a touch sensor that receives touch input, and a proximity sensor that receives proximity gestures.

[0088] The microphone unit 123 can receive the user's voice and sounds generated inside and outside the vehicle.

[0089] The output unit 130 is a device for outputting data from the electronic device 100. Here, the output unit 130 may include all or part of the display unit 131 and the audio output unit 133.

[0090] Display unit 131 is a device that outputs visually recognizable data to electronic device 100. Display unit 131 can be a display unit located on the front of the casing of electronic device 100. Here, display unit 131 can be integrated with electronic device 100 and output visually recognizable data, or it can be provided separately from electronic device 100, such as a head-up display, to output visually recognizable data.

[0091] The audio output unit 133 is a device that outputs data to the electronic device 100 in an auditory manner. The audio output unit 133 can display the data that the electronic device 100 should prompt to the user through a speaker that displays sound.

[0092] The communication unit 180 may be provided to enable the electronic device 100 to communicate with other devices. The communication unit 180 may include all or part of the following: location data unit 181, wireless internet unit 183, broadcast transceiver unit 185, mobile communication unit 186, short-range communication unit 187, and wired communication unit 189.

[0093] The location data unit 181 is a device that obtains location data via a Global Navigation Satellite System (GNSS). A GNSS is a navigation system that calculates the location of a receiving terminal using radio signals received from artificial satellites. Specific examples of GNSS systems can be categorized based on the operating entity, such as the Global Positioning System (GPS), Galileo, GLONASS, COMPASS, Indian Regional Navigation Satellite System (IRNSS), and Quasi-Zenith Satellite System (QZSS). In one embodiment of the present invention, the location data unit 181 of the electronic device 100 obtains location information by receiving signals from a GNSS system providing services to the area where the electronic device 100 is used.

[0094] The Wireless Internet Section 183 is a device for obtaining data or transmitting information by connecting to a wireless internet. The wireless internet that can be connected to via the Wireless Internet Section 183 can include Wireless Local Area Network (WLAN), Wireless Broadband (Wibro), World Interoperability for Microwave Access (WiMAX), High Speed ​​Downlink Packet Access (HSDPA), etc.

[0095] The broadcast transceiver unit 185 is a device for transmitting and receiving broadcast signals through various broadcast systems. Broadcast systems that can be transmitted and received through the broadcast transceiver unit 185 include Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Forward LinkOnly (MediaFLO) proposed by Qualcomm, Digital Video Broadcast Handheld (DVBH), and Integrated Services Digital Broadcast Terrestrial (ISDBT) of Japan. The broadcast signals transmitted and received through the broadcast transceiver unit 185 can include traffic data, lifestyle data, etc.

[0096] Mobile Communications Unit 186 can connect to and communicate with mobile communication networks according to various mobile communication specifications such as 3G, 3rd Generation, 3rd Generation Partnership Project (3GPP), and Long Term Evolution (LTE).

[0097] The short-range communication unit 187 is a device for performing short-range communication. As described above, the short-range communication unit 187 can communicate via Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra WidBand (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity, etc.

[0098] The wired communication unit 189 is an interface device that enables the electronic device 100 to be connected to other devices via a wired connection. The wired communication unit 189 can be a Universal Serial Bus (USB) module that enables communication via a USB port.

[0099] This communication unit 180 can communicate with other devices using at least one of the location data unit 181, wireless internet unit 183, broadcast transceiver unit 185, mobile communication unit 186, short-range communication unit 187, and wired communication unit 189.

[0100] As an example, if the electronic device 100 does not include a camera function, at least one of the short-range communication unit 187 and the wired communication unit 189 can be used to receive images captured by a vehicle camera such as a black box.

[0101] As another example, in the case of communicating with multiple devices, one device can communicate through the short-range communication unit 187, while another device communicates through the wired communication unit 189.

[0102] The detection unit 190 is a device capable of detecting the current state of the electronic device 100. The detection unit 190 may include all or part of the motion detection unit 191 and the light detection unit 193.

[0103] The motion detection unit 191 can detect the movement of the electronic device 100 in three-dimensional space. The motion detection unit 191 may include a three-axis magnetometer and a three-axis accelerometer. The motion data obtained by the motion detection unit 191 and the position data obtained by the position data unit 191 can be combined to calculate a more accurate trajectory of the vehicle with the electronic device 100 attached.

[0104] The light detection unit 193 is a device for measuring the ambient illuminance of the electronic device 100. Using the illuminance data obtained by the light detection unit 193, the brightness of the display unit 195 can be changed in accordance with the ambient brightness.

[0105] The power supply unit 195 is a device for supplying power required for the operation of the electronic device 100 or other devices connected to the electronic device 100. The power supply unit 195 can be a device that receives power from an external power source such as a battery built into the electronic device 100 or a vehicle. Furthermore, depending on the form of power receiving, the power supply unit 195 can be embodied as a wired communication module 119 or as a device that receives power wirelessly.

[0106] On the other hand, the control unit 170 controls the overall operation of the electronic device 100. Specifically, the control unit 170 can control all or some of the storage unit 110, input unit 120, output unit 130, lane line information generation unit 140, roadway position generation unit 150, augmented reality supply unit 160, communication unit 180, and detection unit 190.

[0107] In particular, the control unit 170 can control the lane line information generation unit 140 and the roadway position information generation unit 150 to identify lane line area portions from vehicle driving-related image data, and generate lane line information corresponding to the roadway where the vehicle is located from the image data of the identified lane line area portions, and generate roadway position information of the vehicle using at least one of the generated lane line information and the roadway information of the road where the vehicle is located.

[0108] Here, lane lines can mean the lines on both sides of the roadway that forms the lane where the vehicle is located. Furthermore, the roadway can be formed by lane lines such as first lane, second lane, ... Nth lane, and can mean the road on which vehicles travel.

[0109] The lane line information generation unit 140 can identify lane line areas from image data captured during driving states such as vehicle movement and parking, and can generate lane line information corresponding to each lane line located on both sides of the roadway based on the vehicle's location, from the image data of the lane line areas. The lane line information may include lane line type information and lane line color information corresponding to each lane line located on both sides of the roadway based on the vehicle's location.

[0110] Here, in order to generate lane line information corresponding to the lane where the vehicle is located, the lane line information generation unit 140 can perform binary conversion on the image data of the lane line area and obtain lane line type information from the binary partial image data. Specifically, the lane line information generation unit 140 can analyze the binary partial image data using at least one of lane line temporal continuity information and lane line speed information, thereby identifying whether the two lane lines of the lane where the vehicle is located are solid lines or dashed lines.

[0111] Furthermore, the lane line information generation unit 140 can extract color information corresponding to each lane line whose type has been identified from the image data to generate lane line information.

[0112] Under the control of the control unit 170, the lane position information generation unit 150 can generate lane position information of the vehicle using at least one of the lane line information generated by the lane line information generation unit 140 and the lane information of the road where the vehicle is located.

[0113] Specifically, the lane position information generation unit 150 obtains the lane information of the road where the vehicle is located from map data, and uses the generated lane line information to determine whether the vehicle is located in the first or last lane of the road. If the vehicle is located in the first or last lane, the lane position information of the road can be reflected to generate the lane position information of the vehicle. Then, if the vehicle changes lanes to a lane between the first and last lanes, the lane position information generation unit 150 updates the generated lane position information to the changed lane position information. Then, if the vehicle changes lanes from a lane between the first and last lanes to the first or last lane, the lane position information generation unit 150 regenerates the lane position information of the vehicle based on the lane information of the road where the vehicle is located.

[0114] Here, the roadway information of the road where the vehicle is located may include information on the number of roadways and road type (e.g., highway, urban expressway, local road, ordinary road), etc. The roadway information can be obtained from map data stored in the storage unit 110 within the electronic device 100, or from other external map databases (DBs) outside the electronic device 100, or from other electronic devices 100. As an example, if the electronic device 100 is a black box, the black box can obtain the roadway information from an external navigation device that is communicatively connected to the black box.

[0115] On the other hand, since lane marking information used by different countries varies, the lane position information generation unit 150 can generate lane position information using a lane determination table based on the traffic regulations of each country. Therefore, the lane position information generation unit 150 can generate lane position information based on a lane determination table mapped to pre-set country information.

[0116] On the other hand, the control unit 170 can use the lane line information generated by the lane line information generation unit 140 and the roadway position information generation unit 150 to perform driving-related guidance for the vehicle.

[0117] As an example, the control unit 170 can select appropriate lane departure guidance based on the types of lane lines on both sides of the roadway where the vehicle is located, identified based on lane line information, and output it to the user. Specifically, when the electronic device 100 provides lane departure guidance, the control unit 170 can provide different guidance based on the type and color of the lane line being departed. For example, the control unit 170 can select different images or prompts and output them based on whether the vehicle has crossed the center line, crossed the solid white line, crossed the dashed white line, or crossed the blue line.

[0118] As another example, the control unit 170 can utilize the lane position information generated by the lane position information generation unit 150 and output the generated lane guidance through the output unit 140. Specifically, the control unit 170 can output the lane number of the current vehicle in the lane in the form of images or voice, for example, lane 1, lane 2, ... lane N.

[0119] As another example, the control unit 170 can utilize the lane position information generated by the lane position information generation unit 150 and output the generated lane change guidance through the output unit 140. Specifically, when the electronic device 100 provides vehicle navigation functionality, the control unit 170 can output lane change guidance in the form of images or voice prompts based on the route to the destination and the determined lane position. That is, if the distance to the left or right turn guidance point is less than a predetermined distance, it can determine whether a left or right turn is possible in the current lane, and thus output lane change guidance to the user.

[0120] On the other hand, the control unit 170 can control the augmented reality supply unit 160, enabling the electronic device 100 to perform driving-related guidance based on augmented reality. Here, augmented reality can be a method of providing additional information (e.g., graphic elements indicating points of interest (POIs), graphic elements indicating the path to the destination, etc.) in a visually superimposed manner onto a view of the real world actually seen by the user. In this case, the control unit 170 and the augmented reality supply unit 160 work together to generate indicators for performing driving-related guidance, and output the generated indicators through the output unit 130. As an example, augmented reality can be provided using an image overlay on a head-up display on the vehicle's windshield or other image output devices; in this way, the augmented reality supply unit 160 can generate real-world images or interface images superimposed on the glass. Thus, an augmented reality navigation system or vehicle information system can be implemented.

[0121] In particular, according to one embodiment of the present invention, a path guidance indicator suitable for augmented reality can be constructed in real time through the three-dimensional processing of the path guidance line, thereby realistically and effectively displaying the three-dimensional path guidance indicator on the two-dimensional camera image. This will be described later with reference to the accompanying drawings.

[0122] Figure 2 This is a diagram illustrating a network used to explain a system connected to an electronic device according to an embodiment of the present invention. (Refer to...) Figure 2 The electronic device 100 of one embodiment of the present invention may be embodied as a navigator, a black box, a smartphone or other vehicle augmented reality interface supply device, etc. installed in a vehicle, and may be connected to various communication networks and other electronic devices 61, 62, 63, 64.

[0123] Furthermore, the electronic device 100 can coordinate with the Global Positioning System based on the radio signals received from the artificial satellite 20, thereby calculating the current location and the current time.

[0124] Each artificial satellite 20 can transmit L-band frequencies in different frequency bands. The electronic device 100 can calculate its current position based on the time required for the L-band frequencies transmitted from each artificial satellite 20 to reach the electronic device 100.

[0125] On the other hand, the electronic device 100 can wirelessly connect to the network 30 via the communication unit 180 and by means of the control station 40 (ACR), base station 50 (RAS), etc. If the electronic device 100 is connected to the network 30, it can also connect to other electronic devices 61 and 62 that are indirectly connected to the network 30 and exchange data.

[0126] On the other hand, the electronic device 100 can also be indirectly connected to the network 30 via other devices 63 that have communication functions. For example, if the electronic device does not have a module that can be connected to the network 30, it can communicate with other devices 63 that have communication functions through short-range communication or the like.

[0127] Figure 3 This is a flowchart illustrating a lane line information generation method of an electronic device according to an embodiment of the present invention. (Refer to...) Figure 3First, the electronic device 100 can identify lane line areas from the vehicle's driving-related image data (step S101). Specifically, the lane line information generation unit 140 converts the driving-related images into grayscale images and executes a lane line detection algorithm to determine the identifiable areas in each lane line located on both sides of the vehicle as lane line areas. Here, the vehicle's driving-related images may include images related to the vehicle's parking and driving. Moreover, the vehicle's driving-related images may be images received by the electronic device 100 and captured by a camera module included in the electronic device 100, or images captured by other devices. Furthermore, the vehicle's driving-related images may be red-green-blue (RGB) color images.

[0128] Furthermore, the electronic device 100 can generate lane line information corresponding to the lane where the vehicle is located from the image data of the identified lane line area (step S102). Specifically, the lane line information generation unit 140 can generate lane line information by analyzing the lane line pattern information and lane line color information of the detected lane line area. The lane line information may include at least one of line type information and line color information corresponding to each lane line located on both sides of the lane where the vehicle is located.

[0129] The following is for reference Figures 4 to 7 This section will provide a more detailed explanation of the lane line information generation method.

[0130] Figure 4 A flowchart illustrating a lane line information generation method of an electronic device according to an embodiment of the present invention is provided. (Refer to...) Figure 4 First, the electronic device 100 converts the color image data into a grayscale image (step S201), and can detect lane line regions from the converted grayscale image (step S202).

[0131] Specifically, the lane line information generation unit 140 can extract the area for detecting lane lines from the captured driving-related images. Furthermore, if a part of the road is affected by shadows, it is difficult for the lane line information generation unit 140 to detect lane lines. Therefore, in order to minimize the influence of shadows, the light source of the original image can be corrected in advance.

[0132] Furthermore, the lane line information generation unit 140 can detect areas where lane lines may exist as lane line areas based on the pre-set position or angle of the camera. For example, the lane line information generation unit 140 can determine the lane line area by using the starting point of the lane line. Moreover, the lane line information generation unit 140 can estimate the starting position and length of the lane line area based on the width of the roadway in the driving-related image (the maximum width between the left and right lane line areas) and the camera's viewing angle.

[0133] Furthermore, the lane line information generation unit 140 converts the grayscale image corresponding to the lane line detection area into an edge image, and can detect the lane line area based on the straight line position extracted from the converted edge image. More specifically, the driving-related image can be converted into an edge image using various known algorithms, and the edge image may include the edges displaying multiple straight lines. In this case, the lane line information generation unit 140 can identify the position of the detected straight line as a lane line. Furthermore, the lane line information generation unit 140 can determine the lane line area from multiple candidate straight lines based on the position of a straight line having a predetermined lane line width.

[0134] Figure 5 This illustrates the grayscale image conversion and lane line region detection process. (Refer to...) Figure 5 The system can convert the initially input driving-related images into grayscale images 200, and use edge detection and lane line detection algorithms to detect straight lane line regions 201 and 202. The lane line regions can be divided into left lane line region 201 and right lane line region 202 based on the vehicle's position.

[0135] Re-examine Figure 4 Please provide an explanation.

[0136] Subsequently, if a lane line area is detected, the lane line information generation unit 140 can set a lane line type interest region based on the lane line area (step S203). Specifically, if a lane line area is detected, the lane line information generation unit 140 can set a lane line type interest region (ROI) based on the detected lane line area. The lane line type interest region may mean a portion of the driving-related image that includes the lane line used to determine the type and color of the lane line and the surrounding defined area of ​​the lane line.

[0137] More specifically, Figure 6 This indicates the area of ​​interest for different lane line types in a grayscale image.

[0138] like Figure 6 As shown, the lane marking type interest areas 210 and 220 may include a portion of the previously detected lane marking area and its surrounding area. Furthermore, the lane marking type interest areas can be divided into a left lane marking type interest area 210 and a right lane marking type interest area 220 based on the vehicle's direction of travel.

[0139] For example, the lane line regions detected previously are typically straight lines. When expressed mathematically as y = a × x + b, the lane line type interest area can be represented as a region that includes both y = a × x + b + m and y = a × x + b - m. This differs from the previous simple lane line detection method; it is a method used to generate specific and diverse driving lane line information. Therefore, the lane line information generation unit 140 can expand the detected straight lane line regions to also set the surrounding area of ​​the straight lane line regions as interest areas.

[0140] Re-examine Figure 4 Please provide an explanation.

[0141] Subsequently, the electronic device 100 can perform binary conversion on the lane line type interest area (step S204), map the grayscale image of the binary portion into a one-dimensional region (step S205), and use at least one of visual continuity and speed to identify the type of line (step S206).

[0142] The lane line information generation unit 140 can extract a portion of the grayscale image of the lane line type interest area from the converted grayscale image, and can perform binary conversion on the portion of the grayscale image. The reference value used for binary conversion can be determined based on the average grayscale value of the portion of the grayscale image of the interest area. As a result, the driving lane line information generation module 180 can clearly distinguish the portion of the grayscale image that is only judged to be a lane line.

[0143] Furthermore, the lane line information generation unit 140 can map each line (left and right sides) identified from the binary partial grayscale image into a one-dimensional region. Moreover, by analyzing the pattern of each line mapped into a one-dimensional region, the type of line can be identified.

[0144] More specifically, Figure 7 This represents the binary representation and one-dimensional mapping of the lane line type interest area in a grayscale image.

[0145] like Figure 7 As shown, if the area of ​​interest for lane line types is binaryized, a binaryized image 300 can be obtained. In the binaryized image 300, the parts displayed in white can be identified as lane lines, while the rest can be identified as black.

[0146] Furthermore, each line identified in the binary image 300 can be mapped to a one-dimensional region. The lane line information generation unit 140 can easily determine the type of line using the image 310 mapped to a one-dimensional region.

[0147] For example, the lane line information generation unit 140 can determine whether a line is dashed or solid based on the starting point and length characteristics of each line mapped to one dimension. Furthermore, the lane line information generation unit 140 can determine whether a line is dashed or solid by utilizing the continuity of time and speed of each line mapped to one dimension. Moreover, the lane line information generation unit 140 can first determine whether a line is dashed or solid based on the aforementioned starting point and length characteristics, and then ultimately determine whether a line is dashed or solid by utilizing the continuity of time and speed.

[0148] More specifically, the lane line information generation unit 140 can first determine whether a line is dashed or solid by comparing the lengths of the lines at the starting points of each line. In this case, the lane line information generation unit 140 can determine whether a line is dashed or solid using only one image frame.

[0149] Furthermore, the lane line information generation unit 140 can more clearly determine whether a line is dashed or solid based on whether each line is formed continuously over time. For example, the lane line information generation unit 140 can preset the continuity of the line's movement speed within the image, and determine that a line is dashed if the continuity of each line is less than the preset continuity value.

[0150] Therefore, according to an embodiment of the present invention, it is possible to distinguish whether a line is dashed or solid in advance using a single frame, and to verify this using consecutive frames, thereby ultimately determining the type of line.

[0151] Re-reference Figure 4 The electronic device 100 can detect the color of the aforementioned line portion from the original color image data to obtain the type of identification (step S207).

[0152] The lane line information generation unit 140 can analyze color images to detect the color of the portion corresponding to the previously identified type of line and classify it accordingly. For example, the lane line information generation unit 140 can classify the detected color as white, yellow, or blue.

[0153] Then, the electronic device 100 can generate lane line information corresponding to the roadway where the vehicle is located based on the type of line identified and the color classified (step S208).

[0154] Figure 8 A flowchart illustrating a method for generating lane location information according to an embodiment of the present invention is provided. (Refer to...) Figure 8The electronic device 100 can obtain the lane information of the road where the vehicle is located from map data (step S301). Here, the lane information can be the lane information of the road where the vehicle is currently located, and may include information about the number of lanes on the road where the vehicle is located. Moreover, the lane information can be obtained from map data stored in the storage unit 110 within the electronic device 100, or from other external map databases (DBs) outside the electronic device 100, or from other electronic devices 100.

[0155] Furthermore, the electronic device 100 can use the generated lane line information to determine whether the vehicle is located in the first lane or the last lane of the road (step S302). Specifically, the lane position information generation unit 150 can, for example, Figure 9 The lane determination table shown uses lane line information corresponding to the lane where the vehicle is located to determine whether the vehicle is in the first or last lane of the road.

[0156] That is, the lane determination table may include the first and last lanes determined by the country, the type and color of the left lane, and the type and color of the right lane. Here, for example... Figure 10 The lane determination table shown is illustrative and can be set with different values ​​depending on different settings, countries, or situations.

[0157] On the other hand, when a vehicle is located in the first or last lane, the electronic device 100 can reflect the lane information to generate lane position information of the vehicle (step S303). For example, if it is determined that the vehicle is located in the last lane, the lane position information generation unit 150 can generate the lane position information as the Nth lane. Moreover, if there are 5 lanes corresponding to the road lane information, this can be reflected, and the Nth lane can be generated as the fifth lane.

[0158] Furthermore, if the vehicle's lane changes to a lane between the first and last lanes due to a lane change, the electronic device 100 can update the generated lane position information to reflect the changed lane position (step S304). In this case, the lane position information generation unit 150 can use lane line information to determine whether the vehicle has deviated from the lane lines and, based on this, determine whether the lane has been changed. For example, if it is determined that the vehicle has changed lanes to the left from the fifth lane, the lane position information generation unit 150 will respond accordingly and update the lane position information from the fifth lane to the fourth lane.

[0159] Furthermore, if the vehicle's position changes from the lane between the first and last lanes to either the first or last lane based on a lane change, the electronic device 100 can re-obtain the lane information of the road where the vehicle is located (step S305). Moreover, reflecting the re-obtained lane information, the lane position information of the vehicle can be regenerated (step S306). For example, if it is determined that the vehicle has changed lanes from the fourth lane to the right lane, it moves to the fifth lane, which is the previously set last lane, thereby obtaining the lane information of the road where the vehicle is currently located. Furthermore, if the obtained lane information is the fourth lane, the lane position information of the vehicle can be regenerated as the fourth lane.

[0160] On the other hand, the method for generating lane location information according to an embodiment of the present invention is not limited to the above. Figure 9 Therefore, the above-described order can be partially modified according to another embodiment. As an example, the step of obtaining the lane information of the road where the vehicle is located can be performed in step S304. In this case, if the vehicle is located in the first lane or the last lane, the electronic device 100 can generate lane location information of the vehicle (step S303). For example, if it is determined that the vehicle is located in the last lane, the lane location information generation unit 150 can generate the lane location information as the Nth lane.

[0161] Furthermore, if the vehicle changes lanes to a lane between the first and last lanes, the electronic device 100 can update the lane position information using the generated lane position information and the obtained road lane information (step S304). For example, if it is determined that the vehicle changes lanes from the Nth lane corresponding to the last lane to the left lane, the lane position information generation unit 150 reflects N=5 as the number of lanes for the (N-1)th lane, thereby updating the lane position information to the fourth lane.

[0162] Furthermore, according to another embodiment of the present invention, the lane location information generation unit 150 can, as shown in the example... Figure 10 The lane determination table shown applies lane line information corresponding to the lane where the vehicle is located to determine whether the vehicle is in the first lane, middle lane, or last lane of the road. However, in cases where there are multiple middle lanes (e.g., more than four lanes), it is impossible to accurately determine the lane where a vehicle is located among multiple middle lanes. Therefore, preferably, one embodiment of the present invention can use... Figure 9 The method shown.

[0163] Figure 11A flowchart illustrating a control method for an electronic device according to an embodiment of the present invention. (Refer to...) Figure 11 First, the electronic device 100 can identify the lane line area from the vehicle's driving-related image data (step S401).

[0164] Furthermore, lane line information corresponding to the roadway where the vehicle is located can be generated from the image data of the identified lane line area (step S402).

[0165] Furthermore, at least one of the generated lane line information and the carriageway information of the road where the vehicle is located can be used to generate the carriageway position information of the vehicle (step S403).

[0166] Furthermore, the obtained lane information is used to perform driving-related guidance for the vehicle (step S404).

[0167] Here, the steps for performing driving-related guidance for the vehicle (step S404) may include the step of outputting lane change guidance using the vehicle's navigation path and lane location information.

[0168] Furthermore, the step of performing driving-related guidance for the vehicle (step S404) may include the step of using the lane location information to output the lane guidance where the vehicle is located.

[0169] On the other hand, the control method of the electronic device according to an embodiment of the present invention may further include the step of selecting and outputting appropriate lane departure guidance based on the type of lane lines on both sides of the vehicle's carriageway identified based on lane line information.

[0170] Here, the above output can be executed by generating indicators for performing driving-related guidance and by using augmented reality output.

[0171] On the other hand, previous augmented reality navigators have used a variety of technologies for path guidance, but due to the difficulty in synthesizing the path guidance lines with the actual road environment, there are limitations in the representation of the path guidance lines.

[0172] However, according to an embodiment of the present invention, a path guidance indicator suitable for augmented reality can be constructed in real time through the three-dimensional processing of the path guidance line, thereby enabling the realistic and effective display of a three-dimensional path guidance indicator on a two-dimensional camera image. Hereinafter, an augmented reality supply unit according to an embodiment of the present invention for achieving the above-mentioned objective will be specifically described.

[0173] Figure 12 A block diagram illustrating an embodiment of the augmented reality control unit 160 of the present invention. (Refer to...) Figure 12The augmented reality supply unit 160 may include all or part of the calibration unit 161, the three-dimensional space generation unit 162, the indicator generation unit 163, and the mapping unit 164.

[0174] The calibration unit 161 can perform calibration to estimate camera parameters corresponding to the camera from the captured image obtained from the camera. Here, the camera parameters can be parameters constituting a camera matrix, which represents information indicating the correspondence between real space and the photograph.

[0175] The 3D space generation unit 162 can generate a virtual 3D space based on images captured by a camera. Specifically, the 3D space generation unit 162 can obtain depth information from images captured by a camera based on camera parameters estimated by the calibration unit 161, and can generate a virtual 3D space based on the obtained depth information and the captured images.

[0176] The indicator generation unit 163 can generate indicators for guidance in augmented reality, such as path guidance indicators, lane change guidance indicators, and lane departure guidance indicators.

[0177] In particular, if the indicator generation unit 163 receives a path guidance request for the destination input by the user through the input unit 120, a path guidance indicator for path guidance can be generated on the augmented reality. Here, the path guidance indicator generation unit may include a path guidance line processing unit 163-1, a path guidance line stereoscopic unit 163-2, and a dynamic texture mapping unit 163-3.

[0178] If a path guide line to the destination is generated based on the user's path guidance request, the path guide line processing unit 163-1 can perform path guide line processing based on the radius of the actual driving trajectory of the vehicle.

[0179] Specifically, the electronic device 100 can generate a path guidance line up to the destination based on the user's path guidance request, using map data obtained from the storage unit 110 and map data obtained from other external map databases besides the electronic device 100. Here, the generated path guidance line may include nodes and connecting lines, and can be, for example... Figure 13 The shape shown in part (a). That is, referring to... Figure 13 In part (a), the generated path guide line 1301 can be a straight line in the curved section 1302 that is not similar to the vehicle's trajectory. Therefore, if based on... Figure 13 The path guide lines shown in part (a) are used to synthesize with the camera images and provide augmented reality, resulting in a display that is different from the actual vehicle's driving trajectory.

[0180] Therefore, the path guidance line processing unit 163-1 of one embodiment of the present invention can perform path guidance line processing in a manner that reflects the radius of the actual driving trajectory of the vehicle. Specifically, the path guidance line processing unit 163-1 can remove unnecessary vertices such as vertices corresponding to areas not displayed on the current screen and duplicate points from the generated path guidance line. Moreover, the path guidance line processing unit 163-1 can add vertices to maintain the straightness of the forward path guidance line of the vehicle. Furthermore, the path guidance line processing unit 163-1 can add vertices to the curved sections of the path guidance line to achieve curvature of the curved sections. Moreover, the path guidance line processing unit 163-1 can use the added vertices to generate a path guidance line that reflects the radius of the actual driving trajectory of the vehicle.

[0181] Thus, according to one embodiment of the present invention, it is possible to generate such a... Figure 13 The path guide line shown in section (b). That is, refer to... Figure 13 In part (b), the path guide line 1303 processed can be a gentle shape similar to the vehicle's trajectory in the curved section 1304.

[0182] The path guide line 3D modeling unit 163-2 can perform variable 3D modeling by varying the height of the path guide line generated by the processing unit 163-1 according to the distance. For this, refer to... Figure 14 Let me explain in detail.

[0183] Reference Figure 14 The path guide line 3D generation unit 163-2 can generate virtual path guide lines 1402 and 1403 on both sides of the path guide line 1401 generated by the path guide line processing unit 163-1. Specifically, the path guide line 3D generation unit 163-2 can calculate the unit vector of the vertex of the path guide line 1401 generated by the path guide line processing unit 163-1, and calculate the normal vector perpendicular to the unit vector by means of the internal operation of the calculated unit vector, thereby generating virtual path guide lines 1402 and 1403.

[0184] Furthermore, the path guide line 3Dization unit 163-2 can calculate the height value of the vertices included in the path guide line 1401. In this case, the path guide line 3Dization unit 163-2 can calculate the height value in such a way that the height value of the vertices included in the path guide line 1401 increases proportionally to the distance.

[0185] Furthermore, the path guide line stereoforming unit 163-2 can generate faces by using polygons at the vertices of the path guide line 1401 whose height value has been calculated and the vertices included by the virtual path guide lines 1402 and 1403, thereby enabling stereoforming.

[0186] Therefore, the route guidance sign located far from the current lane can be displayed on the screen in a way that is recognizable to the driver.

[0187] The dynamic texture mapping unit 163-3 can map the 3D data generated by the path-guided 3D mapping unit 163-2 to a texture with displacement according to the vehicle's speed.

[0188] Here, the texture can be a texture that is displaced according to the vehicle's speed. That is, when the vehicle moves along the path, the dynamic texture mapping unit 163-3 changes the mapping position of the texture in the stereo data, thereby generating a texture that is displaced according to the vehicle's speed. In this case, the path guidance indicator displayed on the screen can maximize the effect of appearing to be closely attached to the road surface.

[0189] Based on this work, the indicator generation unit 163 can generate path guidance indicators for path guidance in augmented reality.

[0190] On the other hand, the display technology that has a turning point and passes through the aforementioned turning point can verticalize the three-dimensional part 163-2 generated according to the above-described operation in order to display the image more effectively.

[0191] That is, when turning left, the path guide line 3D unit 163-2 can erect a virtual path guide line 1403 corresponding to the right side of the path guide line 1401 in the vertical direction to determine the 3D data. Conversely, when turning right, the path guide line 3D unit 163-2 can erect a virtual path guide line 1402 corresponding to the left side of the path guide line 1401 in the vertical direction to determine the 3D data.

[0192] On the other hand, the mapping unit 164 can combine the indicator generated by the indicator generation unit 163 on the virtual three-dimensional space generated by the three-dimensional space generation unit 162.

[0193] Figure 15 This is a flowchart illustrating an embodiment of an augmented reality path guidance method of the present invention. (Refer to...) Figure 15 First, the electronic device 100 can receive input from a user requesting route guidance (step S501).

[0194] Furthermore, the electronic device 100 can generate a path guidance line based on the destination information of the path guidance (step S502).

[0195] Furthermore, the electronic device 100 can correct the generated path guide line by reflecting the radius of the actual driving trajectory of the vehicle (step S503). Specifically, step S503 may include: removing unnecessary vertices such as vertices and duplicate points corresponding to areas not displayed on the current screen from the generated path guide line; adding vertices to maintain the straightness of the forward path guide line of the vehicle; adding vertices to the curved section of the path guide line to achieve curvature of the curved section; and using the added vertices to generate a path guide line that reflects the radius of the actual driving trajectory of the vehicle.

[0196] Furthermore, the electronic device 100 can perform variable stereoscopic transformation (step S504) by differentiating the height of the corrected path guide line based on the distance to the vehicle. Specifically, step S504 may include: generating virtual path guide lines on both sides of the processed path guide line; calculating the height value in such a way that the height value of the vertices included in the processed path guide line increases proportionally to the distance; and generating faces using polygons at the vertices of the path guide line whose height values ​​have been calculated and the vertices included in the virtual path guide lines, thereby performing stereoscopic transformation.

[0197] Furthermore, the electronic device 100 can generate a path guidance indicator by mapping a texture with displacement based on the vehicle's speed in a 3D data mapping (step S505).

[0198] Furthermore, the electronic device 100 can use augmented reality to display a path guide indicator on the screen (step S506). Here, reference will be made to... Figure 16 Provide a detailed description of the output screen.

[0199] Figure 16 This is a diagram illustrating a path guidance screen according to an embodiment of the present invention. (Refer to...) Figure 16 In one embodiment of the present invention, the electronic device 100 can simultaneously display an augmented reality path guidance screen (left side screen) and a path guidance screen on a map (right side screen).

[0200] In this case, in order to provide guidance on augmented reality, the augmented reality supply unit 160 can generate indicators superimposed on augmented reality.

[0201] As an example, such as Figure 16 As shown, the augmented reality supply unit 160 can generate a path guidance indicator 1601, a lane change guidance indicator 1602, and a lane departure guidance indicator 1603. Furthermore, the augmented reality supply unit 160 can output the generated indicators to the augmented reality.

[0202] Figure 17This diagram illustrates a configuration where the camera and electronic device are separate components according to an embodiment of the present invention. (Refer to...) Figure 17 The vehicle navigation device 100 and the vehicle black box 200, which are respectively installed, can be configured into a system according to an embodiment of the present invention using wired / wireless communication methods.

[0203] The vehicle navigation system 100 may include: a display unit 145 disposed on the front surface of the housing 191 of the navigation system; navigation operation keys 193; and a navigation microphone 195.

[0204] The vehicle black box 200 can acquire vehicle data during both driving and parking. That is, it can capture images not only while the vehicle is in motion but also when the vehicle is parked. The image resolution obtained by the vehicle black box 200 can be constant or variable. For example, the image resolution can be high before and after an accident, while under normal circumstances, the resolution can be reduced, thereby minimizing the required storage space and storing critical images.

[0205] The vehicle black box 200 may include a black box camera 222, a black box microphone 224, and an attachment 281.

[0206] On the other hand, although Figure 17 The vehicle navigation system 100 and the separately installed vehicle black box 200 are shown to be interconnected via wired / wireless communication. However, the vehicle navigation system 100 and the vehicle black box 200 can also be connected without wired / wireless communication. In this case, if a storage medium for storing images captured by the black box 200 is inserted into the electronic device 100, the electronic device 100 can receive the captured images. On the other hand, the vehicle black box 200 can also have the functions of the vehicle navigation system 100, or the vehicle navigation system 100 can be integrated with a camera. For this, refer to... Figure 18 Let me explain in detail.

[0207] Figure 18 This diagram illustrates an embodiment of the present invention where the camera and electronic device are integrated. (Refer to...) Figure 18 When an electronic device includes a camera function, the user can use the camera portion of the electronic device to photograph the front of the vehicle, and the user can position the electronic device in a manner that allows them to identify the display portion of the electronic device. Thus, a system according to an embodiment of the present invention can be demonstrated.

[0208] Figure 19 This figure illustrates an embodiment of a head-up display and electronic device utilizing an embodiment of the present invention. (Refer to...) Figure 19Electronic devices can connect to the head-up display via wired / wireless communication and display augmented reality guide screens on the head-up display.

[0209] On the other hand, the control method of the electronic device, which can embody the various embodiments of the present invention described above, can be provided to various servers or devices in a state of being stored in various non-transitory computer readable media.

[0210] Non-transitory readable media do not mean short-term data storage media such as registers, caches, or memory, but rather media that store data semi-permanently and can be read by a device. Specifically, the various applications or programs mentioned above can be provided on non-transitory readable media such as optical discs (CDs), digital multifunction optical discs (DVDs), hard disks, Blu-ray discs, universal serial buses, memory cards, and read-only memory.

[0211] Furthermore, while the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. Without departing from the spirit of the present invention as claimed in the invention claims, various modifications can be made to the present invention by those skilled in the art, and these modifications should not be understood separately from the technical concept or prospects of the present invention.

Claims

1. A method for controlling an electronic device, characterized in that, include: The steps to determine the type and color of lane markings on both sides of the roadway from driving-related video data of the vehicle; The step of comparing the type and color of the lane lines on both sides based on the above determination with the stored lane determination table to determine the lane position information of the vehicle in motion. as well as If there is a turning point on the path ahead of the aforementioned vehicle at a predetermined distance, the determined lane position information is compared with the path direction at the turning point to determine whether a lane change for the aforementioned vehicle is necessary. The aforementioned stored lane determination table defines lane markings based on the type and color of the lane lines on both sides of a vehicle. The control method further includes the following steps: The route guidance indicators for the aforementioned vehicles and the lane change guidance indicators indicating that the aforementioned vehicles need to change lanes are displayed in the augmented reality image; Receives input from the user requesting route guidance; Generate a path guide based on the destination information corresponding to the request; The generated path guide line is corrected by reflecting the radius of the actual driving trajectory of the vehicle. Variable stereoscopic transformation is performed by varying the height of the corrected path guide lines based on the distance between the guide lines and the vehicle itself. Texture mapping is performed on the 3D data generated based on variable stereoscopic modeling, and path guidance indicators are generated. The steps involved in performing variable stereoscopic transformation include: The step of generating virtual path guide lines on both sides of the corrected path guide line; The step of calculating the height value in such a way that the height value of the vertices included in the corrected path guide line increases proportionally to the distance to the vehicle; and The step of stereoforming is performed on the vertices of the path guide line whose height value is calculated using polygons, and on the vertices of each virtual path guide line included on both sides of the corrected path guide line.

2. The control method for the electronic device according to claim 1, characterized in that, The steps for determining whether a lane change for the aforementioned vehicles is necessary include: The steps to obtain route information to the destination from map data; The steps to obtain the location information of the aforementioned vehicles; and The step of detecting turning points within a specified distance from the vehicle's location using the vehicle location information and path information obtained above.

3. The control method for the electronic device according to claim 1, characterized in that, It also includes the following steps: if it is determined that the above-mentioned lane change is required, a first lane change guidance indicator is displayed in the augmented reality image, which indicates that a lane change is required.

4. The control method for the electronic device according to claim 1, characterized in that, The process also includes the following steps: if it is determined that a lane change is necessary, a second lane change guidance indicator is displayed in the augmented reality image, indicating that a lane change is required. The aforementioned second lane change guidance indicator starts from the lane the vehicle was in before the lane change and ends at the lane it was in after the lane change.

5. The control method for the electronic device according to claim 1, characterized in that, It also includes the following steps: obtaining the number of carriageways in which the above-mentioned vehicles are traveling from the map data. The steps for determining whether a lane change for the aforementioned vehicles is necessary include: If a vehicle needs to change lanes while it is traveling in the first or last lane, the number of lanes to be changed is determined based on the number of lanes mentioned above.

6. An electronic device, characterized in that, include: The lane line information generation unit determines the type and color of the lane lines on both sides of the roadway from the vehicle's driving-related image data. The lane position information generation unit compares the type and color of the lane lines on both sides based on the above determination with a stored lane determination table to determine the lane position information of the vehicle in motion; and If a turning point exists on the path ahead of the vehicle at a predetermined distance, the control unit compares the determined lane position information with the path direction at the turning point to determine whether a lane change is necessary for the vehicle. The aforementioned stored lane determination table defines lane markings based on the type and color of the lane lines on both sides of a vehicle. The electronic device also includes a display unit that displays the route guidance indicator for the vehicle and the lane change guidance indicator indicating that the vehicle needs to change lanes in the augmented reality image. The input section receives input from users requesting route guidance; The path guidance line generation unit generates a path guidance line based on the destination information corresponding to the request; The path guidance line processing unit corrects the generated path guidance line by reflecting the radius of the actual driving trajectory of the vehicle. The path guide line three-dimensionalization unit performs variable three-dimensionalization by adjusting the height of the corrected path guide line based on the distance between it and the vehicle. The texture mapping unit maps textures onto stereo data generated based on variable stereo modeling and generates path guidance indicators. The path guide line stereoscopic unit generates virtual path guide lines on both sides of the corrected path guide line, calculates the height value of the vertices included in the corrected path guide line in such a way that the height value increases proportionally to the distance to the vehicle, and performs stereoscopic transformation on the vertices of the path guide line whose height value is calculated and the vertices in each virtual path guide line included on both sides of the corrected path guide line using polygons.

7. The electronic device according to claim 6, characterized in that, The control unit obtains the path information up to the destination from the map data, obtains the location information of the vehicle, and uses the obtained vehicle location information and the obtained path information to detect turning points within a specified distance from the vehicle's location.

8. The electronic device according to claim 6, characterized in that, It also includes a display unit that, when it is determined that the above-mentioned lane change is required, displays a first lane change guidance indicator in the augmented reality image, which indicates that a lane change is required.

9. The electronic device according to claim 6, characterized in that, It also includes a display unit that, when it is determined that a lane change is required, displays a second lane change guidance indicator in the augmented reality image. This second lane change guidance indicator indicates that a lane change is required. The aforementioned second lane change guidance indicator starts from the lane the vehicle was in before the lane change and ends at the lane it was in after the lane change.

10. The electronic device according to claim 6, characterized in that, The aforementioned control unit obtains the number of lanes in the roadway where the aforementioned vehicles are traveling from the map data. If a vehicle needs to change lanes while it is traveling in the first or last lane, the control unit determines how many lanes need to be changed based on the number of lanes.

11. A computer-readable recording medium for performing a control method for an electronic device, characterized in that the control method comprises: The steps to determine the type and color of lane markings on both sides of the roadway from driving-related video data of the vehicle; The step of comparing the type and color of the lane lines on both sides based on the above determination with the stored lane determination table to determine the lane position information of the vehicle in motion. as well as If there is a turning point on the path ahead of the aforementioned vehicle at a predetermined distance, the determined lane position information is compared with the path direction at the turning point to determine whether a lane change for the aforementioned vehicle is necessary. The aforementioned stored lane determination table defines lane markings based on the type and color of the lane lines on both sides of a vehicle. The control method further includes the following steps: The route guidance indicators for the aforementioned vehicles and the lane change guidance indicators indicating that the aforementioned vehicles need to change lanes are displayed in the augmented reality image; Receives input from the user requesting route guidance; Generate a path guide based on the destination information corresponding to the request; The generated path guide line is corrected by reflecting the radius of the actual driving trajectory of the vehicle. Variable stereoscopic transformation is performed by varying the height of the corrected path guide lines based on the distance between the guide lines and the vehicle itself. Texture mapping is performed on the 3D data generated based on variable stereoscopic modeling, and path guidance indicators are generated. The steps for performing the variable stereoscopic transformation include: The step of generating virtual path guide lines on both sides of the corrected path guide line; The step of calculating the height value in such a way that the height value of the vertices included in the corrected path guide line increases proportionally to the distance to the vehicle; and The step of stereoforming is performed on the vertices of the path guide line whose height value is calculated using polygons, and on the vertices of each virtual path guide line included on both sides of the corrected path guide line.

Images