Method for measuring distance between vehicles based on vehicle images, distance measuring device, electronic device, computer program, and computer-readable recording medium

Abstract

To provide a measuring method and measuring device for vehicular distance based on vehicle video, electronic apparatus, computer program and computer readable recording medium.SOLUTION: A measuring method of the present invention comprises: a step of acquiring a travel video imaged by an imaging apparatus of a first vehicle during travel; a step of detecting a second vehicle from the acquired travel video; a step of detecting a first characteristic point of a second vehicle region from a first frame corresponding to a frame in which the second vehicle is detected, before a frame in which the second vehicle is not detected, out of a plurality of frames constituting the travel video if the second vehicle is not detected from the travel video; a step of detecting a second characteristic point in a second frame corresponding to a present frame by tracing the detected first characteristic point; a step of calculating a characteristic point change value between the first characteristic point and the second characteristic point; and a step of calculating a distance from the imaging apparatus of the first vehicle to the second vehicle.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a method for measuring a distance between vehicles based on a vehicle image, a vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium. More specifically, the present invention relates to a method for measuring a distance between vehicles, which measures a distance to a vehicle located nearby by tracking feature points in a vehicle image, a vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium. [Background technology]

[0002] The most important thing when a vehicle is traveling is safe driving and prevention of traffic accidents. For this reason, vehicles are equipped with various auxiliary devices that control the vehicle's attitude and the functions of vehicle component devices, as well as safety devices such as safety belts and airbags.

[0003] Recently, vehicles are being equipped with devices such as black boxes that are located in the vehicle and store driving images of the vehicle and data transmitted from various sensors to determine the cause of a vehicle accident when one occurs. Portable terminals such as smartphones and tablet computers can also be equipped with black boxes or navigation applications and are being used as such vehicle devices.

[0004] In recent years, advanced driver assistance systems (ADAS) that assist the driver in driving by using driving images captured while the vehicle is traveling have been developed and spread to ensure safe driving and convenience for the driver.

[0005] Among the functions provided by such ADAS, the Forward Collision Warning System (FCWS) function detects a vehicle ahead of the vehicle on the vehicle's driving path from captured driving footage, measures the distance to the detected vehicle ahead, and notifies the driver of the risk of a collision depending on the distance.

[0006] That is, FCWS requires the detection of a vehicle ahead, which has been conventionally achieved by using an image processing method that uses the shadow of the vehicle ahead in the captured driving video, or a machine learning method that learns and detects a large number of vehicle images, etc. Both of these methods have high detection performance when a vehicle area including the bottom of the vehicle is present in the driving video.

[0007] However, a vehicle may encounter various driving environments while traveling, and a situation may occur in which the bottom end of the vehicle is not included in the driving footage. As an example, when a vehicle is traveling and discovers a vehicle ahead and its driving speed decreases, the distance between the vehicle and the vehicle ahead becomes very close. As shown in FIG. 1, when the distance between vehicle 1 and vehicle ahead 2 is far (2-1), the driving footage captured by vehicle 1 includes a vehicle image including the bottom end of the vehicle ahead. However, when the distance between vehicle 1 and vehicle ahead 2 becomes close (for example, within 10 m) (2-2), the bottom end of the vehicle ahead is not included in the driving footage.

[0008] In this way, when the distance to the vehicle ahead becomes closer while the vehicle is traveling and the lower end of the vehicle ahead is not included in the traveling image, there was a problem that the conventional methods for detecting the vehicle ahead based on shadows or learning were unable to adequately detect the vehicle ahead.

[0009] Meanwhile, such inter-vehicle distance measurement technology is a core technology for autonomous driving of autonomous vehicles, which has been the subject of active discussion recently, and the importance of inter-vehicle distance measurement technology is becoming increasingly important because if a vehicle is unable to adequately detect a vehicle ahead during autonomous driving and is unable to measure the distance between vehicles, it could lead to an accident. Summary of the Invention [Problem to be solved by the invention]

[0010] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a method for measuring inter-vehicle distance based on vehicle images, a vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium, which measure the distance between a target vehicle and the own vehicle by tracking characteristic points, even if the lower end portion of the target vehicle is no longer captured as the distance between the own vehicle and the target vehicle (front vehicle or rear vehicle) to be measured becomes closer and it becomes impossible to detect the target vehicle using driving images.

[0011] Another object of the present invention is to provide a method for measuring a distance between vehicles based on vehicle images, a distance measuring device, an electronic device, a computer program, and a computer-readable recording medium, which use the measured distance to provide guidance based on the distance between vehicles.

[0012] Furthermore, an object of the present invention is to provide a method for measuring a distance between vehicles based on vehicle images, a distance measuring device, an electronic device, a computer program, and a computer-readable recording medium, which generate an autonomous driving control signal for the vehicle using the measured distance. [Means for solving the problem]

[0013] To achieve the above-mentioned object, a method for measuring a distance between vehicles using a processor according to one embodiment of the present invention includes the steps of: acquiring a driving video captured by an image capture device of a first vehicle while the vehicle is moving; detecting a second vehicle from the acquired driving video; if the second vehicle is not detected from the driving video, detecting a first feature point in the area of ​​the second vehicle from a first frame corresponding to a frame in which the second vehicle was detected and which is prior to the frame in which the second vehicle was not detected among a plurality of frames constituting the driving video; detecting a second feature point in a second frame corresponding to a current frame by tracking the detected first feature point; calculating a feature point change value between the first feature point and the second feature point; and calculating the distance between the vehicle from the image capture device of the first vehicle to the second vehicle based on the calculated feature point change value.

[0014] In addition, in the step of detecting the second vehicle, the second vehicle may be detected using a learning model constructed by machine learning or deep learning for a vehicle image.

[0015] In addition, in the step of detecting the first feature points, if the second vehicle is not detected using the constructed learning model as the distance between the first vehicle and the second vehicle becomes closer, a step of detecting the first feature points in the second vehicle area can be performed.

[0016] In the step of detecting the first feature point, a middle area of ​​the vehicle in the second vehicle area in the frame is set as an area of ​​interest, and the first feature point can be detected from the set area of ​​interest.

[0017] In addition, in the step of detecting the second feature point, the second feature point can be detected in the second frame by tracking the second feature point using an optical flow of the detected first feature point.

[0018] The method may further include, when tracking the second feature points using the optical flow, filtering out second feature points not expressed in the second frame and first feature points corresponding to the second feature points not expressed in the second frame.

[0019] In addition, the step of calculating the feature point change value may include the steps of: calculating an average pixel position of the first feature points; calculating a first average pixel distance by averaging pixel distances from the calculated average pixel position of the first feature points to each of the first feature points; calculating an average pixel position of the second feature points; calculating a second average pixel distance by averaging pixel distances from the calculated average pixel position of the second feature points to each of the second feature points; and calculating an average pixel distance ratio between the first average pixel distance and the second average pixel distance.

[0020] And, the step of calculating the inter-vehicle distance may include a step of multiplying the image width of the second vehicle in the first frame by the calculated average pixel distance ratio to calculate the image width of the second vehicle in the second frame.

[0021] In addition, the step of calculating the inter-vehicle distance may further include a step of calculating the inter-vehicle distance from the camera of the first vehicle to the second vehicle based on the image width of the second vehicle in the calculated second frame, the focal length of the first camera, and the predicted width of the second vehicle.

[0022] The step of calculating the inter-vehicle distance may further include the steps of calculating an image width ratio between an image width of the detected second vehicle and an image width of the road on which the second vehicle is located, determining a size class of the second vehicle based on the calculated ratio, and calculating a predicted width of the second vehicle based on the determined size class of the second vehicle.

[0023] In addition, the method may further include a step of generating guidance data for providing guidance on a collision risk level corresponding to a distance difference between the first vehicle and the second vehicle when the calculated inter-vehicle distance is smaller than a preset distance.

[0024] The method may further include generating a control signal for controlling the autonomous driving of the first vehicle based on the calculated inter-vehicle distance.

[0025] Meanwhile, to achieve the above-mentioned object, an inter-vehicle distance measurement device according to one embodiment of the present invention includes an image acquisition unit that acquires a driving image captured by an imaging device of a first vehicle while the first vehicle is traveling; a vehicle detection unit that detects a second vehicle from the acquired driving image; and, if the second vehicle is not detected from the driving image, a feature point detection unit that detects a first feature point of a second vehicle area of ​​the second vehicle from a first frame corresponding to a frame in which the second vehicle was detected and which is prior to the frame in which the second vehicle was not detected among a plurality of frames constituting the driving image, and detects a second feature point in a second frame corresponding to a current frame by tracking the detected first feature point; a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point; and an inter-vehicle distance calculation unit that calculates a distance from the imaging device of the first vehicle to the second vehicle based on the calculated feature point change value.

[0026] The vehicle detection unit may detect the second vehicle using a learning model constructed by machine learning or deep learning for a vehicle image.

[0027] In addition, as the distance between the first vehicle and the second vehicle becomes closer, the feature point detection unit can detect a first feature point in the second vehicle area when the second vehicle is not detected using the constructed learning model.

[0028] Then, the feature point detection unit can set an intermediate region of the vehicle in the second vehicle region in the frame as a region of interest, and detect the first feature point from the set region of interest.

[0029] In addition, the feature point detection unit can detect the second feature point in the second frame by tracking the second feature point using an optical flow of the detected first feature point.

[0030] Then, when tracking the second feature points using the optical flow, the feature point detection unit can filter out second feature points that are not expressed in the second frame and first feature points that correspond to the second feature points that are not expressed in the second frame.

[0031] In addition, the feature point change value calculation unit may include an average pixel distance calculation unit that calculates an average pixel position of the first feature points, calculates a first average pixel distance by averaging pixel distances from the calculated average pixel position of the first feature points to each of the first feature points, calculates an average pixel position of the second feature points, and calculates a second average pixel distance by averaging pixel distances from the calculated average pixel position of the second feature points to each of the second feature points, and a ratio calculation unit that calculates an average pixel distance ratio between the first average pixel distance and the second average pixel distance.

[0032] Then, the vehicle distance calculation unit can calculate the image width of the second vehicle in the second frame by multiplying the image width of the second vehicle in the first frame by the calculated average pixel distance ratio.

[0033] In addition, the vehicle distance calculation unit can calculate the distance from the imaging device of the first vehicle to the second vehicle based on the image width of the second vehicle in the calculated second frame, the focal length of the first imaging device, and the predicted width of the second vehicle.

[0034] Then, the vehicle distance calculation unit can calculate an image width ratio between an image width of the detected second vehicle and an image width of the road on which the second vehicle is located, determine a size class of the second vehicle based on the calculated ratio, and calculate a predicted width of the second vehicle based on the determined size class of the second vehicle.

[0035] The vehicle control system may further include a guidance data generating unit that generates guidance data for providing guidance on a collision risk level corresponding to a distance difference between the first vehicle and the second vehicle when the calculated inter-vehicle distance is smaller than a preset distance.

[0036] The autonomous driving control signal generating unit may further include an autonomous driving control signal generating unit that generates a control signal for controlling the autonomous driving of the first vehicle based on the calculated inter-vehicle distance.

[0037] Meanwhile, according to one embodiment of the present invention for achieving the above-mentioned object, an electronic device that provides guidance to assist a driver based on a vehicle distance includes an output unit that outputs guidance information that can be confirmed by the driver, an image acquisition unit that acquires a driving image captured by a camera device, a vehicle detection unit that detects a second vehicle from the acquired driving image, and a feature point detection unit that, when the second vehicle is not detected from the driving image, detects a first feature point of a second vehicle area of ​​the second vehicle from a first frame corresponding to a frame in which the second vehicle was detected and which is prior to the frame in which the second vehicle was not detected among a plurality of frames constituting the driving image, and detects a second feature point in a second frame corresponding to a current frame by tracking the detected first feature point, a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point, a vehicle distance calculation unit that calculates a vehicle distance from the camera device of the first vehicle to the second vehicle based on the calculated feature point change value, and a control unit that controls the output unit to output a forward vehicle collision alert or a forward vehicle departure alert in accordance with the calculated distance.

[0038] The output unit may further include a display unit that combines the captured driving image with a guide object to output an augmented reality image, and the control unit may control the display unit to generate a guide object for the forward vehicle collision warning and to superimpose the generated guide object for the forward vehicle collision warning on a forward vehicle display area in the augmented reality image.

[0039] Meanwhile, according to one embodiment of the present invention for achieving the above-mentioned object, a computer-readable recording medium having a program for executing the above-mentioned method for measuring a vehicle distance can be provided.

[0040] According to one embodiment of the present invention for achieving the above object, a program for executing the above-mentioned method for measuring a distance between vehicles can be provided. Effect of the Invention

[0041] According to the various embodiments of the present invention described above, as the distance between the vehicle and the target vehicle to be measured decreases, the lower end portion of the target vehicle is no longer captured, making it impossible to detect the target vehicle using driving footage. Even if this happens, the distance between the vehicle and the target vehicle can be accurately measured by tracking feature points.

[0042] Furthermore, according to various embodiments of the present invention, in close range, tracking based on feature points can achieve high processing speeds even on low-spec terminals, enabling real-time processing.

[0043] In addition, according to various embodiments of the present invention, the collision warning function and departure warning function can be performed accurately even if the area of ​​the target vehicle becomes hidden as the distance between the vehicle and the target vehicle becomes closer.

[0044] Furthermore, according to various embodiments of the present invention, even if the area of ​​the target vehicle becomes hidden as the distance between the own vehicle and the target vehicle becomes closer, the distance between the own vehicle and the target vehicle can be accurately calculated, and autonomous driving control of the own vehicle can be accurately performed. [Brief description of the drawings]

[0045] [Figure 1] 1A and 1B are diagrams for explaining driving images captured depending on the distance between the vehicle and a vehicle ahead, and problems with the prior art; [Diagram 2] 1 is a block diagram showing an inter-vehicle distance measuring device according to an embodiment of the present invention; [Diagram 3] 1 is a block diagram showing in more detail an inter-vehicle distance measuring device according to an embodiment of the present invention; [Figure 4] 4 is a diagram illustrating a feature point detection and tracking process according to an embodiment of the present invention; [Diagram 5] 1 is a diagram for explaining the construction of a learning dataset for vehicle detection according to an embodiment of the present invention, and the process of detecting a vehicle using the constructed learning dataset. [Figure 6] 5 is a diagram for explaining a feature point change value calculation unit according to an embodiment of the present invention; FIG. [Figure 7] 10A and 10B are diagrams for explaining a method for calculating an average position of feature points according to an embodiment of the present invention. [Figure 8] 3 is a block diagram showing in more detail a vehicle distance calculation unit according to an embodiment of the present invention. FIG. [Figure 9] 1 is a diagram for explaining a method for measuring a vehicle distance according to an embodiment of the present invention; [Figure 10] FIG. 4 is a diagram showing a ratio between an image width of a second vehicle and an image width of a road on which the second vehicle is located, according to an embodiment of the present invention. [Figure 11] 5 is a diagram illustrating a process of fitting a vehicle region to the detected left and right boundary positions of a vehicle according to an embodiment of the present invention; FIG. [Figure 12] 5 is a conceptual diagram illustrating a process of determining a size class of a second vehicle according to an embodiment of the present invention. FIG. [Figure 13] 4 is a flowchart showing a method for measuring a vehicle distance according to an embodiment of the present invention. [Figure 14]4 is a flowchart specifically illustrating a feature point change value calculation step (S150) according to an embodiment of the present invention. [Figure 15] 4 is a flowchart specifically showing a vehicle distance calculation step (S160) according to one embodiment of the present invention. [Figure 16] 10 is a flowchart showing a method for calculating a vehicle distance according to another embodiment of the present invention. [Figure 17] 1 is a block diagram illustrating an electronic device according to an embodiment of the present invention. [Figure 18] 1 is a diagram illustrating a system network connected to an electronic device according to an embodiment of the present invention; [Figure 19] 13 is a diagram showing a guidance screen for preventing a forward vehicle collision in the electronic device according to the embodiment of the present invention; [Figure 20] 13 is a diagram showing a guidance screen for preventing a forward vehicle collision in the electronic device according to the embodiment of the present invention; [Figure 21] FIG. 13 is a diagram showing an embodiment of an electronic device according to the present invention that does not include a photographing unit. [Figure 22] FIG. 1 is a diagram showing an embodiment of an electronic device according to the present invention, including a photographing unit. [Figure 23] FIG. 1 is a diagram showing an embodiment of the present invention using a HUD (Head-Up Display). [Figure 24] 1 is a block diagram showing an autonomous driving system according to an embodiment of the present invention. [Diagram 25] 1 is a block diagram showing a configuration of an autonomous vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] The following content is merely illustrative of the principles of the present invention. Thus, those skilled in the art can invent various devices that embody the principles of the present invention and fall within the concept and scope of the present invention, even if not explicitly described or illustrated in the present specification. In addition, it should be understood that all conditional terms and embodiments listed in the present specification are, in principle, clearly intended only for the purpose of understanding the concept of the present invention, and are not limited by the embodiments and conditions specifically listed in this specification.

[0047] Moreover, all detailed descriptions reciting specific embodiments, as well as principles, aspects, and embodiments of the present invention, should be understood to be intended to include structural and functional equivalents of such items, and such equivalents should be understood to include not only currently known equivalents, but also equivalents developed in the future, i.e., all elements invented that perform the same function, regardless of structure.

[0048] Thus, for example, the block diagrams herein should be understood to represent conceptual views of illustrative circuitry embodying the principles of the invention. Similarly, all flow charts, state transition diagrams, pseudo code, and the like, may be substantially depicted on a computer readable medium and should be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly depicted.

[0049] The functions of the various elements illustrated in the figures, including functional blocks labeled as processors or similar concepts, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared.

[0050] Additionally, the express use of terms such as processor, control, or similar concepts should not be construed as exclusively referring to hardware capable of executing software, but should be understood to implicitly include, without limitation, digital signal processor (DSP) hardware, ROM, RAM, and non-volatile memory for storing software. Other hardware known and commonly used may also be included.

[0051] In the claims of this specification, elements expressed as means for performing a function described in the detailed description are intended to include any method of performing that function, including, for example, a combination of circuit elements performing that function, or any type of software, including firmware / microcode, combined with appropriate circuitry for executing the software to perform that function. The invention defined by such claims is such that the functionality provided by the various recited means is combined and combined in the manner required by the claims, and any means capable of providing that functionality should be understood to be equivalent to what is understood from this specification.

[0052] The above-mentioned objects, features, and advantages will become more apparent from the following detailed description in conjunction with the accompanying drawings, which will enable those skilled in the art to which the present invention pertains to easily implement the technical ideas of the present invention. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may obscure the gist of the present invention, the detailed description will be omitted.

[0053] Before describing various embodiments of the present invention in detail, the nomenclature used in the present invention will be defined as follows.

[0054] In this specification, the inter-vehicle distance may refer to a distance in real-world coordinates, where the inter-vehicle distance may refer to the distance between a first vehicle and a second vehicle, or more precisely, the distance from a camera installed in the first vehicle to the second vehicle.

[0055] Additionally, in this specification, the width of a vehicle may refer to the width of the vehicle in real-world coordinates or the lateral width of the vehicle in real-world coordinates.

[0056] In addition, in this specification, the image width may refer to the pixel width of an image formed on the imaging surface of the imaging element of the imaging device.

[0057] In addition, in this specification, pixel distance can mean the distance between pixels connected to the imaging surface of the imaging element of the imaging device.

[0058] Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0059] Fig. 2 is a block diagram showing an inter-vehicle distance measuring device according to an embodiment of the present invention. Fig. 3 is a block diagram showing a more detailed inter-vehicle distance measuring device according to an embodiment of the present invention. Referring to Figs. 2 and 3, an inter-vehicle distance measuring device 10 according to an embodiment of the present invention may include all or some of an image acquiring unit 11, a vehicle detecting unit 12, a feature point detecting unit 13, a feature point change value calculating unit 14, an inter-vehicle distance calculating unit 15, a guidance data generating unit 17, a driving control data generating unit 18, and a control unit 19. In addition, the feature point change value calculating unit 14 may include an average pixel distance calculating unit 14-1 and an average pixel distance ratio calculating unit 14-2.

[0060] Here, the inter-vehicle distance measuring device 10 can measure the distance between a first vehicle that is a reference for distance measurement and a second vehicle that is a target for distance measurement. Here, the first vehicle is a vehicle that is a reference for distance measurement and may be named a "reference vehicle" or "own vehicle", and the second vehicle is a vehicle that is a target for distance measurement and may be named a "target vehicle". In addition, the second vehicle is a vehicle located near the first vehicle, and may include a front vehicle located in front of the first vehicle and a rear vehicle located behind the first vehicle.

[0061] Such a vehicle distance measuring device 10 can calculate the distance between a first vehicle and a second vehicle by controlling the activation of feature point detection and tracking functions depending on whether a second vehicle is detected from a driving image captured by a camera of the first vehicle.

[0062] Specifically, when a first vehicle travels on a roadway, a second vehicle may first appear in front of or behind the first vehicle while the first vehicle is traveling. In this case, the inter-vehicle distance measurement device 10 can detect the second vehicle from a traveling video by machine learning or deep learning. Then, the inter-vehicle distance measurement device 10 can calculate the inter-vehicle distance between the first vehicle and the second vehicle detected by machine learning or deep learning.

[0063] However, when the lower end portion of the second vehicle is not captured as the distance between the first vehicle and the second vehicle decreases and the second vehicle is not detected from the driving video by machine learning or deep learning, the inter-vehicle distance measurement device 10 can detect feature points from the driving video and track the detected feature points to calculate the distance between the first vehicle and the second vehicle. Specifically, the inter-vehicle distance measurement device 10 selects a first frame corresponding to a frame in which the second vehicle is detected and which is before a frame in which the second vehicle is not detected from among a plurality of frames constituting the driving video, detects a first feature point from the second vehicle area in the selected first frame, tracks the detected first feature point, thereby detecting a second feature point in a second frame corresponding to the current frame, calculates a feature point change value between the first feature point and the second feature point, and calculates the distance from the imaging device of the first vehicle to the second vehicle based on the calculated feature point change value.

[0064] That is, the operation of the vehicle distance measurement device 10 of the present invention will be described in the order of steps: in a first step, the second vehicle first appears in front of the first vehicle; in a second step, the vehicle distance measurement device 10 detects the second vehicle from the driving footage by machine learning or deep learning and calculates the vehicle distance between the detected second vehicle and the first vehicle; and in a third step, if the vehicle distance measurement device 10 does not detect the second vehicle from the driving footage by machine learning or deep learning, it can calculate the vehicle distance between the second vehicle and the first vehicle by detecting and tracking characteristic points from the driving footage.

[0065] The vehicle distance measurement device 10 can be realized using software, hardware, or a combination of these. As an example, when realized in hardware, the device can be realized using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, micro-controllers, micro-processors, and other electrical units for performing functions.

[0066] For ease of explanation, the following describes in more detail each of the constituent modules that make up the inter-vehicle distance measurement device 10, taking as an example a case in which the second vehicle to be measured is a leading vehicle.

[0067] The image acquiring unit 11 may acquire a driving image captured by a camera installed in the first vehicle. Specifically, the image acquiring unit 11 may acquire a driving image captured by a camera installed in the first vehicle in real time while the first vehicle is traveling. Here, the acquired driving image may include a plurality of lanes separated by lanes, a road composed of a plurality of lanes, and a plurality of vehicles traveling on the road.

[0068] Here, lane markings may refer to the lines on both sides of a lane that a vehicle is located in. A lane may refer to a road that is made up of lanes and on which vehicles travel, such as one lane, two lanes, ..., N lanes, etc.

[0069] The vehicle detection unit 12 can detect the second vehicle from the driving image acquired by the image acquisition unit 11. Specifically, the vehicle detection unit 12 can construct a learning model for vehicle detection by performing machine learning or deep learning on the vehicle image, and detect the second vehicle based on the constructed learning model. Here, the constructed model is an algorithm or program for detecting a vehicle from an image.

[0070] The learning model for vehicle detection can be trained to a more advanced model using the output value indicating the result of vehicle detection. As an example, if the output result is an incorrect answer, the user can input a response to the output result, and the vehicle detection unit 12 can train the learning model for vehicle detection based on the driver's response.

[0071] That is, according to the present invention, a learning model for vehicle detection is generated by performing machine learning or deep learning, and a vehicle can be detected from a driving image using the generated model. Here, a convolution neural network (CNN) algorithm, which is one of neural network models, can be applied to deep learning. In this case, deep learning can perform learning based on augmented data assuming various conditions of a driving image. Here, the condition is defined as a condition for converting an image collected for learning of a neural network model. In particular, since various aspects may be shown depending on factors such as image shift, rotation, brightness change, and blur, data can be augmented taking this into consideration.

[0072] In addition, when multiple vehicles are detected from the driving video, the vehicle detection unit 12 can select a second vehicle to be measured distance from the multiple detected vehicles based on driving state information indicating whether the first vehicle is driving correctly along a specific road or is leaving the specific road.

[0073] As an example, when a first vehicle is traveling on a specific road, the vehicle detection unit 12 can select a second vehicle located on the same road as the first vehicle from a plurality of vehicles included in the traveling image, and detect the selected second vehicle.

[0074] As another example, when a first vehicle is leaving a specific lane, the vehicle detection unit 12 can select a second vehicle that is located on the lane toward which the front of the first vehicle leaving the lane is heading from a plurality of vehicles included in the driving image, and detect the selected second vehicle.

[0075] On the other hand, when the vehicle detection unit 12 detects a second vehicle, the vehicle distance calculation unit 15 can calculate the distance between the detected second vehicle and the first vehicle. That is, when the vehicle detection unit 12 detects a second vehicle from a driving video using a learning model, the functions of the feature point detection unit 13 and the feature point change value calculation unit 14 are not activated, and the vehicle distance calculation unit 15 is activated to calculate the distance between the detected second vehicle and the first vehicle. Here, the vehicle distance calculation unit 15 can calculate the distance between the first vehicle and the detected second vehicle using a vehicle distance calculation algorithm described later.

[0076] However, when the distance between the first vehicle and the second vehicle becomes less than the predetermined distance and they approach each other, the bottom end portion of the second vehicle is no longer captured, and the vehicle detection unit 12 is unable to detect the second vehicle from the driving footage using the learning model, the functions of the feature point detection unit 13 and the feature point change value calculation unit 14 are activated to track the second vehicle by tracking the feature points, and the vehicle distance calculation unit 15 is activated to calculate the distance between the first vehicle and the second vehicle.

[0077] This will be described in more detail with reference to FIG.

[0078] 4 is a diagram for explaining a process of detecting and tracking a feature point according to an embodiment of the present invention. Referring to FIG. 4, when the distance between the first vehicle 21 and the second vehicle 22 is equal to or greater than a predetermined distance (22-1), the driving image 23-1 captured by the camera of the first vehicle 21 includes the bottom end portion of the second vehicle. Therefore, the vehicle detection unit 12 can detect the second vehicle image 23-2 from the driving image 23-1 using a learning model constructed by machine learning or deep learning.

[0079] In this case, the data set required for learning to detect the vehicle may be constructed by classifying the vehicle rear image data set collected according to the vehicle type (sedan, SUV, truck, large car, etc.) according to the detection distance (short distance, medium distance, long distance) to construct learning data. The vehicle detection unit 12 may generate a classifier by learning the constructed learning data using a learning-based method (machine learning, deep learning, etc.). The vehicle detection unit 12 may detect the second vehicle image 23-2 from the driving image 23-1 using the generated classifier.

[0080] The vehicle detection operation of the vehicle detection unit 12 will be described in more detail with reference to FIG.

[0081] FIG. 5 is a diagram illustrating a process of constructing a learning data set for vehicle detection and detecting a vehicle using the constructed learning data set.

[0082] 5, first, a vehicle learning dataset is constructed (S1000), selective data learning is performed using the constructed vehicle learning dataset (S1500), and then a classifier for classifying vehicles can be generated (S1700). Then, when an image is input by a camera 1700 (S1950), the learning system for vehicle detection according to an embodiment of the present invention can detect vehicles using the generated classifier (S1900).

[0083] The specific steps performed at each step are shown in the diagram on the right.

[0084] First, the vehicle learning dataset construction step (S1000) will be described in detail. The learning system for vehicle detection according to the embodiment of the present invention may acquire an image to be learned (S1010), crop a vehicle area to be learned from the learned image (S1030), and perform annotation on vehicle attributes included in the cropped vehicle area. In this case, the vehicle attributes to be annotated may be the type of vehicle, the distance of the vehicle in the image, etc. Then, the learning system for vehicle detection according to the embodiment of the present invention may generate a learning dataset based on the cropped vehicle image and its attributes (S1070).

[0085] Specific steps of the selective dataset learning step (S1500) are as follows. The learning system for vehicle detection according to the embodiment of the present invention may extract features from the constructed dataset (S1510). In this case, the method of extracting features may be (i) grayscale intensity, (ii) red, green, blue (RGB) color information, (iii) hue, saturation, value (HSV) color information, (iv) YIQ color information, (v) edge information (grayscale, binary, eroded binary), etc. Then, the learning system for vehicle detection according to the embodiment of the present invention may classify vehicles using the extracted features (S1530), and generate a classifier for classifying vehicles after enforcing the process of classifying vehicles through learning (S1550) (S1700).

[0086] Finally, the vehicle detection step (S1900) will be described in detail as follows: The learning system for vehicle detection according to the embodiment of the present invention extracts features from an image input by the camera 1700 (S1950) (S1920), detects a vehicle using a classifier for the extracted features (S1930), and outputs the detection result (S1970).

[0087] On the other hand, when the second vehicle image 23-2 is detected by the vehicle detection unit 12, the inter-vehicle distance calculation unit 15 can calculate the distance between the detected second vehicle 22 and the first vehicle 21.

[0088] However, vehicles encounter various driving environments while traveling, and when the distance between the first vehicle 21 and the second vehicle 22 becomes close while the first vehicle 21 is traveling (for example, when they approach within 10 m) (22-2), the driving image captured by the camera of the first vehicle 21 does not include the bottom end part of the second vehicle, so the vehicle detection unit 12 cannot detect the second vehicle image using a learning model constructed by machine learning or deep learning.

[0089] In this way, when the vehicle detection unit 12 does not detect the second vehicle image from the running image, the feature point detection unit 13 can select the first frame 24-1 corresponding to the frame in which the second vehicle was detected, which is before the frame in which the second vehicle was not detected, from among the multiple frames constituting the running image. Then, the feature point detection unit 13 can detect the feature points using the selected first frame 24-1. In this case, the feature point detection unit 13 can detect the feature points using the first frame 24-1 that has not been subjected to region of interest processing, or can detect the feature points using the first frame 24-2 that has been subjected to region of interest processing. As an example, as shown in FIG. 4, the feature point detection unit 13 can set the second vehicle region in the first frame 24-1 as a region of interest, generate the first frame 24-2 that has been subjected to region of interest processing, and detect the first feature point 24-3 from the first frame 24-2 that has been subjected to region of interest processing. Although the reference numeral 24-3 in Fig. 4 is described as indicating only one point, the first feature point may refer to all points that are displayed distinctly in the first frame 24-2 that has been subjected to the region of interest processing. In this case, the feature point detection unit 13 may set the middle region of the vehicle in the second vehicle region of the first frame 24-2 that has been subjected to the region of interest processing as the region of interest, and detect the first feature point 24-3 from the set region of interest. Here, the middle region of the vehicle may include the license plate region of the vehicle formed on the rear surface of the vehicle, and may include the rear bumper region and trunk region of the vehicle that are separated by a predetermined distance from the license plate region of the vehicle.

[0090] Here, the feature point detection unit 13 can detect the first feature point by using a Harris corner detection method or a features-from-accelerated-segment test (FAST) corner detection method.

[0091] Thereafter, the feature point detection unit 13 may detect a second feature point in a second frame corresponding to the current frame by tracking the detected first feature point 24-3. In this case, the feature point detection unit 13 may detect the second feature point using a second frame 25-1 that has not been subjected to region of interest processing, or may detect the feature point using a second frame 25-2 that has been subjected to region of interest processing. As an example, as shown in FIG. 4, the feature point detection unit 13 may set a second vehicle region as a region of interest to generate a second frame 25-2 that has been subjected to region of interest processing, and detect a second feature point 25-3 from the second frame 25-2 that has been subjected to region of interest processing. Although the reference numeral 25-3 in FIG. 4 is described as indicating only one point, the second feature point may refer to all points that are displayed in a distinct manner in the second frame 25-2 that has been subjected to region of interest processing.

[0092] At this time, the feature point detection unit 13 can detect the second feature point 25-3 in the second frame 25-2 that has been subjected to region of interest processing by tracking the second feature point 25-3 using the optical flow of the detected first feature point 24-3.

[0093] Meanwhile, when tracking the second feature point 25-3 using the optical flow, the feature point detection unit 13 may filter out the second feature point 25-3 that is not expressed in the second frame 25-2 that has been subjected to the region of interest processing, and the first feature point 24-3 that corresponds to the second feature point that is not expressed in the second frame 25-2 that has been subjected to the region of interest processing. That is, when the distance between the first vehicle 21 and the second vehicle 22 becomes closer, the second feature point 25-3 that corresponds to some of the first feature points 24-3 detected from the first frame 24-2 that has been subjected to the region of interest processing (for example, the feature point located at the bottom end of the vehicle among the detected first feature points 24-3) is not expressed in the second frame 25-2 that has been subjected to the region of interest processing. Therefore, when tracking the second feature point 25-3 using optical flow, the feature point detection unit 13 can increase the calculation speed by filtering out and removing the second feature point 25-3 that is not expressed in the second frame 25-2 that has been processed as a region of interest, and the first feature point 24-3 that corresponds to the second feature point that is not expressed in the second frame 25-2 that has been processed as a region of interest.

[0094] Meanwhile, the feature point change value calculation unit 14 can calculate the feature point change value between the first feature point and the second feature point. Here, the feature point change value calculation unit 14 can include an average pixel distance calculation unit 14-1 and an average pixel distance ratio calculation unit 14-2. The operation of the feature point change value calculation unit 14 will be described in more detail with reference to FIG. 6.

[0095] FIG. 6 is a diagram for explaining a feature point change value calculation unit according to an embodiment of the present invention. Referring to FIG. 6, the average pixel distance calculation unit 14-1 may calculate an average pixel position 24-5 of the first feature point 24-3 and a first average pixel distance obtained by averaging pixel distances from the average position to each of the first feature points 24-3. Specifically, the average pixel distance calculation unit 14-1 may calculate the coordinate value of the average pixel position 24-5 by averaging the coordinate values ​​of the pixel positions of the first feature point 24-3 in the region of interest 24-4 of the first frame 24-1. Here, the region of interest 24-4 may be a middle region of the vehicle, and as an example, the region of interest may include a license plate region of the vehicle formed on the rear surface of the vehicle and may include a rear bumper region and a trunk region of the vehicle that are separated by a predetermined distance from the license plate region of the vehicle. Also, although the reference numeral 24-3 in FIG. 6 is described as referring to only one point, the first feature point may refer to all points that are distinguished and displayed in the region of interest 24-4 except for the reference numeral 24-5.

[0096] Then, the average pixel distance calculation unit 14-1 can calculate the coordinate values ​​of the average pixel position 24-5 by averaging the coordinate values ​​of the pixel positions of the first feature points 24-3 in the region of interest 24-4.

[0097] The calculation process of such an average pixel position will be described with reference to Fig. 7. As an example, when the feature points are composed of a first point having pixel position coordinate values ​​of (x, y), a second point having pixel position coordinate values ​​of (x', y'), and a third point having pixel position coordinate values ​​of (x'', y'') based on a two-dimensional plane coordinate system, the average pixel distance calculation unit 14-1 can calculate the coordinate values ​​of the average pixel position (mx, my) by arithmetically averaging the coordinate values ​​of the pixel positions of the first point, the second point, and the third point.

[0098] 6, the average pixel distance calculation unit 14-1 can calculate a first average pixel distance by averaging the pixel distances from the calculated average pixel position 24-5 to each of the first feature points 24-3. Here, the first frame 24-1 is a frame corresponding to a frame in which the second vehicle is detected, which is before a frame in which the second vehicle is not detected, among a plurality of frames constituting the running video.

[0099] Also, the average pixel distance calculation unit 14-1 may calculate an average pixel position 25-5 of the second feature point 25-3 tracked using the optical flow, and a second average pixel distance obtained by averaging pixel distances from the average pixel position to each of the second feature points 25-3. Specifically, the average pixel distance calculation unit 14-1 may calculate the coordinate value of the average pixel position 25-5 by averaging pixel position coordinate values ​​of the second feature point 25-3 in the region of interest 25-4 of the second frame 25-1. Here, the feature point indicated by the dotted line in FIG. 6 represents the position of the first feature point 24-3 in the region of interest 24-4 of the first frame 24-1. And the point indicated by the solid line represents the second feature point 25-3, and although the reference numeral 25-3 in FIG. 6 is described as indicating only one point, the second feature point may mean all points displayed in the region of interest 25-4, except for the reference numeral 25-5.

[0100] Then, the average pixel distance calculation unit 14-1 sets a region of interest 25-4 including the second feature point 25-3, and calculates the coordinate value of the average pixel position 25-5 by averaging the coordinate values ​​of the pixel position of the feature point 25-3 in the region of interest 25-4. Then, the second average pixel distance can be calculated by averaging the pixel distances from the calculated average pixel position 25-5 to each of the feature points 25-3. Here, the second frame 25-1 is the current frame.

[0101] Meanwhile, in the above example, the intermediate regions of the vehicle in the frames 24-1 and 25-1 are set as the regions of interest 24-4 and 25-4, and feature points are detected, tracked, and the average pixel distance is calculated, but the present invention is not limited to this example. According to another embodiment of the present invention, the entire frames 24-1 and 25-1 may be set as the region of interest, and the feature points may be detected, tracked, and the average distance may be calculated.

[0102] Meanwhile, when the first average pixel distance and the second average pixel distance are calculated through the above-mentioned operation, the average pixel distance ratio calculation unit 14-2 may calculate an average pixel distance ratio between the first average pixel distance and the second average pixel distance. Specifically, the average pixel distance ratio calculation unit 14-2 may calculate the average pixel distance ratio by dividing the second average pixel distance by the first average pixel distance as shown in the following mathematical formula 1.

[0103] JPEG0007658745000001.jpg26170

[0104] Here, Ratio 1 may refer to the average pixel distance ratio, curAvgDist may refer to the second average pixel distance, and preAvgDist may refer to the first average pixel distance.

[0105] Meanwhile, the inter-vehicle distance calculation unit 15 can calculate the inter-vehicle distance between the first vehicle and the second vehicle. Specifically, the inter-vehicle distance calculation unit 15 can calculate the distance from the image capture device of the first vehicle to the second vehicle based on the image width of the second vehicle, the focal length of the image capture device provided in the first vehicle, and the predicted width of the second vehicle. The inter-vehicle distance calculation unit 15 will be described in more detail with reference to Figs. 8 to 12.

[0106] 6, the feature point change value is calculated using the first frame 24-1 that has not been subjected to region of interest processing and the second frame 25-1 that has not been subjected to region of interest processing, but the present invention is not limited thereto. According to another embodiment of the present invention, the feature point change value may be calculated using the first frame 24-2 that has been subjected to region of interest processing and the second frame 25-2 that has been subjected to region of interest processing.

[0107] Fig. 8 is a block diagram showing in more detail the vehicle distance calculation unit 15 according to an embodiment of the present invention. Fig. 9 is a diagram for explaining a method for measuring a vehicle distance according to an embodiment of the present invention. Referring to Figs. 8 and 9, the vehicle distance calculation unit 15 may include an image width ratio calculation unit 15-1, a vehicle size class calculation unit 15-2, a vehicle width calculation unit 15-3, and a distance calculation unit 15-4.

[0108] A camera device 50 for capturing driving images of the first vehicle (not shown) may be installed in the first vehicle. Here, the camera device 50 may be implemented as a car dash cam or a car video recorder that is installed in the first vehicle and captures the surroundings of the vehicle during driving, parking, etc. Alternatively, the camera device 50 may be implemented as a camera formed in a navigation device that provides route guidance to the driver of the first vehicle, or a camera built into the driver's mobile device.

[0109] The photographing device 50 may include a lens unit 51 and an image sensor 52, and may further include all or some of a lens unit driver, an aperture, an aperture driver, an image sensor controller, and an image processor, which are not shown in Fig. 6. Here, the lens unit 51 may perform a function of collecting an optical signal, and the optical signal transmitted through the lens unit 51 reaches an imaging area of ​​the image sensor 52 to form an optical image. Here, the image sensor 52 may be a CCD (Charge Coupled Device), a CIS (Complementary Metal Oxide Semiconductor Image Sensor), or a high-speed image sensor that converts an optical signal into an electrical signal.

[0110] Meanwhile, the vehicle distance calculation unit 15 can calculate the distance between the camera device 50 installed in the first vehicle and the second vehicle 30 based on the following mathematical formula 2 using the driving image captured by the camera device 50 of the first vehicle.

[0111] JPEG0007658745000002.jpg21170

[0112] Here, D may be the distance from the camera installed in the first vehicle to the second vehicle, W may be the width of the second vehicle, f may be the focal length of the camera, and w may be the image width of the second vehicle.

[0113] That is, the distance (D) from the camera installed in the first vehicle to the second vehicle may mean the distance from the camera installed in the first vehicle to the second vehicle in real-world coordinates.

[0114] And the width (W) of the second vehicle may mean the width of the second vehicle in real-world coordinates.

[0115] The image width may refer to the pixel width of the second vehicle connected to the imaging surface of the imaging element 52 of the photographing device 50. Here, the image width w of the second vehicle may be the same value as VehicleW in Equation 3 described below.

[0116] Meanwhile, the vehicle distance calculation unit 15 first calculates a ratio between the image width of the second vehicle 30 and the image width of the road on which the second vehicle 30 is located from the driving image captured by the camera device 50 of the first vehicle, determines a size class of the second vehicle 30 from a plurality of size classes based on the calculated ratio, and calculates the width (W) of the second vehicle 30 based on the determined size class of the second vehicle. The operation of the vehicle distance calculation unit 15 will be described in more detail with reference to FIG.

[0117] 10 is a diagram showing a ratio between an image width of a second vehicle and an image width of a roadway on which the second vehicle is located, according to an embodiment of the present invention. Referring to FIG. 10, a driving image 45 captured by a camera device 50 of a first vehicle may include a second vehicle 30 traveling in front of the first vehicle, a roadway 40 on which the second vehicle is traveling, and a left lane 41 and a right lane 42 separating the roadway 40 from other roadways.

[0118] At this time, the image width ratio calculation unit 15-1 can calculate the image width (VehicleW) of the second vehicle 30. Specifically, when the vehicle detection unit 12 detects a vehicle from a traveling image using a pre-constructed learning model, the image width ratio calculation unit 15-1 can identify the left boundary 31 and the right boundary 32 of the second vehicle 30 in the image of the detected second vehicle 30. Such boundary identification will be described in more detail with reference to FIG. 11.

[0119] Referring to FIG. 11, when a vehicle is detected from an image frame (W×H) 81 acquired by a camera using a learning model, the detected vehicle area is cropped (82), and a Sobel operation is performed on the cropped area (w'×h') 83 to detect vertical edges (84).

[0120] Then, a vertical histogram accumulation value is calculated from the detected vertical edges, and the point where the largest histogram value is located can be detected as the left and right boundary positions of the vehicle (85).

[0121] Then, the vehicle region can be fitted to the detected left and right boundary positions of the vehicle (86).

[0122] On the other hand, the image width ratio calculation unit 15-1 can determine the image width between the identified left boundary 31 and the identified right boundary 32 as the image width (VehicleW) of the second vehicle.

[0123] In addition, the image width ratio calculation unit 15-1 may identify the left lane 41 and the right lane 42 of the road 40 on which the second vehicle 30 is traveling in the acquired driving image 45. The image width ratio calculation unit 15-1 may set a line 33 indicating the position of the second vehicle 30 on the road. Here, the line 33 indicating the position of the second vehicle 30 on the road may be realized as a line extending from the bottom end of the second vehicle 30 in the driving image 45. As an example, it may be realized as a line extending from the bottom end of the left wheel and the bottom end of the right wheel of the second vehicle 30.

[0124] Meanwhile, a first point 43 where the line 33 indicating the position of the second vehicle 30 on the road intersects with the left lane 41, and a second point 44 where the line 33 intersects with the right lane 42 are determined, and the image width between the first point 43 and the second point 44 can be determined as the image width (Lane W) of the road on which the second vehicle 30 is located.

[0125] Meanwhile, when the image width of the second vehicle (VehicleW) and the image width of the lane on which the second vehicle 30 is located (LaneW) are calculated, the image width ratio calculation unit 15-1 can calculate the ratio between the image width of the second forward vehicle and the image width of the lane on which the second vehicle is located by applying the following mathematical formula 3.

[0126] JPEG0007658745000003.jpg21170

[0127] Here, VehicleW may mean the image width of the second vehicle, LaneW may mean the image width of the lane on which the second vehicle is located, and Ratio2 may mean the ratio between the image width of the second vehicle and the image width of the lane on which the second vehicle is located.

[0128] In this way, when the distance between the first and second vehicles is closer, the image width of the second vehicle and the image width of the road on which the second vehicle is located become larger, and when the distance between the first and second vehicles is farther, the image width of the second vehicle and the image width of the road on which the second vehicle is located become smaller. However, since the above ratio is proportional to the size of the second vehicle without being affected by the distance between the first and second vehicles, according to the present invention, this can be used as an index for calculating the size of the second vehicle.

[0129] Meanwhile, when the ratio between the image width of the second vehicle and the image width of the road on which the second vehicle is located is calculated according to the above example, the vehicle distance calculation unit 15 can determine the size class of the second vehicle from among a plurality of size classes. This will be described in more detail with reference to FIG.

[0130] 12 is a conceptual diagram illustrating a process of determining a size class of a second vehicle according to an embodiment of the present invention. Referring to FIG. 12, a vehicle size class calculation unit 15-2 classifies the ratio value into a plurality of sections and calculates a size class of the vehicle based on a threshold table in which the size class of the second vehicle is matched to each of the plurality of sections.

[0131] As an example, the threshold table may be classified into three ranges based on a first value and a second value, and if the calculated ratio is smaller than the first value, a first size class corresponding to a small vehicle may be matched, if the calculated ratio is greater than the first value and smaller than the second value, a second size class corresponding to a medium vehicle may be matched, and if the calculated ratio is greater than the second value, a third size class corresponding to a large vehicle may be matched.

[0132] In this case, the vehicle size class calculation unit 15-2 may determine the size class of the second vehicle as the first size class when the ratio calculated by the image width ratio calculation unit 15-1 is smaller than the first value. And, the vehicle size class calculation unit 15-2 may determine the size class of the second vehicle as the second size class when the ratio calculated by the image width ratio calculation unit 15-1 is larger than the first value and smaller than the second value. And, the vehicle size class calculation unit 15-2 may determine the size class of the second vehicle as the third size class when the ratio calculated by the image width ratio calculation unit 15-1 is larger than the second value. As an example, the first value may be 48% and the second value may be 60%.

[0133] The vehicle width calculation unit 15-3 may determine the width of the second vehicle based on the size class of the second vehicle. Specifically, the storage unit may store vehicle widths for a plurality of size classes as shown in Table 1 below, and the vehicle width calculation unit 15-3 may determine the width of the second vehicle (VehicleW) by detecting the vehicle width corresponding to the determined size class from among the vehicle widths already stored in the storage unit.

[0134] JPEG0007658745000004.jpg29170

[0135] Then, the distance calculation unit 15-4 can calculate the distance between the imaging device 50 and the second vehicle 30 by dividing the focal length (f) of the imaging device 50 by the image width (w) of the second vehicle 30 and multiplying it by the width (W) of the second vehicle 30 calculated by the vehicle width calculation unit 15-3, as shown in the above mathematical equation 2.

[0136] Meanwhile, when the distance between the photographing device 50 and the second vehicle 30 is calculated, the distance calculation unit 15-4 can calculate the distance value between the first vehicle on which the photographing device 50 is installed and the second vehicle 30 by appropriately correcting the distance value between the photographing device 50 and the second vehicle 30 in order to accurately calculate the inter-vehicle distance. According to the present invention, it is possible to reduce an error in the inter-vehicle distance between the first vehicle and the second vehicle, and to more accurately measure the inter-vehicle distance.

[0137] That is, in order to obtain equal distance values ​​calculated based on the above mathematical formula 2 for small, medium, and large vehicles of different widths that are positioned at equal distances from the first vehicle, the exact width of each vehicle must be known. However, since it is not possible to check all the specifications of all vehicle types in conventional image recognition and detection, the vehicle width has been processed with a specific constant value set in advance to measure the inter-vehicle distance without considering the actual width of numerous vehicles with different widths (e.g., small, medium, and large vehicles), resulting in an inaccurate measured inter-vehicle distance value.

[0138] However, according to the present invention, to solve this problem, the ratio of the image width of the vehicle ahead to the image width of the road is used to classify the vehicle ahead into small, medium and large vehicles, and based on the classification results, the vehicle distance is measured based on the average width assigned to small, medium and large vehicles, respectively, thereby reducing errors and measuring the vehicle distance more accurately.

[0139] On the other hand, when the expected vehicle width of the second vehicle is calculated as described above, in an environment where the second vehicle is detected by the vehicle detection unit 12, the image width of the second vehicle can be continuously measured to calculate the distance between the camera of the first vehicle and the second vehicle.

[0140] However, as the distance between the first vehicle and the second vehicle approaches less than a predetermined distance, the lower end portion of the second vehicle is no longer captured, and if the vehicle detection unit 12 cannot detect the second vehicle from the driving image using the learning model, it becomes impossible to measure the image width of the second vehicle.

[0141] Therefore, according to the present invention, when the lower end portion of the second vehicle is not captured as the distance between the first vehicle and the second vehicle approaches less than a predetermined distance and the second vehicle cannot be detected from the driving image using the learning model in the vehicle detection unit 12, the image width of the second vehicle can be predicted based on the average distance ratio calculated in the average pixel distance ratio calculation unit 14-2. This will be described in detail with further reference to FIG. 6.

[0142] Referring to FIG. 6 above, the first frame 24-1 is a frame among a plurality of frames constituting a driving image, which corresponds to a frame in which a second vehicle is detected, preceding a frame in which the second vehicle is not detected, and the vehicle detection unit 12 can detect the second vehicle 30 in the first frame 24-1, and the image width ratio calculation unit 15-1 can calculate the image width 24-6 of the second vehicle.

[0143] However, since the second frame 25-1 is the current frame and is a frame in which the second vehicle is not detected, the image width ratio calculation unit 15-1 can calculate a predicted value of the image width of the second vehicle in the second frame 25-1 by applying the image width 24-6 of the second vehicle and the average pixel distance ratio calculated by the average pixel distance ratio calculation unit 14-2 to the following mathematical formula 4.

[0144] JPEG0007658745000005.jpg17170

[0145] Here, curVehicleW may mean the image width of the second vehicle in the second frame 25-1, Ratio 1 may mean the average pixel distance ratio calculated by mathematical formula 1, and preVehicleW may mean the image width of the second vehicle in the first frame 24-1.

[0146] Then, the distance calculation unit 15-4 can calculate the distance between the image capture device 50 and the second vehicle 30 by dividing the focal length (f) of the image capture device 50 by the image width (curVehicleW) of the second vehicle in the second frame 25-1, and multiplying it by the width (W) of the second vehicle 30 calculated by the vehicle width calculation unit 15-3, as shown in the above mathematical equation 2.

[0147] As a result, according to the present invention, even if the lower end portion of the second vehicle is no longer captured as the distance between the first vehicle and the second vehicle becomes closer and it becomes impossible to detect the target vehicle from the driving footage, the distance between the vehicle and the target vehicle can be accurately measured by tracking feature points.

[0148] In another embodiment of the present invention, the vehicle distance calculation unit 15 can monitor the distance between the second vehicle and the first vehicle by calculating the ratio between the image width 24-6(w) of the second vehicle detected from the first frame 24-1 and the image width 25-6(w) of the second vehicle detected from the second frame 25-1. The control unit 19 can provide the driver with various functions related to vehicle driving, such as collision notification and adaptive cruise control, using the distance monitored by the vehicle distance calculation unit 15.

[0149] On the other hand, when the distance calculated by the vehicle distance calculation unit 15 is smaller than a preset distance, the guidance data generation unit 17 can generate guidance data for providing information on the collision risk level corresponding to the distance difference between the first vehicle and the second vehicle.

[0150] Furthermore, the driving control data generation unit 18 can generate a control signal for controlling the autonomous driving of the first vehicle based on the distance calculated by the inter-vehicle distance calculation unit 15.

[0151] The operations of the guidance data generating unit 17 and the driving control data generating unit 18 will be described later with reference to the control unit 19.

[0152] The control unit 19 controls the overall operation of the vehicle distance measurement device 10. Specifically, the control unit 19 can control all or a part of the image acquisition unit 11, the vehicle detection unit 12, the feature point detection unit 13, the feature point change value calculation unit 14, the vehicle distance calculation unit 15, the guidance data generation unit 17, and the driving control data generation unit 18.

[0153] In particular, the control unit 19 controls the vehicle detection unit 12 to detect a second vehicle from a driving image captured by the camera of the first vehicle while the vehicle is moving, and when the second vehicle is not detected from the driving image, controls the feature point detection unit 13 to select a first frame corresponding to a frame in which the second vehicle was detected and which is prior to the frame in which the second vehicle was not detected from among a plurality of frames constituting the driving image, detect a first feature point from the second vehicle area in the selected first frame, and track the detected first feature point to detect a second feature point in a second frame corresponding to the current frame, controls the feature point change value calculation unit 14 to calculate a feature point change value between the first feature point and the second feature point, and controls the vehicle distance calculation unit 15 to calculate a distance from the camera of the first vehicle to the second vehicle based on the calculated feature point change value.

[0154] In addition, when the control unit 19 acquires the following distance information between the first vehicle and the second vehicle, the control unit 19 can control the guidance data generating unit 17 to generate guidance data for assisting the driver of the first vehicle in safe driving based on the acquired information. Specifically, when the following distance calculated by the following distance calculating unit 15 is smaller than a preset distance, the guidance data generating unit 17 can generate guidance data for informing the driver of the distance difference between the first vehicle and the second vehicle. As an example, the guidance data generated by the guidance data generating unit 17 can be data for warning by voice that the following distance is a distance that requires attention, or data for providing guidance by image.

[0155] As another example, when the inter-vehicle distance calculated by the inter-vehicle distance calculation unit 15 is smaller than a preset distance, the guidance data generation unit 17 can generate data for informing of a collision risk level corresponding to the distance difference between the first vehicle and the second vehicle. As an example, the distance difference between the first vehicle and the second vehicle can be divided into a plurality of levels, and when the inter-vehicle distance is smaller than a first value, data for informing of a first risk level, which is higher than the first risk level, can be generated when the inter-vehicle distance is larger than the first value and smaller than a second value, and data for informing of a third risk level, which is higher than the second risk level, can be generated when the inter-vehicle distance is larger than the second value.

[0156] Meanwhile, when the inter-vehicle distance information between the first vehicle and the second vehicle is acquired, the control unit 19 can control the driving control data generating unit 18 to generate driving control data for controlling the autonomous driving of the first vehicle based on the acquired information. Specifically, when the first vehicle is operating in the autonomous driving mode and the inter-vehicle distance calculated by the inter-vehicle distance calculating unit 15 is smaller than a preset distance, the control unit 19 can control the driving control data generating unit 18 to generate driving control data for controlling the autonomous driving of the first vehicle (e.g., command data for controlling the speed of the first vehicle to decrease from a current speed to a predetermined speed or for controlling the first vehicle to stop, etc.). Here, the driving control data generated by the driving control data generating unit 18 can be transmitted to an autonomous driving control unit that generally controls the autonomous driving of the first vehicle, and the autonomous driving control unit of the first vehicle can control various units (brake, steering wheel, electric motor, engine, etc.) provided in the first vehicle based on this information, thereby controlling the first vehicle to drive autonomously.

[0157] Hereinafter, the method for measuring a vehicle distance according to one embodiment of the present invention will be described in more detail with reference to Figs.

[0158] 13 is a flowchart showing a method for measuring a distance between vehicles according to an embodiment of the present invention. Referring to FIG. 13, first, a driving image captured by a camera of a first vehicle while the first vehicle is moving can be acquired (S110).

[0159] Then, it may be determined whether a second vehicle is detected from the acquired driving image (S120). Here, the detection of the second vehicle from the acquired driving image may be performed using a learning model constructed by machine learning or deep learning for the vehicle image.

[0160] When a second vehicle is detected from the acquired driving image (S120:Y), a distance from the camera of the first vehicle to the detected second vehicle can be calculated (S170). Here, the distance calculation step (S170) calculates an image width ratio between an image width of the detected second vehicle and an image width of the road, determines a size class of the second vehicle based on the calculated image width ratio, calculates a predicted width of the second vehicle based on the determined size class of the second vehicle, and applies the image width of the second vehicle, the focal length of the first camera, and the predicted width of the second vehicle to the above-mentioned mathematical formula 2, thereby calculating a distance from the camera of the first vehicle to the second vehicle.

[0161] However, if the second vehicle is not detected from the acquired running image (S120:N), a first frame corresponding to a frame in which the second vehicle was detected and preceding the frame in which the second vehicle was not detected is selected from among the multiple frames constituting the running image, and a first feature point can be detected from the second vehicle area in the selected first frame (S130). That is, in the step of detecting the first feature point (S130), if the second vehicle is not detected using the constructed learning model as the distance between the first vehicle and the second vehicle becomes closer, a step of detecting the first feature point in the second vehicle area can be performed. In the step of detecting the first feature point (S130), a middle area of ​​the vehicle in the second vehicle area in the first frame can be set as an area of ​​interest, and the first feature point can be detected from the set area of ​​interest.

[0162] Then, by tracking the detected first feature point, it is possible to detect a second feature point in a second frame corresponding to the current frame (S140). Specifically, in the step of detecting the second feature point (S140), it is possible to detect the second feature point in the second frame by tracking the second feature point using the optical flow of the detected first feature point.

[0163] Furthermore, according to an embodiment of the present invention, when tracking the second feature points using the optical flow, the method may further include filtering out the second feature points not expressed in the second frame and the first feature points corresponding to the second feature points not expressed in the second frame.

[0164] Then, a feature point change value between the first feature point and the second feature point can be calculated (S150). The step of calculating the feature point change value (S150) will be described later with reference to FIG.

[0165] Then, based on the calculated feature point change value, the distance from the image capture device of the first vehicle to the second vehicle can be calculated (S160). Here, the step of calculating the distance (S160) will be described in more detail with reference to FIG.

[0166] FIG. 14 is a flowchart specifically illustrating the step of calculating a feature point change value (S150) according to an embodiment of the present invention. Referring to FIG. 14, an average pixel position of the first feature points may be calculated (S210). Then, a first average pixel distance may be calculated by averaging pixel distances from the calculated average pixel position to each of the first feature points (S220). Specifically, the average pixel distance calculation unit 14-1 may calculate a coordinate value of the average pixel position by averaging coordinate values ​​of pixel positions of the first feature points, and may calculate a first average pixel distance by averaging pixel distances between the calculated average pixel position and each of the first feature points. Here, the first frame may be a frame corresponding to a frame in which the second vehicle is detected and a frame before a frame in which the second vehicle is not detected, among a plurality of frames constituting a traveling image, and may be a frame before the second frame.

[0167] Then, an average pixel position of the second feature points can be calculated (S230). Then, a second average pixel distance can be calculated by averaging distances from the calculated average pixel position to each of the second feature points (S240). Specifically, the average pixel distance calculation unit 14-1 can calculate the coordinate value of the average pixel position by averaging the coordinate values ​​of the pixel positions of the second feature points, and calculate the second average pixel distance by averaging pixel distances between the calculated average pixel position and each of the second feature points. Here, the second frame can be a frame after the first frame.

[0168] Then, an average pixel distance ratio between the first average pixel distance and the second average pixel distance may be calculated (S250). Specifically, the average pixel distance ratio calculation unit 14-2 may calculate the average pixel distance ratio by dividing the second average pixel distance by the first average pixel distance as shown in the above-mentioned Equation 1.

[0169] 15 is a flowchart showing the inter-vehicle distance calculation step (S160) in more detail according to an embodiment of the present invention. Referring to FIG 15, a second vehicle is detected from a first frame (S310), and an image width of the second vehicle in the first frame can be calculated (S320).

[0170] Then, a predicted value of the image width of the second vehicle in the second frame can be calculated based on the image width of the second vehicle in the first frame and the average pixel distance ratio calculated by the average pixel distance ratio calculation unit (S330). That is, since the second frame is a frame in which the second vehicle is not detected, the image width ratio calculation unit 15-1 can calculate a predicted value of the image width of the second vehicle in the second frame by applying the image width of the second vehicle and the average pixel distance ratio calculated by the average pixel distance ratio calculation unit 14-2 to the above-mentioned mathematical formula 4.

[0171] Then, based on the calculated image width of the second vehicle in the second frame, the focal length of the camera of the first vehicle, and the predicted width of the second vehicle, the distance from the camera of the first vehicle to the second vehicle can be calculated (S340). Specifically, the distance calculation unit 15-4 can calculate the distance between the camera of the first vehicle and the second vehicle by dividing the focal length (f) of the camera of the first vehicle by the image width (curVehicleW) of the second vehicle in the second frame, as in the above mathematical formula 2, and multiplying it by the predicted width of the second vehicle 30 calculated by the vehicle width calculation unit 15-3.

[0172] Here, the process of calculating the predicted width of the second vehicle may include the steps of calculating an image width ratio between an image width of the second vehicle detected by the vehicle detection unit 12 and an image width of the road on which the second vehicle is located, determining a size class of the second vehicle based on the calculated ratio, and calculating the predicted width of the second vehicle based on the determined size class of the second vehicle. The predicted width of the second vehicle may be calculated and stored in advance while the second vehicle is detected by the vehicle detection unit 12.

[0173] FIG. 16 is a flowchart showing a method for measuring a vehicle distance according to another embodiment of the present invention.

[0174] Referring to Fig. 16, the inter-vehicle distance measurement device 10 first receives an input of a current frame (i-th frame) from the image acquisition unit 11 (S1100). The inter-vehicle distance measurement device 10 detects a second vehicle from the input i-th frame using a learning model (S1105). If a second vehicle is detected in the i-th frame (S1110: Y), the inter-vehicle distance measurement device 10 calculates a distance to the detected second vehicle using the learning model (S1115) and stores the i-th frame and the detected vehicle area (S1120). At this time, the detected vehicle area may be set in the form of a rectangle, a circle, or a polygon, but is not limited thereto.

[0175] Then, when the i-th frame and vehicle area are stored, the inter-vehicle distance measurement device 10 updates i to i+1 (S1125) and receives the i+1-th frame as the current frame (S1100).

[0176] However, if the second vehicle is not detected in step S1110 (S1110:N), the inter-vehicle distance measurement device 10 checks whether there is a vehicle area stored in the immediately preceding frame (i-1th frame) (S1130). If there is a vehicle area stored in the i-1th frame in S1130 (S1130:Y), the inter-vehicle distance measurement device 10 extracts a first feature point from the region of interest of the vehicle area in the i-1th frame (S1135), extracts a second feature point corresponding to the first feature point extracted from the i-1th frame from the i-th frame (S1140), and calculates the distance to the second vehicle using the difference between the average pixel position of the first feature point and the average pixel position of the second feature point (S1145).

[0177] On the other hand, if there is no vehicle area stored in the i-1th frame in step S1130 (S1130:N), the inter-vehicle distance measurement device 10 determines that the second vehicle was not previously present and receives input of a new current frame acquired from the image acquisition unit 11 (S1125).

[0178] On the other hand, the inter-vehicle distance measuring device 10 is realized as one module of an electronic device that outputs various kinds of guidance information to assist the driver in driving, and can perform a route guidance function. This will be described in more detail with reference to Figs. 15 to 17.

[0179] 17 is a block diagram showing an electronic device according to an embodiment of the present invention. Referring to FIG. 17, the electronic device 100 includes all or some of a storage unit 110, an input unit 120, an output unit 130, a vehicle distance measurement unit 140, an augmented reality providing unit 160, a control unit 170, a communication unit 180, a sensing unit 190, and a power supply unit 195.

[0180] Here, the electronic device 100 can be realized as various devices such as a smartphone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), smart glasses, projector glasses, navigation, a car dash cam or a car video recorder which is a video capture device for a vehicle, etc., capable of providing driving-related guidance to a driver of the vehicle, and can be installed in the vehicle.

[0181] Driving-related guidance may include various guidance to assist the vehicle driver in driving, such as route guidance, lane departure guidance, lane keeping guidance, forward vehicle departure guidance, signal light change guidance, forward vehicle rear-end collision prevention guidance, lane change guidance, lane guidance, curve guidance, etc.

[0182] Here, the route guidance may include augmented reality route guidance that provides route guidance by combining various information such as the user's position, direction, etc. with an image captured in front of a moving vehicle, and 2D (2-dimensional) or 3D (3-dimensional) route guidance that provides route guidance by combining various information such as the user's position, direction, etc. with 2D (2-dimensional) or 3D (3-dimensional) map data.

[0183] Furthermore, the route guidance may include aerial map route guidance that performs route guidance by combining various information such as aerial map data and a user's position, direction, etc. Here, the route guidance may be interpreted as a concept including route guidance when the user is walking or running as well as when the user is driving a vehicle.

[0184] The lane departure guidance is to provide guidance as to whether or not the vehicle in motion has left its lane.

[0185] The lane keeping guidance is to guide the vehicle back to the lane it was originally traveling on.

[0186] The leading vehicle departure guidance is to inform the driver of the departure of a vehicle ahead of the stopped vehicle. Here, the leading vehicle departure guidance can be performed using the distance between the vehicles calculated by the vehicle distance measuring unit 140.

[0187] In addition, the signal light change guidance is to inform the driver of the change in the signal of the signal light located in front of the stopped vehicle. For example, when the red light indicating a stop signal changes to a green light indicating a departure signal, the driver can be notified.

[0188] In addition, the front vehicle collision prevention guidance is a guidance to prevent a collision with a front vehicle when the distance between a stopped or moving vehicle and a vehicle located in front of the vehicle becomes within a certain distance. Here, the front vehicle collision prevention guidance can be performed using the inter-vehicle distance calculated by the inter-vehicle distance measurement unit 140.

[0189] The lane change guidance is to guide the vehicle to change from the lane on which the vehicle is currently located to another lane in order to guide the vehicle to the destination.

[0190] The road guidance is to guide the vehicle along the road on which the vehicle is currently located.

[0191] The curve guidance is to inform the driver that the road on which the vehicle will be traveling in the future will be curved.

[0192] Driving-related images, such as images of the front of the vehicle, that can provide various kinds of guidance may be captured by a camera mounted on the vehicle or a camera of a smartphone. Here, the camera may be a camera that is integrally formed with the electronic device 100 mounted on the vehicle and captures images of the front of the vehicle.

[0193] As another example, the camera may be a camera mounted on a vehicle separately from the electronic device 100 to capture an image in front of the vehicle. In this case, the camera may be a separate vehicle-mounted image capturing device mounted facing the front of the vehicle. The electronic device 100 may receive an image captured by the separate vehicle-mounted image capturing device through wired / wireless communication, or the electronic device 100 may receive an image captured by the vehicle-mounted image capturing device when a storage medium for storing images captured by the vehicle-mounted image capturing device is inserted into the electronic device 100.

[0194] Based on the above, the electronic device 100 according to an embodiment of the present invention will be described in more detail below.

[0195] The storage unit 110 serves to store various data and applications necessary for the operation of the electronic device 100. In particular, the storage unit 110 may store data necessary for the operation of the electronic device 100, such as an OS, a route search application, map data, etc. The storage unit 110 may also store data generated by the operation of the electronic device 100, such as searched route data, received images, etc.

[0196] The storage unit 110 may be realized by a built-in storage element such as a RAM (Random Access Memory), flash memory, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electronically Erasable and Programmable ROM), register, hard disk, removable disk, memory card, USIM (Universal Subscriber Identity Module), etc., as well as a removable storage element such as a USB memory.

[0197] The input unit 120 serves to convert a physical input from outside the electronic device 100 into a specific electrical signal. Here, the input unit 120 may include all or a part of a user input unit 121 and a microphone unit 123.

[0198] The user input unit 121 may receive a user input such as a touch, a push operation, etc. Here, the user input unit 121 may be realized using at least one of various types of buttons, a touch sensor that receives a touch input, and a proximity sensor that receives an approaching motion.

[0199] The microphone unit 123 can receive the user's voice and sounds generated inside and outside the vehicle.

[0200] The output unit 130 is a device that outputs data of the electronic device 100 to a user in the form of video and / or audio. Here, the output unit 130 may include all or part of a display unit 131 and an audio output unit 133.

[0201] The display unit 131 is a device that outputs visually recognizable data to a user. The display unit 131 may be realized as a display unit provided on the front surface of a housing of the electronic device 100. In addition, the display unit 131 may be formed integrally with the electronic device 100 to output visually recognized data, or may be provided separately from the electronic device 100 like a HUD (Head Up Display) to output visually recognized data.

[0202] The audio output unit 133 is a device that outputs data that can be audibly recognized by the user. The audio output unit 133 can be realized by a speaker that expresses data to be notified to the user of the electronic device 100 as sound.

[0203] The inter-vehicle distance measuring unit 140 can perform the functions of the inter-vehicle distance measuring device 10 described above.

[0204] The augmented reality providing unit 160 may provide an augmented reality view mode. Here, the augmented reality is a method of visually overlaying additional information (e.g., a graphic element indicating a point of interest (POI), a graphic element indicating a risk of collision with a forward vehicle, a graphic element indicating a distance between vehicles, a graphic element indicating a curve, various additional information for assisting a driver in safe driving, etc.) on a screen including the real world that the user is actually viewing.

[0205] The augmented reality providing unit 160 may include all or some of a calibration unit, a 3D space generating unit, an object generating unit, and a mapping unit.

[0206] The calibration unit may perform calibration to estimate camera parameters corresponding to the camera from an image captured by the camera. Here, the camera parameters are parameters constituting a camera matrix, which is information representing a relationship between a real-world space and a photograph, and may include camera extrinsic parameters and camera intrinsic parameters.

[0207] The 3D space generation unit can generate a virtual 3D space based on the captured image captured by the camera. Specifically, the 3D space generation unit can generate the virtual 3D space by applying the camera parameters estimated by the calibration unit to the 2D captured image.

[0208] The object generating unit may generate objects for guidance in augmented reality, for example, a forward collision prevention guidance object, a route guidance object, a lane change guidance object, a lane departure guidance object, a curve guidance object, and the like.

[0209] The mapping unit may map the object generated by the object generation unit to the virtual 3D space generated by the 3D space generation unit. Specifically, the mapping unit may determine a position in the virtual 3D space of the object generated by the object generation unit, and map the object to the determined position.

[0210] Meanwhile, the communication unit 180 may be provided to allow the electronic device 100 to communicate with other devices. The communication unit 180 may include all or some of a location data unit 181, a wireless Internet unit 183, a broadcast transceiver unit 185, a mobile communication unit 186, a short-range communication unit 187, and a wired communication unit 189.

[0211] The location data unit 181 is a device that acquires location data through a Global Navigation Satellite System (GNSS). The GNSS refers to a navigation system that can calculate the location of a receiving terminal using radio signals received from an artificial satellite. Specific examples of the GNSS may include Global Positioning System (GPS), Galileo, Global Orbiting Navigational Satellite System (GLONASS), COMPASS, Indian Regional Navigational Satellite System (IRNSS), and Quasi-Zenith Satellite System (QZSS), depending on the operator. The location data unit 181 of the system according to an embodiment of the present invention may acquire location data by receiving a GNSS signal serving an area where the electronic device 100 is used. Alternatively, the location data unit 181 may acquire location data by communicating with a base station or an Access Point (AP) other than the GNSS.

[0212] The wireless Internet unit 183 is a device that connects to the wireless Internet to obtain or transmit data. The wireless Internet unit 183 can connect to the Internet network according to various communication protocols defined for transmitting and receiving wireless data, such as WLAN (Wireless LAN), Wibro (Wireless broadband), Wimax (World interoperability for microwave access), and HSDPA (High Speed ​​Downlink Packet Access).

[0213] The broadcast transmission / reception unit 185 is a device that transmits and receives broadcast signals by various broadcast systems. The broadcast systems that can be transmitted and received by the broadcast transmission / reception unit 185 may be DMBT (Digital Multimedia Broadcasting Terrestrial), DMBS (Digital Multimedia Broadcasting Satellite), MediaFLO (Media Forward Link Only), DVBH (Digital Video Broadcast Handheld), ISDBT (Integrated Services Digital Broadcast Terrestrial), etc. The broadcast signals transmitted and received by the broadcast transmission / reception unit 185 may include traffic data, life data, etc.

[0214] The mobile communication unit 186 can connect to a mobile communication network according to various mobile communication standards such as 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evolution (LTE) to perform voice and data communication.

[0215] The short-distance communication unit 187 is a device for short-distance communication. As described above, the short-distance communication unit 187 can communicate by Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (IrDA, Infrared Data Association), UWB (Ultra WideBand), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), etc.

[0216] The wired communication unit 189 is an interface device that can connect the electronic device 100 to other devices via a wired connection, and can be a USB module that can communicate via a USB port.

[0217] The communication unit 180 can communicate with other devices using at least one of a location data unit 181, a wireless Internet unit 183, a broadcast transmission / reception unit 185, a mobile communication unit 186, a short-range communication unit 187, and a wired communication unit 189.

[0218] As an example, if the electronic device 100 does not have a camera function, images captured by a vehicle-mounted video capture device such as a car dash cam or a car video recorder can be received using at least one of the short-range communication unit 187 and the wired communication unit 189.

[0219] As another example, when communicating with a plurality of devices, it is also possible to communicate with one of the devices via short-range communication unit 187 and with the other via wired communication unit 189.

[0220] The sensing unit 190 is a device capable of detecting a current state of the electronic device 100. The sensing unit 190 may include a motion sensing unit 191 and a light sensing unit 193, in whole or in part.

[0221] The motion sensing unit 191 can detect the motion of the electronic device 100 in a three-dimensional space. The motion sensing unit 191 can include a three-axis geomagnetic sensor and a three-axis acceleration sensor. By combining the motion data acquired by the motion sensing unit 191 with the position data acquired by the position data unit 181, the trajectory of the vehicle to which the electronic device 100 is attached can be calculated more accurately.

[0222] The light sensing unit 193 is a device that measures the illuminance of the surroundings of the electronic device 100. Using the illuminance data acquired by the light sensing unit 193, the brightness of the display unit 131 can be changed in response to the surrounding brightness.

[0223] The power supply unit 195 is a device that supplies power required for the operation of the electronic device 100 or the operation of another device connected to the electronic device 100. The power supply unit 195 may be a battery built into the electronic device 100 or a device that receives power from an external power source such as a vehicle. In addition, the power supply unit 195 may be realized as the wired communication module 119 or may be realized as a device that receives power wirelessly depending on the form of receiving power.

[0224] The control unit 170 controls the overall operation of the electronic device 100. Specifically, the control unit 170 may control all or some of the storage unit 110, the input unit 120, the output unit 130, the inter-vehicle distance measurement unit 140, the augmented reality providing unit 160, the communication unit 180, the sensing unit 190, and the power supply unit 195.

[0225] Specifically, the control unit 170 may control the output unit 130 to output a front vehicle collision notification or a front vehicle departure notification according to the inter-vehicle distance calculated by the inter-vehicle distance measurement unit 140. As an example, the output unit 130 may include a display unit 131 that combines a captured driving image with a guide object to output an augmented reality image. In this case, the control unit 170 may control the display unit 131 to generate a guide object for the front vehicle collision notification or the front vehicle departure notification, and to display the generated guide object by superimposing it on a front vehicle display area of ​​the augmented reality image.

[0226] When performing forward vehicle collision warning, the displayed guidance object may be different depending on the collision risk level corresponding to the distance difference between the first vehicle and the second vehicle. As an example, the distance difference between the first vehicle and the second vehicle may be divided into a plurality of levels, and if the inter-vehicle distance is smaller than a first value, a guidance object for providing guidance on the first risk level may be displayed, if the inter-vehicle distance is larger than the first value and smaller than a second value, a guidance object for providing guidance on the second risk level that is more dangerous than the first risk level may be displayed, and if the inter-vehicle distance is larger than the second value, a guidance object for providing guidance on the third risk level that is more dangerous than the second risk level may be displayed.

[0227] Alternatively, when a forward vehicle departure notification is performed, the displayed guidance object may be different depending on a departure request level corresponding to a distance difference between the first vehicle and the second vehicle. As an example, the distance difference between the first vehicle and the second vehicle may be divided into a plurality of levels, and if the inter-vehicle distance is smaller than a first value, a guidance object for providing guidance for the first departure request level, which requires a faster departure than the first departure request level, may be displayed if the inter-vehicle distance is larger than the first value and smaller than a second value, and a guidance object for providing guidance for the third departure request level, which requires a faster departure than the second departure request level, may be displayed if the inter-vehicle distance is larger than the second value.

[0228] 18 is a diagram illustrating a system network connected to an electronic device according to an embodiment of the present invention. Referring to FIG. 18, the electronic device 100 according to an embodiment of the present invention may be implemented as various devices provided in a vehicle, such as a navigation system, a vehicle-use image capturing device, a smartphone, or other devices providing an augmented reality interface for a vehicle, and may be connected to various communication networks and other electronic devices 61 to 64.

[0229] In addition, the electronic device 100 can calculate the current location and the current time zone in response to radio signals received from the artificial satellite 70 in conjunction with a GPS module.

[0230] Each satellite 70 can transmit a different L-band frequency band, and the electronic device 100 can calculate its current location based on the time it takes for the L-band frequency transmitted from each satellite 70 to reach the electronic device 100.

[0231] Meanwhile, the electronic device 100 can wirelessly connect to the network 90 via a control station 80 (ACR), a base station 85 (RAS), an AP (Access Point), etc., by the communication unit 180. When the electronic device 100 is connected to the network 90, it can also indirectly connect to other electronic devices 61 and 62 connected to the network 90 to exchange data.

[0232] The electronic device 100 can also be indirectly connected to the network 90 via another device 63 having a communication function. For example, when the electronic device 100 is not provided with a module capable of connecting to the network 90, the electronic device 100 can communicate with the other device 63 having a communication function via a short-range communication module or the like.

[0233] 19 and 20 are diagrams illustrating a guidance screen for preventing a collision with a front vehicle of an electronic device according to an embodiment of the present invention. Referring to Fig. 19 and Fig. 20, the electronic device 100 may generate a guidance object indicating a risk of a vehicle collision according to a distance between the host vehicle and a front vehicle, and output the generated guidance object as an augmented reality.

[0234] For example, as shown in FIG. 19, when the distance between the vehicle and the vehicle ahead is equal to or greater than a predetermined distance, the electronic device 100 may generate and display a warning notification object 1501 that notifies the user that the user needs to pay attention to the situation.

[0235] In addition, as shown in FIG. 20, when the distance between the vehicle and the preceding vehicle becomes within a predetermined distance and the risk of collision with the preceding vehicle increases, the electronic device 100 may generate and display a danger notification object 1601 that notifies the user of the risk of collision.

[0236] Here, the warning notification object 1501 and the danger notification object 1601 are distinguished from each other by different colors and sizes to enhance the visibility of the driver. Also, the notification objects 1501 and 1601 may be realized as texture images and displayed as augmented reality, for example.

[0237] Furthermore, in order to allow the driver to more easily recognize the distance to the vehicle ahead, the electronic device 100 may digitize the distance between the vehicle ahead and the vehicle ahead calculated by the vehicle ahead distance measuring unit 140 and display it on a screen. For example, the vehicle ahead distance measuring unit 140 may calculate the distance between the vehicle ahead and the vehicle ahead, and the electronic device 100 may generate guide objects 1502 and 1602 indicating the distance between the vehicles ahead and display them on a screen.

[0238] Additionally, the electronic device 100 can output various kinds of guidance using voice.

[0239] 21 is a diagram showing an embodiment of an electronic device according to the present invention that does not include a photographing unit. Referring to FIG. 21, a vehicle-mounted image photographing device 200 provided separately from the electronic device 100 can configure a system according to the present invention using wired / wireless communication.

[0240] The electronic device 100 may include a display portion 131 provided on the front surface of the housing 191, a user input portion 121, and a microphone portion 123.

[0241] The vehicle video capture device 200 may include a camera 222 , a microphone 224 , and a mounting portion 281 .

[0242] Fig. 22 is a diagram showing an embodiment of an electronic device according to an embodiment of the present invention, in which an image capturing unit is provided. Referring to Fig. 22, when the electronic device 100 includes the image capturing unit 150, the image capturing unit 150 of the electronic device 100 captures an image of the area in front of the vehicle, and allows the user to recognize the display unit of the electronic device 100. In this way, a system according to an embodiment of the present invention can be realized.

[0243] Fig. 23 is a diagram showing an embodiment using a HUD (Head-Up Display) according to an embodiment of the present invention. Referring to Fig. 23, the HUD can display an augmented reality guidance screen on a head-up display through wired / wireless communication with another device.

[0244] For example, augmented reality may be provided by a HUD using the windshield of a vehicle, or an image overlay using another image output device, and the augmented reality providing unit 160 may generate an interface image to be overlaid on the real image or glass in this manner, thereby realizing an augmented reality navigation or vehicle infotainment system.

[0245] Meanwhile, the inter-vehicle distance measuring device 10 can be realized as one module of a system for autonomous driving and can perform a route guidance function, which will be described in more detail with reference to Figs. 24 and 25.

[0246] Fig. 24 is a block diagram showing the configuration of an autonomous vehicle according to an embodiment of the present invention. Referring to Fig. 24, an autonomous vehicle 2000 according to this embodiment may include a control device 2100, sensing modules 2004a, 2004b, 2004c, and 2004d, an engine 2006, and a user interface 2008.

[0247] The autonomous vehicle 2000 can have an autonomous mode or a manual mode. As an example, a user input received by the user interface 2008 can switch from manual mode to autonomous mode or vice versa.

[0248] When the vehicle 2000 is operated in an autonomous driving mode, the autonomous vehicle 2000 can be operated under the control of the control device 2100.

[0249] In this embodiment, the control device 2100 may include a controller 2120 including a memory 2122 and a processor 2124 , a sensor 2110 , a wireless communication device 2130 , and an object detection device 2140 .

[0250] In this embodiment, the object detection device 2140 is a device for detecting an object located outside the vehicle 2000. The object detection device 2140 can detect an object located outside the vehicle 2000 and generate object information based on the detection result.

[0251] The object information may include information regarding the presence or absence of an object, position information of the object, distance information between the vehicle and the object, and relative speed information between the vehicle and the object.

[0252] The object may be a lane, another vehicle, a pedestrian, a traffic signal, a light, a road, a structure, a speed bump, a topographical feature, an animal, etc., and may include various objects located outside the vehicle 2000. Here, the traffic signal is a concept including a traffic light, a traffic sign board, and a picture or text painted on the road surface. And, the light may be light generated by a lamp provided on another vehicle, light generated by a street lamp, or sunlight.

[0253] The structures may be objects that are located around the road and fixed to the ground. For example, the structures may include street lights, roadside trees, buildings, utility poles, traffic lights, bridges, etc. The topographical features may include mountains, hills, etc.

[0254] The object detection device 2140 may include a camera module. The controller 2120 may extract object information from an external image captured by the camera module and process the information related to the object information.

[0255] The object detection device 2140 may further include an imaging device for recognizing the external environment. In addition to LIDAR, RADAR, GPS devices, odometry devices, and other computer vision devices, ultrasonic sensors, and infrared sensors may be used, and these devices may be operated selectively or simultaneously as needed to enable more precise detection.

[0256] Also, the sensor 2110 may be connected to sensing modules 2004a, 2004b, 2004c, and 2004d that detect the internal / external environment of the vehicle to acquire various sensing information. Here, the sensor 2110 may include an attitude sensor (e.g., a yaw sensor, a roll sensor, and a pitch sensor), a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight detection sensor, a heading sensor, a gyro sensor, a position module, a vehicle forward / reverse sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor based on steering wheel rotation, a vehicle internal temperature sensor, a vehicle internal humidity sensor, an ultrasonic sensor, an illuminance sensor, an accelerator pedal position sensor, and a brake pedal position sensor.

[0257] As a result, the sensor 2110 can acquire sensing signals related to vehicle posture information, vehicle collision information, vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle inclination information, vehicle forward / backward information, battery information, fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, steering wheel rotation angle, vehicle exterior illuminance, pressure applied to the accelerator pedal, pressure applied to the brake pedal, etc.

[0258] In addition, the sensor 2110 may further include an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake temperature sensor (ATS), a water temperature sensor (WTS), a throttle position sensor (TPS), a TDC sensor, a crankshaft position sensor (CAS), etc.

[0259] In this manner, the sensor 2110 can generate vehicle status information based on the sensing data.

[0260] The wireless communication device 2130 is configured to realize wireless communication between the autonomous vehicle 2000. For example, the wireless communication device 2130 enables the autonomous vehicle 2000 to communicate with a user's mobile phone or other wireless communication devices 2130, other vehicles, a central device (traffic control device), a server, etc. The wireless communication device 2130 can transmit and receive wireless signals according to a connection wireless protocol. The wireless communication protocol can be, but is not limited to, Wi-Fi, Bluetooth, Long-Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global Systems for Mobile Communications (GSM).

[0261] In addition, in this embodiment, the autonomous vehicle 2000 can also realize inter-vehicle communication by the wireless communication device 2130. That is, the wireless communication device 2130 can communicate with other vehicles on the road and with other vehicles by vehicle-to-vehicle (V2V) communication. The autonomous vehicle 2000 can transmit and receive information such as driving warnings and traffic information by vehicle-to-vehicle communication, and can also request information from or receive requests from other vehicles. For example, the wireless communication device 2130 can perform V2V communication by a dedicated short-range communication (DSRC) device or a Celluar-V2V (C-V2V) device. In addition to inter-vehicle communication, communication between a vehicle and other things (e.g., electronic devices carried by pedestrians) (V2X, Vehicle to Everything communication) can also be realized by the wireless communication device 2130.

[0262] In this embodiment, the controller 2120 is a unit that controls the overall operation of each unit in the vehicle 2000, and may be configured when the vehicle is manufactured by a vehicle manufacturer, or may be additionally configured after manufacture to perform an autonomous driving function. Alternatively, the controller 2120 configured at the time of manufacture may be upgraded to include a configuration for performing a continuous additional function. Such a controller 2120 may be named an ECU (Electronic Control Unit).

[0263] The controller 2120 can collect various data from the connected sensors 2110, the object detection device 2140, the wireless communication device 2130, etc., and transmit control signals based on the collected data to the sensors 2110, the engine 2006, the user interface 2008, the wireless communication device 2130, and the object detection device 2140 included as other components in the vehicle. In addition, although not shown, the controller 2120 can also transmit control signals to an acceleration device, a braking system, a steering device, or a navigation device related to the running of the vehicle.

[0264] In this embodiment, the controller 2120 can control the engine 2006, for example, to detect the speed limit of the road on which the autonomous vehicle 2000 is traveling and control the engine 2006 so that the traveling speed does not exceed the speed limit, or to control the engine 2006 to accelerate the traveling speed of the autonomous vehicle 2000 within a range that does not exceed the speed limit.

[0265] In addition, the controller 2120 can detect the distance to a vehicle located ahead while the autonomous vehicle 2000 is traveling, and control the engine 2006 to control the traveling speed according to the distance. Specifically, the autonomous vehicle 2000 can be equipped with an inter-vehicle distance measurement device 10 according to an embodiment of the present invention, and the inter-vehicle distance measurement device 10 can measure the distance between the vehicle 1000 and a target vehicle, and transmit the measured inter-vehicle distance value to the controller 2120.

[0266] In this case, the controller 2120 can control the autonomous driving of the vehicle 2000 by controlling deceleration, acceleration, and constant speed of the vehicle 2000 based on the following distance information acquired from the following distance measurement device 10. Specifically, if the acquired following distance is smaller than a preset distance, the controller 2120 can control the speed of the vehicle 2000 to be reduced from the current speed to a predetermined speed, or control various units (brake, steering wheel, etc.) provided in the vehicle 2000 to stop the vehicle 2000. In other words, the controller 2120 can control the autonomous driving of the vehicle 2000 based on the following distance acquired from the following distance measurement device 10.

[0267] In addition, according to yet another embodiment of the present invention, the controller 2120 can also control the driving speed by issuing a command to the driving device of the vehicle 2000 so that the inter-vehicle distance obtained from the inter-vehicle distance measurement device 10 is maintained at a preset constant distance.

[0268] According to still another embodiment of the present invention, when the following distance acquired from the following distance measurement device 10 is greater than a preset distance, the controller 2120 can control various units (brake, steering wheel, etc.) provided in the vehicle 2000 to increase the speed of the vehicle 2000 from the current speed to a predetermined speed. That is, the controller 2120 can control the autonomous driving of the vehicle 2000 based on the following distance acquired from the following distance measurement device 10.

[0269] The inter-vehicle distance measuring device 10 can be configured as one module in a control device 2100 of an autonomous vehicle 2000. In other words, the memory 2122 and the processor 2124 of the control device 2100 may be configured to realize the inter-vehicle distance measuring method according to the present invention in a software manner.

[0270] If there is another vehicle or an obstacle ahead of the vehicle, the engine 2006 or the brake system can be controlled to slow down the traveling vehicle, and in addition to the speed, the trajectory, the driving route, and the steering angle can be controlled. Alternatively, the controller 2120 can generate necessary control signals according to other external environment recognition information such as the vehicle's driving lane, driving signals, etc., to control the traveling of the vehicle.

[0271] In addition to generating its own control signals, the controller 2120 can also control the running of the vehicle by communicating with surrounding vehicles or a central server and transmitting instructions for controlling peripheral devices based on the information received.

[0272] In addition, when the position or angle of view of the camera module 2150 is changed, accurate recognition of a vehicle or lane according to the present embodiment may be difficult, so in order to prevent this, the controller 2120 may generate a control signal to control the camera module 2150 to perform calibration. Therefore, in the present embodiment, the controller 2120 generates a calibration control signal to the camera module 2150, so that the normal mounting position, direction, angle of view, etc. of the camera module 2150 can be continuously maintained even if the mounting position of the camera module 2150 is changed due to vibration or impact caused by the movement of the autonomous vehicle 2000. The controller 2120 may generate a control signal to calibrate the camera module 2150 when the initial mounting position, direction, angle of view information of the camera module 2150 stored in advance and the initial mounting position, direction, angle of view information of the camera module 2150 measured during the traveling of the autonomous vehicle 2000 change by more than a threshold value.

[0273] In this embodiment, the controller 2120 may include a memory 2122 and a processor 2124. The processor 2124 may execute software stored in the memory 2122 in response to a control signal from the controller 2120. Specifically, the controller 2120 may store data and instructions for performing a method for measuring a distance between vehicles according to the present invention in the memory 2122, and the instructions may be executed by the processor 2124 to realize one or more methods disclosed herein.

[0274] In this case, the memory 2122 may be stored in a non-volatile recording medium executable by the processor 2124. The memory 2122 may store software and data in an appropriate internal or external device. The memory 2122 may be composed of a random access memory (RAM), a read only memory (ROM), a hard disk, or a memory 2122 device connected to a dongle.

[0275] The memory 2122 may store at least an operating system (OS), user applications, and executable instructions, and may also store application data and array data structures.

[0276] The processor 2124 may be a microprocessor or any suitable electronic processor, and may be a controller, microcontroller, or a state machine.

[0277] The processor 2124 may be implemented by a combination of computing devices, which may be digital signal processors, microprocessors, or any suitable combination thereof.

[0278] Meanwhile, the autonomous vehicle 2000 may further include a user interface 2008 for user input to the control device 2100 described above. The user interface 2008 may allow the user to input information through appropriate interaction. For example, the user interface 2008 may be realized by a touch screen, a keypad, an operation button, etc. The user interface 2008 may transmit the input or command to the controller 2120, and the controller 2120 may perform a control operation of the vehicle in response to the input or command.

[0279] Additionally, user interface 2008 may enable autonomous vehicle 2000 to communicate with devices external to autonomous vehicle 2000 via wireless communication device 2130. For example, user interface 2008 may be capable of interfacing with a mobile phone, tablet computer, or other computing device.

[0280] Further, in this embodiment, the autonomous vehicle 2000 is described as including the engine 2006, but may include other types of propulsion systems. For example, the vehicle may be powered by electric energy, hydrogen energy, or a hybrid system that combines these. Thus, the controller 2120 may include the propulsion mechanisms of the propulsion system of the autonomous vehicle 2000 and provide control signals to the configuration of each propulsion mechanism.

[0281] Hereinafter, with reference to FIG. 25, a detailed configuration of the control device 2100 that performs the method for measuring the inter-vehicle distance according to the present invention will be described in more detail.

[0282] The control device 2100 includes a processor 2124. The processor 2124 may be a general-purpose single or multi-chip microprocessor, a special purpose microprocessor, a microcontroller, a programmable gate array, or the like. The processor may also be referred to as a central processing unit (CPU). In this embodiment, the processor 2124 may also be used as a combination of multiple processors.

[0283] The controller 2100 also includes a memory 2122. The memory 2122 may be any electronic component capable of storing electronic information. The memory 2122 may also include a single memory as well as a combination of memories 2122.

[0284] Data and instructions 2122a for performing the method for measuring a distance between vehicles according to the present invention may be stored in the memory 2122. When the processor 2124 executes the instructions 2122a, all or a part of the instructions 2122a and data 2122b required to execute the instructions may be loaded (2124a, 2124b) onto the processor 2124.

[0285] The controller 2100 may include a transmitter 2130a, a receiver 2130b, or a transceiver 2130c to allow transmission and reception of signals. One or more antennas 2132a, 2132b may be electrically coupled to the transmitter 2130a, the receiver 2130b, or each transceiver 2130c, which may additionally include an antenna.

[0286] The controller 2100 may include a digital signal processor (DSP) 2170. The DSP 2170 enables the vehicle to quickly process digital signals.

[0287] The control device 2100 may include a communication interface 2180. The communication interface 2180 may include one or more ports and / or communication modules for coupling other devices to the control device 2100. The communication interface 2180 may allow a user and the control device 2100 to interact.

[0288] The various components of the controller 2100 may be coupled together by one or more buses 2190, which may include a power bus, a control signal bus, a status signal bus, a data bus, etc. Under the control of the processor 2124, the components may communicate information to each other via the bus 2190 to perform their intended functions.

[0289] Meanwhile, in the above embodiment, for convenience of explanation, the calculation of the distance between the reference vehicle and the preceding vehicle is described as an example, but the present invention is not limited thereto. The method for measuring the distance between vehicles according to the present invention can be similarly applied to the calculation of the distance between the reference vehicle and the following vehicle.

[0290] On the other hand, when terms such as "first," "second," "third," and "fourth" are used in the specification and claims, they are used to distinguish between similar components and to describe a particular sequence or order of occurrence, although not necessarily so. Such terms used should be understood to be interchangeable under appropriate circumstances, such that the embodiments of the invention described herein can operate in other sequences than those shown or described herein. Similarly, when a method is described herein as including a series of steps, the order of such steps presented herein is not necessarily the order in which such steps may be performed, and any described steps may be omitted and / or any other steps not described herein may be added to the method. For example, a first component may be named a second component, and similarly, a second component may be named a first component, without departing from the scope of the invention.

[0291] Additionally, in the specification and claims, terms such as "left", "right", "front", "rear", "top", "bottom", "above", "below", and the like are used for purposes of explanation and not necessarily for describing fixed relative positions. Terms so used should be understood to be interchangeable under appropriate circumstances such that the embodiments of the invention described herein can operate in other orientations than those shown or described herein, for example. The term "coupled" as used herein is defined as directly or indirectly connected in an electrical or non-electrical manner. Herein, objects described as "adjacent" to one another can be in physical contact with one another, in close proximity to one another, or in the same general area or region as one another, as appropriate in the context in which the phrase is used. Herein, the phrase "in one embodiment" refers to the same embodiment, although this is not necessarily the case.

[0292] In addition, in the specification and claims, the terms "connected," "couple," "fastened," "fasten," "coupled," "join," and various variations of these expressions are used to mean either being directly connected to other components or being indirectly connected via other components.

[0293] In contrast, when an element is referred to as being "directly coupled" or "directly connected" to another element, it should be understood that there are no other elements in between.

[0294] In addition, the suffixes "module" and "section" for components used in this specification are given or mixed for the sole consideration of ease in drafting the specification, and do not in themselves have meanings or roles that are distinct from each other.

[0295] In addition, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. As used in this specification, the singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, the terms "comprise" or "include" should not be interpreted as including all of the various components or steps described in the specification, but may not include some of the components or some of the steps, or may further include additional components or steps.

[0296] The present invention has been described so far with reference to its preferred embodiments. All the embodiments and conditional examples disclosed in this specification are described so that a person skilled in the art having ordinary knowledge in the technical field of the present invention can easily understand the principles and concepts of the present invention, and a person skilled in the art can understand that the present invention can be realized in modified forms without departing from the essential characteristics of the present invention.

[0297] Accordingly, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense, and the scope of the present invention is indicated in the appended claims, not the foregoing description, and all differences within the scope of the appended claims are to be construed as being within the scope of the present invention.

[0298] Meanwhile, the method for measuring a distance between vehicles according to the various embodiments of the present invention described above may be implemented as a program and provided to a server or device, whereby each device can connect to the server or device in which the program is stored and download the program.

[0299] In addition, the methods according to the various embodiments of the present invention described above may be implemented as a program and stored in various non-transitory computer readable media for provision. A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc. In particular, the various applications or programs described above may be stored in non-transitory readable media such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc. for provision.

[0300] Furthermore, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and it goes without saying that various modifications can be made by a person having ordinary knowledge in the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical ideas and prospects of the present invention. [Explanation of symbols]

[0301] 10 Inter-vehicle distance measuring device 11 Video Acquisition Department 12 Vehicle detection unit 13 Feature point detection unit 14 Feature point change value calculation unit 15 Vehicle distance calculation unit

Claims

1. A method for measuring a vehicle distance using a processor, comprising: acquiring a driving image captured by an image capturing device of a first vehicle while the first vehicle is moving; detecting a second vehicle from the acquired driving image; when the second vehicle is not detected from the traveling video, detecting a first feature point of a second vehicle area of ​​the second vehicle from a first frame corresponding to a frame in which the second vehicle is detected and preceding a frame in which the second vehicle is not detected among a plurality of frames constituting the traveling video; detecting a second feature point in a second frame corresponding to the current frame by tracking the detected first feature point; calculating a feature point change value between the first feature point and the second feature point; calculating a distance between the first vehicle and the second vehicle based on the calculated feature point change value.

2. In the step of detecting the second vehicle, The method of claim 1 , further comprising: detecting the second vehicle using a learning model constructed by machine learning or deep learning for a vehicle image.

3. In the step of detecting the first feature point, The method for measuring the distance between vehicles as described in claim 2, characterized in that as the distance between the first vehicle and the second vehicle becomes closer, if the second vehicle is not detected using the constructed learning model, a step of detecting a first feature point in the second vehicle area is performed.

4. In the step of detecting the first feature point, The method for measuring a distance between vehicles according to claim 1 , further comprising the steps of: setting an intermediate region of the vehicle as a region of interest in a second vehicle region in a frame; and detecting the first feature point from the set region of interest.

5. In the step of detecting the second feature point, 5. The method for measuring a distance between vehicles according to claim 1, further comprising: detecting the second feature point in the second frame by tracking the second feature point using an optical flow of the detected first feature point.

6. 6. The method for measuring a distance between vehicles according to claim 5, further comprising a step of filtering out second feature points not appearing in the second frame and first feature points corresponding to second feature points not appearing in the second frame when tracking the second feature points using the optical flow.

7. The step of calculating the feature point change value includes: calculating a mean pixel position of the first feature points; calculating a first average pixel distance by averaging pixel distances from the calculated average pixel position of the first feature points to each of the first feature points; calculating an average pixel position of the second feature points; calculating a second average pixel distance by averaging distances from the calculated average pixel position of the second feature points to each of the second feature points; 7. The method for measuring a distance between vehicles according to claim 1, further comprising the step of: calculating an average pixel distance ratio between the first average pixel distance and the second average pixel distance.

8. The step of calculating the vehicle distance includes:

8. The method for measuring a distance between vehicles according to claim 7, further comprising a step of calculating an image width of the second vehicle in the second frame by multiplying the image width of the second vehicle in the first frame by the calculated average pixel distance ratio.

9. The step of calculating the vehicle distance includes:

9. The method for measuring a distance between vehicles as described in claim 8, further comprising a step of calculating a distance between the camera of the first vehicle and the second vehicle based on the calculated image width of the second vehicle in the second frame, a focal length of the first camera, and a predicted width of the second vehicle.

10. The step of calculating the vehicle distance includes: Calculating an image width ratio between an image width of the detected second vehicle and an image width of a road on which the second vehicle is located; determining a size class of the second vehicle based on the calculated ratio; and The method of claim 9, further comprising: calculating an expected width of the second vehicle based on the determined size class of the second vehicle.

11. The method for measuring a distance between vehicles described in any one of claims 1 to 10, further comprising a step of generating guidance data for providing guidance on a collision risk level corresponding to the distance difference between the first vehicle and the second vehicle when the calculated distance between the vehicles is smaller than a preset distance.

12. The method for measuring a distance between vehicles according to claim 1 , further comprising a step of generating a control signal for controlling the autonomous driving of the first vehicle based on the calculated distance between vehicles.

13. A vehicle distance measuring device, an image acquisition unit that acquires a traveling image captured by an image capture device of the first vehicle while the first vehicle is traveling; a vehicle detection unit that detects a second vehicle from the acquired driving image; a feature point detection unit that, when the second vehicle is not detected from the driving video, detects a first feature point of a second vehicle area of ​​the second vehicle from a first frame corresponding to a frame in which the second vehicle is detected and which is a frame prior to the frame in which the second vehicle is not detected among a plurality of frames constituting the driving video, and detects a second feature point in a second frame corresponding to a current frame by tracking the detected first feature point; a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point; a vehicle distance calculation unit that calculates a distance from the image capture device of the first vehicle to the second vehicle based on the calculated feature point change value.

14. The vehicle detection unit The device according to claim 13, further comprising: a learning model constructed by machine learning or deep learning for a vehicle image to detect the second vehicle.

15. The feature point detection unit The inter-vehicle distance measurement device according to claim 14, characterized in that as the distance between the first vehicle and the second vehicle becomes closer, if the second vehicle is not detected using the constructed learning model, a first feature point in the second vehicle area is detected.

16. The feature point detection unit The inter-vehicle distance measurement device according to claim 13 , characterized in that in a second vehicle area in a frame, a middle area of ​​the vehicle is set as an area of ​​interest, and the first feature point is detected from the set area of ​​interest.

17. The feature point detection unit 17. The inter-vehicle distance measurement device according to claim 13, further comprising: a step of detecting the second feature point in the second frame by tracking the second feature point using an optical flow of the detected first feature point.

18. The feature point detection unit 18. The inter-vehicle distance measurement device according to claim 17, characterized in that, when tracking the second feature points using the optical flow, second feature points not appearing in the second frame and first feature points corresponding to the second feature points not appearing in the second frame are filtered.

19. The feature point change value calculation unit Calculating an average pixel position of the first feature points, and calculating a first average pixel distance by averaging distances from the calculated average pixel position of the first feature points to each of the first feature points; an average pixel distance calculation unit that calculates an average pixel position of the second feature points and calculates a second average pixel distance by averaging distances from the calculated average pixel position of the second feature points to each of the second feature points; 19. The inter-vehicle distance measurement device according to claim 13, further comprising: a ratio calculation unit that calculates an average pixel distance ratio between the first average pixel distance and the second average pixel distance.

20. The vehicle distance calculation unit The inter-vehicle distance measurement device according to claim 19, characterized in that the image width of the second vehicle in the second frame is calculated by multiplying the image width of the second vehicle in the first frame by the calculated average pixel distance ratio.

21. The vehicle distance calculation unit The vehicle distance measurement device according to claim 20, further comprising: a first imaging device that detects an image of the second vehicle; a first imaging device that detects an image of the second vehicle; a first imaging device that detects an image of the second vehicle;

22. The vehicle distance calculation unit 22. The inter-vehicle distance measurement device of claim 21, further comprising: calculating an image width ratio between an image width of the detected second vehicle and an image width of a road on which the second vehicle is located; determining a size class of the second vehicle based on the calculated ratio; and calculating a predicted width of the second vehicle based on the determined size class of the second vehicle.

23. 23. The vehicle distance measurement device according to claim 13, further comprising a guidance data generation unit that generates guidance data for providing guidance on a collision risk level corresponding to a distance difference between the first vehicle and the second vehicle when the calculated vehicle distance is smaller than a preset distance.

24. 24. The inter-vehicle distance measurement device according to claim 13, further comprising an autonomous driving control signal generation unit that generates a control signal for controlling the autonomous driving of the first vehicle based on the calculated inter-vehicle distance.

25. An electronic device that provides guidance for assisting a driver based on a vehicle distance, an output unit that outputs guidance information that can be confirmed by the driver; an image acquisition unit that acquires a driving image captured by an image capture device; a vehicle detection unit that detects a second vehicle from the acquired driving image; a feature point detection unit that, when the second vehicle is not detected from the driving video, detects a first feature point of a second vehicle area of ​​the second vehicle from a first frame corresponding to a frame in which the second vehicle is detected and which is a frame prior to the frame in which the second vehicle is not detected among a plurality of frames constituting the driving video, and detects a second feature point in a second frame corresponding to a current frame by tracking the detected first feature point; a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point; a vehicle distance calculation unit that calculates a vehicle distance from the image capture device of the first vehicle to the second vehicle based on the calculated feature point change value; a control unit that controls the output unit to output a forward vehicle collision alert or a forward vehicle departure alert in accordance with the calculated distance.

26. The output unit further includes a display unit that combines the captured driving image with a guide object to output an augmented reality image, The control unit is 26. The electronic device of claim 25, further comprising: controlling the display unit to generate a guide object for the forward vehicle collision notification, and to superimpose and display the generated guide object for the forward vehicle collision notification on a forward vehicle display area in the augmented reality image.

27. A computer-readable recording medium having a program recorded thereon for executing the method for measuring a vehicle distance according to any one of claims 1 to 12.

28. A program stored in a computer-readable recording medium, the program causing a vehicle to execute the method for measuring a vehicle distance according to any one of claims 1 to 12.

Images