Measurement method of inter-vehicle distance based on vehicle image, inter-vehicle distance measuring device, electronic device, computer program, and computer-readable recording medium

Abstract

To provide an inter-vehicle distance measurement method that measures distance between an object vehicle and a user's vehicle based on a vehicle image.SOLUTION: An inter-vehicle measurement method includes: an image acquisition section 11 for acquiring a traveling image photographed by a photographing device of a first traveling vehicle; a vehicle detection section 12 for detecting a second vehicle from the acquired traveling image; a feature point detection section 13 for detecting a second feature point in a second frame corresponding to a current frame by detecting a first feature point in a second vehicle area from a first frame corresponding to a frame in which the second vehicle is detected before a frame in which the second vehicle is not detected among a plurality of frames constituting the driving image when the second vehicle is not detected from the traveling image, and by tracking the first detected feature point; a feature point change value calculation section 14 for calculating a feature point change value between the first feature point and the second feature point; and an inter-vehicle distance calculation section 15 for calculating distance from a photographing device of the first vehicle to the second vehicle on the basis of the calculated feature point change value.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a method for measuring the distance between vehicles based on vehicle images, a vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium. More specifically, the present invention relates to a method for measuring the distance between vehicles located at a short distance by tracking feature points of vehicle images, a vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium.

Background Art

[0002] When a vehicle is running, the most important things are safe driving and prevention of traffic accidents. For this purpose, various auxiliary devices for controlling the attitude of the vehicle and the functions of vehicle components, and safety devices such as seat belts and airbags are installed in the vehicle.

[0003] Moreover, recently, devices such as black boxes are installed in vehicles to store the driving images of the vehicles and the data transmitted from various sensors, so that the causes of vehicle accidents can be investigated when vehicle accidents occur. Portable terminals such as smartphones and tablet computers can also be equipped with black boxes or navigation applications, and are being used as such vehicle devices.

[0004] Therefore, in recent years, advanced driver assistance systems (ADAS) that assist vehicle drivers during vehicle operation using driving images captured during vehicle operation have been developed and popularized to achieve safe driving and driver convenience.

[0005] Among the functions provided by such ADAS, the Forward Collision Warning System (FCWS) function detects a preceding vehicle located ahead in the driving route from the captured driving video, measures the distance to the detected preceding vehicle, and guides the driver that there is a risk of collision according to the distance.

[0006] That is, for FCWS, detection of a preceding vehicle is necessary. Conventionally, for detection of a preceding vehicle, a video processing method using the shadow of the preceding vehicle in the captured driving video or a machine learning method that learns and detects images of many vehicles has been used. Both of these two methods have high detection performance when a vehicle area including the lower end portion of the vehicle exists in the driving video.

[0007] However, a vehicle encounters various driving environments during driving, and a situation where the lower end portion of the vehicle is not included in the driving video may occur. As an example, in the process of discovering a preceding vehicle during driving of the vehicle and the driving speed decreasing, the distance between the vehicle and the preceding vehicle becomes very close. As shown in FIG. 1, when the distance between vehicle 1 and preceding vehicle 2 is far (2-1), the driving video captured by vehicle 1 includes a vehicle video including the lower end portion of the preceding vehicle. However, when the distance between vehicle 1 and preceding vehicle 2 becomes close (for example, within 10 m) (2-2), the lower end portion of the preceding vehicle is not included in the driving video.

[0008] Thus, when the distance to the preceding vehicle becomes close during driving of the vehicle and the lower end portion of the preceding vehicle is not included in the driving video, there has been a problem that the conventional method for detecting a preceding vehicle based on shadow or learning cannot sufficiently detect the preceding vehicle.

[0009] On the other hand, such an inter-vehicle distance measurement technology is a core technology for autonomous driving of an autonomous vehicle that has been actively discussed recently. If a preceding vehicle cannot be sufficiently detected during autonomous driving and the inter-vehicle distance cannot be measured, it may lead to an accident, so the importance of the inter-vehicle distance measurement technology is increasing more and more. Summary of the Invention

Problems to be Solved by the Invention

[0010] The present invention has been made to solve the above problems, and an object of the present invention is to track a target vehicle by tracking feature points even when the lower end portion of the target vehicle is not photographed and detection of the target vehicle using a running video becomes impossible as the distance between the own vehicle and the target vehicle (front vehicle or rear vehicle) to be measured approaches, thereby measuring the distance between the target vehicle and the own vehicle. Another object is to provide a method for measuring an inter-vehicle distance based on a vehicle video, an inter-vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium.

[0011] Another object of the present invention is to provide a method for measuring an inter-vehicle distance based on a vehicle video, an inter-vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium that provide guidance based on the inter-vehicle distance using the measured distance.

[0012] Furthermore, an object of the present invention is to provide a method for measuring an inter-vehicle distance based on a vehicle video, an inter-vehicle distance measuring device, an electronic device, a computer program, and a computer-readable recording medium that generate an autonomous driving control signal for the own vehicle using the measured distance.

Means for Solving the Problems

[0013] A method for measuring the inter-vehicle distance using a processor according to an embodiment of the present invention to achieve the above object includes: a step of acquiring a driving video captured by a photographing device of a first vehicle during driving; a step of detecting a second vehicle from the acquired driving video; a step of, when the second vehicle is not detected from the driving video, detecting a first feature point in a second vehicle region from a first frame corresponding to a frame in which the second vehicle was detected, which is before the frame in which the second vehicle was not detected, among a plurality of frames constituting the driving video; a step of detecting a second feature point in a second frame corresponding to the current frame by tracking the detected first feature point; a step of calculating a feature point change value between the first feature point and the second feature point; and a step of calculating an inter-vehicle distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value.

[0014] And in the step of detecting the second vehicle, the second vehicle can be detected by using a learning model constructed by machine learning or deep learning for the vehicle video.

[0015] Also, in the step of detecting the first feature point, when the second vehicle is not detected by using the constructed learning model as the distance between the first vehicle and the second vehicle approaches, the step of detecting the first feature point in the second vehicle region can be performed.

[0016] And in the step of detecting the first feature point, in the second vehicle region in the frame, an intermediate region of the vehicle can be set as a region of interest, and the first feature point can be detected from the set region of interest.

[0017] Also, in the step of detecting the second feature point, the second feature point in the second frame can be detected by tracking the second feature point using the optical flow of the detected first feature point.

[0018] When tracking the second feature point using the optical flow, the method may further include filtering a second feature point not appearing in the second frame and a first feature point corresponding to the second feature point not appearing in the second frame.

[0019] The step of calculating the feature point change value may include: calculating an average pixel position of the first feature point; calculating a first average pixel distance obtained by averaging pixel distances from the calculated average pixel position of the first feature point to each first feature point; calculating an average pixel position of the second feature point; calculating a second average pixel distance obtained by averaging pixel distances from the calculated average pixel position of the second feature point to each second feature point; and calculating an average pixel distance ratio between the first average pixel distance and the second average pixel distance.

[0020] The step of calculating the inter-vehicle distance may include calculating a video width of the second vehicle in the second frame by multiplying a video width of the second vehicle in the first frame by the calculated average pixel distance ratio.

[0021] The step of calculating the inter-vehicle distance may further include calculating an inter-vehicle distance from the imaging device of the first vehicle to the second vehicle based on the calculated video width of the second vehicle in the second frame, the focal length of the first imaging device, and a predicted width of the second vehicle.

[0022] The step of calculating the inter-vehicle distance may further include calculating a video width ratio between the detected video width of the second vehicle and a video width of the road on which the second vehicle is located; determining a size class of the second vehicle based on the calculated ratio; and calculating a predicted width of the second vehicle based on the determined size class of the second vehicle.

[0023] Further, when the calculated inter-vehicle distance is smaller than a preset distance, it may further include a step of generating guidance data for guiding a collision risk level corresponding to a distance difference between the first vehicle and the second vehicle.

[0024] And it may further include a step of generating a control signal for controlling autonomous driving of the first vehicle based on the calculated inter-vehicle distance.

[0025] On the other hand, an inter-vehicle distance measurement device according to an embodiment of the present invention for achieving the above object includes a video acquisition unit that acquires a driving video captured by a photographing device of a first vehicle during driving, a vehicle detection unit that detects a second vehicle from the acquired driving video, and when the second vehicle is not detected from the driving video, among a plurality of frames constituting the driving video, a first feature point of a second vehicle region of the second vehicle is detected from a first frame corresponding to a frame in which the second vehicle was detected before the frame in which the second vehicle was not detected, and by tracking the detected first feature point, a second feature point in a second frame corresponding to the current frame is detected. A feature point detection unit, a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point, and an inter-vehicle distance calculation unit that calculates a distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value.

[0026] And the vehicle detection unit can detect the second vehicle using a learning model constructed by machine learning or deep learning for vehicle videos.

[0027] Further, when the second vehicle is not detected using the constructed learning model as the distance between the first vehicle and the second vehicle approaches, the feature point detection unit can detect a first feature point of the second vehicle region.

[0028] Then, in the second vehicle area in the frame, the feature point detection unit sets the intermediate area of the vehicle as the area of interest, and can detect the first feature point from the set area of interest.

[0029] Also, the feature point detection unit can detect the second feature point in the second frame by tracking the second feature point using the optical flow of the detected first feature point.

[0030] Then, when tracking the second feature point using the optical flow, the feature point detection unit can filter the second feature point not appearing in the second frame and the first feature point corresponding to the second feature point not appearing in the second frame.

[0031] The feature point change value calculation unit may include an average pixel distance calculation unit that calculates the average pixel position of the first feature point, calculates a first average pixel distance obtained by averaging the pixel distances from the calculated average pixel position of the first feature point to each first feature point, calculates the average pixel position of the second feature point, and calculates a second average pixel distance obtained by averaging the pixel distances from the calculated average pixel position of the second feature point to each second feature point, and a ratio calculation unit that calculates an average pixel distance ratio between the first average pixel distance and the second average pixel distance.

[0032] Then, the inter-vehicle distance calculation unit can calculate the video width of the second vehicle in the second frame by multiplying the video width of the second vehicle in the first frame by the calculated average pixel distance ratio.

[0033] Also, the inter-vehicle distance calculation unit can calculate the distance from the imaging device of the first vehicle to the second vehicle based on the calculated video width of the second vehicle in the second frame, the focal length of the first imaging device, and the predicted width of the second vehicle.

[0034] Then, the inter-vehicle distance calculation unit calculates a video width ratio between the detected video width of the second vehicle and the video width of the road section where the second vehicle is located, determines the size class of the second vehicle based on the calculated ratio, and can calculate the predicted width of the second vehicle based on the determined size class of the second vehicle.

[0035] Further, when the calculated inter-vehicle distance is smaller than a preset distance, it can further include a guidance data generation unit that generates guidance data for guiding a collision risk level corresponding to the distance difference between the first vehicle and the second vehicle.

[0036] Then, it can further include an autonomous driving control signal generation unit that generates a control signal for controlling the autonomous driving of the first vehicle based on the calculated inter-vehicle distance.

[0037] On the other hand, an electronic device for providing guidance for assisting a driver based on an inter-vehicle distance according to an embodiment of the present invention for achieving the above object includes an output unit that outputs guidance information that can be confirmed by the driver, a video acquisition unit that acquires a driving video captured by a photographing device, a vehicle detection unit that detects a second vehicle from the acquired driving video, and when the second vehicle is not detected from the driving video, among a plurality of frames constituting the driving video, a first feature point of a second vehicle region of the second vehicle is detected from a first frame corresponding to a frame in which the second vehicle was detected before the frame in which the second vehicle was not detected, and a second feature point in a second frame corresponding to the current frame is detected by tracking the detected first feature point, a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point, an inter-vehicle distance calculation unit that calculates an inter-vehicle distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value, and a control unit that controls the output unit to output a forward vehicle collision notification or a forward vehicle departure notification according to the calculated distance.

[0038] The output unit further includes a display unit that outputs an augmented reality image by combining the captured driving video and the guidance object, and the control unit generates a guidance object for the front vehicle collision notification and displays the generated guidance object for the front vehicle collision notification on the front vehicle display area in the augmented reality image. The display unit can be controlled to be overlaid and displayed.

[0039] On the other hand, according to an embodiment of the present invention for achieving the above object, a computer-readable recording medium on which a program for executing the above-described inter-vehicle distance measurement method is recorded can be provided.

[0040] Further, according to an embodiment of the present invention for achieving the above object, a program for executing the above-described inter-vehicle distance measurement method can be provided.

Effects of the Invention

[0041] According to various embodiments of the present invention described above, as the distance between the host vehicle and the target vehicle to be measured approaches, the lower end portion of the target vehicle is not photographed, and even when it becomes impossible to detect the target vehicle using the driving video, the distance between the host vehicle and the target vehicle can be accurately measured by tracking the feature points.

[0042] Further, according to various embodiments of the present invention, at a short distance, a high processing speed can be achieved even with a low-spec terminal by tracking based on feature points, and real-time processing is possible.

[0043] Further, according to various embodiments of the present invention, even when the area of the target vehicle is hidden as the distance between the host vehicle and the target vehicle approaches, the collision notification function and the departure notification function can be accurately performed.

[0044] Further, according to various embodiments of the present invention, even when the area of the target vehicle is hidden as the distance between the host vehicle and the target vehicle approaches, the distance between the host vehicle and the target vehicle can be accurately calculated, and autonomous driving control for the host vehicle can be accurately performed.

Brief Description of the Drawings

[0045]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Modes for Carrying Out the Invention

[0046] The following content merely exemplifies the principles of the present invention. Therefore, those skilled in the art can invent various devices that implement the principles of the present invention and fall within the concept and scope of the present invention, even if not explicitly described or illustrated in this specification. Also, it should be understood that all conditional terms and embodiments listed in this specification are, in principle, intended only for the purpose of understanding the concept of the present invention and are not limited by the specifically listed embodiments and conditions as such.

[0047] Moreover, it should be understood that not only the principles, aspects, and embodiments of the present invention, but also all the detailed descriptions listing specific embodiments are intended to include structural and functional equivalents of such matters. Also, such equivalents should be understood to include not only currently known equivalents but also equivalents developed in the future, that is, all elements invented to perform the same function regardless of structure.

[0048] Therefore, for example, the block diagrams in this specification should be understood to show a conceptual view of exemplary circuits embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudocode, etc. can be substantially shown on a computer-readable medium and should be understood to show various processes performed by a computer or processor, regardless of whether the computer or processor is explicitly illustrated.

[0049] The functions of the various elements illustrated in a figure including a processor or functional blocks represented as similar concepts can be provided not only by dedicated hardware but also by the use of hardware having the ability to execute software in association with appropriate software. When provided by a processor, the said functions can be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which can be shared.

[0050] Also, the clear use of terms presented as processor, control, or similar concepts shall not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, shall be understood to implicitly include digital signal processor (DSP) hardware, ROM, RAM, and non-volatile memory for storing software. Other well-known and conventional hardware can also be included.

[0051] In the claims of this specification, components expressed as means for performing the functions described in the detailed description are intended to include all forms of software including, for example, combinations of circuit elements for performing the above functions or all methods for performing functions including firmware / microcode, etc., and are coupled to appropriate circuits for executing the software so as to perform the above functions. The present invention defined by such claims is understood to be equivalent to any means that can be grasped from this specification that can provide the above functions because the functions provided by variously listed means are combined and combined in the manner required by the claims.

[0052] The above objects, features, and advantages will become more apparent from the following detailed description in connection with the accompanying drawings, whereby those of ordinary skill in the technical field to which the present invention pertains should be able to easily implement the technical idea of the present invention. Also, when it is determined that a specific description of known technology related to the present invention may obscure the gist of the present invention in explaining the present invention, the detailed description thereof will be omitted.

[0053] Before explaining in detail various embodiments of the present invention, the names used in the present invention are defined as follows.

[0054] In this specification, the inter-vehicle distance may mean the distance on real-world coordinates. Here, the inter-vehicle distance may be the distance between the first vehicle and the second vehicle, or more precisely, the distance from the imaging device installed on the first vehicle to the second vehicle.

[0055] In addition, in this specification, the width of the vehicle may mean the width of the vehicle on the real-world coordinates or the lateral width of the vehicle on the real-world coordinates.

[0056] In addition, in this specification, the video width may mean the pixel width of the image formed on the imaging surface of the imaging device of the photographing device.

[0057] In addition, in this specification, the pixel distance may mean the distance between pixels formed on the imaging surface of the imaging device of the photographing device.

[0058] Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0059] FIG. 2 is a block diagram showing an inter-vehicle distance measurement device according to an embodiment of the present invention. FIG. 3 is a block diagram showing more specifically the inter-vehicle distance measurement device according to an embodiment of the present invention. Referring to FIGS. 2 and 3, the inter-vehicle distance measurement device 10 according to an embodiment of the present invention may include all or part of a video acquisition unit 11, a vehicle detection unit 12, a feature point detection unit 13, a feature point change value calculation unit 14, an inter-vehicle distance calculation unit 15, a guidance data generation unit 17, a travel control data generation unit 18, and a control unit 19. Further, the feature point change value calculation unit 14 may include an average pixel distance calculation unit 14-1 and an average pixel distance ratio calculation unit 14-2.

[0060] Here, the inter-vehicle distance measurement device 10 can measure the distance between a first vehicle that is a reference for distance measurement and a second vehicle that is an object of distance measurement. Here, the first vehicle is a vehicle that serves as a reference for distance measurement and may be named a "reference vehicle" or "own vehicle", and the second vehicle is a vehicle that is an object of distance measurement and may be named an "object vehicle". Further, the second vehicle is a vehicle located near the first vehicle and may include a vehicle in front of the first vehicle and a vehicle behind the first vehicle.

[0061] The inter-vehicle distance measuring device 10 can calculate the distance between the first vehicle and the second vehicle by controlling the activation of the feature point detection and tracking functions according to whether the second vehicle is detected from the driving video captured by the imaging device of the first vehicle.

[0062] Specifically, the first vehicle travels on a lane, and during the travel of the first vehicle, the second vehicle may first appear in front of or behind the first vehicle. At this time, the inter-vehicle distance measuring device 10 can detect the second vehicle from the driving video by Machine Learning or Deep Learning. Then, the inter-vehicle distance measuring device 10 can calculate the inter-vehicle distance between the second vehicle detected by Machine Learning or Deep Learning and the first vehicle.

[0063] However, as the distance between the first vehicle and the second vehicle approaches, the lower end portion of the second vehicle is no longer captured, and when the second vehicle is not detected from the driving video by Machine Learning or Deep Learning, the inter-vehicle distance measuring device 10 can detect feature points from the driving video, track the detected feature points, and calculate the distance between the first vehicle and the second vehicle. Specifically, the inter-vehicle distance measuring device 10 selects a first frame corresponding to the frame in which the second vehicle is detected, which is before the frame in which the second vehicle is not detected, from among a plurality of frames constituting the driving video, detects a first feature point from the second vehicle region in the selected first frame, and by tracking the detected first feature point, detects a second feature point in a second frame corresponding to the current frame, calculates the feature point change value between the first feature point and the second feature point, and based on the calculated feature point change value, can calculate the distance from the imaging device of the first vehicle to the second vehicle.

[0064] That is, the operation of the inter-vehicle distance measuring device 10 according to the present invention will be described in the order of steps. As a first step, a second vehicle first appears in front of the first vehicle. As a second step, the inter-vehicle distance measuring device 10 detects the second vehicle from the running video by machine learning or deep learning, and calculates the inter-vehicle distance between the detected second vehicle and the first vehicle. As a third step, when the second vehicle is not detected from the running video by the inter-vehicle distance measuring device 10 by machine learning or deep learning, the inter-vehicle distance between the second vehicle and the first vehicle can be calculated by detecting and tracking feature points from the running video.

[0065] Such an inter-vehicle distance measuring device 10 can be realized using software, hardware, or a combination thereof. As an example, in the case of a hardware implementation, it can be realized using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, micro-controllers, micro-processors, and other electrical units for performing other functions.

[0066] Hereinafter, for the sake of convenience of explanation, taking the case where the second vehicle to be measured for distance is the preceding vehicle as an example, each constituent module constituting the inter-vehicle distance measuring device 10 will be described in more detail.

[0067] The image acquisition unit 11 can acquire the driving image captured by the imaging device of the first vehicle. Specifically, the image acquisition unit 11 can acquire in real time the driving image captured by the imaging device installed on the first vehicle while the first vehicle is running. Here, the acquired driving image can include a plurality of lanes differentiated along the lane, a road composed of a plurality of lanes, and a plurality of vehicles running on the road.

[0068] Here, the lane marking can respectively mean the lines on both sides that form the lane where the vehicle is located. Also, the lane can mean a road formed by lanes and on which vehicles run, such as a single-lane road, a two-lane road, …, an N-lane road, etc.

[0069] The vehicle detection unit 12 can detect the second vehicle from the driving image acquired by the image acquisition unit 11. Specifically, the vehicle detection unit 12 performs learning on the vehicle image by machine learning or deep learning to construct a learning model for vehicle detection, and can detect the second vehicle based on the constructed learning model. Here, the constructed model is an algorithm or program for detecting vehicles from images.

[0070] And the learning model for vehicle detection can be further learned into a more advanced model using the output value indicating the result of vehicle detection. As an example, when the output result is a false answer, the user can input a response to the output result, and the vehicle detection unit 12 can learn the learning model for vehicle detection based on the driver's response.

[0071] That is, according to the present invention, machine learning or deep learning is performed to generate a learning model for vehicle detection, and the generated model can be used to detect vehicles from driving images. Here, in deep learning, the CNN (Convolution Neural Network) algorithm, which is one of the neural network models, can be applied. At this time, deep learning can perform learning based on augmented data assuming various conditions of the driving image. Here, the conditions are defined as conditions for converting the images collected for learning of the neural network model. Specifically, since various aspects can be shown by elements such as image shift, rotation, brightness change, and blur, the data can be augmented in consideration of this.

[0072] Further, when a plurality of vehicles are detected from the driving image, the vehicle detection unit 12 can select a second vehicle to be the distance measurement target from the plurality of detected vehicles based on the driving state information indicating whether the first vehicle is accurately driving on or leaving a specific lane.

[0073] As an example, when the first vehicle is driving on a specific lane, the vehicle detection unit 12 can select a second vehicle located in the same lane as the first vehicle from the plurality of vehicles included in the driving image, and detect the selected second vehicle.

[0074] As another example, when the first vehicle is leaving a specific lane, the vehicle detection unit 12 can select a second vehicle located in the lane towards which the front of the first vehicle leaving the lane is heading from the plurality of vehicles included in the driving image, and detect the selected second vehicle.

[0075] On the one hand, when the second vehicle is detected by the vehicle detection unit 12, the inter-vehicle distance calculation unit 15 can calculate the distance between the detected second vehicle and the first vehicle. That is, when the vehicle detection unit 12 detects the second vehicle from the driving video using the learning model, the functions of the feature point detection unit 13 and the feature point change value calculation unit 14 are not activated, but the inter-vehicle distance calculation unit 15 is activated to calculate the distance between the detected second vehicle and the first vehicle. Here, the inter-vehicle distance calculation unit 15 can calculate the distance between the first vehicle and the detected second vehicle using the inter-vehicle distance calculation algorithm described later.

[0076] However, as the distance between the first vehicle and the second vehicle becomes less than the predetermined distance and they get closer, the lower end portion of the second vehicle is no longer photographed. When the vehicle detection unit 12 cannot detect the second vehicle from the driving video using the learning model, the functions of the feature point detection unit 13 and the feature point change value calculation unit 14 are activated to track the second vehicle by tracking the feature points, and the inter-vehicle distance calculation unit 15 is activated to calculate the distance between the first vehicle and the second vehicle.

[0077] This will be described more specifically with reference to FIG. 4.

[0078] FIG. 4 is a diagram for explaining the detection and tracking process of feature points according to an embodiment of the present invention. Referring to FIG. 4, when the distance between the first vehicle 21 and the second vehicle 22 is greater than or equal to the predetermined distance (22-1), the driving video 23-1 taken by the imaging device of the first vehicle 21 includes the lower end portion of the second vehicle. Therefore, the vehicle detection unit 12 can detect the second vehicle video 23-2 from the driving video 23-1 using the learning model constructed by machine learning or deep learning.

[0079] At this time, the dataset required for learning for vehicle detection can construct learning data by classifying the rear image dataset of vehicles collected according to the vehicle type (Sedan, SUV, Truck, Large car, etc.) according to the detection distance (short distance, medium distance, long distance). Then, the vehicle detection unit 12 can generate a classifier created by learning the constructed learning data by a learning-based method (such as Machine Learning or deep learning). Then, the vehicle detection unit 12 can detect the second vehicle video 23-2 from the driving video 23-1 using the generated classifier.

[0080] The vehicle detection operation of such a vehicle detection unit 12 will be described more specifically with reference to FIG. 5.

[0081] FIG. 5 is a diagram for explaining the process of constructing a learning dataset for vehicle detection and detecting a vehicle using the constructed learning dataset.

[0082] Referring to FIG. 5, first, a vehicle learning dataset is constructed (S1000). After performing selective data learning (S1500) using the constructed vehicle learning dataset, a classifier for classifying vehicles can be generated (S1700). Then, when an image is input by the camera 1700 (S1950), the learning system for vehicle detection according to the embodiment of the present invention can detect a vehicle using the generated classifier (S1900).

[0083] The specific steps performed for each step are as shown in the right figure.

[0084] First, the vehicle learning dataset construction step (S1000) will be specifically described. The learning system for vehicle detection according to an embodiment of the present invention acquires video to be learned (S1010), crops the vehicle area to be learned from the learned video (S1030), and can perform annotation on the attributes of the vehicles included in the cropped vehicle area. At this time, the attributes of the vehicles for which annotation is performed can be the type of vehicle, the distance of the vehicle in the video, and the like. Then, the learning system for vehicle detection according to an embodiment of the present invention can generate a learning dataset based on the cropped vehicle video and its attributes (S1070).

[0085] The specific steps of the selective dataset learning step (S1500) are described as follows. The learning system for vehicle detection according to an embodiment of the present invention can extract features from the constructed dataset (S1510). At this time, as the method for extracting features, methods such as (i) Grayscale intensity, (ii) Red, Green, Blue (RGB) color information, (iii) Hue, Saturation, Value (HSV) color information, (iv) YIQ color information, and (v) Edge information (grayscale, binary, eroded binary) can be used. Then, the learning system for vehicle detection according to an embodiment of the present invention classifies the vehicles using the extracted features (S1530), and after strengthening the process of classifying the vehicles by learning (S1550), a classifier for classifying the vehicles can be generated (S1700).

[0086] Finally, the vehicle detection step (S1900) will be specifically described as follows. The learning system for vehicle detection according to the embodiment of the present invention extracts features (S1920) from the video (S1950) input by the camera 1700, detects the vehicle using a classifier for the extracted features (S1930), and can output the detected result (S1970).

[0087] On the other hand, when the second vehicle video 23-2 is detected by the vehicle detection unit 12, the inter-vehicle distance calculation unit 15 can calculate the distance between the detected second vehicle 22 and the first vehicle 21.

[0088] However, the vehicle will encounter various driving environments during driving. When the distance between the first vehicle 21 and the second vehicle 22 becomes short during the driving of the first vehicle 21 (for example, when it is close within 10 m) (22-2), the driving video captured by the imaging device of the first vehicle 21 does not include the lower end portion of the second vehicle. Therefore, the vehicle detection unit 12 cannot detect the second vehicle video using the learning model constructed by machine learning or deep learning.

[0089] Thus, when the second vehicle image is not detected from the driving image by the vehicle detection unit 12, the feature point detection unit 13 can select a first frame 24-1 corresponding to a frame in which the second vehicle was detected, which is before the frame in which the second vehicle was not detected, among a plurality of frames constituting the driving image. Then, the feature point detection unit 13 can detect feature points using the selected first frame 24-1. At this time, the feature point detection unit 13 can detect feature points using the first frame 24-1 that has not been subjected to region of interest processing, or can detect feature points using the first frame 24-2 that has been subjected to region of interest processing. As an example, as shown in FIG. 4, the feature point detection unit 13 sets the second vehicle region as the region of interest in the first frame 24-1 to generate a first frame 24-2 that has been subjected to region of interest processing, and can detect a first feature point 24-3 from the first frame 24-2 that has been subjected to region of interest processing. And although the reference numeral 24-3 in FIG. 4 is described as indicating only one point, the first feature point can mean all the points that are distinguished and displayed in the first frame 24-2 that has been subjected to region of interest processing. At this time, the feature point detection unit 13 can set the intermediate region of the vehicle as the region of interest in the second vehicle region of the first frame 24-2 that has been subjected to region of interest processing, and can detect the first feature point 24-3 from the set region of interest. Here, the intermediate region of the vehicle includes the license plate region of the vehicle formed on the rear surface of the vehicle, and can include the rear bumper region and the trunk region of the vehicle that are at a predetermined distance from the license plate region of the vehicle.

[0090] Here, the feature point detection unit 13 can detect the first feature point using the Harris corner detection method or the FAST (features-from-accelerated-segment test) corner detection method.

[0091] Thereafter, the feature point detection unit 13 can detect the second feature points in the second frame corresponding to the current frame by tracking the detected first feature points 24-3. At this time, the feature point detection unit 13 can detect the second feature points using the second frame 25-1 that has not been subjected to the region of interest processing, or can detect the feature points using the second frame 25-2 that has been subjected to the region of interest processing. As an example, as shown in FIG. 4, the feature point detection unit 13 sets the second vehicle region as the region of interest and generates the second frame 25-2 that has been subjected to the region of interest processing, and can detect the second feature points 25-3 from the second frame 25-2 that has been subjected to the region of interest processing. Although the reference numeral 25-3 in FIG. 4 is described as indicating only one point, the second feature points can mean all the points that are distinguished and displayed in the second frame 25-2 that has been subjected to the region of interest processing.

[0092] At this time, the feature point detection unit 13 can detect the second feature points 25-3 in the second frame 25-2 that has been subjected to the region of interest processing by tracking the second feature points 25-3 using the optical flow of the detected first feature points 24-3.

[0093] On the other hand, when the feature point detection unit 13 tracks the second feature point 25-3 using the optical flow, it can filter the second feature point 25-3 that does not appear in the second frame 25-2 subjected to the region of interest processing, and the first feature point 24-3 corresponding to the second feature point that does not appear in the second frame 25-2 subjected to the region of interest processing. That is, when the distance between the first vehicle 21 and the second vehicle 22 becomes closer, among the first feature points 24-3 detected from the first frame 24-2 subjected to the region of interest processing, some feature points (for example, the feature points located at the lower end of the vehicle among the detected first feature points 24-3) The corresponding second feature point 25-3 does not appear in the second frame 25-2 subjected to the region of interest processing. Therefore, when the feature point detection unit 13 tracks the second feature point 25-3 using the optical flow, it filters and removes the second feature point 25-3 that does not appear in the second frame 25-2 subjected to the region of interest processing, and the first feature point 24-3 corresponding to the second feature point that does not appear in the second frame 25-2 subjected to the region of interest processing, thereby increasing the operation execution speed.

[0094] On the other hand, the feature point change value calculation unit 14 can calculate the feature point change value between the first feature point and the second feature point. Here, the feature point change value calculation unit 14 can include an average pixel distance calculation unit 14-1 and an average pixel distance ratio calculation unit 14-2. The operation of such a feature point change value calculation unit 14 will be described more specifically with reference to FIG. 6.

[0095] FIG. 6 is a diagram for explaining a feature point change value calculation unit according to an embodiment of the present invention. Referring to FIG. 6, the average pixel distance calculation unit 14-1 can calculate the average pixel position 24-5 of the first feature point 24-3 and the first average pixel distance obtained by averaging the pixel distances from the average position to each of the first feature points 24-3. Specifically, the average pixel distance calculation unit 14-1 can calculate the coordinate value of the average pixel position 24-5 by averaging the coordinate values of the pixel positions of the first feature point 24-3 in the region of interest 24-4 of the first frame 24-1. Here, the region of interest 24-4 can be an intermediate region of the vehicle. As an example, the region of interest includes the license plate region of the vehicle formed on the rear surface of the vehicle, and can include the rear bumper region and the trunk region of the vehicle at a predetermined distance from the license plate region of the vehicle. And although the reference numeral 24-3 in FIG. 6 is described as indicating only one point, the first feature point can mean all the points separately displayed in the region of interest 24-4 excluding the reference numeral 24-5.

[0096] And the average pixel distance calculation unit 14-1 can calculate the coordinate value of the average pixel position 24-5 by averaging the coordinate values of the pixel positions of the first feature point 24-3 within the region of interest 24-4.

[0097] The calculation process of such an average pixel position will be described with reference to FIG. 7. As an example, when the feature points are composed of a first point with (x, y) as the coordinate value of the pixel position, a second point with (x', y') as the coordinate value of the pixel position, and a third point with (x'', y'') as the coordinate value of the pixel position based on a two-dimensional plane coordinate system, the average pixel distance calculation unit 14-1 can calculate (mx, my), which is the coordinate value of the average pixel position, by arithmetically averaging the coordinate values of the pixel positions of the first point, the second point, and the third point.

[0098] On the other hand, returning to FIG. 6, the average pixel distance calculation unit 14-1 can calculate a first average pixel distance obtained by averaging the pixel distances from the calculated average pixel position 24-5 to the respective first feature points 24-3. Here, the first frame 24-1 is a frame corresponding to the frame in which the second vehicle was detected, among the plurality of frames constituting the driving video, before the frame in which the second vehicle was not detected.

[0099] Also, the average pixel distance calculation unit 14-1 can calculate an average pixel position 25-5 of the second feature points 25-3 tracked using optical flow, and a second average pixel distance obtained by averaging the pixel distances from the average pixel position to the respective second feature points 25-3. Specifically, the average pixel distance calculation unit 14-1 can calculate the coordinate value of the average pixel position 25-5 by averaging the pixel position coordinate values of the second feature points 25-3 in the region of interest 25-4 of the second frame 25-1. Here, the feature points indicated by the dotted line in FIG. 6 represent the positions of the first feature points 24-3 in the region of interest 24-4 of the first frame 24-1. And the points indicated by the solid line represent the second feature points 25-3. Although the reference numeral 25-3 in FIG. 6 is described as referring to only one point, the second feature points can mean all the points separately displayed in the region of interest 25-4 excluding the reference numeral 25-5.

[0100] Then, the average pixel distance calculation unit 14-1 sets a region of interest 25-4 including the second feature points 25-3, and can calculate the coordinate value of the average pixel position 25-5 by averaging the coordinate values of the pixel positions of the feature points 25-3 within the region of interest 25-4. And a second average pixel distance obtained by averaging the pixel distances from the calculated average pixel position 25-5 to the respective feature points 25-3 can be calculated. Here, the second frame 25-1 is the current frame.

[0101] On the one hand, according to the above example, taking the setting of the intermediate region of the vehicle as the region of interest 24-4 and 25-4 in frames 24-1 and 25-1, and performing the detection, tracking, and calculation of the average pixel distance of feature points as an example, it is not limited thereto. According to other embodiments of the present invention, the entire frames 24-1 and 25-1 can be set as the region of interest, and the above-mentioned detection, tracking, and calculation of the average distance of feature points can also be performed.

[0102] On the one hand, when the first average pixel distance and the second average pixel distance are calculated by the above-mentioned operations, the average pixel distance ratio calculation unit 14-2 can calculate the average pixel distance ratio between the first average pixel distance and the second average pixel distance. Specifically, the average pixel distance ratio calculation unit 14-2 can calculate the average pixel distance ratio by dividing the second average pixel distance by the first average pixel distance as shown in the following mathematical formula 1.

[0103] JPEG2025102858000002.jpg26170

[0104] Here, Ratio 1 can represent the average pixel distance ratio, curAvgDist can represent the second average pixel distance, and preAvgDist can represent the first average pixel distance.

[0105] On the one hand, the inter-vehicle distance calculation unit 15 can calculate the inter-vehicle distance between the first vehicle and the second vehicle. Specifically, the inter-vehicle distance calculation unit 15 can calculate the distance from the imaging device of the first vehicle to the second vehicle based on the image width of the second vehicle, the focal length of the imaging device provided on the first vehicle, and the predicted width of the second vehicle. Such an inter-vehicle distance calculation unit 15 will be described more specifically with reference to FIGS. 8 to 12.

[0106] On the one hand, in FIG. 6, the calculation of the feature point change value using the first frame 24-1 not processed in the region of interest and the second frame 25-1 not processed in the region of interest was described as an example. However, the present invention is not limited to this. According to another embodiment of the present invention, the feature point change value can also be calculated using the first frame 24-2 processed in the region of interest and the second frame 25-2 processed in the region of interest.

[0107] FIG. 8 is a block diagram more specifically showing the inter-vehicle distance calculation unit 15 according to an embodiment of the present invention. FIG. 9 is a diagram for explaining a method of measuring the inter-vehicle distance according to an embodiment of the present invention. Referring to FIGS. 8 and 9, the inter-vehicle distance calculation unit 15 can include a video width ratio calculation unit 15-1, a vehicle size class calculation unit 15-2, a vehicle width calculation unit 15-3, and a distance calculation unit 15-4.

[0108] In the first vehicle (not shown), a photographing device 50 for photographing the traveling video of the first vehicle can be installed. Here, the photographing device 50 is installed in the first vehicle and can be realized by an in-vehicle camera (Car dash cam) or a car video recorder that photographs the periphery of the vehicle in situations such as vehicle traveling and parking. Alternatively, the photographing device 50 may be realized by a camera formed in a navigation device that provides route guidance to the driver of the first vehicle, or a camera built into the driver's mobile device.

[0109] Such a photographing device 50 can include a lens unit 51 and an imaging device 52, and although not shown in FIG. 6, it can further include all or part of a lens unit driving unit, a diaphragm, a diaphragm driving unit, an imaging device control unit, and an image processor. Here, the lens unit 51 can perform a function of condensing an optical signal, and the optical signal transmitted through the lens unit 51 reaches the imaging region of the imaging device 52 to form an optical image. Here, as the imaging device 52, a CCD (Charge Coupled Device), a CIS (Complementary Metal Oxide Semiconductor Image Sensor), a high-speed image sensor, or the like that converts an optical signal into an electrical signal can be used.

[0110] On the other hand, the inter-vehicle distance calculation unit 15 can calculate the distance between the photographing device 50 installed in the first vehicle and the second vehicle 30 by using the traveling video image photographed by the photographing device 50 of the first vehicle based on the following mathematical formula 2.

[0111] JPEG2025102858000003.jpg21170

[0112] Here, D can be the distance from the photographing device installed in the first vehicle to the second vehicle, W can be the width of the second vehicle, f can be the focal length of the photographing device, and w can be the video width of the second vehicle.

[0113] That is, the distance (D) from the photographing device installed in the first vehicle to the second vehicle can mean the distance from the photographing device installed in the first vehicle to the second vehicle on the real-world coordinates.

[0114] And the width (W) of the second vehicle can mean the width of the second vehicle on the real-world coordinates.

[0115] And the video width can mean the pixel width of the second vehicle formed on the imaging surface of the imaging device 52 of the photographing device 50. Here, the video width w of the second vehicle can be the same value as VehicleW in the following mathematical formula 3.

[0116] On the other hand, the inter-vehicle distance calculation unit 15 first calculates, from the driving video acquired by the imaging device 50 of the first vehicle, the ratio between the video width of the second vehicle 30 and the video width of the road on which the second vehicle 30 is located, and based on the calculated ratio, determines the size class of the second vehicle 30 from a plurality of size classes, and can calculate the width (W) of the second vehicle 30 based on the determined size class of the second vehicle. The operation of such an inter-vehicle distance calculation unit 15 will be described more specifically with reference to FIG. 10.

[0117] FIG. 10 is a diagram showing the ratio between the video width of the second vehicle and the video width of the road on which the second vehicle is located according to an embodiment of the present invention. Referring to FIG. 10, the driving video 45 captured by the imaging device 50 of the first vehicle may include a second vehicle 30 traveling in front of the first vehicle, a road 40 on which the second vehicle is traveling, a left lane 41 and a right lane 42 that distinguish the road 40 from other roads.

[0118] At this time, the video width ratio calculation unit 15-1 can calculate the video width (VehicleW) of the second vehicle 30. Specifically, when the vehicle detection unit 12 detects a vehicle from the driving video using a pre-constructed learning model, the video width ratio calculation unit 15-1 can identify the left boundary 31 and the right boundary 32 of the detected second vehicle 30 in the video of the second vehicle 30. Such boundary identification will be described more specifically with reference to FIG. 11.

[0119] Referring to FIG. 11, when a vehicle is detected from the image frame (W X H) 81 acquired by the camera using a learning model, the detected vehicle area is cropped (82), and a Sobel operation is performed on the cropped area (w’ X h’) 83 to detect a vertical edge (84).

[0120] Then, a vertical histogram cumulative value is calculated from the detected vertical edges, and the point where the largest histogram value is located can be detected as the left and right boundary positions of the vehicle (85).

[0121] Then, a vehicle area can be fitted (86) to the left and right boundary positions of the detected vehicle.

[0122] On the other hand, the video width ratio calculation unit 15-1 can determine the video width between the identified left boundary 31 and the identified right boundary 32 as the video width (VehicleW) of the second vehicle.

[0123] Also, the video width ratio calculation unit 15-1 can identify the left lane 41 and the right lane 42 of the road 40 on which the second vehicle 30 is traveling in the acquired driving video 45. Then, the video width ratio calculation unit 15-1 can set a line 33 indicating the position of the second vehicle 30 on the road. Here, the line 33 indicating the position of the second vehicle 30 on the road can be realized by a line obtained by extending the lowermost end of the second vehicle 30 in the driving video 45. As an example, it can be realized by a line obtained by extending the lower ends of the left and right wheels of the second vehicle 30.

[0124] On the other hand, a first point 43 where the line 33 indicating the position of the second vehicle 30 on the road contacts the left lane 41 and a second point 44 where the line contacts the right lane 42 are determined, and the video width between the first point 43 and the second point 44 can be determined as the video width (LaneW) of the road on which the second vehicle 30 is located.

[0125] On the other hand, when the video width (VehicleW) of the second vehicle and the video width (LaneW) of the road on which the second vehicle 30 is located are calculated, the video width ratio calculation unit 15-1 can calculate the ratio between the video width of the second vehicle ahead and the video width of the road on which the second vehicle is located by applying the following mathematical formula 3.

[0126] JPEG2025102858000004.jpg21170

[0127] Here, VehicleW means the image width of the second vehicle, LaneW means the image width of the lane where the second vehicle is located, and Ratio2 may mean the ratio of the image width of the second vehicle to the image width of the lane where the second vehicle is located.

[0128] Thus, when the distance between the first vehicle and the second vehicle becomes closer, the image width of the second vehicle and the image width of the lane where the second vehicle is located become larger. When the distance between the first vehicle and the second vehicle becomes farther, the image width of the second vehicle and the image width of the lane where the second vehicle is located become smaller. However, since the above ratio is proportional to the size of the second vehicle without being affected by the distance between the first vehicle and the second vehicle, according to the present invention, this can be used as an index for calculating the size of the second vehicle.

[0129] On the other hand, as shown in the above example, when the ratio of the image width of the second vehicle to the image width of the lane where the second vehicle is located is calculated, the inter-vehicle distance calculation unit 15 can determine the size grade of the second vehicle among a plurality of size grades. This will be described more specifically with reference to FIG. 12.

[0130] FIG. 12 is a conceptual diagram for explaining the process of determining the size grade of the second vehicle according to an embodiment of the present invention. Referring to FIG. 12, the vehicle size grade calculation unit 15-2 can classify the ratio values into a plurality of sections, and calculate the vehicle size grade based on a threshold table in which the size grade of the second vehicle is matched to each of the plurality of sections.

[0131] As an example, the threshold table can be classified into three sections based on the first value and the second value. When it is smaller than the first value, the first size grade corresponding to a small vehicle is matched. When the calculated ratio is larger than the first value and smaller than the second value, the second size grade corresponding to a medium vehicle is matched. When the calculated ratio is larger than the second value, the third size grade corresponding to a large vehicle can be matched.

[0132] At this time, when the ratio calculated by the video width ratio calculation unit 15-1 is smaller than the first value, the vehicle size class calculation unit 15-2 can determine the size class of the second vehicle as the first size class. Then, when the ratio calculated by the video width ratio calculation unit 15-1 is larger than the first value and smaller than the second value, the vehicle size class calculation unit 15-2 can determine the size class of the second vehicle as the second size class. And when the ratio calculated by the video width ratio calculation unit 15-1 is larger than the second value, the vehicle size class calculation unit 15-2 can determine the size class of the second vehicle as the third size class. As an example, the first value can be 48% and the second value can be 60%.

[0133] The vehicle width calculation unit 15-3 can determine the width of the second vehicle based on the size class of the second vehicle. Specifically, as shown in Table 1 below, the storage unit can store the vehicle width for each of a plurality of size classes. At this time, the vehicle width calculation unit 15-3 can determine the width (VehicleW) of the second vehicle by detecting the vehicle width corresponding to the determined size class among the vehicle widths already stored in the storage unit.

[0134] JPEG2025102858000005.jpg29170

[0135] Then, as shown in the above mathematical formula 2, the distance calculation unit 15-4 divides the focal length (f) of the imaging device 50 by the video width (w) of the second vehicle 30 and multiplies it by the width (W) of the second vehicle 30 calculated by the vehicle width calculation unit 15-3, thereby calculating the distance between the imaging device 50 and the second vehicle 30.

[0136] On the other hand, when the distance between the imaging device 50 and the second vehicle 30 is calculated, the distance calculation unit 15-4 appropriately corrects the distance value between the imaging device 50 and the second vehicle 30 for accurate calculation of the inter-vehicle distance, thereby calculating the distance value between the first vehicle on which the imaging device 50 is installed and the second vehicle 30. According to such a present invention, the error of the inter-vehicle distance between the first vehicle and the second vehicle can be reduced, and the inter-vehicle distance can be measured more accurately.

[0137] That is, in each of small-sized vehicles, medium-sized vehicles, and large-sized vehicles located at an equal distance from the first vehicle and having different widths, in order for the distance values calculated based on the above mathematical formula 2 to be equal, the exact widths of the respective vehicles must be known. However, in conventional video recognition and detection, since all specifications of all vehicle types cannot be confirmed, conventionally, without considering the actual widths of a large number of vehicles (for example, small-sized vehicles, medium-sized vehicles, and large-sized vehicles) with different vehicle widths, the vehicle width was processed with a specific constant value set in advance to measure the inter-vehicle distance, resulting in the problem that the measured inter-vehicle distance value was not accurate.

[0138] However, according to the present invention, in order to solve such a problem, by using the ratio of the video width of the preceding vehicle to the video width of the road, the preceding vehicle is classified into small-sized vehicles, medium-sized vehicles, and large-sized vehicles, and based on the classified results, the inter-vehicle distance is measured based on the average widths respectively assigned to small-sized vehicles, medium-sized vehicles, and large-sized vehicles, thereby reducing errors and enabling more accurate measurement of the inter-vehicle distance.

[0139] On the other hand, when the predicted vehicle width of the second vehicle is calculated as described above, in an environment where the second vehicle is detected by the vehicle detection unit 12, by continuously measuring the video width of the second vehicle, the distance between the imaging device of the first vehicle and the second vehicle can be calculated.

[0140] However, as the distance between the first vehicle and the second vehicle approaches less than a predetermined distance, the lower end portion of the second vehicle is no longer photographed, and when the second vehicle cannot be detected from the driving video using the learning model by the vehicle detection unit 12, the video width of the second vehicle cannot be measured.

[0141] Therefore, according to the present invention, as the distance between the first vehicle and the second vehicle approaches less than a predetermined distance and the lower end portion of the second vehicle is no longer photographed, and when the second vehicle cannot be detected from the driving video using the learning model by the vehicle detection unit 12, the video width of the second vehicle can be predicted based on the ratio of the average distance calculated by the average pixel distance ratio calculation unit 14-2. This will be specifically described with further reference to FIG. 6.

[0142] Referring to FIG. 6 above, the first frame 24-1 is a frame corresponding to the frame in which the second vehicle was detected, before the frame in which the second vehicle was not detected, among the plurality of frames constituting the driving video. The vehicle detection unit 12 can detect the second vehicle 30 within the first frame 24-1, and the video width ratio calculation unit 15-1 can calculate the video width 24-6 of the second vehicle.

[0143] However, since the second frame 25-1 is the current frame and the second vehicle is not detected in this frame, the video width ratio calculation unit 15-1 applies the video width 24-6 of the second vehicle and the average pixel distance ratio calculated by the average pixel distance ratio calculation unit 14-2 to the following mathematical formula 4, thereby being able to calculate the predicted value of the video width of the second vehicle in the second frame 25-1.

[0144] JPEG2025102858000006.jpg17170

[0145] Here, curVehicleW means the video width of the second vehicle in the second frame 25-1, Ratio 1 means the average pixel distance ratio calculated by Mathematical Formula 1, and preVehicleW may mean the video width of the second vehicle in the first frame 24-1.

[0146] Then, the distance calculation unit 15-4 divides the focal length (f) of the imaging device 50 by the video width (curVehicleW) of the second vehicle in the second frame 25-1 as in the above Mathematical Formula 2, and multiplies it by the width (W) of the second vehicle 30 calculated by the vehicle width calculation unit 15-3, thereby being able to calculate the distance between the imaging device 50 and the second vehicle 30.

[0147] Thus, according to the present invention, as the distance between the first vehicle and the second vehicle approaches, the lower end portion of the second vehicle is not photographed, and even when it becomes impossible to detect the target vehicle from the driving video, the distance between the host vehicle and the target vehicle can be accurately measured by tracking the feature points.

[0148] In another embodiment of the present invention, the inter-vehicle distance calculation unit 15 can monitor the distance between the second vehicle and the first vehicle by calculating the ratio of the video width 24-6 (w) of the second vehicle detected from the first frame 24-1 to the video width 25-6 (w) of the second vehicle detected from the second frame 25-1. The control unit 19 can provide the driver with various functions related to the running of the vehicle, such as collision notification and Adaptive Cruise Control, using the distance monitored by the inter-vehicle distance calculation unit 15.

[0149] On the other hand, when the distance calculated by the inter-vehicle distance calculation unit 15 is smaller than a preset distance, the guidance data generation unit 17 can generate guidance data for guiding the collision risk level corresponding to the distance difference between the first vehicle and the second vehicle.

[0150] In addition, the driving control data generation unit 18 can generate a control signal for controlling the autonomous driving of the first vehicle based on the distance calculated by the inter-vehicle distance calculation unit 15.

[0151] The operations of the guidance data generation unit 17 and the driving control data generation unit 18 will be described later based on the control unit 19.

[0152] The control unit 19 controls the overall operation of the inter-vehicle distance measuring device 10. Specifically, the control unit 19 can control all or part of the video acquisition unit 11, the vehicle detection unit 12, the feature point detection unit 13, the feature point change value calculation unit 14, the inter-vehicle distance calculation unit 15, the guidance data generation unit 17, and the driving control data generation unit 18.

[0153] In particular, the control unit 19 controls the vehicle detection unit 12 to detect the second vehicle from the driving video captured by the imaging device of the first vehicle during driving. If the second vehicle is not detected from the driving video, among the plurality of frames constituting the driving video, a first frame corresponding to the frame in which the second vehicle was detected, which is before the frame in which the second vehicle was not detected, is selected. A first feature point is detected from the second vehicle region in the selected first frame, and the feature point detection unit 13 is controlled to detect a second feature point in the second frame corresponding to the current frame by tracking the detected first feature point. The feature point change value calculation unit 14 is controlled to calculate the feature point change value between the first feature point and the second feature point, and the inter-vehicle distance calculation unit 15 can be controlled to calculate the distance from the imaging device of the first vehicle to the second vehicle based on the calculated feature point change value.

[0154] In addition, when the inter-vehicle distance information between the first vehicle and the second vehicle is acquired, the control unit 19 can control the guidance data generation unit 17 to generate guidance data for assisting the safe driving of the first vehicle driver based on this. Specifically, when the inter-vehicle distance calculated by the inter-vehicle distance calculation unit 15 is smaller than a preset distance, the guidance data generation unit 17 can generate guidance data for guiding the distance difference between the first vehicle and the second vehicle. As an example, the guidance data generated by the guidance data generation unit 17 can be data for warning by voice that the inter-vehicle distance is a distance that requires attention, or data for guiding by image.

[0155] As another example, when the inter-vehicle distance calculated by the inter-vehicle distance calculation unit 15 is smaller than a preset distance, the guidance data generation unit 17 can generate data for guiding a collision risk level corresponding to the distance difference between the first vehicle and the second vehicle. As an example, the distance difference between the first vehicle and the second vehicle is divided into a plurality of levels. When the inter-vehicle distance is smaller than a first value, data for guiding a first risk level is generated. When the inter-vehicle distance is larger than the first value and smaller than a second value, data for guiding a second risk level with a higher degree of risk than the first risk level is generated. When the inter-vehicle distance is larger than the second value, data for guiding a third risk level with a higher degree of risk than the second risk level can be generated.

[0156] On the other hand, when the inter-vehicle distance information between the first vehicle and the second vehicle is acquired, the control unit 19 can control the driving control data generation unit 18 to generate driving control data for controlling the autonomous driving of the first vehicle based on this information. Specifically, when the first vehicle is operating in the autonomous driving mode and the inter-vehicle distance calculated by the inter-vehicle distance calculation unit 15 is smaller than a preset distance, the control unit 19 can control the driving control data generation unit 18 to generate driving control data (for example, instruction data for controlling the speed of the first vehicle to be reduced from the current speed to a predetermined speed or for controlling the first vehicle to stop). Here, the driving control data generated by the driving control data generation unit 18 can be transmitted to the autonomous driving control unit that comprehensively controls the autonomous driving of the first vehicle. The autonomous driving control unit of the first vehicle can control various units (brake, steering wheel, electric motor, engine, etc.) provided in the first vehicle based on this information, so as to control the first vehicle to perform autonomous driving.

[0157] Hereinafter, with reference to FIGS. 13 to 16, the method for measuring the inter-vehicle distance according to an embodiment of the present invention will be described more specifically.

[0158] FIG. 13 is a flowchart showing a method for measuring an inter-vehicle distance according to an embodiment of the present invention. Referring to FIG. 13, first, a running video captured by a photographing device of a first vehicle during running can be acquired (S110).

[0159] Then, it can be determined whether a second vehicle is detected from the acquired running video (S120). Here, the detection of the second vehicle from the acquired running video can be performed using a learning model constructed by machine learning or deep learning for vehicle videos.

[0160] When the second vehicle is detected from the acquired running video (S120: Y), the distance from the photographing device of the first vehicle to the detected second vehicle can be calculated (S170). Here, in the distance calculation step (S170), the video width ratio between the video width of the detected second vehicle and the video width of the road is calculated, the size class of the second vehicle is determined based on the calculated video width ratio, the predicted width of the second vehicle is calculated based on the determined size class of the second vehicle, and by applying the video width of the second vehicle, the focal length of the first photographing device, and the predicted width of the second vehicle to the above-mentioned mathematical formula 2, the distance from the photographing device of the first vehicle to the second vehicle can be calculated.

[0161] However, when the second vehicle is not detected from the acquired running video (S120: N), among the plurality of frames constituting the running video, a first frame corresponding to the frame in which the second vehicle was detected before the frame in which the second vehicle was not detected is selected, and a first feature point can be detected from the second vehicle region in the selected first frame (S130). That is, in the step of detecting the first feature point (S130), as the distance between the first vehicle and the second vehicle approaches, when the second vehicle is not detected using the constructed learning model, the step of detecting the first feature point in the second vehicle region can be performed. In the step of detecting such a first feature point (S130), the middle region of the vehicle is set as the region of interest in the second vehicle region in the first frame, and the first feature point can be detected from the set region of interest.

[0162]

[0162] And by tracking the detected first feature points, the second feature points in the second frame corresponding to the current frame can be detected (S140). Specifically, in the step of detecting the second feature points (S140), the second feature points in the second frame can be detected by tracking the second feature points using the optical flow of the detected first feature points.

[0163] Also, according to an embodiment of the present invention, when tracking the second feature points using the optical flow, the step of filtering the second feature points not appearing in the second frame and the first feature points corresponding to the second feature points not appearing in the second frame can be further included.

[0164]

[0162] And the feature point change value between the first feature points and the second feature points can be calculated (S150). Here, the step of calculating the feature point change value (S150) will be described later with reference to FIG. 14.

[0165]

[0162] And based on the calculated feature point change value, the distance from the imaging device of the first vehicle to the second vehicle can be calculated (S160). Here, the step of calculating the distance (S160) will be described more specifically with reference to FIG. 15.

[0166] FIG. 14 is a flowchart more specifically showing the step (S150) of calculating the characteristic point change value according to an embodiment of the present invention. Referring to FIG. 14, the average pixel position of the first characteristic points can be calculated (S210). Then, a first average pixel distance obtained by averaging the pixel distances from the calculated average pixel position to each of the first characteristic points can be calculated (S220). Specifically, the average pixel distance calculation unit 14-1 can calculate the coordinate value of the average pixel position by averaging the coordinate values of the pixel positions of the first characteristic points, and calculate a first average pixel distance obtained by averaging the pixel distances between the calculated average pixel position and each of the first characteristic points. Here, the first frame is a frame corresponding to the frame in which the second vehicle is detected, before the frame in which the second vehicle is not detected, among the plurality of frames constituting the driving video, and can be a frame before the second frame.

[0167] Then, the average pixel position of the second characteristic points can be calculated (S230). Then, a second average pixel distance obtained by averaging the distances from the calculated average pixel position to each of the second characteristic points can be calculated (S240). Specifically, the average pixel distance calculation unit 14-1 can calculate the coordinate value of the average pixel position by averaging the coordinate values of the pixel positions of the second characteristic points, and calculate a second average pixel distance obtained by averaging the pixel distances between the calculated average pixel position and each of the second characteristic points. Here, the second frame can be a frame after the first frame.

[0168] Then, an average pixel distance ratio between the first average pixel distance and the second average pixel distance can be calculated (S250). Specifically, the average pixel distance ratio calculation unit 14-2 can calculate the average pixel distance ratio by dividing the second average pixel distance by the first average pixel distance as in the above-mentioned mathematical formula 1.

[0169] FIG. 15 is a flowchart showing more specifically the inter-vehicle distance calculation step (S160) according to an embodiment of the present invention. Referring to FIG. 15, the second vehicle is detected from the first frame (S310), and the video width of the second vehicle in the first frame can be calculated (S320).

[0170] Then, based on the video width of the second vehicle in the first frame and the average pixel distance ratio calculated by the average pixel distance ratio calculation unit, a predicted value of the video width of the second vehicle in the second frame can be calculated (S330). That is, since the second frame is a frame in which the second vehicle is not detected, the video width ratio calculation unit 15-1 applies the video width of the second vehicle and the average pixel distance ratio calculated by the average pixel distance ratio calculation unit 14-2 to the above-mentioned mathematical formula 4, whereby a predicted value of the video width of the second vehicle in the second frame can be calculated.

[0171] Then, based on the calculated video width of the second vehicle in the second frame, the focal length of the imaging device of the first vehicle, and the predicted width of the second vehicle, the distance from the imaging device of the first vehicle to the second vehicle can be calculated (S340). Specifically, the distance calculation unit 15-4 divides the focal length (f) of the imaging device of the first vehicle by the video width (curVehicleW) of the second vehicle in the second frame as in the above-mentioned mathematical formula 2, and multiplies by the predicted width of the second vehicle 30 calculated by the vehicle width calculation unit 15-3, whereby the distance between the imaging device of the first vehicle and the second vehicle can be calculated.

[0172] Here, the process of calculating the predicted width of the second vehicle can be composed of a step of calculating the video width ratio between the video width of the second vehicle detected by the vehicle detection unit 12 and the video width of the road on which the second vehicle is located, a step of determining the size class of the second vehicle based on the calculated ratio, and a step of calculating the predicted width of the second vehicle based on the determined size class of the second vehicle. Such a predicted width of the second vehicle can be calculated and stored in advance while the second vehicle is detected by the vehicle detection unit 12.

[0173] FIG. 16 is a flowchart showing a method for measuring an inter-vehicle distance according to another embodiment of the present invention.

[0174] Referring to FIG. 16, first, the inter-vehicle distance measuring device 10 receives an input of the current frame (the i-th frame) from the video acquisition unit 11 (S1100). Then, the inter-vehicle distance measuring device 10 detects the second vehicle from the input i-th frame using the learning model (S1105). If there is a second vehicle detected in the i-th frame (S1110: Y), after calculating the distance to the second vehicle detected using the learning model (S1115), the i-th frame and the detected vehicle area are stored (S1120). At this time, the detected vehicle area can be set in the form of a rectangle, a circle, or a polygon, but is not limited thereto.

[0175] Then, when the i-th frame and the vehicle area are stored, the inter-vehicle distance measuring device 10 updates i to i + 1 (S1125) and receives the (i + 1)-th frame as the current frame (S1100).

[0176] However, if the second vehicle is not detected in step S1110 (S1110: N), the inter-vehicle distance measuring device 10 checks whether there is a vehicle area stored in the immediately preceding frame (the (i - 1)-th frame) (S1130). And in S1130, if the inter-vehicle distance measuring device 10 has a vehicle area stored in the (i - 1)-th frame (S1130: Y), it extracts the first feature point from the region of interest of the vehicle area in the (i - 1)-th frame (S1135), extracts the second feature point corresponding to the first feature point extracted from the (i - 1)-th frame from the i-th frame (S1140), and then calculates the distance to the second vehicle using the difference between the average pixel positions of the first feature point and the second feature point (S1145).

[0177] On the other hand, if there is no vehicle area stored in the (i - 1)-th frame in the step S1130 (S1130:N), the inter-vehicle distance measuring device 10 determines that the second vehicle did not exist previously, and receives an input of a new current frame acquired from the video acquisition unit 11 (S1125).

[0178] On the other hand, such an inter-vehicle distance measuring device 10 is realized as a module of an electronic device that outputs various guidance information for assisting the driver's driving, and can perform a route guidance function. This will be described more specifically with reference to FIGS. 15 to 17.

[0179] FIG. 17 is a block diagram showing an electronic device according to an embodiment of the present invention. Referring to FIG. 17, the electronic device 100 includes all or part of a storage unit 110, an input unit 120, an output unit 130, an inter-vehicle distance measuring unit 140, an augmented reality providing unit 160, a control unit 170, a communication unit 180, a sensing unit 190, and a power supply unit 195.

[0180] Here, the electronic device 100 can be realized by various devices such as a smartphone, a tablet computer, a notebook computer, a PDA (personal digital assistant), a PMP (portable multimedia player), smart glasses, project glasses, a navigation device, an in-vehicle camera (Car dash cam) or a car video recorder which can provide driving-related guidance to the driver of the vehicle, and can be provided in the vehicle.

[0181] The driving-related guidance can include various guidance for assisting the driving of the vehicle driver, such as route guidance, lane departure guidance, road keeping guidance, forward vehicle start guidance, traffic signal change guidance, forward vehicle collision prevention guidance, road change guidance, road guidance, curve guidance, and the like.

[0182] Here, the route guidance can include augmented reality route guidance that combines various information such as the user's position and direction with the video captured in front of the running vehicle to provide route guidance, 2D (2-Dimensional) or 3D (3-Dimensional) route guidance that combines various information such as the user's position and direction with 2D (2-Dimensional) or 3D map data to provide route guidance.

[0183] Furthermore, the route guidance can include an aeronautical chart route guidance that combines various information such as the user's position and direction with aeronautical chart data to provide route guidance. Here, the route guidance can be interpreted as a concept that includes not only the case where the user is driving in a vehicle but also the route guidance when the user is walking or running.

[0184] Also, the lane departure guidance is to guide whether the running vehicle has left the lane.

[0185] Also, the road maintenance guidance is to guide the vehicle to return to the original road on which it is running.

[0186] Also, the forward vehicle departure guidance is to guide whether the vehicle located in front of the stopped vehicle has started. Here, the forward vehicle departure guidance can be performed using the inter-vehicle distance calculated by the inter-vehicle distance measurement unit 140.

[0187] Also, the traffic signal change guidance is to guide whether there is a change in the signal of the traffic signal located in front of the stopped vehicle. As an example, when the red light indicating a stop signal changes to a green light indicating a departure signal, it can be guided.

[0188] Also, the forward vehicle collision prevention guidance is to guide to prevent collision with the vehicle located in front of the stopped or running vehicle when the distance to the vehicle in front reaches within a certain distance. Here, the forward vehicle collision prevention guidance can be performed using the inter-vehicle distance calculated by the inter-vehicle distance measurement unit 140.

[0189] Also, the lane change guidance is to guide a change from the lane in which the vehicle is located to another lane for route guidance to the destination.

[0190] Also, the lane guidance is to guide the lane in which the vehicle is currently located.

[0191] Also, the curve guidance is to guide that the road on which the vehicle will travel after a predetermined time is a curve.

[0192] The driving-related video such as the front video of the vehicle that enables provision of such various guidances can be captured by a camera mounted on the vehicle or a camera of a smartphone. Here, the camera can be integrally formed with the electronic device 100 mounted on the vehicle and can be a camera that captures the front of the vehicle.

[0193] As another example, the camera can be mounted on the vehicle separately from the electronic device 100 and can be a camera that captures the front of the vehicle. At this time, the camera can be another video capturing device for a vehicle mounted facing the front of the vehicle. The electronic device 100 can receive the input of the captured video through wired / wireless communication with the separately mounted video capturing device for a vehicle, or when a storage medium storing the captured video of the video capturing device for a vehicle is inserted into the electronic device 100, the electronic device 100 can receive the input of the captured video.

[0194] Hereinafter, based on the above-described content, the electronic device 100 according to an embodiment of the present invention will be described in more detail.

[0195] The storage unit 110 functions to store various data and applications necessary for the operation of the electronic device 100. In particular, the storage unit 110 can store data necessary for the operation of the electronic device 100, for example, an OS, a route search application, map data, etc. Also, the storage unit 110 can store data generated by the operation of the electronic device 100, for example, searched route data, received video, etc.

[0196] Such a storage unit 110 can be realized not only by built-in storage elements such as RAM (Random Access Memory), flash memory, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electronically Erasable and Programmable ROM), registers, hard disks, removable disks, memory cards, USIM (Universal Subscriber Identity Module), etc., but also by removable storage elements such as USB memories.

[0197] The input unit 120 functions to convert physical inputs from the outside of the electronic device 100 into specific electrical signals. Here, the input unit 120 can include all or part of the user input unit 121 and the microphone unit 123.

[0198] The user input unit 121 can receive user inputs such as touches and push operations. Here, the user input unit 121 can be realized using at least one of various button forms, a touch sensor that receives touch inputs, and a proximity sensor that receives approaching motions.

[0199] The microphone unit 123 can receive the user's voice and sounds generated inside and outside the vehicle.

[0200] The output unit 130 is a device that outputs the data of the electronic device 100 to the user as video and / or audio. Here, the output unit 130 can include all or part of the display unit 131 and the audio output unit 133.

[0201] The display unit 131 is a device that outputs data that can be visually recognized by the user. The display unit 131 can be realized by a display unit provided on the front surface of the housing of the electronic device 100. Further, the display unit 131 may be integrally formed with the electronic device 100 to output visually recognizable data, or may be provided separately from the electronic device 100 like a HUD (Head Up Display) to output visually recognizable data.

[0202] The audio output unit 133 is a device that outputs data that can be aurally recognized by the user. The audio output unit 133 can be realized by a speaker that expresses the data to be notified to the user of the electronic device 100 in sound.

[0203] The inter-vehicle distance measurement unit 140 can perform the function of the above-described inter-vehicle distance measurement device 10.

[0204] The augmented reality providing unit 160 can provide an augmented reality view mode. Here, augmented reality is a method of visually superimposing additional information (for example, graphic elements indicating a point of interest (POI), graphic elements guiding the risk of collision with a vehicle ahead, graphic elements indicating the inter-vehicle distance, graphic elements guiding a curve, various additional information for assisting the safe driving of the driver, etc.) on a screen including the real world that the user is actually looking at.

[0205] Such an augmented reality providing unit 160 can include all or part of a calibration unit, a 3D space generation unit, an object generation unit, and a mapping unit.

[0206] The calibration unit can perform calibration for estimating camera parameters corresponding to the camera from the captured video captured by the camera. Here, the camera parameters are parameters constituting a camera matrix, which is information representing the relationship between the real world space and the photograph, and can include camera extrinsic parameters and camera intrinsic parameters.

[0207] The 3D space generation unit can generate a virtual 3D space based on the captured video captured by the camera. Specifically, the 3D space generation unit can generate a virtual 3D space by applying the camera parameters estimated by the calibration unit to the 2D captured video.

[0208] The object generation unit can generate objects for guidance in augmented reality, such as objects for preventing frontal vehicle collisions, route guidance objects, lane change guidance objects, lane departure guidance objects, curve guidance objects, and the like.

[0209] The mapping unit can map the objects generated by the object generation unit to the virtual 3D space generated by the 3D space generation unit. Specifically, the mapping unit can determine the positions of the objects generated by the object generation unit in the virtual 3D space and perform mapping of the objects at the determined positions.

[0210] On the other hand, the communication unit 180 can be provided for the electronic device 100 to communicate with other devices. The communication unit 180 can include all or part of a position data unit 181, a wireless Internet unit 183, a broadcast transmission / reception unit 185, a mobile communication unit 186, a short-range communication unit 187, and a wired communication unit 189.

[0211] The position data unit 181 is a device that acquires position data by means of GNSS (Global Navigation Satellite System). GNSS means a navigation system that can calculate the position of a receiving terminal by using radio signal received from artificial satellites. Specific examples of GNSS can be GPS (Global Positioning System), Galileo, GLONASS (Global Orbiting Navigational Satellite System), COMPASS, IRNSS (Indian Regional Navigational Satellite System), QZSS (Quasi-Zenith Satellite System), etc., depending on the operating entity. The position data unit 181 of the system according to an embodiment of the present invention can receive GNSS signals that are serviced in the area where the electronic device 100 is used and acquire position data. Alternatively, in addition to GNSS, the position data unit 181 can also acquire position data by communicating with a base station or an AP (Access Point).

[0212] The wireless Internet unit 183 is a device that connects to the wireless Internet to acquire or transmit data. The wireless Internet unit 183 can connect to the Internet network according to various communication protocols defined to perform wireless data transmission and reception, such as WLAN (Wireless LAN), Wibro (Wireless broadband), Wimax (World interoperability for microwave access), HSDPA (High Speed Downlink Packet Access).

[0213] The broadcast transceiver unit 185 is a device that transmits and receives broadcast signals via various broadcast systems. The broadcast systems that can be transmitted and received by the broadcast transceiver unit 185 can be DMBT (Digital Multimedia Broadcasting Terrestrial), DMBS (Digital Multimedia Broadcasting Satellite), MediaFLO (Media Forward Link Only), DVBH (Digital Video Broadcast Handheld), ISDBT (Integrated Services Digital Broadcast Terrestrial), etc. The broadcast signals transmitted and received by the broadcast transceiver unit 185 can include traffic data, life data, etc.

[0214] The mobile communication unit 186 can connect to a mobile communication network and perform voice and data communication according to various mobile communication standards such as 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), etc.

[0215] The short - range communication unit 187 is a device for short - range communication. As described above, the short - range communication unit 187 can communicate via Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (IrDA, Infrared Data Association), UWB (Ultra WideBand), ZigBee, NFC (Near Field Communication), Wi - Fi (Wireless - Fidelity), etc.

[0216] The wired communication unit 189 is an interface device that can connect the electronic device 100 to other devices in a wired manner. The wired communication unit 189 can be a USB module that can communicate via a USB port.

[0217] The communication unit 180 can communicate with other devices using at least one of a position data unit 181, a wireless Internet unit 183, a broadcast transmission / reception unit 185, a mobile communication unit 186, a short-range communication unit 187, and a wired communication unit 189.

[0218] As an example, when the electronic device 100 does not have a camera function, video captured by a vehicle video capturing device such as a car dash cam or a car video recorder can be received using at least one of the short-range communication unit 187 and the wired communication unit 189.

[0219] As another example, when communicating with multiple devices, one of them can communicate via the short-range communication unit 187 and the other can communicate via the wired communication unit 189.

[0220] The sensing unit 190 is a device that can detect the current state of the electronic device 100. The sensing unit 190 can include all or part of a motion sensing unit 191 and a light sensing unit 193.

[0221] The motion sensing unit 191 can detect the motion of the electronic device 100 in three-dimensional space. The motion sensing unit 191 can include a three-axis geomagnetic sensor and a three-axis acceleration sensor. By combining the motion data obtained by the motion sensing unit 191 with the position data obtained by the position data unit 181, the trajectory of the vehicle to which the electronic device 100 is attached can be calculated more accurately.

[0222] The light sensing unit 193 is a device that measures the ambient illuminance of the electronic device 100. Using the illuminance data obtained by the light sensing unit 193, the brightness of the display unit 131 can be changed corresponding to the ambient brightness.

[0223] The power supply unit 195 is a device that supplies the power necessary for the operation of the electronic device 100 or the operation of other devices connected to the electronic device 100. The power supply unit 195 can be a device that receives power supply from a battery built in the electronic device 100 or an external power supply such as a vehicle. Also, the power supply unit 195 may be realized by the wired communication module 119 or may be realized by a device that receives power supply wirelessly depending on the form of receiving the power supply.

[0224] The control unit 170 controls the overall operation of the electronic device 100. Specifically, the control unit 170 can control all or part of the storage unit 110, the input unit 120, the output unit 130, the inter-vehicle distance measurement unit 140, the augmented reality providing unit 160, the communication unit 180, the sensing unit 190, and the power supply unit 195.

[0225] Specifically, the control unit 170 can control the output unit 130 to output a forward vehicle collision notification or a forward vehicle departure notification according to the inter-vehicle distance calculated by the inter-vehicle distance measurement unit 140. As an example, the output unit 130 can include a display unit 131 that outputs an augmented reality image by combining the captured driving video and the guiding object. At this time, the control unit 170 can generate a guiding object for the forward vehicle collision notification or the forward vehicle departure notification, and control the display unit 131 to superimpose and display the generated guiding object on the forward vehicle display area of the augmented reality image.

[0226] When performing a forward vehicle collision notification, different guiding objects can be displayed according to the collision risk level corresponding to the distance difference between the first vehicle and the second vehicle. As an example, the distance difference between the first vehicle and the second vehicle is divided into a plurality of levels. When the inter-vehicle distance is smaller than the first value, a guiding object for guiding the first risk level is displayed. When the inter-vehicle distance is larger than the first value and smaller than the second value, a guiding object for guiding the second risk level with a higher risk degree than the first risk level is displayed. When the inter-vehicle distance is larger than the second value, a guiding object for guiding the third risk level with a higher risk degree than the second risk level can be displayed.

[0227] Alternatively, when notifying the departure of the vehicle ahead, different guiding objects can be displayed according to the departure request level corresponding to the distance difference between the first vehicle and the second vehicle. As an example, the distance difference between the first vehicle and the second vehicle is divided into multiple levels. When the inter-vehicle distance is smaller than the first value, a guiding object for guiding the first departure request level is displayed. When the inter-vehicle distance is larger than the first value and smaller than the second value, a guiding object for guiding the second departure request level that requires a faster departure than the first departure request level is displayed. When the inter-vehicle distance is larger than the second value, a guiding object for guiding the third departure request level that requires a faster departure than the second departure request level can be displayed.

[0228] FIG. 18 is a diagram for explaining a system network connected to an electronic device according to an embodiment of the present invention. Referring to FIG. 18, an electronic device 100 according to an embodiment of the present invention can be realized by various devices provided in a vehicle, such as navigation, a vehicle video camera, a smartphone, or other vehicle extended reality interface providing devices, and can be connected to various communication networks and other electronic devices 61 to 64.

[0229] In addition, the electronic device 100 can calculate the current position and the current time zone in conjunction with the GPS module according to the radio wave signal received from the artificial satellite 70.

[0230] Each artificial satellite 70 can transmit L-band frequencies with different frequency bands. The electronic device 100 can calculate the current position based on the time it takes for the L-band frequency transmitted from each artificial satellite 70 to reach the electronic device 100.

[0231] On the one hand, the electronic device 100 can be wirelessly connected to the network 90 via the communication unit 180 through a control station 80 (ACR), a base station 85 (RAS), an AP (Access Point), etc. When the electronic device 100 is connected to the network 90, it can also be indirectly connected to other electronic devices 61 and 62 connected to the network 90 to exchange data.

[0232] In addition, the electronic device 100 can also be indirectly connected to the network 90 via another device 63 having a communication function. For example, when the electronic device 100 is not equipped with a module that can be connected to the network 90, it can communicate with another device 63 having a communication function through a short-range communication module or the like.

[0233] FIGS. 19 and 20 are diagrams showing a front vehicle collision prevention guidance screen of an electronic device according to an embodiment of the present invention. Referring to FIGS. 19 and 20, the electronic device 100 can generate a guidance object indicating the vehicle collision risk level according to the distance between the own vehicle and the preceding vehicle, and output the generated guidance object as augmented reality.

[0234] As an example, as shown in FIG. 19, when the distance between the own vehicle and the preceding vehicle is equal to or greater than a predetermined distance, the electronic device 100 can generate and display a caution guidance object 1501 that guides that the user needs to pay attention.

[0235] Also, as shown in FIG. 20, when the distance between the own vehicle and the preceding vehicle is close within a predetermined distance and the risk of collision with the preceding vehicle increases, the electronic device 100 can generate and display a danger guidance object 1601 that guides that there is a risk of collision.

[0236] Here, the caution guidance object 1501 and the danger guidance object 1601 are distinguished from each other by different colors and sizes, which can improve the visibility of the driver. And, as an example, the guidance objects 1501 and 1601 are realized by a texture image and can be presented as augmented reality.

[0237] Furthermore, so that the driver can more easily recognize the distance to the vehicle ahead, the electronic device 100 can also digitize the inter-vehicle distance between the host vehicle and the vehicle ahead calculated by the inter-vehicle distance measurement unit 140 and display it on the screen. As an example, the inter-vehicle distance measurement unit 140 calculates the inter-vehicle distance between the host vehicle and the vehicle ahead, and the electronic device 100 can generate and display guide objects 1502 and 1602 indicating the inter-vehicle distance on the screen.

[0238] In addition, the electronic device 100 can also output various guides using sound.

[0239] FIG. 21 is a diagram showing an implementation form when the electronic device according to an embodiment of the present invention does not include a photographing unit. Referring to FIG. 21, a vehicle video photographing device 200 provided separately from the electronic device 100 can constitute a system according to an embodiment of the present invention using a wired / wireless communication method.

[0240] The electronic device 100 can include a display unit 131 provided on the front surface of the housing 191, a user input unit 121, and a microphone unit 123.

[0241] The vehicle video photographing device 200 can include a camera 222, a microphone 224, and a mounting unit 281.

[0242] FIG. 22 is a diagram showing an implementation form when the electronic device according to an embodiment of the present invention includes a photographing unit. Referring to FIG. 22, when the electronic device 100 includes the photographing unit 150, the photographing unit 150 of the electronic device 100 photographs the front of the vehicle, and it is a device for enabling the user to recognize the display unit of the electronic device 100. Thereby, a system according to an embodiment of the present invention can be realized.

[0243] FIG. 23 is a diagram showing an implementation form using a HUD (Head-Up Display) according to an embodiment of the present invention. Referring to FIG. 23, the HUD can display an augmented reality guide screen on the head-up display by means of wired / wireless communication with other devices.

[0244] As an example, augmented reality can be provided by, for example, a HUD using the windshield of a vehicle or a video overlay using another video output device, and the augmented reality providing unit 160 can generate a real-world video or an interface image overlaid on the glass, etc. In this way, an augmented reality navigation or a vehicle infotainment system can be realized.

[0245] On the other hand, the inter-vehicle distance measurement device 10 is realized as one module of a system for autonomous driving and can perform a route guidance function. This will be described in more detail with reference to FIGS. 24 and 25.

[0246] FIG. 24 is a block diagram showing the configuration of an autonomous driving vehicle according to an embodiment of the present invention. Referring to FIG. 24, the autonomous driving vehicle 2000 according to this embodiment can include a control device 2100, sensing modules 2004a, 2004b, 2004c, 2004d, an engine 2006, and a user interface 2008.

[0247] The autonomous driving vehicle 2000 can be provided with an autonomous driving mode or a manual mode. As an example, it is possible to switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode according to an input from the user received by the user interface 2008.

[0248] When the vehicle 2000 is operating in the autonomous driving mode, the autonomous driving vehicle 2000 can be operated under the control of the control device 2100.

[0249] In this embodiment, the control device 2100 can include a controller 2120 including a memory 2122 and a processor 2124, a sensor 2110, a wireless communication device 2130, and an object detection device 2140.

[0250] In this embodiment, the object detection device 2140 is a device for detecting an object located outside the vehicle 2000. The object detection device 2140 can detect an object located outside the vehicle 2000 and generate object information based on the detection result.

[0251] The object information can include information regarding the presence or absence of an object, position information of the object, distance information between the vehicle and the object, and relative speed information between the vehicle and the object.

[0252] The object can be a lane, another vehicle, a pedestrian, a traffic signal, light, a road, a structure, a speed bump, terrain features, an animal, etc., and can include various objects located outside the vehicle 2000. Here, the traffic signal is a concept including a traffic light, a traffic sign board, a pattern or text drawn on the road surface. And the light can be light generated by a lamp provided on another vehicle, light generated by a street lamp, or sunlight.

[0253] And the structure can be an object located around the road and fixed to the ground. For example, the structure can include a street lamp, a roadside tree, a building, a utility pole, a traffic signal, a bridge. The terrain features can include a mountain, a hill, etc.

[0254] Such an object detection device 2140 can include a camera module. The controller 2120 can extract object information from an external image captured by the camera module and cause the controller 2120 to process information regarding the same.

[0255] In addition, the object detection device 2140 can further include an imaging device for recognizing the external environment. In addition to LIDAR, RADAR, a GPS device, an odometry device, and other computer vision devices, an ultrasonic sensor and an infrared sensor can be used, and these devices can operate selectively or simultaneously as necessary to enable more precise detection.

[0256] In addition, the sensor 2110 is connected to sensing modules 2004a, 2004b, 2004c, 2004d that detect the internal / external environment of the vehicle, and can acquire various sensing information. Here, the sensor 2110 can include an attitude sensor (e.g., a yaw sensor, a roll sensor, a pitch sensor), a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight detection sensor, a heading sensor, a gyro sensor, a position module, a vehicle forward / backward sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, an ultrasonic sensor, an illuminance sensor, an accelerator pedal position sensor, a brake pedal position sensor, etc.

[0257] Thereby, the sensor 2110 can acquire sensing signals regarding vehicle attitude information, vehicle collision information, vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle inclination information, vehicle forward / backward information, battery information, fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, rotation angle of the steering wheel, external illuminance of the vehicle, pressure applied to the accelerator pedal, pressure applied to the brake pedal, etc.

[0258] In addition, the sensor 2110 can further include, among others, an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), a throttle position sensor (TPS), a TDC sensor, a crankshaft position sensor (CAS), etc.

[0259] Thus, the sensor 2110 can generate the state information of the vehicle based on the sensing data.

[0260] The wireless communication device 2130 is configured to realize wireless communication between the autonomous driving vehicles 2000. For example, it enables the autonomous driving vehicle 2000 to communicate with the user's mobile phone, or other wireless communication devices 2130, other vehicles, a central device (traffic control device), a server, etc. The wireless communication device 2130 can transmit and receive wireless signals according to a connectionless wireless protocol. The wireless communication protocol can be Wi-Fi, Bluetooth, Long-Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA) (registered trademark), Global Systems for Mobile Communications (GSM), but the communication protocol is not limited to this.

[0261] Also, in this embodiment, the autonomous driving vehicle 2000 can also realize vehicle-to-vehicle communication by the wireless communication device 2130. That is, the wireless communication device 2130 can communicate with other vehicles on the road and other vehicles through vehicle-to-vehicle (V2V) communication. The autonomous driving vehicle 2000 can transmit and receive information such as driving warnings and traffic information through vehicle-to-vehicle communication, and can also request information from other vehicles or receive requests. For example, the wireless communication device 2130 can perform V2V communication by a dedicated short-range communication (DSRC) device or a C-V2V (Celluar-V2V) device. In addition to vehicle-to-vehicle communication, communication between the vehicle and other things (for example, electronic devices carried by pedestrians, etc.) (V2X, Vehicle to Everything communication) can also be realized by the wireless communication device 2130.

[0262] In this embodiment, the controller 2120 is a unit that controls the overall operation of each unit within the vehicle 2000. It may be configured when manufactured by the vehicle manufacturer, or may be additionally configured after manufacture to perform the function of autonomous driving. Alternatively, through the upgrade of the controller 2120 configured during manufacture, a configuration for performing continuous additional functions can be included. Such a controller 2120 may also be named an ECU (Electronic Control Unit).

[0263] The controller 2120 collects various data from the connected sensors 2110, object detection device 2140, wireless communication device 2130, etc., and based on the collected data, transmits control signals to the sensors 2110, engine 2006, user interface 2008, wireless communication device 2130, object detection device 2140 included as other components within the vehicle. Also, although not shown, control signals can also be transmitted to an acceleration device, braking system, steering device, or navigation device related to the driving of the vehicle.

[0264] In this embodiment, the controller 2120 can control the engine 2006. For example, when the autonomous driving vehicle 2000 detects the speed limit of the road during driving, it can control the engine 2006 so that the driving speed does not exceed the speed limit, or control the engine 2006 to accelerate the driving speed of the autonomous driving vehicle 2000 within a range not exceeding the speed limit.

[0265] Also, the controller 2120 can detect the distance to the vehicle located ahead during the driving of the autonomous driving vehicle 2000 and control the engine 2006 to control the driving speed according to the inter-vehicle distance. Specifically, the autonomous driving vehicle 2000 can be equipped with the inter-vehicle distance measurement device 10 according to an embodiment of the present invention. The inter-vehicle distance measurement device 10 can measure the distance between the vehicle 1000 and the target vehicle and transmit the measured inter-vehicle distance value to the controller 2120.

[0266] At this time, the controller 2120 can control the autonomous driving of the vehicle 2000 by controlling the deceleration, acceleration, and constant speed of the vehicle 2000 based on the inter-vehicle distance information acquired from the inter-vehicle distance measuring device 10. Specifically, when the acquired inter-vehicle distance is smaller than the preset distance, the controller 2120 can control various units (such as brakes, steering wheels, etc.) provided in the vehicle 2000 to reduce the speed of the vehicle 2000 from the current speed to a predetermined speed, or to stop the vehicle 2000. That is, the controller 2120 can control the autonomous driving of the vehicle 2000 based on the inter-vehicle distance acquired from the inter-vehicle distance measuring device 10.

[0267] Further, according to still another embodiment of the present invention, the controller 2120 can also control the traveling speed by generating a command word to the driving device of the vehicle 2000 so that the inter-vehicle distance acquired from the inter-vehicle distance measuring device 10 maintains a preset fixed distance.

[0268] Further, according to still another embodiment of the present invention, when the inter-vehicle distance acquired from the inter-vehicle distance measuring device 10 is larger than the preset distance, the controller 2120 can control various units (such as brakes, steering wheels, etc.) provided in the vehicle 2000 to increase the speed of the vehicle 2000 from the current speed to a predetermined speed. That is, the controller 2120 can control the autonomous driving of the vehicle 2000 based on the inter-vehicle distance acquired from the inter-vehicle distance measuring device 10.

[0269] Such an inter-vehicle distance measuring device 10 can be configured as a module within the control device 2100 of the autonomous driving vehicle 2000. That is, the memory 2122 and the processor 2124 of the control device 2100 may also realize the method for measuring the inter-vehicle distance according to the present invention in software.

[0270] When there is another vehicle or an obstacle in front of the vehicle, the engine 2006 or the braking system can be controlled to decelerate the moving vehicle, and in addition to the speed, the trajectory, the driving route, and the steering angle can be controlled. Alternatively, the controller 2120 can generate necessary control signals according to the recognition information of other external environments such as the driving lane and driving signals of the vehicle, and control the driving of the vehicle.

[0271] In addition to generating its own control signals, the controller 2120 can also communicate with surrounding vehicles or a central server, and transmit commands for controlling peripheral devices based on the received information, so as to control the driving of the vehicle.

[0272] Also, when the position of the camera module 2150 is changed or the viewing angle is changed, it may be difficult to accurately recognize the vehicle or the lane according to the present embodiment. Therefore, in order to prevent this, the controller 2120 can also generate a control signal for controlling the camera module 2150 to perform calibration. Thus, in the present embodiment, the controller 2120 can generate a calibration control signal to the camera module 2150, so that even if the mounting position of the camera module 2150 is changed due to vibrations and impacts generated by the movement of the autonomous driving vehicle 2000, the normal mounting position, direction, viewing angle, etc. of the camera module 2150 can be continuously maintained. When the pre-stored initial mounting position, direction, viewing angle information of the camera module 2150 and the initial mounting position, direction, viewing angle information of the camera module 2150 measured during the driving of the autonomous driving vehicle 2000 change by more than a threshold value, the controller 2120 can generate a control signal to control the calibration of the camera module 2150.

[0273] In this embodiment, the controller 2120 can include a memory 2122 and a processor 2124. The processor 2124 can execute the software stored in the memory 2122 according to the control signals of the controller 2120. Specifically, the controller 2120 stores data and instructions for performing the method for measuring the inter-vehicle distance according to the present invention in the memory 2122, and the instructions can be executed by the processor 2124 to implement one or more methods disclosed herein.

[0274] At this time, the memory 2122 can be stored in a non-volatile recording medium executable by the processor 2124. The memory 2122 can store software and data by appropriate internal and external devices. The memory 2122 can be composed of a RAM (random access memory), a ROM (read only memory), a hard disk, and a memory 2122 device connected to a dongle.

[0275] The memory 2122 can store at least an operating system (OS, Operating system), user applications, and executable instructions. The memory 2122 can also store application data and array data structures.

[0276] The processor 2124 is a microprocessor or an appropriate electronic processor and can be a controller, a microcontroller, or a state machine.

[0277] The processor 2124 can be realized by a combination of computing devices, and the computing devices can be composed of a digital signal processor, a microprocessor, or an appropriate combination thereof.

[0278] On the one hand, the autonomous vehicle 2000 can further include a user interface 2008 for user input to the above-described control device 2100. The user interface 2008 can enable the user to input information through appropriate interactions. For example, it can be implemented by a touch screen, a keypad, operation buttons, etc. The user interface 2008 transmits the input or command to the controller 2120, and the controller 2120 can perform vehicle control operations in response to the input or command.

[0279] Also, the user interface 2008 can enable the autonomous vehicle 2000 to communicate with an external device of the autonomous vehicle 2000 through the wireless communication device 2130. For example, the user interface 2008 can be made interoperable with a mobile phone, a tablet computer, or other computer devices.

[0280] Furthermore, in this embodiment, although the autonomous vehicle 2000 has been described as including an engine 2006, it is also possible to include other types of propulsion systems. For example, the vehicle may be operated by electric energy, or may be operated by hydrogen energy or a hybrid system combining these. Therefore, the controller 2120 includes a propulsion mechanism by the propulsion system of the autonomous vehicle 2000, and can provide control signals thereby to the configuration of each propulsion mechanism.

[0281] Hereinafter, with reference to FIG. 25, the detailed configuration of the control device 2100 that performs the vehicle distance measurement method according to the present invention will be described in more detail.

[0282] The control device 2100 includes a processor 2124. The processor 2124 may be a general-purpose single or multi-chip microprocessor, a dedicated microprocessor, a microcontroller, a programmable gate array, or the like. The processor may also be referred to as a central processing unit (CPU). Also, in the present embodiment, the processor 2124 can be used as a combination of a plurality of processors.

[0283] Further, the control device 2100 includes a memory 2122. The memory 2122 may be any electronic component capable of storing electronic information. The memory 2122 can also include a combination of memories 2122 in addition to a single memory.

[0284] Data and instruction words 2122a for performing the method for measuring the inter-vehicle distance according to the present invention may be stored in the memory 2122. When the processor 2124 executes the instruction word 2122a, all or part of the instruction word 2122a and the data 2122b necessary for performing the instruction may be loaded (2124a, 2124b) onto the processor 2124.

[0285] The control device 2100 may include a transmitter 2130a, a receiver 2130b, or a transceiver 2130c for allowing transmission and reception of signals. One or more antennas 2132a, 2132b may be electrically connected to the transmitter 2130a, the receiver 2130b, or each transceiver 2130c, and may additionally include an antenna.

[0286] The control device 2100 may include a digital signal processor (DSP) 2170. The DSP 2170 enables the vehicle to quickly process digital signals.

[0287] The control device 2100 may include a communication interface 2180. The communication interface 2180 may include one or more ports and / or communication modules for connecting other devices to the control device 2100. The communication interface 2180 can enable interaction between the user and the control device 2100.

[0288] The various components of the control device 2100 may all be connected by one or more buses 2190, and the buses 2190 may include a power bus, a control signal bus, a status signal bus, a data bus, etc. According to the control of the processor 2124, the components can transmit mutual information via the buses 2190 to perform the intended functions.

[0289] On the other hand, in the above embodiment, for the sake of convenience of explanation, calculating the distance between the reference vehicle and the vehicle ahead is taken as an example for explanation, but it is not limited thereto. The method for measuring the inter-vehicle distance according to the present invention is similarly applicable to calculating the distance between the reference vehicle and the vehicle behind.

[0290] On the other hand, in the specification and the claims, when terms such as "first", "second", "third", and "fourth" are described, they are used to distinguish between similar components, and although not necessarily so, they are used to describe a specific sequential or occurrence order. It should be understood that such terms used in this way are interchangeable in a suitable environment so that the embodiments of the present invention described herein can operate in a sequence other than, for example, that illustrated or described here. Similarly, here, when a method is described as including a series of steps, the order of such steps presented here is not necessarily the order in which such steps can be executed, any arbitrarily described step may be omitted, and / or any other step not described here may be added to the method. For example, without departing from the scope of the present invention, the first component can be named the second component, and similarly, the second component can also be named the first component.

[0291] Also, in the specification and claims, terms such as "left side", "right side", "front", "rear", "upper part", "bottom part", "above", "below", etc. are used for the purpose of explanation and are not necessarily for describing invariant relative positions. Such terms should be understood as being interchangeable in an appropriate environment so that the embodiments of the present invention described herein can operate in other directions, for example, directions other than those illustrated or described herein. The term "connected" used herein is defined as being directly or indirectly connected by electrical or non-electrical means. Here, objects described as "adjacent" to each other can physically contact each other, be close to each other, or exist in the same general range or area appropriately in the context in which the sentence is used. Here, the phrase "in one embodiment" means, although not necessarily so, the same embodiment.

[0292] Also, in the specification and claims, "connected", "connecting", "fastened", "fastening", "coupled", "coupling", etc., and various variations of these expressions are used in the sense of including being directly connected to other components or being indirectly connected with other components interposed therebetween.

[0293] In contrast, when a certain component is referred to as being "directly connected" or "directly attached" to another component, it should be understood that there are no other components therebetween.

[0294] Also, the suffixes "module" and "part" for components used in this specification are given or mixed only in consideration of the ease of preparing the specification and do not have distinct meanings or roles from each other.

[0295] Also, the terms used in this specification are for explaining the embodiments and not for limiting the present invention. The singular expressions used in this specification include plural expressions unless the context clearly indicates a different meaning. In this application, terms such as "configured to" or "including" should not be construed as necessarily including all of the various components or various steps described in the specification, and some of those components or some of those steps may not be included, or it should be construed that additional components or steps may be further included.

[0296] So far, the present invention has been described centering around its preferred embodiments. All of the embodiments and conditional exemplifications disclosed in this specification are described so that those of ordinary skill in the technical field of the present invention can easily understand the principles and concepts of the present invention. A person skilled in the art can understand that the present invention can be realized in a modified form without departing from the essential characteristics of the present invention.

[0297] Therefore, the disclosed embodiments should be considered from an illustrative perspective rather than a limiting perspective. The scope of the present invention is shown not in the foregoing description but in the claims, and any differences within the equivalent scope should be construed as being included in the present invention.

[0298] On the other hand, the inter-vehicle distance measurement methods according to the various embodiments of the present invention described above can be realized by a program and provided to a server or a device. Thereby, each device can connect to a server or a device storing the program and download the program.

[0299] In addition, the methods according to the various embodiments of the present invention described above can be implemented by a program and stored and provided in various non-transitory computer readable media. A non-transitory computer readable medium means a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short moment, such as a register, a cache, or a memory. Specifically, the various applications or programs described above can be stored and provided in non-transitory computer readable media such as CDs, DVDs, hard disks, Blu-ray disks, USBs, memory cards, ROMs, and the like.

[0300] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. Needless to say, various modifications can be made by those having ordinary knowledge in the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. Such modifications should not be individually understood apart from the technical idea and prospect of the present invention.

Explanation of Reference Numerals

[0301] 10 Inter-vehicle distance measuring device 11 Image acquisition unit 12 Vehicle detection unit 13 Feature point detection unit 14 Feature point change value calculation unit 15 Inter-vehicle distance calculation unit

Claims

1. A method for measuring the inter-vehicle distance using a processor, comprising: obtaining a driving video captured by a photographing device of a first vehicle during driving; detecting a second vehicle from the obtained driving video; when the second vehicle is not detected from the driving video, detecting a first feature point of a second vehicle region of the second vehicle from a first frame corresponding to a frame in which the second vehicle was detected, which is before the frame in which the second vehicle was not detected, among a plurality of frames constituting the driving video; detecting a second feature point in a second frame corresponding to the current frame by tracking the detected first feature point; calculating a feature point change value between the first feature point and the second feature point; calculating an inter-vehicle distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value.

2. In the step of detecting the second vehicle, the method for measuring the inter-vehicle distance according to claim 1, wherein the second vehicle is detected using a learning model constructed by machine learning or deep learning for a vehicle video.

3. In the step of detecting the first feature point, the method for measuring the inter-vehicle distance according to claim 2, wherein when the second vehicle is not detected using the constructed learning model as the distance between the first vehicle and the second vehicle approaches, the step of detecting the first feature point of the second vehicle region is performed.

4. In the step of detecting the first feature point, the method for measuring the inter-vehicle distance according to any one of claims 1 to 3, wherein in the second vehicle region in a frame, an intermediate region of the vehicle is set as a region of interest, and the first feature point is detected from the set region of interest.

5. In the step of detecting the second feature point, the method for measuring the inter-vehicle distance according to any one of claims 1 to 4, wherein the second feature point in the second frame is detected by tracking the second feature point using an optical flow of the detected first feature point.

6. When tracking the second feature point using the optical flow, the method further includes a step of filtering a second feature point not appearing in the second frame and a first feature point corresponding to the second feature point not appearing in the second frame. The method for measuring the inter-vehicle distance according to claim 5 is characterized in that.

7. The step of calculating the feature point change value includes: calculating an average pixel position of the first feature point; calculating a first average pixel distance obtained by averaging pixel distances from the average pixel position of the calculated first feature point to each first feature point; calculating an average pixel position of the second feature point; calculating a second average pixel distance obtained by averaging distances from the average pixel position of the calculated second feature point to each second feature point; calculating an average pixel distance ratio between the first average pixel distance and the second average pixel distance. The method for measuring the inter-vehicle distance according to any one of claims 1 to 6 is characterized in that.

8. The step of calculating the inter-vehicle distance includes: calculating the image width of the second vehicle in the second frame by multiplying the image width of the second vehicle in the first frame by the calculated average pixel distance ratio. The method for measuring the inter-vehicle distance according to claim 7 is characterized in that.

9. The step of calculating the inter-vehicle distance includes: further calculating an inter-vehicle distance from the imaging device of the first vehicle to the second vehicle based on the calculated image width of the second vehicle in the second frame, the focal length of the first imaging device, and the predicted width of the second vehicle. The method for measuring the inter-vehicle distance according to claim 8 is characterized in that.

10. The step of calculating the inter-vehicle distance includes: calculating an image width ratio between the detected image width of the second vehicle and the image width of the road on which the second vehicle is located; determining a size class of the second vehicle based on the calculated ratio; further calculating a predicted width of the second vehicle based on the determined size class of the second vehicle. The method for measuring the inter-vehicle distance according to claim 9 is characterized in that.

11. When the calculated inter-vehicle distance is smaller than a preset distance, the method further includes generating guidance data for guiding a collision risk level corresponding to a distance difference between the first vehicle and the second vehicle. The method for measuring an inter-vehicle distance according to any one of claims 1 to 10.

12. The method for measuring an inter-vehicle distance according to any one of claims 1 to 11, further comprising generating a control signal for controlling autonomous driving of the first vehicle based on the calculated inter-vehicle distance.

13. An inter-vehicle distance measuring device, a video acquisition unit that acquires a traveling video captured by a photographing device of a first vehicle during traveling; a vehicle detection unit that detects a second vehicle from the acquired traveling video; if the second vehicle is not detected from the traveling video, among a plurality of frames constituting the traveling video, a first feature point in a second vehicle region of the second vehicle is detected from a first frame corresponding to a frame in which the second vehicle was detected before the frame in which the second vehicle was not detected, and by tracking the detected first feature point, a second feature point in a second frame corresponding to the current frame is detected. A feature point detection unit; a feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point; An inter-vehicle distance measuring device, comprising an inter-vehicle distance calculation unit that calculates a distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value.

14. The vehicle detection unit detects the second vehicle by using a learning model constructed by machine learning (Machine Learning) or deep learning (Deep Learning) for a vehicle video. The inter-vehicle distance measuring device according to claim 13.

15. The feature point detection unit detects a first feature point in the second vehicle region when the second vehicle is not detected by using the constructed learning model as the distance between the first vehicle and the second vehicle approaches. The inter-vehicle distance measuring device according to claim 14.

16. The feature point detection unit sets an intermediate region of the vehicle as a region of interest in the second vehicle region in a frame, and detects the first feature point from the set region of interest. The inter-vehicle distance measuring device according to any one of claims 13 to 15.

17. The feature point detection unit The inter-vehicle distance measuring device according to any one of claims 13 to 16, wherein the second feature point in the second frame is detected by tracking the second feature point using the optical flow of the detected first feature point.

18. The feature point detection unit The inter-vehicle distance measuring device according to claim 17, wherein when tracking the second feature point using the optical flow, second feature points not appearing in the second frame and first feature points corresponding to the second feature points not appearing in the second frame are filtered.

19. The feature point change value calculation unit calculates the average pixel position of the first feature point, calculates a first average pixel distance obtained by averaging the distances from the calculated average pixel position of the first feature point to each first feature point, a mean pixel distance calculation unit that calculates the mean pixel position of the second feature point and calculates a second mean pixel distance obtained by averaging the distances from the calculated mean pixel position of the second feature point to each second feature point; The inter-vehicle distance measuring device according to any one of claims 13 to 18, comprising: a ratio calculation unit that calculates a mean pixel distance ratio between the first mean pixel distance and the second mean pixel distance.

20. The inter-vehicle distance calculation unit The inter-vehicle distance measuring device according to claim 19, wherein the video width of the second vehicle in the second frame is calculated by multiplying the video width of the second vehicle in the first frame by the calculated mean pixel distance ratio.

21. The inter-vehicle distance calculation unit The inter-vehicle distance measuring device according to claim 20, wherein the distance from the imaging device of the first vehicle to the second vehicle is calculated based on the calculated video width of the second vehicle in the second frame, the focal length of the first imaging device, and the predicted width of the second vehicle.

22. The inter-vehicle distance calculation unit The inter-vehicle distance measuring device according to claim 21, wherein a video width ratio between the detected video width of the second vehicle and the video width of the road on which the second vehicle is located is calculated, a size class of the second vehicle is determined based on the calculated ratio, and the predicted width of the second vehicle is calculated based on the determined size class of the second vehicle.

23. When the calculated inter-vehicle distance is smaller than a preset distance, it further includes a guidance data generation unit for generating guidance data for guiding a collision risk level corresponding to a distance difference between the first vehicle and the second vehicle. The inter-vehicle distance measuring device according to any one of claims 13 to 22.

24. It further includes an autonomous driving control signal generation unit for generating a control signal for controlling the autonomous driving of the first vehicle based on the calculated inter-vehicle distance. The inter-vehicle distance measuring device according to any one of claims 13 to 23.

25. An electronic device that provides guidance for assisting a driver based on an inter-vehicle distance, An output unit that outputs guidance information that can be confirmed by the driver, A video acquisition unit that acquires a driving video captured by a photographing device, A vehicle detection unit that detects a second vehicle from the acquired driving video, When the second vehicle is not detected from the driving video, among a plurality of frames constituting the driving video, a first feature point of the second vehicle region of the second vehicle is detected from a first frame corresponding to a frame in which the second vehicle was detected before the frame in which the second vehicle was not detected, and by tracking the detected first feature point, a second feature point in a second frame corresponding to the current frame is detected. A feature point detection unit, A feature point change value calculation unit that calculates a feature point change value between the first feature point and the second feature point, An inter-vehicle distance calculation unit that calculates an inter-vehicle distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value, A control unit that controls the output unit to output a forward vehicle collision notification or a forward vehicle departure notification according to the calculated distance. An electronic device.

26. The output unit further includes a display unit that combines the captured driving video and a guidance object to output an augmented reality image, The control unit, Generates a guidance object for the forward vehicle collision notification, and controls the display unit to superimpose and display the generated guidance object for the forward vehicle collision notification in a forward vehicle display area in the augmented reality image. The electronic device according to claim 25.

27. A computer-readable recording medium on which a program for executing the inter-vehicle distance measurement method according to any one of claims 1 to 13 is recorded.

28. A program stored in a computer-readable recording medium and causing the vehicle distance measurement method according to any one of claims 1 to 13 to be executed.

Images