Distance estimation device and operation method thereof and host vehicle device

Abstract

Provided are a distance estimation device including at least one camera, an operation method thereof, and a host vehicle device. The operation method includes: acquiring a bounding box corresponding to a target vehicle based on an image acquired by the at least one camera; acquiring a first straight-line distance to the target vehicle; acquiring a first world width of the target vehicle based on the first straight-line distance and the width of the bounding box; acquiring a second ratio of an area corresponding to a rear surface of the target vehicle in an area of ​​the bounding box based on a first ratio; calculating a second world width of the target vehicle based on the second ratio; calculating an estimated distance to the target vehicle based on the second world width and the second ratio; and controlling the host vehicle based on the estimated distance to the target vehicle, wherein the first ratio refers to a ratio of a rear surface to a side surface of the target vehicle.

Description

[0001] This application claims the benefit of priority from Korean Patent Application No. 10-2019-0096914 filed on August 8, 2019, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. Technical Field

[0002] Various example embodiments of the inventive concept relate to distance estimation devices, systems, non-transitory computer-readable media and / or operating methods thereof, and more particularly, to a device for estimating a distance (vehicle distance) to a forward vehicle (front vehicle) using a camera and an operating method of the device. Background Art

[0003] An image capturing device including an image sensor may be included in various types of electronic devices such as a smart phone, a personal computer (PC), a surveillance camera, and a vehicle, etc., or may be used as an independent electronic device.

[0004] The autonomous vehicle can detect the distance to peripheral vehicles (eg, other vehicles approaching the autonomous vehicle) by using an image sensor, and can be controlled based on the detected distance, thereby performing stable driving.

[0005] However, in order to identify the observed object as a peripheral vehicle, the autonomous vehicle mainly relies on identifying the characteristic information of the observed object (such as taillights and license plates, etc.) to determine that it is a peripheral vehicle. Therefore, when the characteristic information is blocked due to deteriorating weather, traffic conditions, lighting conditions, etc., it is difficult for the autonomous vehicle to identify the peripheral vehicle. Therefore, the necessity of identifying the peripheral vehicle and estimating the accurate distance to the identified peripheral vehicle is increasing. Summary of the invention

[0006] At least one example embodiment of the inventive concept provides a distance estimation device (vehicle distance estimation device), which calculates the ratio of a rearward area (rear area) of a target vehicle from a bounding box corresponding to the target vehicle, and accurately measures a straight-line distance (straight-line vehicle distance) and a lateral distance (lateral vehicle distance) to the target vehicle based on the calculated ratio of the rearward area. Some example embodiments of the inventive concept provide an operating method of a distance estimation device and a non-transitory computer-readable medium thereof.

[0007] According to an aspect of at least one example embodiment of the inventive concept, there is provided an operating method of a distance estimation device, the distance estimation device including at least one camera, the operating method including: generating a bounding box corresponding to a target vehicle based on an image acquired using the at least one camera using a processing circuit; determining a first straight-line distance to the target vehicle using the processing circuit; calculating a first world width of the target vehicle based on the first straight-line distance and a width of the bounding box using the processing circuit; calculating a first ratio based on a width of a rear surface of the target vehicle and a width of a side surface of the target vehicle in the acquired image using the processing circuit; calculating a second ratio based on the processing circuit, the second ratio corresponding to the rear surface of the target vehicle, the second ratio based on the width of the bounding box and the first ratio; calculating a second world width of the target vehicle based on the second ratio using the processing circuit; calculating an estimated distance (estimated vehicle distance) to the target vehicle based on the second world width and the second ratio using the processing circuit; and controlling a host vehicle based on the estimated distance to the target vehicle using the processing circuit.

[0008] According to another aspect of at least one example embodiment of the inventive concept, a distance estimation device is provided, the distance estimation device comprising: at least one camera configured to acquire at least one image related to a target vehicle; and a processing circuit configured to: generate a bounding box corresponding to the target vehicle, determine a first straight-line distance to the target vehicle, calculate a first world width of the target vehicle based on the first straight-line distance and a width of the bounding box, calculate a first ratio (the first ratio is based on a width of a rear surface of the target vehicle and a width of a side surface of the target vehicle in the acquired image), calculate a second ratio (the second ratio corresponds to the rear surface of the target vehicle, and the second ratio is based on the width of the bounding box and the first ratio), calculate a second world width of the target vehicle based on the second ratio, calculate an estimated distance to the target vehicle based on the second world width and the second ratio, and output the estimated distance to a host vehicle.

[0009] According to another aspect of at least one example embodiment of the inventive concept, a main vehicle device is provided, the main vehicle device comprising: at least one camera configured to acquire at least one image related to a target vehicle; and a processing circuit configured to: generate a bounding box corresponding to the target vehicle, determine a first straight-line distance to the target vehicle, calculate a first world width of the target vehicle based on the first straight-line distance and a width of the bounding box, calculate a first ratio (the first ratio is based on a width of a rear surface of the target vehicle and a width of a side surface of the target vehicle in the acquired image), calculate a second ratio (the second ratio corresponds to the rear surface of the target vehicle, and the second ratio is based on the width of the bounding box and the first ratio), calculate a second world width of the target vehicle based on the second ratio, and estimate a forward distance and a lateral distance to the target vehicle based on the second world width and the second ratio, and control the main vehicle device based on the estimated forward distance and lateral distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Example embodiments of the inventive concept will be more clearly understood through the following detailed description taken in conjunction with the accompanying drawings, in which:

[0011] Figure 1 is a block diagram of a distance estimation device according to at least one example embodiment;

[0012] Figure 2 is a side view of a host vehicle including a distance estimation device according to at least one example embodiment;

[0013] Figure 3 shows an example of an image of a target vehicle projected onto a distance estimation device according to at least one example embodiment;

[0014] Figure 4 shows an example of a side surface of each of a target vehicle and a distance estimation device according to at least one example embodiment;

[0015] Figure 5 shows an example of a bird's eye view according to at least one example embodiment;

[0016] Fig. 6A Another example of a bird's eye view according to at least one example embodiment is shown;

[0017] Figure 6B Another example of a bird's eye view according to at least one example embodiment is shown;

[0018] Figure 7 is a flowchart illustrating the operation of a distance estimation device according to at least one example embodiment;

[0019] Figure 8shows a bird's eye view of a host vehicle including a plurality of distance estimation devices according to at least one example embodiment; and

[0020] Fig. 9 An example of successive refinements performed on a bounding box is shown according to at least one example embodiment. DETAILED DESCRIPTION

[0021] Hereinafter, various exemplary embodiments will be described in detail with reference to the accompanying drawings. Before providing a detailed description, terms will be described below.

[0022] A host vehicle may refer to a vehicle in which a distance estimation device according to at least one example embodiment is embedded. The host vehicle may be referred to as various terms such as ego-vehicle, self-vehicle, self-driving vehicle, autonomous vehicle, autonomous driving vehicle, etc.

[0023] The target vehicle may refer to at least one vehicle other than the host vehicle. In detail, the target vehicle may refer to a vehicle photographed by a camera included in the distance estimation apparatus according to at least one example embodiment and for which distance estimation is to be performed.

[0024] The image plane may refer to a two-dimensional (2D) area of ​​the real world projected by a camera of a distance estimation device according to at least one example embodiment. The image plane may be a 2D area and may therefore have coordinates distinguishable in pixels. The coordinates may be referred to as image coordinates. For example, when the upper left end of the image plane is set as the origin, the right direction (or right direction / right direction) may be represented as the x-axis, and the lower direction (or lower direction / lower direction) may be represented as the y-axis.

[0025] The world coordinates may refer to coordinates used to represent the real world corresponding to the external environment of the camera of the distance estimation device. According to various example embodiments, when the camera of the distance estimation device is set as the origin (for example, the camera position is set as the origin of the coordinate system), the world coordinates may be referred to as camera coordinates (or vice versa). The camera coordinates or world coordinates in which the camera of the distance estimation device is set as the origin may have an x-axis and a y-axis, or an X-axis, a Y-axis, and a Z-axis. For example, the X-axis may correspond to the front direction facing the camera (or referred to as the front direction / front direction), the Y-axis may correspond to the left direction relative to the front of the camera (or referred to as the left direction / left direction), and the Z-axis may correspond to the upper direction relative to the front of the camera (or referred to as the upper direction / upper direction), but the example embodiments are not limited thereto.

[0026] The full length may refer to the front-to-back length of the vehicle (e.g., if the vehicle is a passenger car, it refers to the full length of the vehicle from the "hood" of the vehicle to the "trunk" of the vehicle). In other words, the full length may refer to the horizontal length when the camera faces the target vehicle in the lateral direction (e.g., when the camera faces the side door of the target vehicle).

[0027] The full width may refer to the left-right length of the vehicle (e.g., if the vehicle is a passenger car, it refers to the distance between the "driver's side door" and the "passenger's side door" of the vehicle). That is, the full width may represent the horizontal length when the camera faces the target vehicle in the rear direction (e.g., when the camera faces the "trunk" of the target vehicle).

[0028] The full height may refer to the vertical length of the vehicle. That is, the full height may represent the vertical length from the ground in contact with the vehicle to the upper cover of the vehicle (for example, if the vehicle is a passenger car, it represents the distance from the bottom surface of the tire of the vehicle to the roof of the vehicle). For example, the camera of the distance estimation device may be set at the upper cover of the vehicle, in which case the height of the camera may correspond to the full height.

[0029] The world width may correspond to the full width of the target vehicle in the world coordinates. In addition, the world width may refer to the lateral length of the rear region (or rear region) of the target vehicle in the world coordinates.

[0030] The pixel width may refer to the lateral length of the target vehicle projected onto the image plane. For example, when the target vehicle is located in the front direction of the camera, the pixel width may represent the lateral length of the image of the rear surface of the target vehicle in pixels. As another example, when the target vehicle is located at a side that deviates from the front direction of the camera by a certain angle, the pixel width may represent the lateral length of the image of the target vehicle including the rear surface and the side surface.

[0031] Figure 1 is a block diagram of a distance estimation apparatus 100 according to at least one example embodiment.

[0032] Reference Figure 1 , the distance estimation device 100 may include a camera 110 and / or an image signal processing circuit (ISP) 120 (eg, at least one image signal processor), etc., but example embodiments are not limited thereto but may include a greater number or a lesser number of constituent elements.

[0033] The camera 110 may be embedded in the main vehicle, and may recognize the external environment of the main vehicle (e.g., capture an image of the external environment of the main vehicle, sense the external environment of the main vehicle, etc.). For example, the camera 110 may convert light corresponding to the external environment in the front direction or several directions into electrical energy to generate an image, and may transmit the image to the ISP 120. The camera 110 may be embedded in an electronic device, or may be implemented as an electronic device, but is not limited thereto. The electronic device may be implemented as, for example, a personal computer (PC), an Internet of Things (IoT) device, or a portable electronic device, etc. The portable electronic device may be, for example, a mobile phone, a smart phone, a tablet PC, a personal digital assistant (PDA), an enterprise digital assistant (EDA), a digital still camera, a digital video camera, an audio device, a portable multimedia player (PMP), a personal navigation device (PND), an MP3 player, a handheld game console, an e-book, a wearable device, an augmented reality device, a virtual reality device, etc. In addition, in some example embodiments, the camera 110 may be embedded and / or integrated into one or more components (e.g., a windshield, a roof, etc.) of the main vehicle itself. Furthermore, in some example embodiments, the camera 110 may be two or more cameras connected to the ISP 120 and located in various locations on the vehicle, such as a front bumper and a rear bumper, etc.

[0034] ISP 120 may include: a processing circuit, such as hardware including a logic circuit; a hardware / software combination, such as at least one processor running software; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), etc. ISP 120 may identify peripheral vehicles approaching the main vehicle, and may estimate the distance to each peripheral vehicle based on an image and / or multiple images acquired from camera 110. For example, ISP 120 may identify peripheral vehicles from among multiple objects included in the acquired image, and may generate a bounding box representing the identified vehicle. In addition, ISP 120 may calculate the proportion of the image of the rear surface of the vehicle included in the bounding box, and may estimate the distance to the vehicle included in the bounding box based on the calculated proportion of the image. ISP 120 may be a dedicated processing circuit specially designed and / or specially programmed to perform the above operations. The operations will be described in detail below.

[0035] Figure 2 is a side view of a host vehicle 200 including a distance estimation device according to at least one example embodiment. Figure 1 The same or similar description.

[0036] Reference Figure 1 and Figure 2 , the host vehicle 200 may include the distance estimation device 100 and / or the vehicle controller 210, etc., but example embodiments are not limited thereto.

[0037] The vehicle controller 210 may control the overall travel of the host vehicle 200. For example, the vehicle controller 210 may receive information indicating a straight-line distance received from the distance estimation device 100, and may control the speed and / or direction of the host vehicle 200 based on the received information. In other words, the vehicle controller 210 may control the accelerator, brake, and / or steering of the host vehicle 200 based on the received information. The straight-line distance may refer to the distance between the target vehicle and the host vehicle 200. For example, the straight-line distance may be represented as an X-axis component in a world coordinate, but is not limited thereto. According to some example embodiments, the vehicle controller 210 may confirm that the straight-line distance to the target vehicle is less than a threshold distance. In response to confirming that the straight-line distance to the target vehicle is less than the threshold distance, the vehicle controller 210 may perform control for reducing the speed of the host vehicle 200 and / or changing the direction in which the vehicle is traveling. To this end, the vehicle controller 210 may generate a control signal indicating a deceleration and may transmit the control signal to a braking system, and / or may generate a control signal indicating a change in the position of a steering wheel and may transmit the control signal to a steering wheel, etc. The vehicle controller 210 may include processing circuits, such as hardware including logic circuits; a hardware / software combination, such as at least one processor running software; or a combination thereof. For example, the processing circuits may more specifically include, but are not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc. In some example embodiments, the vehicle controller 210 may be integrated with the image signal processing circuit 120 and / or perform the functions of the image signal processing circuit 120, or vice versa.

[0038] Figure 3 An example of an image of a target vehicle projected onto a distance estimation device is shown according to at least one example embodiment.

[0039] Reference Figure 3 , the distance estimation image 300 may include at least one target vehicle. For convenience, a description will be given based on the target vehicle 310. However, example embodiments are not limited thereto, for example, in some example embodiments, distance estimation is performed independently for each of a plurality of target vehicles, etc.

[0040] According to at least one example embodiment, the target vehicle 310 may be located in the front side direction of the host vehicle 200. That is, the target vehicle 310 included in the distance estimation image 300 may correspond to a rear side image of the target vehicle 310.

[0041] According to various example embodiments, a bounding box 320 may be generated by the ISP 120 based on one or more images captured by the camera 110. The bounding box 320 may refer to a quadrilateral region including a specific object or location information about a specific object, but example embodiments are not limited thereto, and the bounding box 320 may also be a region of other shapes / types such as a polygonal region, a cubic region, etc. In the case where a target vehicle 310 is detected, the bounding box 320 may be displayed as a region including at least the target vehicle 310. For example, the bounding box 320 may be represented as four edges, such as a bottom edge, a top edge, a left edge, and a right edge, but is not limited thereto. The bounding box 320 may be generated by the ISP 120 based on an object detection algorithm including image processing, pattern recognition, machine learning, and / or deep learning, etc. According to at least one example embodiment, the number of target vehicles is not limited to one, and therefore, the number of bounding boxes may be set to a plurality.

[0042] The distance estimation image 300 may include a plurality of lanes 360 (e.g., road lanes), but is not limited thereto, and the distance estimation image 300 may include other objects (e.g., curbs, road signs, traffic lights, pedestrians, etc.). The plurality of lanes 360 may be parallel to each other. When parallel lines in the world coordinates are projected onto the image plane, the parallel lines may be concentrated at one point. The one point to which the parallel lines are concentrated may be referred to as a vanishing point 370.

[0043] According to various example embodiments, the size of the target vehicle 310 may vary based on the physical distance between the camera and the target vehicle 310. For example, as the target vehicle 310 becomes farther away from the camera 110, the target vehicle 310 may be closer to the vanishing point 370, and thus, the size of the object projected onto the distance estimation image 300 may decrease. As another example, as the target vehicle 310 becomes closer to the camera 110, the target vehicle 310 may be farther away from the vanishing point 370, and thus, the size of the object projected onto the distance estimation image 300 may increase.

[0044] Figure 4 An example of a side surface of each of a target vehicle and a distance estimation device according to at least one example embodiment is shown.

[0045] Reference Figure 4, the shape of the target vehicle 310 may be provided to the image sensor 430 through the lens 420 of the camera 110. For example, when the camera 110 is set as the origin of the world coordinates, the target vehicle 310 may be positioned at an initial straight-line distance Z from the lens 420 along the x-axis in front of the host vehicle 200. The initial straight-line distance Z may be expressed as the following equation 1.

[0046] [Equation 1]

[0047]

[0048] Here, f may refer to an intrinsic parameter of the camera 110 and may correspond to a focal length, H C Δy may correspond to the height of the lens 420. Δy may correspond to the distance in pixels between the vanishing point 370 of the image plane and the bottom edge 330 of the bounding box 320 of the target vehicle 310.

[0049] The lens 420 may be disposed at a height H spaced apart from the ground 410 by the lens 420. C , or in other words, the lens 420 may be located at a height H relative to the ground 410 C For example, when the camera 110 is set in the main vehicle ( Figure 2 When the vehicle is at the upper cover (e.g., roof) of the host vehicle 200, the specific separation distance may be the same as that of the host vehicle ( Figure 2 However, the present exemplary embodiment is not limited thereto. As another example, when the camera 110 is disposed outside the host vehicle 200, the specific interval distance may correspond to a value smaller than the full height, etc.

[0050] The image sensor 430 may receive incident light through the lens 420 of the camera 110 and may generate an image plane (eg, Figure 3 According to some example embodiments, the camera 110 may be a pinhole camera, but example embodiments are not limited thereto. In an example when the camera 110 is a pinhole camera, a vertically inverted image of the target vehicle 310 may be projected. The vertically inverted image may be displayed as a bounding box 320 corresponding to the target vehicle 310. The lens 420 may be physically separated from the image sensor 430. The spacing distance may correspond to the focal length f.

[0051] Light 440 incident along the optical axis of lens 420 may form Figure 3 That is, the vanishing point 370 can be determined based on the height of the lens 420. Figure 3 The magnitude of the y-axis component of the vanishing point is 370.

[0052] According to various example embodiments, H C It can be expressed as the following equation 2.

[0053] [Equation 2]

[0054] H C =(R -1 ·t) 3

[0055] Here, R can refer to the rotation matrix and t can refer to the translation vector. (·) 3 Can point to the third element of a quantity.

[0056] According to various example embodiments, the method of estimating the distance based on equation 1 may be referred to as a height-based method. However, the height-based method may include a situation in which the host vehicle 200 and the target vehicle 310 are traveling on the same plane, and due to this, errors may be caused when the height-based method is applied to a real driving environment (e.g., a real-world driving environment or actual driving experience, etc.). Therefore, a method based on the relationship (e.g., ratio) between the pixel width of the bounding box displayed in the image plane and the full width of the target vehicle in the world coordinate will be described below.

[0057] Figure 5 An example of a bird's eye view is shown according to at least one example embodiment.

[0058] Reference Figure 5 , a target vehicle 310 and a lens 420 and an image sensor 430 both included in the camera 110 are shown. Figure 5 The situation when the camera 110 and the target vehicle 310 are viewed from above is shown, and the Figure 3 and Figure 4 The same or similar description.

[0059] According to various example embodiments, target vehicle 310 may be positioned in front of host vehicle 200 at straight-line distance Z. When target vehicle 310 and host vehicle 200 travel in the same lane, straight-line distance Z may be calculated based on Equation 3 below.

[0060] [Equation 3]

[0061]

[0062] Here, f may refer to a focal length, W may refer to a world width of the target vehicle 310 , and w may refer to a pixel width of a bounding box generated in an image plane.

[0063] According to various example embodiments, the method of estimating the straight-line distance by using Equation 3 may be referred to as a width-based method. However, when the target vehicle 310 is traveling in the same direction in an adjacent lane, rather than in the same lane as the host vehicle 200, the bounding box 320 may include an image of the rear surface of the target vehicle 310 and an image of the side surface of the target vehicle 310. That is, the pixel width of the bounding box 320 may not be completely mapped to the world width of the rear surface of the target vehicle 310 (e.g., unlike the above scenario where the target vehicle 310 and the host vehicle 200 are traveling in the same lane, the pixel width of the bounding box 320 may not be equal to the world width of the rear surface of the target vehicle 310), and at least a portion of the length of the bounding box 320 (e.g., at least a portion of the bounding box 320) may include the pixel length of the side surface of the target vehicle 310. Or, in other words, the pixel width of the bounding box 320 is a sum of the pixel width of the image of the rear surface of the target vehicle 310 and the pixel width of the image of the side surface of the target vehicle 310.

[0064] Therefore, the ratio of the side surface of the target vehicle 310 in the image in the bounding box 320 (e.g., the ratio between the size of the side surface in the image and the width of the bounding box 320) and the ratio of the rear surface of the target vehicle in the image in the bounding box 320 (e.g., the ratio between the size of the rear surface of the target vehicle in the image and the width of the bounding box 320) may be different based on the straight-line distance between the target vehicle 310 and the host vehicle 200. For example, when the target vehicle 310 is traveling along an adjacent lane, for example, when the target vehicle 310 is in front of the host vehicle 200 and away from the host vehicle 200, the straight-line distance between the target vehicle 310 and the host vehicle 200 is sufficiently long. In this case, the size of the bounding box 320 may be reduced, and the image of the side surface of the target vehicle 310 in the bounding box 320 may also be sufficiently reduced. In another case, when the target vehicle 310 is traveling in parallel with the host vehicle 200 (e.g., the two vehicles are traveling substantially side by side), the straight-line distance between the host vehicle 200 and the target vehicle 310 traveling along the adjacent lane is short. In this case, the size of the bounding box 320 of the image acquired by the camera 110 may be large, and the size of the image of the side surface of the target vehicle 310 in the bounding box 320 may be much higher than the size of the image of the rear surface of the target vehicle 310. That is, the bounding box 320 may include only the image of the side surface of the target vehicle 310, or may mainly include the image of the side surface of the target vehicle 310 relative to the image of the rear surface of the target vehicle 310. A detailed method of determining and / or acquiring the world width of the target vehicle 310 traveling along an adjacent lane will be described below.

[0065] Fig. 6A Another example of a bird's eye view according to at least one example embodiment is shown.

[0066] Fig. 6A The main vehicle 200 and the target vehicle 310 are shown as viewed from above, and the Figures 3 to 5 The same or similar description.

[0067] Reference Fig. 6A , the main vehicle 200 and the target vehicle 310 traveling in the same direction can be disclosed. It can be understood that the target vehicle 310 is at a distance Z in front of the main vehicle 200. 0 , and the target vehicle 310 is traveling along an adjacent lane on the left side of the host vehicle 200, but example embodiments are not limited thereto.

[0068] Referring to Equation 1, the straight-line distance Z 0 It can be expressed as the following equation 4 and equation 5.

[0069] [Equation 4]

[0070] f: Δy=Z 0 :H C

[0071] [Equation 5]

[0072]

[0073] According to various example embodiments, 0 may refer to the pixel width of the bounding box 320 corresponding to the target vehicle 310, W 0 It may refer to the world width of the world coordinates corresponding to the bounding box 320. The geometric proportional relationship between them may be expressed as the following equation 6, and the world width of the target vehicle 310 projected onto the host vehicle 200 may be expressed as the following equation 7.

[0074] [Equation 6]

[0075] w 0 :W 0 =f:Z 0

[0076] [Equation 7]

[0077]

[0078] According to various example embodiments, host vehicle 200 may determine whether host vehicle 200 is traveling in the same lane as target vehicle 310 or in an adjacent lane based on images acquired by camera 110. Figure 3In the case where light incident on the optical axis of the image is displayed on the image coordinates, the light may correspond to the x-axis center of the image acquired by the camera 110. Δx may be defined as a smaller value between the pixel distance between the x-axis center and the left edge 340 of the bounding box 320 and the pixel distance between the x-axis center and the right edge 350 of the bounding box 320. For example, when the target vehicle 310 is traveling to the left front of the host vehicle 200, the bounding box 320 may be generated on the right side of the image acquired by the camera 110, and thus, Δx may correspond to the pixel distance from the x-axis center to the left edge 340 of the bounding box 320. As another example, when the target vehicle 310 is traveling to the right front of the host vehicle 200 (e.g., in front of the host vehicle 200 and traveling to the right side of the host vehicle 200), the bounding box 320 may be generated on the left side of the image acquired by the camera 110, and thus, Δx may correspond to the pixel distance from the x-axis center to the right edge 350 of the bounding box 320. However, when the target vehicle 310 is traveling forward in the same lane as the host vehicle 200, the bounding box 320 may be generated near the center of the image acquired by the camera 110. That is, the x-axis center may be set between the left edge 340 and the right edge 350 of the bounding box 320, in which case Δx may be defined as 0. Therefore, the host vehicle 200 may compare the x-axis center of the image acquired by the camera 110 and the position of the bounding box 320, and when Δx is 0, the host vehicle 200 may determine that the target vehicle 310 is traveling forward in the same lane. When Δx is not 0, the host vehicle 200 may determine that the target vehicle 310 is traveling forward in an adjacent lane.

[0079] In addition, when the target vehicle 310 is traveling forward in the same lane as the host vehicle 200, the bounding box 320 may include only the image of the rearward region of the target vehicle 310. In the case where the ratio of the width of the image of the rear surface of the target vehicle 310 to the width of the bounding box 320 is defined as α, when Δx is 0, α may correspond to 1. This is because the host vehicle 200 is traveling behind the target vehicle 310 and only the image of the rear surface of the target vehicle 310 is acquired.

[0080] However, when Δx is not 0, that is, when the target vehicle 310 is traveling forward along the adjacent lane relative to the host vehicle 200, α may not be 1, but may have any value between 0 and 1. According to the above description, α may be a ratio between the width of the image of the rear surface of the target vehicle 310 in the bounding box 320 and the width of the bounding box 320, and thus, the ratio between the width of the image of the side surface of the target vehicle 310 and the width of the bounding box 320 may correspond to 1-α.

[0081] According to various example embodiments, the ratio of the full width of the target vehicle to the full length of the target vehicle may be defined as k. In this case, the proportional relationship between the ratio of the world width corresponding to the length of the rear surface of the target vehicle 310 in the image on which the target vehicle 310 is projected and the ratio α of the image of the rear surface of the target vehicle 310 in the bounding box 320 may be expressed as Equation 8 below.

[0082] [Equation 8]

[0083] W: (kW·tanθ) = α: (1-α)

[0084] According to the above description, Δx can be defined as the shorter distance between the pixel distance between the x-axis center (optical axis) and the left edge 340 of the bounding box 320 and the pixel distance between the x-axis center and the right edge 350 of the bounding box 320, and thus, can be expressed as Equation 8 can be expressed as in Equation 9 below.

[0085] [Equation 9]

[0086]

[0087] According to various example embodiments, the α value may be calculated when the k value is arbitrarily determined. The k value may be a ratio of the full length of the target vehicle 310 to the full width of the target vehicle 310 and may be different for each vehicle manufacturer, vehicle classification type, and / or vehicle model, etc.

[0088] [Table 1]

[0089] Classification Overall length [mm] Overall width [mm] k Compact car 3595 1595 2.253918 Semi-midsize sedan 4550 1775 2.56338 Mid-size sedan 4855 1865 2.603217 Mid-size SUV 4700 1880 2.5 Full-size sedan 5155 1935 2.664083 bus 10975 2495 4.398798 Refrigerated Truck 5440 1740 3.126437 Truck 10125 2495 4.058116

[0090] Referring to Table 1, it can be seen that the k value as a ratio of full width to full length (e.g., k is a ratio of real-world width to real-world length) is different for each vehicle type, but the full-width values ​​in Table 1 do not exceed a maximum value of 2.5 m. The expected maximum full-width value and the expected minimum full-width value (e.g., an expected range of full-width values) can be set to expected values, such as values ​​set according to laws and / or regulations related to the size and / or dimensions of vehicles in various countries or regions. Therefore, the host vehicle 200 can set the k value to an expected value (and / or an arbitrary value). For example, the host vehicle 200 can set the k value to 2.5, and then the α value can be calculated.

[0091] According to various example embodiments, when the host vehicle 200 sets the k value to a desired value and / or an arbitrary value and calculates the α value, the world width of the target vehicle 310 may be expressed as the following Equation 10 based on the proportional relationship of the bounding box 320 .

[0092] [Equation 10]

[0093] W=α·W 0

[0094] With reference to Table 1, according to various example embodiments, it can be seen that the range of k values ​​is different for each vehicle type. For example, in ordinary cars including compact cars, quasi-midsize cars, midsize cars, full-size cars, and midsize sports utility vehicles (SUVs), it can be seen that the range of k values ​​is between 2.2 and 2.7. However, in trucks, buses, or refrigerated trucks, it can be seen that the range of k values ​​is between 3.1 and 4.4. Therefore, when the k value corresponding to an ordinary car is applied to a special vehicle such as a truck or a bus, an error may occur in identifying the target vehicle for a vehicle including an autonomous driving system, and the error may lead to serious consequences such as traffic accidents. Therefore, a method for determining a suitable k value corresponding to a vehicle type may be desirable and beneficial, and further, when an inappropriate k value is set, a method for correcting an inappropriate k value may be desirable and beneficial.

[0095] According to various example embodiments, the range of the full width shown in Table 1 may be used to determine whether correction of the k value is desired and / or required. That is, considering that the width of the road is basically limited and the width of the vehicle is limited by law, a world width with an appropriate range may be set to reduce and / or prevent distance estimation based on an inappropriate k value.

[0096] According to various example embodiments, the desired minimum world width and the desired maximum world width may be set based on the desired width value. For example, referring to Table 1, 1595 mm, which is the minimum full width available for the currently released vehicle, may be set as the minimum world width, and 2495 mm, which is the maximum full width available for the currently released vehicle, may be set as the maximum world width, but example embodiments are not limited thereto. For example, even in the case where the target vehicle 310 is a vehicle such as a bus or a truck having a high k value, when the world width and α are calculated based on a condition in which a low k value corresponding to a compact vehicle or a medium-sized vehicle is used, the world width of the target vehicle 310 calculated based on the low k value may also be less than the minimum world width or greater than the maximum world width. This may be the same as the case where a high k value is set despite the target vehicle 310 being a compact vehicle or a medium-sized vehicle. Or in other words, if a k value that does not match and / or does not correspond to the vehicle type of the target vehicle is used, the world width of the target vehicle may be calculated incorrectly (e.g., outside the acceptable world width range).

[0097] According to various example embodiments, the ISP 120 may calculate the world width of the target vehicle 310 by using a desired and / or arbitrary k value, and may determine whether the calculated world width is within the minimum world width W.min Value and maximum world width W max When the calculated world width is greater than the minimum world width and less than the maximum world width, the ISP 120 may determine the calculated world width as the world width of the target vehicle 310. However, when the calculated world width is less than the minimum world width or greater than the maximum world width, the ISP 120 may determine that a correction to the k value is desired and / or required, and may generate a signal indicating an update to the world width or a signal indicating a correction to the k value.

[0098] According to various example embodiments, ISP 120 may update the world width. That is, because the calculated world width is outside the range of the minimum world width to the maximum world width, the calculated world width may be changed to the minimum world width or the maximum world width. For example, when the world width calculated based on an inappropriate k value is 3100 mm, ISP 120 may update the calculated world width to an updated value corresponding to 2495 mm as the maximum world width value. As another example, when the world width calculated based on an inappropriate k value is 1300 mm, ISP 120 may update the calculated world width to an updated value corresponding to 1595 mm as the minimum world width value.

[0099] According to at least one example embodiment, when the world width is updated to the maximum world width value, the corrected k value may be expressed as Equation 11 below.

[0100] [Equation 11]

[0101]

[0102] Equation 11 can be derived by the following equations 12 and 13. An example of updating the world width to the maximum world width value in equation 11 is shown, but example embodiments are not limited thereto. In the case of updating the world width to the minimum world width value, the maximum world width can be replaced by the minimum world width in equation 11.

[0103] [Equation 12]

[0104] W max +kW max tanθ=W o

[0105] [Equation 13]

[0106]

[0107] According to various example embodiments, when there is no information about the world width corresponding to the target vehicle 310 or the information about the world width corresponding to the target vehicle 310 is insufficient, the ISP 120 may generate an update signal. That is, in the case of detecting the target vehicle 310 based on the image acquired from the distance estimation device 100, the ISP 120 may determine whether the image of the previous frame image is used to detect the target vehicle 310 and whether a world width value between the minimum world width and the maximum world width is stored. If there is no world width value corresponding to the detected target vehicle 310 stored in the memory, the ISP 120 may generate an update signal. The distance estimation device 100 may perform operations related to determining and / or acquiring the world width of the target vehicle in response to the update signal. This will be described below with reference to Figure 7 Give a description.

[0108] According to various example embodiments, when ISP 120 includes information about the world width W and the ratio k of the full width to the full length of the target vehicle 310, ISP 120 may calculate the ratio α of the pixel width of the rear surface of the target vehicle 310 in the bounding box 320 by using Equation 14 below.

[0109] [Equation 14]

[0110]

[0111] In the case where α is calculated based on Equation 14, the ISP 120 may calculate the width of the image corresponding to the rear surface of the target vehicle 310 in the bounding box 320 by using the pixel width of the bounding box 320 of the target vehicle 310 .

[0112] [Equation 15]

[0113] w=α·w 0

[0114] Because the target vehicle 310 and the host vehicle 200 are traveling along adjacent lanes, the distance between the target vehicle 310 and the host vehicle 200 may include a forward distance component (or referred to as a front distance component) and a lateral distance component (or referred to as a side distance component) between the two vehicles. The forward distance component may be calculated based on the following equation 16.

[0115] [Equation 16]

[0116]

[0117] Furthermore, the lateral distance component can be expressed as in the following Equation 17.

[0118] [Equation 17]

[0119]

[0120] In Equation 17, Δx can be used to find

[0121] [Equation 18]

[0122]

[0123] In the above-mentioned example embodiment, an example of setting the k value to an expected value and / or an arbitrary value has been described, but the example embodiment is not limited thereto. The ISP 120 may confirm the type of the target vehicle 310 detected by image processing (e.g., confirming the type of the target vehicle based on the shape of the vehicle, or using image processing to confirm the characteristics of the vehicle, etc.). For example, the confirmable vehicle type may include a sedan, a special vehicle, a bus, etc. Examples of sedans may include compact sedans, quasi-midsize sedans, midsize sedans, full-size sedans, midsize SUVs, etc., and examples of special vehicles may include trucks, refrigerated trucks, tractor trailers, etc. The ISP 120 may confirm the type of the target vehicle 310, and may set an expected and / or predetermined k value based on the confirmed type. For example, when the confirmed type corresponds to a sedan, the ISP 120 may set the k value to an expected value and / or an arbitrary value between 2.2 and 2.7, but the example embodiment is not limited thereto. As another example, when the confirmed type corresponds to a special vehicle or a bus, the ISP 120 may set the k value to a desired value and / or an arbitrary value between 3.1 and 4.5, but example embodiments are not limited thereto. When the ISP 120 confirms the type of the target vehicle 310 and sets the k value within the range corresponding to the confirmed type, the possibility of calculating a world width outside the range of the minimum world width to the maximum world width can be significantly reduced.

[0124] Figure 6B Another example of a bird's eye view according to at least one example embodiment is shown.

[0125] Figure 6B The situation when the host vehicle 200 and the target vehicle 310 are seen from above is shown, and the Figures 3 to 6A The same or similar description as described.

[0126] Reference Figure 6B , the driving direction of the target vehicle 310 may be different from the driving direction of the host vehicle 200. Fig. 6A , the target vehicle 310 and the host vehicle 200 may travel in adjacent lanes, and the travel directions of the two vehicles may be the X-axis direction of the world coordinates and may be the same. Figure 6BIn the embodiment, the driving directions of the host vehicle 200 and the target vehicle 310 may be different. In addition, for example, when the host vehicle 200 is driving along the X-axis direction of the world coordinate system, the target vehicle 310 may be driving at an angle of 45 degrees in the negative Y-axis direction. In other words, Figure 6B A bird's-eye view corresponding to the time (ie, the moment) when the target vehicle 310 changes lanes to enter the same lane as that of the host vehicle 200 may be shown.

[0127] According to various example embodiments, when the target vehicle 310 is traveling while maintaining a traveling angle, the ISP 120 may set a time interval by comparing the traveling angle with the target vehicle 310. Fig. 6A The angle θ is added or from Fig. 6A The ISP 120 can obtain a new angle θ by subtracting the driving angle from the angle θ. The new angle θ is calculated based on the Δx value that changes based on the travel angle of the target vehicle 310. Other operations for determining, calculating and / or obtaining the forward distance (or front distance) and the lateral distance (or lateral distance) to the target vehicle 310 can be performed with Fig. 6A are the same, therefore, their descriptions are omitted.

[0128] In the above exemplary embodiment, an example in which the driving angle of the target vehicle 310 changes has been described, but the exemplary embodiment is not limited thereto. The exemplary embodiment can also be applied to a case in which the driving angle of the host vehicle 200 and / or the target vehicle 310 changes (and / or changes over time).

[0129] Figure 7 is a flowchart illustrating the operation of a distance estimation apparatus according to at least one example embodiment.

[0130] Reference Figure 7 , the ISP 120 may generate, determine, calculate, and / or obtain a bounding box 320 of the target vehicle 310 in operation S701. The bounding box 320 may be generated, determined, calculated, and / or obtained by an artificial intelligence (AI) algorithm including machine learning and / or deep learning, etc., but example embodiments are not limited thereto.

[0131] In operation S702, ISP 120 may determine whether the value of the world width is known or whether ISP 120 has received an update signal. When ISP 120 receives the update signal, ISP 120 may determine that a world width based on an inappropriate k value is calculated for the detected vehicle. In addition, ISP 120 may determine whether there is a world width value for the target vehicle 310 corresponding to the bounding box 320. ISP 120 may determine whether the target vehicle 310 is in the previous image frame. ISP 120 may determine that the target vehicle 310 has entered the field of view of the camera 110 in the current image frame, and may determine that there is no world width value. If ISP 120 determines that the world width value is unknown and / or an update signal has been received, ISP 120 proceeds to operation S703. If ISP 120 determines that the world width value is known and an update signal has not been received, ISP 120 proceeds to operation S708.

[0132] In operation S703, the ISP 120 may calculate an initial straight-line distance Z between the host vehicle 200 and the target vehicle 310. 0 ISP 120 may calculate the initial straight-line distance based on a geometric proportional relationship between the height at which camera 110 is installed (e.g., full height), the focal length of camera 110, and the y-axis pixel distance from the bottom edge of bounding box 320 to the vanishing point, but example embodiments are not limited thereto. Figures 4 to 6B As discussed in , the initial straight-line distance may be provided based on the condition that the target vehicle 310 and the host vehicle 200 are traveling along the same lane, but is not limited thereto.

[0133] In operation S704, the ISP 120 may calculate an initial world width W 0 The ISP 120 may calculate the initial world width based on a geometric proportional relationship between the horizontal (width direction) pixel distance of the bounding box 320, the focal length of the camera 110, and the initial straight line distance calculated in operation S703, but example embodiments are not limited thereto. Figures 4 to 6B As discussed, the initial world width may be provided based on a condition that the target vehicle 310 and the host vehicle 200 travel in the same lane, but example embodiments are not limited thereto.

[0134] In operation S705, the ISP 120 may set the k value to a desired value and / or an arbitrary value, and may calculate the α value based on the k value. Here, the k value may refer to the ratio of the full width to the full length of the target vehicle 310, and the α value may refer to the ratio of the width occupied by the image of the rear surface of the target vehicle 310 in the bounding box 320 to the pixel width of the bounding box 320.

[0135] In operation S706 , the ISP 120 may calculate a world width of the target vehicle 310 based on the calculated α value.

[0136] In operation S707, ISP 120 may determine whether the calculated world width is within an acceptable world width range, such as between a desired minimum world width and a desired maximum world width. The acceptable world width range (e.g., between a desired minimum world width and a desired maximum world width) may be based on known specifications (e.g., known width data) for a variety of vehicles and / or a variety of vehicle classification types. If ISP 120 determines that the calculated world width is within an acceptable world width range, ISP 120 proceeds to operation S708. Otherwise, ISP 120 proceeds to operation S709.

[0137] In operation S708, the ISP 120 may determine and use the calculated world width as the world width of the target vehicle 310. Subsequently, although not shown, the ISP 120 may obtain a forward distance component between the target vehicle 310 and the host vehicle 200 based on a ratio of a width occupied by the image of the rear surface of the target vehicle 310 in the bounding box 320 to the width of the bounding box 320 based on the calculated world width.

[0138] In operation S709, the ISP 120 may use the minimum world width or the maximum world width as the initial world width W 0 The α value is calculated again. For example, when the world width calculated in operation S706 is larger than the expected maximum world width, the ISP 120 may set the value of the calculated world width to the expected maximum world width value, may calculate the α value, and may correct the k value, but example embodiments are not limited thereto. As another example, when the calculated world width is smaller than the expected minimum world width, the ISP 120 may set the value of the calculated world width to the expected minimum world width value, etc.

[0139] Figure 8 A bird's eye view of a host vehicle 200 including a plurality of distance estimation devices is shown, according to at least one exemplary embodiment.

[0140] Reference Figure 8, the main vehicle 200 may include multiple distance estimation devices. For example, the main vehicle 200 may include a first distance estimation device 810 and a second distance estimation device 820, but is not limited thereto. According to other example embodiments, the main vehicle 200 may include more distance estimation devices, and each distance estimation device may be configured to acquire images of other areas surrounding the main vehicle 200 (such as a lateral area (e.g., a left area and / or a right area), a rearward area (e.g., a rearward area), etc.) and images of one or more target vehicles 310 located in a forward (front) area.

[0141] like Figure 8 As shown in , according to at least one example embodiment, the ISP 120 may estimate the forward distance to the target vehicle 310 based on at least one of the information acquired by the first distance estimation device 810 and the information acquired by the second distance estimation device 820. For example, the ISP 120 may estimate the forward distance to the target vehicle 310 based on at least one of the first world width acquired from the first distance estimation device 810 and the second world width acquired from the second distance estimation device 820. The first distance estimation device 810 may calculate the world width of the target vehicle 310 based on the k1 value, and the second distance estimation device 820 may calculate the world width of the target vehicle 310 based on the k2 value, wherein the definitions of k1 and k2 are similar to the above-mentioned definition of k.

[0142] For example, the value of the world width calculated by the first distance estimation device 810 based on the k1 value may be within the expected world width range, such as between the expected minimum world width and the expected maximum world width, and the value of the world width calculated by the second distance estimation device 820 based on the k2 value may be within the expected world width range, such as from the expected minimum world width to the expected maximum world width. In this case, the ISP 120 may use the forward distance value obtained by the first distance estimation device 810 and / or the second distance estimation device 820. That is, the ISP 120 may determine the first forward distance obtained by the first distance estimation device 810 as the distance between the host vehicle 200 and the target vehicle 310, may determine the second forward distance obtained by the second distance estimation device 820 as the distance between the host vehicle 200 and the target vehicle 310, and / or may determine the value obtained by calculating the average of the first forward distance and the second forward distance as the distance between the host vehicle 200 and the target vehicle 310.

[0143] As another example, the value of the world width calculated based on the k1 value by the first distance estimation device 810 may be less than the desired minimum world width or greater than the desired maximum world width. The value of the world width calculated based on the k2 value by the second distance estimation device 820 may be within the desired world width range, for example, between the minimum world width and the maximum world width. In this case, the ISP 120 may determine the k1 value of the first distance estimation device 810 as an inappropriate value, and may determine the world width obtained using the second distance estimation device 820 as (for example, used as) the world width of the target vehicle 310. The ISP 120 can calculate the forward distance to the target vehicle 310 by using the world width calculated by the second distance estimation device 820. In the above example embodiment, an example is shown in which the k1 value of the first distance estimation device 810 is inappropriate and the k2 value of the second distance estimation device 820 is appropriate, but the present example embodiment is not limited thereto. For example, when the k2 value of the second distance estimation device 820 is inappropriate and the k1 value of the first distance estimation device 810 is appropriate, the ISP 120 can determine the world width obtained using the first distance estimation device 810 as (for example, used as) the world width of the target vehicle 310, and can estimate the distance between the target vehicle 310 and the main vehicle 200 based on the determined world width.

[0144] Fig. 9 An example of successive refinements performed on a bounding box is shown according to at least one example embodiment.

[0145] Reference Fig. 9 , the distance estimation device 100 may acquire continuous images (e.g., multiple images, a sequence of images, etc.) based on continuous driving. As an example, when the host vehicle 200 is traveling in a lane, the host vehicle 200 may acquire continuous images of three forward target vehicles by using the camera 110. In the example, the host vehicle 200 may continuously acquire images of target vehicles from the n-2th frame to the nth frame corresponding to the current frame over time.

[0146] According to various example embodiments, the ISP 120 may detect a bounding box 930 of an n-th frame corresponding to the current time. In the n-2-th frame or the n-1-th frame corresponding to the previous time, the bounding boxes 910 and 920 may be set to include the desired minimum area corresponding to the target vehicle, but the bounding box 930 of the n-th frame may be set to include the area corresponding to the target vehicle and the peripheral area of ​​the target vehicle. In this case, as described above, the distance estimation apparatus 100 according to at least one example embodiment may estimate the forward distance to the target vehicle based on the pixel width of the detected bounding box, and therefore, when the bounding box is set to include the undesired peripheral area of ​​the target vehicle as with the bounding box 930 of the n-th frame, the size of the pixel width of the bounding box 930 (e.g., the pixel width value) may be inaccurate, and therefore, the forward distance estimated based on the inaccurate size of the pixel width may be inaccurate.

[0147] According to various example embodiments, the ISP 120 may determine the reliability of a bounding box (e.g., bounding box 930 of the n-th frame) acquired in the current time frame by using (e.g., based on) an average value of pixel widths of bounding boxes (e.g., bounding box 910 of the n-2th frame and bounding box 920 of the n-1th frame) acquired in a previous time frame (e.g., in a previous image frame, an earlier image frame, etc.). When the pixel width of each bounding box detected from the previous time frame is x, the pixel width of the bounding box detected from the current time frame is y, and the difference between x and y is greater than a desired threshold, the ISP 120 may determine that an error has occurred in determining the size (e.g., pixel width) of the bounding box detected in the current time frame (e.g., current image frame). For example, when the pixel width of the bounding box 910 detected from the n-2 frame is 90, the pixel width of the bounding box 920 detected from the n-1 frame is 91, and the pixel width of the bounding box 930 detected from the n frame is 120, if the difference between the pixel widths of the n-2 frame and the n frame is greater than the desired threshold, the ISP 120 can determine that an error occurred in the process of detecting the bounding box 930. The desired threshold can be set based on empirical data and / or can be set by a user. This is because, during continuous driving, the size of the bounding box corresponding to the target vehicle traveling in parallel cannot increase rapidly between the time interval of one frame or the time interval of some frames. In addition, if the difference between the pixel widths of any previous image frame and the current image frame exceeds the desired threshold, the ISP 120 can determine that an error occurred in estimating the forward distance to the target vehicle based on the calculated world width and the pixel widths of the bounding boxes 910, 920, and 930 corresponding to the continuous time stream.

[0148] According to various example embodiments, ISP 120 may correct an image of an image frame of a current time based on one or more image frames corresponding to at least one previous time. For example, ISP 120 may perform image filtering such as low-pass filtering, Kalman filtering, etc. on the world width calculated from the current frame. In the case of using low-pass filtering or Kalman filtering, ISP 120 may correct the pixel width of bounding box 930 detected as being larger than the actual pixel width in the n-th frame or smaller than the actual pixel width in the n-th frame, or may correct the world width calculated based on the pixel width of bounding box 930 detected as being larger than the actual pixel width or smaller than the actual pixel width, etc.

[0149] According to various example embodiments, the host vehicle 200 may calculate the forward distance from the target vehicle 310, and may control the host vehicle 200 based on the result of the calculation. For example, the host vehicle 200 may further include an autonomous driving controller (not shown) as well as the vehicle controller 210. The autonomous driving controller (not shown) may reflect the calculation result when generating a driving path of the host vehicle 200, determining a driving direction of the host vehicle 200, and changing a lane (e.g., turning the vehicle), or increasing or decreasing the speed of the host vehicle 200 (e.g., applying a brake or accelerator of the vehicle).

[0150] According to various example embodiments, the autonomous driving controller (not shown) may determine the risk level of the target vehicle 310 based on the forward distance and the lateral distance from the target vehicle 310, adjust the speed of the host vehicle 200 by using the vehicle controller 210 based on the risk level, and change the lane of the host vehicle 200 relative to the lane of the target vehicle 310 based on the risk level. The autonomous driving controller (not shown) may determine whether the current situation corresponds to a situation where the forward distance from the target vehicle 310 is less than the expected distance and / or the predetermined distance, a situation where the number of lanes to be changed by the target vehicle 310 is as large as the determination result based on Δx, or a situation where the target vehicle 310 is a vehicle type with a possibility that a load falls onto the driving road, and the like, and may calculate the risk level of the target vehicle 310 based on the determination result. The autonomous driving controller (not shown) may control the travel of the host vehicle 200 based on the risk level, and thus, may stably prepare for the risk of sudden accidents, collisions, etc. caused by peripheral vehicles. According to some example embodiments, the autonomous driving controller may be integrated with the vehicle controller 210 and / or the ISP 120 and / or provide the functionality of the vehicle controller 210 and / or the ISP 120, or vice versa. The autonomous driving controller may include: a processing circuit, such as hardware including a logic circuit; a hardware / software combination, such as at least one processor running software; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), and the like.

[0151] While the inventive concept has been particularly shown and described with reference to various example embodiments of the inventive concept, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the claims.

Claims

1. A method for operating a distance estimation device, the distance estimation device comprising at least one camera, the method comprising: generating, using processing circuitry, a bounding box corresponding to the target vehicle based on images acquired using the at least one camera; determining a first straight-line distance to a target vehicle using a processing circuit; calculating, using processing circuitry, a first world width of the target vehicle based on the first straight-line distance and the width of the bounding box; calculating, using processing circuitry, a first ratio based on a width of a rear surface of the target vehicle and a width of a side surface of the target vehicle in the acquired image; calculating, using the processing circuit, a second ratio corresponding to a rear surface of the target vehicle, the second ratio being based on a width of the bounding box and the first ratio; calculating, using the processing circuit, a second world width of the target vehicle based on the second ratio; calculating, using processing circuitry, an estimated distance to the target vehicle based on the second world width and the second scale; as well as The host vehicle is controlled using processing circuitry based on the estimated distance to the target vehicle.

2. The operating method according to claim 1, further comprising: A determination is made, using the processing circuit, whether the value of the second world width is less than a desired minimum world width or greater than a desired maximum world width.

3. The operating method according to claim 2, further comprising: In response to the second world width being smaller than the desired minimum world width, changing, using the processing circuit, the value of the second world width to the value of the desired minimum world width; as well as In response to the second world width being greater than the desired maximum world width, the value of the second world width is changed, by the processing circuit, to a value of the desired maximum world width.

4. The operating method according to claim 1, further comprising: A first straight-line distance and a first world width are determined using processing circuitry based on the target vehicle being located in the forward region.

5. The operating method according to claim 1, further comprising: The first linear distance is determined, using processing circuitry, based on a height of the at least one camera, a focal length of the at least one camera, and a vertical distance between a lower end of the bounding box and a vanishing point of the acquired image.

6. The operating method according to claim 5, further comprising: A first world width is determined, using processing circuitry, based on the focal length, the width of the bounding box, and the first linear distance.

7. The operating method according to claim 1, wherein: In response to the optical axis of the at least one camera being within the left and right ranges of the bounding box, determining, using the processing circuit, whether the target vehicle and the distance estimation device are located in the same lane.

8. The operating method according to claim 1, further comprising: determining, using processing circuitry, a vehicle type of a target vehicle from a plurality of vehicle types based on the acquired image; as well as setting, using processing circuitry, a value of a third ratio based on the determined vehicle type corresponding to the target vehicle, the third ratio corresponding to a real-world length of a side surface and a rear surface of the target vehicle, The value of the third ratio is different for each of the plurality of vehicle types.

9. The operating method according to claim 5, further comprising: determining, using the processing circuit, a pixel width of a rearward region of the target vehicle within the bounding box based on the second ratio; as well as A forward distance component of a distance between the distance estimation device and the target vehicle is calculated using processing circuitry based on the determined pixel width of the rearward region, the focal length, and the second world width.

10. A distance estimation device, comprising: at least one camera configured to acquire at least one image associated with a target vehicle; as well as The processing circuit is configured to: Generate a bounding box corresponding to the target vehicle, Determine the first straight-line distance to the target vehicle, Calculate the first world width of the target vehicle based on the first straight-line distance and the width of the bounding box, calculating a first ratio based on a width of a rear surface of the target vehicle and a width of a side surface of the target vehicle in the acquired image, calculating a second ratio, the second ratio corresponding to the rear surface of the target vehicle, the second ratio being based on the width of the bounding box and the first ratio, Calculate a second world width of the target vehicle based on the second scale, calculating an estimated distance to the target vehicle based on the second world width and the second scale, and The estimated distance is output to the host vehicle.

11. The distance estimation device according to claim 10, wherein: The processing circuit is further configured to: Determine whether the value of the second world width is smaller than the desired minimum world width or larger than the desired maximum world width.

12. The distance estimation device according to claim 11, wherein: The processing circuit is further configured to: In response to the second world width being smaller than the desired minimum world width, changing the value of the second world width to the value of the desired minimum world width; and In response to the second world width being greater than the desired maximum world width, the value of the second world width is changed to the value of the desired maximum world width.

13. The distance estimation device according to claim 12, wherein: The processing circuit is further configured to: In response to the value of the second world width being changed to the value of the desired minimum world width or the value of the desired maximum world width, a second scale is calculated based on the changed value of the second world width.

14. The distance estimation device according to claim 10, wherein: The processing circuit is further configured to: A first straight-line distance and a first world width are determined based on the target vehicle being located in the forward area.

15. The distance estimation device according to claim 10, wherein: The processing circuit is further configured to: The first linear distance is determined based on a height of the at least one camera, a focal length of the at least one camera, and a vertical distance between a lower end of the bounding box and a vanishing point of the acquired image.

16. The distance estimation device according to claim 15, wherein: The processing circuit is further configured to: The first world width is determined based on the focal length, the width of the bounding box, and the first linear distance.

17. The distance estimation device according to claim 10, wherein: The processing circuit is further configured to: In response to the optical axis of the at least one camera being within the left and right ranges of the bounding box, it is determined whether the target vehicle and the distance estimation device are located in the same lane.

18. The distance estimation device according to claim 10, wherein: The processing circuit is further configured to: determining a vehicle type of a target vehicle from a plurality of vehicle types based on the acquired image; as well as setting a value of a third ratio based on the determined vehicle type corresponding to the target vehicle, the third ratio corresponding to the real-world lengths of the side surface and the rear surface of the target vehicle, The value of the third ratio is different for each of the plurality of vehicle types.

19. The distance estimation device according to claim 15, wherein: The processing circuit is further configured to: Determine the pixel width of the rearward region of the target vehicle in the bounding box based on the second ratio, and A forward distance component of the distance between the distance estimation device and the target vehicle is calculated based on the determined pixel width of the rearward area, the focal length, and the second world width.

20. A host vehicle device, the host vehicle device comprising: at least one camera configured to acquire at least one image associated with a target vehicle; as well as The processing circuit is configured to: Generate a bounding box corresponding to the target vehicle, Determine the first straight-line distance to the target vehicle, Calculate the first world width of the target vehicle based on the first straight-line distance and the width of the bounding box, calculating a first ratio based on a width of a rear surface of the target vehicle and a width of a side surface of the target vehicle in the acquired image, calculating a second ratio, the second ratio corresponding to the rear surface of the target vehicle, the second ratio being based on the width of the bounding box and the first ratio, Calculate a second world width of the target vehicle based on the second scale, Estimate the forward distance and lateral distance to the target vehicle based on the second world width and the second scale, and Host vehicle equipment is controlled based on the estimated forward distance and lateral distance.

Images