Apparatus and method for distance correction using reliability weights, and computer-readable recording medium having program for implementing the recorded thereon

Abstract

A reliability-based distance correction device and method, and a computer-readable recording medium storing a program for implementing the method are disclosed. A reliability-based distance correction device according to an embodiment of the present invention includes an input unit that receives an image from a camera, a distance coordinate estimation unit that estimates initial distance coordinates from the received image, a region of interest setting unit that sets a region of interest in the image, a sectioning unit that divides a forward distance range within the set region of interest into a plurality of sections, a section-by-section correction unit that performs correction on the initial distance coordinates by reflecting the characteristics of each section for the section-by-section correction unit, and a final distance determination unit that determines the final distance coordinates by reflecting the operation results of the section-by-section correction unit.

Description

Technology Field

[0001] The present invention relates to a reliability-based distance correction device and method and a computer-readable recording medium storing a program for implementing the method. More specifically, the invention relates to a reliability-based distance correction device and method that improves errors through real-time correction based on reliability when estimating distance using a vehicle camera, and a computer-readable recording medium storing a program for implementing the method. Background Technology

[0003] Camera-based distance estimation technology, mounted on various mobile devices such as vehicles, robots, and drones, performs environmental recognition and driving control by converting the pixel coordinates of objects within an image into actual distances based on a geometric projection model. Since this camera-based distance estimation technology is based on monocular or stereo cameras, it is sensitive to pixel-level errors, and there is a problem in that error propagation characteristics increase non-linearly with observation distance.

[0004] For example, in near-distance regions, even very small pixel deviations are amplified into large distance errors due to projection geometry, while in far-distance regions, the depth gradient decreases, leading to reduced geometric sensitivity. These structural limitations are pointed out as chronic problems inherent to camera-only distance estimation technologies.

[0005] As a measure to improve the accuracy of such camera-based distance estimation technology, a method that performs an iterative correction process, such as CIDC (Camera Image Distance Correction), has been proposed. While this method has the advantage of improving cumulative accuracy by repeatedly calculating correction values ​​based on an improved distance distribution, it suffers from several drawbacks: excessive computational load resulting from iterative correction across the entire area; error amplification caused by repeated corrections even in low-reliability regions; the failure to reflect the dynamic characteristics of moving objects and changes in road slope, leading to situations where identical pixel changes induce different distance variations; and insufficient robustness against changes in environmental conditions.

[0006] As such, conventional distance estimation technologies fail to simultaneously meet the requirements for improved accuracy, real-time processing, and robustness. In particular, because they fail to account for relative differences in pixel-level reliability, they are not serving as a fundamental solution for camera-based distance estimation technology. Prior art literature

[0008] Registered Patent Publication No. 10-2427463 Japanese Registered Patent Publication JP4771891 The problem to be solved

[0009] To solve the aforementioned problems, the technical objective of the present invention is to provide a reliability-based distance correction device and method that improve accuracy, robustness, and computational efficiency by performing iterative correction based on pixel-by-pixel reliability when estimating the distance to a forward object using a vehicle camera, and a computer-readable recording medium storing a program for implementing this method.

[0010] The problems solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0012] As a means for solving the aforementioned technical problem, a reliability-based distance correction device according to an embodiment of the present invention comprises: an input unit that receives an image from a camera; a distance coordinate estimation unit that estimates initial distance coordinates from the received image; a region of interest setting unit that sets a region of interest in the image; a sectioning unit that divides a forward distance range within the set region of interest into a plurality of sections; a section-by-section correction unit that performs correction on the initial distance coordinates by reflecting the characteristics of each section for the sectioned sections; and a final distance determination unit that determines the final distance coordinates by reflecting the operation results of the section-by-section correction unit.

[0013] The above-mentioned area of ​​interest setting unit can set an area of ​​interest using at least one of lane recognition results, object detection results, and HD map information.

[0014] The above section can be divided into a short-range section, a medium-range section, and a long-range section based on the center of the camera.

[0015] The above-mentioned section-by-section correction unit may include a reliability calculation unit that calculates the reliability of each pixel and a correction amount calculation unit that calculates the correction amount according to a preset correction scale and the number of repetitions for each section.

[0016] The above final distance determining unit can determine the final distance coordinates by reflecting the calculated reliability and the calculated correction amount.

[0017] The above reliability calculation unit can calculate reliability using the following formula:

[0018]

[0019] Here, w(u,v) is the confidence level of pixel(u,v), is the normalized edge strength, is normalized brightness / contrast information, is normalized object detection confidence, is the respective weight of edge intensity, brightness / contrast information, and object detection confidence.

[0020] The above correction amount calculation unit can determine correction amount candidates using the following formula:

[0021]

[0022] Here, is the correction amount candidate vector calculated in the k-th iteration, is a correction scale corresponding to the distance interval, is the distance error vector at the k-th iteration.

[0023] The above correction amount calculation unit can determine the correction amount using the following formula:

[0024]

[0025] Here, is the correction amount, and w(u,v) is the confidence level of pixel(u,v).

[0026] The above final distance determining unit can determine the final distance coordinates using the following formula:

[0027]

[0028] Here, is the distance estimate updated after the (k+1)th iteration, is the distance estimate for the k-th iteration.

[0029] Meanwhile, a reliability-based distance correction method according to another embodiment of the present invention includes the steps of receiving an image from a camera, estimating initial distance coordinates from the received image, setting a region of interest in the image, dividing a forward distance range within the set region of interest into a plurality of segments, performing correction on the initial distance coordinates by reflecting the characteristics of each segment for the segmented plurality of segments, and determining the final distance coordinates by reflecting the operation results of the segment-specific correction unit.

[0030] The above setting step may set an area of ​​interest using at least one of lane recognition results, object detection results, and HD map information.

[0031] The above-mentioned segmentation step can divide the area into a short-range section, a medium-range section, and a long-range section based on the center of the camera.

[0032] The step of performing the above correction may include a step of calculating the reliability of each pixel and a step of calculating the correction amount according to a preset correction scale and the number of repetitions for each interval.

[0033] The above determining step can determine the final distance coordinates by reflecting the calculated reliability and the calculated correction amount.

[0034] The step of calculating the above reliability can calculate the reliability using the following formula:

[0035]

[0036] Here, w(u,v) is the confidence level of pixel(u,v), is the normalized edge strength, is normalized brightness / contrast information, is normalized object detection confidence, is the respective weight of edge intensity, brightness / contrast information, and object detection confidence.

[0037] The step of calculating the above correction amount can determine correction amount candidates using the following formula:

[0038]

[0039] Here, is the correction amount candidate vector calculated in the k-th iteration, is a correction scale corresponding to the distance interval, is the distance error vector at the k-th iteration.

[0040] The step of calculating the above correction amount can determine the correction amount using the following formula:

[0041]

[0042] Here, is the correction amount, and w(u,v) is the confidence level of pixel(u,v).

[0043] The above determining step can determine the final distance coordinates using the following formula:

[0044]

[0045] Here, is the distance estimate updated after the (k+1)th iteration, is the distance estimate for the k-th iteration.

[0046] Additionally, according to another embodiment of the present invention, a computer-readable recording medium is disclosed that records a program for realizing a reliability-based distance correction method according to any one of claims 10 to 18. Effects of the invention

[0048] According to the present invention, by dividing a region of interest based on distance and applying a correction scale and a number of iterations to each section to perform correction, the invention provides a reliability-based distance correction device and method that solve the structural limitations of a geometric projection model and the problem of magnification of pixel-unit errors, and a computer-readable recording medium that records a program for realizing this method.

[0049] In addition, computational efficiency is significantly improved by performing calculations only on the region of interest, and balanced correction performance is provided across the entire distance range by applying different correction amounts depending on the forward distance.

[0050] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0052] FIG. 1 is a block diagram of an autonomous driving system including a reliability-based distance correction device according to an embodiment of the present invention, FIG. 2 is a detailed block diagram of the autonomous driving device illustrated in FIG. 1, FIG. 3 is a block diagram of a reliability-based distance correction device according to an embodiment of the present invention, FIG. 4 is a diagram illustrating the operation of a distance coordinate estimation unit constituting a reliability-based distance correction device according to an embodiment of the present invention. FIG. 5 is a diagram illustrating the operation of a region of interest setting unit constituting a reliability-based distance correction device according to an embodiment of the present invention. FIG. 6 is a flowchart for explaining the operation procedure of the region of interest setting unit illustrated in FIG. 3, FIGS. 7 and 8 are drawings for explaining the operation of a partition section constituting a reliability-based distance correction device according to an embodiment of the present invention. FIG. 9 is a detailed block diagram of a section-by-section correction unit constituting a reliability-based distance correction device according to an embodiment of the present invention. FIG. 10 is a graph showing an error damping curve in a reliability-based distance correction device according to an embodiment of the present invention, and, FIG. 11 is a flowchart illustrating a reliability-based distance correction method according to one embodiment of the present invention. Specific details for implementing the invention

[0053] The above objects, other objects, features, and advantages of the present invention will be easily understood through the following preferred embodiments associated with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to ensure that the disclosed content is thorough and complete, and to ensure that the spirit of the invention is sufficiently conveyed to those skilled in the art.

[0054] In this specification, when a component is described as being on another component, it means that it may be formed directly on the other component or that a third component may be interposed between them.

[0055] Where terms such as "first," "second," etc. are used in this specification to describe components, these components shall not be limited by such terms. These terms are used merely to distinguish one component from another. The embodiments described and illustrated herein also include complementary embodiments.

[0056] Furthermore, when it is stated that the first element (or component) operates or is executed on (ON) the second element (or component), it should be understood that the first element (or component) operates or is executed in the environment where the second element (or component) operates or is executed, or operates or is executed through direct or indirect interaction with the second element (or component).

[0057] Where any element, component, device, or system is described as including a component consisting of a program or software, it should be understood that, even without explicit mention, that element, component, device, or system includes hardware (e.g., memory, CPU, etc.) or other programs or software (e.g., an operating system or drivers required to run the hardware) necessary for the execution or operation of that program or software.

[0058] Furthermore, unless otherwise specified regarding the implementation of any element (or component), it should be understood that the element (or component) may be implemented in software, hardware, or any form of both software and hardware.

[0059] Furthermore, the terms used herein are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, 'comprises' and / or 'comprising' do not exclude the presence or addition of one or more other components to the mentioned components.

[0061] FIG. 1 is a block diagram of an autonomous driving system including a reliability-based distance correction device according to an embodiment of the present invention, and FIG. 2 is a detailed block diagram of the autonomous driving device shown in FIG. 1.

[0062] Referring to FIG. 1, an autonomous driving system (1000) including a reliability-based distance correction device according to one embodiment of the present invention includes a camera (100), a distance correction device (200), and an autonomous driving device (300).

[0063] The camera (100) is a device for acquiring images as one of various sensors installed in the autonomous driving system (1000). The camera (100) is not specifically limited in type and may be any one of a monocular camera, a stereo camera, a wide-angle camera, an infrared camera, an event camera, and an image sensor such as CMOS and CCD.

[0064] The distance correction device (200) estimates initial distance coordinates from an image input from a camera (100) and generates final distance coordinates (corrected distance estimates) by performing an iterative correction algorithm based on a region of interest (ROI), distance segment division, and pixel reliability. That is, the distance correction device (200) corresponds to a device that performs correction to increase the accuracy of image-based distance estimation.

[0065] By means of the distance correction device (200), fast and precise correction is possible at close range relative to the camera, stable correction is possible at medium range, and overcorrection can be prevented at long range. In addition, the distance correction device (200) automatically reduces the amount of correction for areas with low reliability, and real-time performance can be ensured through ROI-based processing. The distance correction device (200) that implements these effects will be described in more detail in FIGS. 3 to 8, which will be described later.

[0066] The autonomous driving device (300) includes one or more processors and memories for perceiving a driving environment, determining a driving situation, and controlling a vehicle, and performs driving control of the vehicle using the final distance coordinates received from the distance correction device (200). Referring to FIG. 2, the autonomous driving device (300) includes a perception module (310), a judgment module (320), a vehicle control module (330), and a map-based path module (340).

[0067] The perception module (310) performs object detection, lane recognition, and traffic light recognition by utilizing sensor data such as a camera (100), radar, and lidar. When the perception module (310) receives corrected distance coordinates from the distance correction device (200) and operates, the perception accuracy can be improved.

[0068] The judgment module (320) analyzes the current driving situation using object information, lane information, and driving environment information provided by the perception module (310), and performs driving decision-making for the autonomous vehicle, such as determining the collision risk, determining the distance between vehicles, determining whether to maintain the lane, and determining the path change, by reflecting the final distance coordinates provided by the distance correction device (200).

[0069] The judgment module (320) calculates the risk of collision with an object using the final distance coordinates provided by the distance correction device (200), determines whether a safe distance is secured, and calculates the Time-To-Collision (TTC). The judgment module (320) can determine the risk of lane departure and the necessity of changing lanes by fusing the lane data and the final distance coordinate data of the perception module (310). Additionally, the judgment module (320) can determine an obstacle avoidance path and select policies such as detour, acceleration, and stopping.

[0070] The vehicle control module (330) generates control signals to perform driving control of the vehicle according to the driving decision of the judgment module (320), and controls vehicle operations such as steering, acceleration, deceleration, and braking. In addition, the vehicle control module (330) communicates with one or more processors (not shown) and a vehicle ECU (Electronic Control Unit) (not shown) to generate and transmit vehicle operation control commands. Here, the ECU includes an Engine Control Unit, a Transmission Control Unit, a Brake Control Unit, a Steering Control Unit, an ADS Controller, and a Body Control Unit.

[0071] More specifically, the vehicle control module (330) can control driving based on the final distance coordinates, which are the output values ​​of the distance correction device (200). Since the final distance coordinates represent the exact distance to the object, stable control can be performed based on them.

[0072] Additionally, the vehicle control module (330) converts the decision of the judgment module (320) into actual vehicle operation. For example, it controls braking when there is a risk of collision, controls steering when a lane change is required, and controls acceleration and deceleration when maintaining speed is required.

[0073] The map-based route module (340) generates a driving route of a vehicle or updates the route using a high-precision map (HD Map), a navigation map, and pre-stored road information. Additionally, the map-based route module (340) can perform route planning based on the judgment result of the judgment module (320).

[0074] The map-based path module (340) establishes a high-precision map-based path plan. For example, it can generate a path by reflecting lane information, road shape information, sign information, and speed limit information. Additionally, the map-based path module (340) can recalculate the path based on obstacles, traffic congestion, and road irregularities by reflecting lane change requests, collision avoidance, etc., determined by the judgment module (320).

[0075] Additionally, the map-based path module (340) can determine an accurate distance-based path to surrounding objects using the final distance coordinates of the distance correction device (200), determine an accurate location when changing lanes on narrow roads, and follow a precise path.

[0077] FIG. 3 is a block diagram of a reliability-based distance correction device according to one embodiment of the present invention.

[0078] Referring to FIG. 3, a reliability-based distance correction device (200) according to one embodiment of the present invention includes an input unit (210), a distance coordinate estimation unit (220), a region of interest setting unit (230), a section unit (240), a section-by-section correction unit (250), a final distance determination unit (260), a storage unit (270), and a control unit (280).

[0079] The input unit (210) receives an image from the camera (100) and provides this image data to the distance coordinate estimation unit (220). The image input to the input unit (210) may be in various forms, such as an RGB image, a grayscale image, and a depth image.

[0080] The input unit (210) preprocesses the image data after receiving the image from the camera (100). The preprocessing operation includes resolution scaling, distortion correction, and noise filtering. When noise filtering is performed on the image data in the input unit (210), an effect of improved accuracy can be expected when the distance coordinate estimation unit (220) estimates the initial distance coordinates. Additionally, the input unit (210) can manage the frame number or timestamp of the image data to ensure time consistency with the autonomous driving system (1000).

[0081] The distance coordinate estimation unit (220) calculates initial distance coordinates based on a geometric projection model using the image received through the input unit (210) and the camera (100) intrinsic parameters (K) and extrinsic parameters (R, T). That is, the distance coordinate estimation unit (220) calculates the pixels (u,v) as three-dimensional coordinates on an actual road plane. It is converted into. The distance coordinate estimation unit (220) will be explained in more detail in FIG. 4, which will be described later.

[0082] The region of interest setting unit (230) sets the area related to driving as a region of interest (ROI) based on the image received through the input unit (210) and the initial distance coordinates estimated by the distance coordinate estimation unit (220). The region of interest setting unit (230) can reduce the amount of computation and improve the accuracy of distance correction by omitting correction or performing only minimal correction for areas outside the ROI. The region of interest setting unit (230) will be described in more detail in FIG. 5, which will be described later.

[0083] The partitioning section (240) divides the forward distance range within the area of ​​interest set by the area of ​​interest setting section (230) into multiple sections. At this time, the multiple sections may include a short distance section, a medium distance section, and a long distance section based on the center of the camera (100). The operation of the partitioning section (240) will be explained in more detail in FIGS. 7 and 8, which will be described later.

[0084] The section-by-section correction unit (250) performs correction on the initial distance coordinates by reflecting the characteristics of each section for a plurality of sections partitioned by the sectioning unit (240). That is, the section-by-section correction unit (250) performs actual calculations by applying reliability-based iterative correction to the distance sections. The section-by-section correction unit (250) will be explained in more detail in FIG. 9, which will be described later.

[0085] The final distance determining unit (260) determines the final distance coordinates by reflecting the operation results of the section-by-section correction unit (250). More specifically, the final distance determining unit (260) determines the final distance coordinates by reflecting the reliability and correction amount calculated by the section-by-section correction unit (250).

[0086] The storage unit (270) stores all information necessary for the operation of the reliability-based distance correction device (200). For example, the storage unit (270) may store camera parameters (internal parameters and external parameters), correction scale information, repetition count information, weights for reliability calculation, and distance correction result data. The storage unit (270) may be implemented as one or more memories, non-volatile storage devices, or data registers.

[0087] The control unit (280) controls the overall operation of the reliability-based distance correction device (200). That is, the control unit (280) controls the signal input and output between the input unit (210), the distance coordinate estimation unit (220), the area of ​​interest setting unit (230), the section unit (240), the section-by-section correction unit (250), the final distance determination unit (260), and the storage unit (270).

[0088] The control unit (280) controls the entire operation so that distance coordinate estimation, region of interest setting, distance segmentation, correction by segment, and final distance determination operations are performed sequentially and logically. Additionally, the control unit (280) can perform adaptive control considering environmental changes. For example, the control unit (280) can control the region of interest setting unit (230) to update external parameters when pitch / roll changes are detected, reflect profiles for night and rainy conditions, and adjust the ROI.

[0090] FIG. 4 is a diagram illustrating the operation of a distance coordinate estimation unit constituting a reliability-based distance correction device according to an embodiment of the present invention.

[0091] The distance coordinate estimation unit (220) obtains an image in a camera coordinate system and back-projects the pixel coordinates (u,v) using an intrinsic / extrinsic camera projection and a road plane condition (z=0) to obtain an initial distance coordinate corresponding to the pixel. Calculates.

[0092] The distance coordinate estimation unit (220) calculates the ray formed by pixel coordinates (u,v) in space using camera intrinsic parameters (K) and extrinsic parameters (R, T). This is shown in Equation 1.

[0093] [Mathematical Formula 1]

[0094]

[0095] Subsequently, the distance coordinate estimation unit (220) calculates the intersection point between the ray and the road plane (z=0) under the assumption that the road is flat. By doing so, the initial distance coordinate This is decided.

[0097] FIG. 5 is a diagram illustrating the operation of a region of interest setting unit constituting a reliability-based distance correction device according to one embodiment of the present invention, and FIG. 6 is a flowchart illustrating the operation procedure of the region of interest setting unit illustrated in FIG. 3.

[0098] The area of ​​interest setting unit (230) sets an area of ​​interest to be corrected based on at least one of lane recognition results, object detection results, or HD map information using an image received through the input unit (210) and initial distance coordinates estimated by the distance coordinate estimation unit (220).

[0099] FIG. 5 illustrates an example of an image received through the input unit (210), and an example of a portion of the entire image being set as an ROI. The ROI is an area selected only for the actual distance correction required, and is an area set to include lanes, objects, and HD map-based structures by analyzing them. In this embodiment, a rectangular ROI setting is illustrated, but depending on the situation, the shape of the ROI can be changed to a polygon or a circle other than a rectangle. The area of ​​interest setting unit (230) can create an ROI in the direction of travel from the center of the lane based on the lane recognition results.

[0100] Referring to FIG. 6, the operation procedure of the interest region setting unit (230) is explained.

[0101] The region of interest setting unit (230) searches for candidate regions for ROI setting within the entire frame of the image received from the input unit (210) (S410). More specifically, the region of interest setting unit (230) analyzes lanes, objects, and HD map-based structures from the image. Since the image of objects is directly linked to safe driving, distance correction is very important, so it must be included in the ROI. Here, objects may include vehicles ahead, pedestrians, bicycles, and obstacles on the road.

[0102] The area of ​​interest setting unit (230) searches for candidate areas and then generates an ROI based on the lane (S420). The ROI at this time includes the actual lane area and is set to include the area in the direction of travel from the center of the lane based on the lane recognition result.

[0103] The area of ​​interest setting unit (230) expands the ROI based on objects (S430). Here, objects may include vehicles ahead, pedestrians, bicycles, and obstacles on the road. Since objects located in front of the vehicle are closely related to safe driving, the ROI is expanded so that they must be included in the ROI.

[0104] The area of ​​interest setting unit (230) expands the ROI based on objects and then corrects the ROI based on an HD map (S440). More specifically, the ROI is corrected by considering the road structure, areas outside the lane boundaries are removed, the shape of the ROI is adjusted according to the road structure such as curvature and merge sections, and attitude changes are reflected based on GPS and IMU.

[0105] Finally, the region of interest setting unit (230) integrates the lane-based ROI, the object-based ROI, and the HD map-based correction result to form the final ROI (S450). That is, the final ROI corresponds to a specific part of the image, as exemplified in FIG. 5.

[0106] Since correction is not performed on the entire image later due to the operation of the region of interest setting unit (230), the amount of computation can be reduced by 60 to 90%, and correction can be concentrated only on information of interest such as lanes, objects and road structures, and overcorrection and noise accumulation in areas outside the ROI can be prevented, thereby improving robustness.

[0108] FIGS. 7 and 8 are drawings for explaining the operation of a partition section constituting a reliability-based distance correction device according to an embodiment of the present invention.

[0109] Referring to FIG. 7, we examine a method of dividing multiple sections based on forward distance within the region of interest in the region of interest setting unit (230). In this embodiment, it is exemplified that a portion of the input image including the lane area where the vehicle is driving is set as the region of interest.

[0110] The section (240) is divided into three sections, a short-range area (A1), a medium-range area (A2), and a long-range area (A3), to apply different correction characteristics depending on the distance within the area of ​​interest.

[0111] The near area (A1) corresponds to the area closest to the camera (100) and is an area where pixel error is greatly amplified by distance error. Accordingly, the near area (A1) is an area requiring active correction, so the largest correction scale and maximum number of iterations are applied.

[0112] The mid-range region (A2) is an intermediate region between the near and far ranges, where pixel error is reduced to a certain level and stability is increased. Accordingly, balanced correction is performed in the mid-range region (A2).

[0113] The far-field area (A3) corresponds to the area furthest from the camera (100) and is an area where the correction effect is limited due to low geometric sensitivity. Accordingly, over-correction is prevented and stability is ensured for the far-field area (A3).

[0114] FIG. 8 illustrates a concept of dividing a distance space into near, mid, and far sections based on the forward distance (d) relative to the camera (100). As illustrated, a distance axis (d) is represented along the space extending forward from the camera (100), and each area is distinguished as the distance increases.

[0115] Near This corresponds to the area where the correction scale is set to the largest and iterative correction is performed most actively.

[0116] Mid-range is It corresponds to, and the correction scale is set to an intermediate level, ensuring that the distance error deceleration speed and stability are maintained in balance.

[0117] Far is It corresponds to, and uses a very small correction scale so that the correction amount is not excessively applied, and sets the number of iterations to the lowest possible level to prevent excessive computation.

[0118] The section (240) divides the distance space into near, mid, and far sections according to the forward distance, and the lengths of the near and mid sections, i.e., d1 and d2, can be dynamically adjusted according to the change in vehicle speed.

[0119] When the vehicle is traveling at high speed, object or lane information at a distance is more important than information at a short distance. Conversely, when the vehicle is traveling at low speed, short-distance correction becomes more important. Based on this principle, the section (240) can dynamically adjust and determine d1 and d2, which were previously determined as fixed values ​​according to changes in vehicle speed. At this time, the section (240) can receive the vehicle speed from the autonomous driving device (300), i.e., the ECU or CAN.

[0120] When adjusting the near and mid sections by reflecting the vehicle speed in the section (240), d1 and d2 can be determined by Equation 2.

[0121] [Mathematical Formula 2]

[0122]

[0123] In mathematical equation 2, d 1_base is the boundary value between short and medium range set by default when vehicle speed is excluded, d 2_base is the boundary value between medium and long distances set by default when vehicle speed is excluded, k1 is the correction factor of d1, k2 is the correction factor of d2, and v is the vehicle speed.

[0124] Referring to FIGS. 7 and 8, the reliability-based distance correction device (200) performs precise correction only on the area of ​​interest set according to a preset standard, rather than on the entire image captured by the camera (100), and applies different correction scales and repetition counts for each distance interval.

[0126] FIG. 9 is a detailed block diagram of a section-by-section correction unit constituting a reliability-based distance correction device according to one embodiment of the present invention.

[0127] Referring to FIG. 9, the section-by-section correction unit (250) constituting the reliability-based distance correction device (200) according to one embodiment of the present invention includes a reliability calculation unit (251) and a correction amount calculation unit (253).

[0128] The reliability calculation unit (251) calculates the reliability of each pixel. More specifically, the reliability calculation unit (251) calculates a reliability value (w(u,v)) based on pixel characteristics such as edge strength, brightness / contrast, and object detection reliability for pixels (u,v) of an input image.

[0129] The reliability calculation unit (251) is edge strength ( ), Brightness / Contrast( ) and object detection reliability( It takes ) as input and calculates the confidence level of each pixel.

[0130] Edge strength is intended to reflect the characteristic that precise distance estimation is possible as the boundary line becomes clearer, showing high values ​​in road boundaries and object contours, and low values ​​in flat areas with a lot of noise.

[0131] Brightness / contrast is intended to reflect the characteristic that the reliability of distance estimation decreases in areas that are too dark or too bright; therefore, reliability is higher in areas with appropriate illumination and contrast.

[0132] Object detection reliability is intended to reflect the feature that allows for the use of deep learning-based object detection techniques and enables more precise correction in areas with a high probability of object existence.

[0133] The reliability calculation unit (251) can calculate the reliability using a weighted sum model according to Equation 3.

[0134] [Mathematical Formula 3]

[0135]

[0136] In mathematical equation 3, w(u,v) is the confidence level of pixel(u,v), is the normalized edge strength, is normalized brightness / contrast information, is normalized object detection confidence, is the respective weight for edge intensity, brightness / contrast information, and object detection confidence. Also, am.

[0137] The correction amount calculation unit (253) calculates the correction amount according to the correction scale and the number of repetitions set for each section. More specifically, the correction amount calculation unit (253) calculates the correction amount and weighted correction amount applied at each repetition step using the distance section information classified by the sectioning unit (240) and the reliability calculated by the reliability calculation unit (251).

[0138] The correction amount calculation unit (253) has a correction scale value for each section, that is, a correction scale for a short distance section ( ), correction scale for mid-distance sections( ) and correction scale for distant sections( ), confidence value(w(u,v)), current distance error( Receives ) as input.

[0139] The current distance error can be determined by mathematical formula 4.

[0140] [Mathematical Formula 4]

[0141]

[0142] In mathematical formula 4, is the reference distance coordinate or correction target value.

[0143] The correction amount calculation unit (253) can determine the correction amount candidate by mathematical formula 5.

[0144] [Mathematical Formula 5]

[0145]

[0146] In mathematical formula 5, is the correction amount candidate vector calculated in the k-th iteration, is a correction scale corresponding to the distance interval, is the distance error vector at the k-th iteration.

[0147] The correction amount calculation unit (253) determines the final correction amount by applying reliability weighting. This is shown in Equation 6.

[0148] [Mathematical Formula 6]

[0149]

[0150] In mathematical formula 6, is the above correction amount, and w(u,v) is the confidence level of pixel(u,v).

[0151] The reliability calculated by the reliability calculation unit (251) and the correction amount calculated by the correction amount calculation unit (253) are transmitted to the final distance determination unit (260). Accordingly, the final distance determination unit (260) determines the final distance coordinates by reflecting the reliability and correction amount. The final distance determination unit (260) can determine the final distance coordinates by mathematical formula 7.

[0152] [Mathematical Formula 7]

[0153]

[0154] In mathematical formula 7, is the distance estimate updated after the (k+1)th iteration, is the distance estimate for the k-th iteration.

[0156] FIG. 10 is a graph showing an error damping curve in a reliability-based distance correction device according to one embodiment of the present invention.

[0157] In this embodiment, the reduction in distance error for each distance segment as iterative correction proceeds is compared and shown. In the graph, the X-axis represents the number of iterations k, and the Y-axis represents the distance error e. k am.

[0158] It can be seen that the error decreases most rapidly in the short-range section. It decreased to 0.4 in Iteration 1, to 0.2 in Iteration 2, and approached 0 in Iteration 5.

[0159] It can be seen that in the mid-range, it decreases steadily, albeit more slowly than in the short-range. It is approximately 0.55 in Iteration 1, approximately 0.17 in Iteration 3, and approximately 0.08 in Iteration 5. This indicates that, as a result of setting the number of iterations in the middle range, a pattern of stable decrease without overshooting appears.

[0160] It can be seen that the rate of decrease is lowest in the long distance section. It is about 0.75 in Iteration 1, about 0.41 in Iteration 3, and about 0.22 in Iteration 5. This is because the geometric sensitivity of the camera (100) is lower at longer distances, so the correction effect is smaller, and this result occurs because a small correction scale was applied to the long distance section, thereby preventing the amplification of noise.

[0161] This graph demonstrates that applying different corrections based on distance intervals yields significant results. It exhibits rapid error convergence in the near-distance range, stable reduction in the mid-distance range, and noise suppression through a gentle slope in the far-distance range. Furthermore, by performing correction based on reliability, a larger correction amount is applied as pixel reliability increases, forming a fast and stable convergence path; it was also confirmed that the correction algorithm operates stably without diverging.

[0163] FIG. 11 is a flowchart illustrating a reliability-based distance correction method according to one embodiment of the present invention.

[0164] The distance correction device (200) receives an image from the camera (100) (S510). The image input from the camera (100) is used as a base input value for distance estimation and correction.

[0165] The distance coordinate estimation unit (220) estimates the initial distance coordinates using image data input through the input unit (210) (S520). More specifically, the pixel (u, v) is back-projected to the initial distance coordinates X0 = (x0, y0) using camera parameters (K, R, T). At this stage, there is a possibility of actual error occurring.

[0166] The region of interest setting unit (230) sets a region of interest (ROI) in the image (S530). The region of interest setting unit (230) selects the ROI based on lanes, objects, and HD maps, and for areas outside the ROI, it either omits correction or performs light correction. This process contributes to reducing computational load and improving accuracy.

[0167] The section (240) divides the forward distance range within the area of ​​interest set by the area of ​​interest setting section (230) into multiple sections (S540). Here, the multiple sections may include a short distance section, a medium distance section, and a long distance section.

[0168] The section-by-section correction unit (250) performs correction on the initial distance coordinates by reflecting the characteristics of each section for a plurality of sectioned sections (S550). In step S550, the operation of applying the current distance error calculation section-by-section correction scale, applying reliability, calculating the correction amount, and updating the distance coordinates is repeated for Iteration k of Graph 10, and the correction is terminated when the error falls below a threshold value after the repeated correction.

[0169] Finally, the final distance determination unit (260) determines the final distance coordinates by reflecting the operation results of the section correction unit (250) (S560). The point at which the final distance coordinates are determined is the point at which the error becomes very low after Iteration 5.

[0170] As described above, the reliability-based distance correction device (200) has resolved the structural limitations of the geometric projection model and the problem of pixel-unit error amplification in the camera image-based distance estimation process. More specifically, by dividing the forward distance range into multiple sections of short distance, medium distance, and long distance, and applying different correction scales and repetition counts for each section, it suppresses the problems of error accumulation and overcorrection that occurred in the existing method due to the failure to reflect differences in distance characteristics. As a result, it is possible to provide balanced correction performance across the entire distance range.

[0171] Furthermore, computational efficiency has been significantly improved by applying correction based on a region of interest. While conventional methods performed correction on the entire image, this invention establishes a region of interest based on lanes, objects, and HD maps, and performs intensive correction only on pixels within this region. As a result, reduced computational load, improved real-time performance, prevention of noise accumulation in areas requiring no correction, and increased system resource efficiency can be expected.

[0172] In addition, reliability-weighted correction was applied to suppress uncertainty and enhance robustness. As a result, the amount of correction is reduced in areas of high uncertainty, such as blur, backlighting, and image noise, while precise correction is performed in areas of high reliability, such as overall contours, thereby enabling noise suppression and stable correction results overall.

[0174] The reliability-based distance correction method according to the present invention can be implemented as a program to realize the same. In addition, such a program may include programmable machine instructions processed by a processor and may be implemented in a high-level programming language, an object-oriented programming language, assembly language, machine language, etc.

[0175] In addition, a program for implementing a reliability-based distance correction method may be stored on a computer-readable recording medium. Here, the computer-readable recording medium may be any device that includes instructions, in whole or in part, that are executable by a computer, and is in a form that is readable by a computer and capable of storing data. For example, the computer-readable recording medium may include CD-ROM, RAM, ROM, flash memory, magnetic disk, optical disk, hard disk drive, and solid-state disk.

[0177] Those skilled in the art to which the present invention pertains will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0179] 200: Reliability-based distance correction device 210: Input section 220: Distance coordinate estimation section 230: Region of Interest Setting Section 240: Compartment 250: Sectional correction unit 260: Final distance determining unit 270: Storage section 280: Control unit

Claims

Claim 1 A reliability-based distance correction device comprising: an input unit for receiving an image from a camera; a distance coordinate estimation unit for estimating initial distance coordinates from the received image; a region of interest setting unit for setting a region of interest in the image; a sectioning unit for dividing a forward distance range within the set region of interest into multiple sections; a section-by-section correction unit for performing correction on the initial distance coordinates by reflecting the characteristics of each section for the multiple sections; and a final distance determination unit for determining the final distance coordinates by reflecting the operation results of the section-by-section correction unit; wherein the section-by-section correction unit comprises: a reliability calculation unit for calculating the reliability of each pixel; and a correction amount calculation unit for calculating the correction amount according to a correction scale and the number of repetitions for each section; and wherein the reliability calculation unit calculates the reliability using the following formula. Here, w(u,v) is the confidence level of pixel(u,v), is the normalized edge strength, is normalized brightness / contrast information, is normalized object detection confidence, is the respective weight of the edge intensity, brightness / contrast information, and object detection reliability. Claim 2 A reliability-based distance correction device according to claim 1, wherein the interest area setting unit sets the interest area using at least one of a lane recognition result, an object detection result, and HD map information. Claim 3 A reliability-based distance correction device according to claim 1, wherein the above-mentioned section is divided into a short-range section, a medium-range section, and a long-range section based on the center of the camera. Claim 4 delete Claim 5 A reliability-based distance correction device according to claim 1, wherein the final distance determining unit determines the final distance coordinates by reflecting the calculated reliability and the calculated correction amount. Claim 6 delete Claim 7 A reliability-based distance correction device according to claim 1, wherein the correction amount calculation unit determines a correction amount candidate using the following formula: Here, is the correction amount candidate vector calculated in the k-th iteration, is a correction scale corresponding to the distance interval, is the distance error vector at the k-th iteration. Claim 8 A reliability-based distance correction device according to claim 7, wherein the correction amount calculation unit determines the correction amount using the following formula: Here, is the above correction amount, and w(u,v) is the confidence level of pixel(u,v). Claim 9 In claim 7, the reliability-based distance correction device is characterized in that the final distance determining unit determines the final distance coordinates using the following formula: Here, is the distance estimate updated after the (k+1)th iteration, is the distance estimate for the k-th iteration. Claim 10 A reliability-based distance correction method applied to a device that corrects distance by communicating with a camera, comprising: receiving an image from the camera; estimating initial distance coordinates from the received image; setting a region of interest in the image; dividing a forward distance range within the set region of interest into a plurality of segments; performing correction on the initial distance coordinates by reflecting the characteristics of each segment for the plurality of segments; and determining final distance coordinates by reflecting the result of the correction; wherein the step of performing the correction includes: calculating the reliability of each pixel; and calculating the correction amount according to a pre-set correction scale and the number of iterations for each segment; wherein the step of calculating the reliability is characterized by calculating the reliability using the following formula. Here, w(u,v) is the confidence level of pixel(u,v), is the normalized edge strength, is normalized brightness / contrast information, is normalized object detection confidence, is the respective weight of the edge intensity, brightness / contrast information, and object detection reliability. Claim 11 A reliability-based distance correction method according to claim 10, wherein the setting step is characterized by setting the area of ​​interest using at least one of lane recognition results, object detection results, and HD map information. Claim 12 A reliability-based distance correction method according to claim 10, wherein the above-mentioned segmenting step is characterized by dividing the area into a short-range segment, a medium-range segment, and a long-range segment based on the center of the camera. Claim 13 delete Claim 14 A reliability-based distance correction method according to claim 10, wherein the determining step is characterized by determining the final distance coordinates by reflecting the calculated reliability and the calculated correction amount. Claim 15 delete Claim 16 In claim 10, the step of calculating the correction amount is characterized by determining a candidate correction amount using the following formula, thereby providing a reliability-based distance correction method: Here, is the correction amount candidate vector calculated in the k-th iteration, is a correction scale corresponding to the distance interval, is the distance error vector at the k-th iteration. Claim 17 In claim 16, the step of calculating the correction amount is characterized by determining the correction amount using the following formula in a reliability-based distance correction method: Here, is the above correction amount, and w(u,v) is the confidence level of pixel(u,v). Claim 18 In claim 16, the determination step is characterized by determining the final distance coordinates using the following formula in a reliability-based distance correction method: Here, is the distance estimate updated after the (k+1)th iteration, is the distance estimate for the k-th iteration. Claim 19 A computer-readable recording medium having a program for implementing a reliability-based distance correction method according to any one of paragraphs 10 through 12, 14, and 16 through 18.

Images