SYSTEM AND METHOD FOR CONTROLLING AUTONOMOUS VEHICLE PARKING

Abstract

A system for performing autonomous parking of a vehicle (1), comprising the system: a camera (100) which is configured to capture an image of the surroundings of the vehicle (1) with a parking line, and a control unit (200) which is set up for this purpose: deriving spatial recognition data based on the vehicle environment image as an input, to derive a feature point that corresponds to the parking line, based on the surrounding image and the spatial recognition data, to determine a candidate parking line based on a bundling of the feature point, and to steer the vehicle (1) in such a way that parking is carried out in a parking space which has the candidate parking line, wherein the control device (200) is configured to identify several candidate parking lines (L61, L62), to identify a first area (Z61) which is defined by the endpoint of the candidate parking line as a boundary, to identify a second area (Z62) which lies between the candidate parking lines and at which the candidate parking line is provided, and to determine the parking area based on the ratio of the number of pixels in the first area (Z61) and the number of pixels in the second area (Z62).

Description

Technical field

[0001] The present invention or disclosure relates to a system and a control method thereof for carrying out autonomous parking. background

[0002] Autonomous driving technology is a technology for automatically driving a vehicle by detecting road conditions without a driver manually controlling a brake, steering wheel, or accelerator pedal.

[0003] Autonomous driving technology is a core technology for the realization of intelligent vehicles (so-called...)."Smart Cars"), which incorporate technologies such as Highway Driving Assistance (HDA - a technology that automatically maintains the distance between cars), Rear Vehicle Detection (BSD - a technology that detects vehicles near the vehicle while reversing and issues a warning), Automatic Emergency Braking (AEB - a technology that activates the braking system if the vehicle in front is not detected), Lane Departure Warning System (LDWS), Lane Keeping Assist System (LKAS - a technology that compensates for leaving the lane without signaling), Advanced Intelligent Speed ​​Control (ASCC - a technology that maintains the distance between cars at a set speed and drives at a constant speed), Traffic Jam Assist (TJA), Park Collision Avoidance Assist (PCA), and an autonomous parking system (so-called..."Remote Smart Parking Assist" - a remote-controlled, intelligent parking assistance system.

[0004] In the technology for detecting surrounding objects and parking spaces for autonomous parking control of vehicles, parking has so far been carried out using ultrasound signals.

[0005] In recent years, active research has been conducted on an autonomous parking system that performs the parking by additionally using a camera.

[0006] Furthermore, a system for performing autonomous parking of a vehicle is known from the publication "Uniform user interface for semiautomatic parking slot marking recognition" by Ho Gi Jung, Yun Hee Lee and Jaihie, Kim, published in IEEE Transactions on Vehicular Technology, Volume 59, 2010, Issue 2, pp. 616-626. The system comprises: a camera configured to capture an image of the vehicle's surroundings with a parking line, and a control unit configured to: derive spatial recognition data based on the vehicle's surroundings image as input, derive a feature point corresponding to the parking line based on the surroundings image and the spatial recognition data, determine a candidate parking line based on a clustering of the feature point, and control the vehicle so that parking is performed in a parking space that has the candidate parking line.Furthermore, DE 10 2015 117 535 A1 reveals a driver assistance device with parking assistance. Explanation of the invention

[0007] It is an object of the present invention or disclosure (hereinafter also referred to as: disclosure) to provide a system and a control method which are capable of performing an accurate autonomous parking operation by learning an image obtained from a camera and using the learned data.

[0008] The present invention provides a system for performing autonomous parking of a vehicle according to claim 1 and a method for controlling automatic parking of a vehicle according to claim 9. Advantageous embodiments are described in the dependent claims.

[0009] According to one aspect of the present disclosure, a system for performing autonomous parking of a vehicle (e.g., a passenger vehicle), which is equipped to perform autonomous parking (e.g., parking and / or exiting a parking space), may comprise: a camera configured to capture an image of the vehicle's surroundings, which has a parking line; a control unit configured to derive spatial recognition data based on the vehicle's surroundings as an input; a feature point corresponding to the parking line (e.g., corresponding to and / or belonging to the parking line) based on the surroundings and the spatial recognition data; and a candidate parking line (e.g., a candidate for a parking line or parking line candidates) based on bundles or bales.The control unit is designed to determine the grouping (hereinafter referred to as clustering) of the feature point and to control the vehicle so that parking is carried out in a parking area that includes the candidate parking line. According to the invention, the control unit is configured to determine several candidate parking lines, to determine a first area defined by the endpoint of the candidate parking line(s) as a boundary, to determine a second area located between the candidate parking lines and for which the candidate parking line is provided, and to determine the parking area based on the ratio of the number of pixels in the first area to the number of pixels in the second area.

[0010] The control unit can be configured to remove noise from the vehicle's environment image using a predetermined first filter and to extract an edge based on a gradient of each pixel contained in the vehicle's environment image.

[0011] The control unit can be configured to classify objects contained in the vehicle's surroundings into at least one category based on spatial recognition data.

[0012] The control unit can be configured to derive a plurality of feature points corresponding to the parking line from the vehicle's surroundings and the spatial recognition data using a predetermined second filter that corresponds to the area of ​​the parking line (e.g., depending on the area of ​​the parking line).

[0013] The control device can be configured to determine the reliability of each of the multiple feature points based on the consistency of the direction values ​​of each of the multiple feature points corresponding to the parking line, and to determine the candidate parking line based on the (e.g., the at least one) feature point whose reliability exceeds a predetermined value.

[0014] The control unit can be configured to determine the parking area based on the feature point that corresponds to the parking line, if the feature point corresponds to the parking line contained in the vehicle's environment image and the spatial recognition data.

[0015] The control device can be configured to determine the parking area based on the feature point and a ratio between the number of pixels that correspond to the parking line (e.g., correspond to and / or belong to the parking line) and the number of pixels of the feature point.

[0016] The control unit can be configured to determine the candidate parking line based on a first feature point and a second feature point if the overlap ratio of the first feature point, which is determined based on the vehicle's surroundings image, and the second feature point, which is determined based on the spatial recognition data, exceeds a predetermined value.

[0017] A control method according to the invention for a vehicle that performs autonomous parking (e.g., parking and / or exiting a parking space) can comprise: obtaining an image of the vehicle's surroundings, which includes a parking line; deriving spatial recognition data based on the vehicle's surroundings image as input; deriving a feature point that corresponds to the parking line (e.g., corresponds to and / or belongs to the parking line) based on the image of the surroundings and the spatial recognition data; determining a candidate parking line based on bundles or groups of feature points; and controlling the vehicle so that parking is performed in a parking area that includes the candidate parking line.According to the invention, controlling the vehicle to perform parking in the parking area comprises: determining several candidate parking lines, determining a first area which is provided with the endpoint of the candidate parking line(s) as a boundary, determining a second area which lies between the candidate parking lines and for which the candidate parking line is provided, and determining the parking area based on the ratio of the number of pixels in the first area and the number of pixels in the second area.

[0018] Determining the candidate parking line may involve: removing noise from the vehicle's surrounding image using a predetermined first filter, and extracting an edge based on a gradient of each pixel contained in the vehicle's surrounding image.

[0019] Determining the candidate parking line can involve: classifying objects contained in the vehicle's surroundings into at least one category based on spatial recognition data.

[0020] Determining the candidate parking line can involve: deriving a plurality of feature points corresponding to the parking line from the vehicle's surroundings and spatial recognition data using a predetermined second filter that corresponds to the area of ​​the parking line (e.g., depending on the area of ​​the parking line).

[0021] Determining the candidate parking line can involve: determining the reliability of each of the multiple feature points based on the consistency of the direction values ​​of each of the multiple feature points corresponding to the parking line, and determining the candidate parking line based on the (e.g., at least one) feature point whose reliability exceeds a predefined value.

[0022] Determining the candidate parking line can involve: Determining the parking area based on the feature point that corresponds to the parking line, if the feature point corresponds to the parking line contained in the vehicle's environment image and the spatial recognition data.

[0023] Steering the vehicle to park in the parking space may involve: determining the parking space based on the feature point and a ratio between the number of pixels that correspond to the parking line (e.g., correspond to and / or belong to the parking line) and the number of pixels of the feature point.

[0024] The candidate parking line determination process can include: Determining the candidate parking line based on a first feature point and a second feature point, if the overlap ratio of the first feature point, which is determined based on the vehicle's surroundings image, and the second feature point, which is determined based on spatial recognition data, exceeds a predetermined value. Brief description of the drawings Fig. Figure 1 is a control block diagram according to an exemplary embodiment of the present disclosure. Fig. Figure 2 is a diagram describing a process for deriving feature points that correspond to parking lines, according to an exemplary embodiment of the present disclosure. Fig. Figure 3 is a diagram describing a process for removing erroneously recorded feature points according to an exemplary embodiment of the present disclosure. Fig. Figure 4 is a diagram describing a process for determining a candidate parking line based on a ratio of the number of pixels corresponding to the parking line and the number of pixels of the feature point according to an exemplary embodiment of the present disclosure. Fig. Figure 5 is a diagram to explain an overlap process of a feature point determined based on an environment image and a feature point determined based on spatial recognition data according to an exemplary embodiment of the present disclosure. Fig. 6A and Fig. Figure 6B are views explaining a process for determining a parking space and for performing autonomous parking in the corresponding parking space according to an exemplary embodiment of the present disclosure. Fig. Figure 7 is a flowchart describing a method for performing autonomous parking according to an exemplary embodiment of the present disclosure. Detailed description

[0025] In the following description, identical reference numerals throughout refer to the same elements. This description does not describe all elements of the embodiments, and in the field of disclosure to which an exemplary embodiment of the present disclosure relates, there is no overlap between the general content or the embodiments. Terms such as "unit," "module," "element," and "block" may be implemented as hardware or software. According to embodiments, a plurality of "units," "modules," "elements," and "blocks" may be implemented as a single component, or a single "unit," "module," "element," and "block" may comprise multiple components.

[0026] It is understood that when an element is described as being “connected” to another element, it may be connected to the other element directly or indirectly, with the indirect connection including a “connection via a wireless communication network”.

[0027] If a part “contains” or “includes” an element, the part may further contain other elements without excluding the other elements, unless there is a specific description to the contrary.

[0028] Furthermore, if a part “has” a certain component, this also means that other components may be included, rather than excluding other components, unless explicitly stated otherwise.

[0029] When the entire description refers to an element being positioned "on" another element, this includes not only the case where the element is in contact with the other element, but also the case where another element is located between the two elements.

[0030] Terms such as first and second are used to distinguish one component from other components, and the component is not limited by the terms described above.

[0031] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0032] Each step uses an identification number to simplify explanation, and the identification number does not describe the order of each step, and each step can be performed differently than in the specified order unless a particular sequence is clearly indicated in the context.

[0033] The following describes a functional principle and embodiments of the present disclosure with reference to the accompanying drawings.

[0034] Fig. Figure 1 is a control block diagram according to an exemplary embodiment of the present disclosure.

[0035] With reference to Fig. 1. A vehicle 1 according to an exemplary embodiment may have a camera 100, a control unit 200 and a driving device 300.

[0036] Vehicle 1 can be designed to perform autonomous parking.

[0037] The Camera 100 can have multiple channels and capture an image around the vehicle (e.g., an image of the vehicle's surroundings).

[0038] The camera 100 according to one embodiment can be provided around a vehicle (e.g. in the form of individual cameras at individual points along the circumference of the body and around the circumference) to operate a surround view monitor (SVM).

[0039] The camera 100 can have a CCD camera (CCD = English "charge-coupled device") or a CMOS color image sensor, which is installed in the vehicle 1.

[0040] Here, both CCD and CMOS refer to sensors that convert light entering through the camera lens into electrical signals and store them.

[0041] The camera 100 can capture an image around the vehicle (e.g., an image of the vehicle's surroundings) which shows the parking line.

[0042] The control unit 200 can derive spatial recognition data by learning the surrounding images as input values.

[0043] Spatial recognition data can refer to data derived by performing deep or multi-layered (machine) learning using an environment image as input data.

[0044] In particular, the spatial recognition data can refer to a deep learning-based spatial recognition result for each of the 4-channel camera images or a spatial recognition result in the form of an environment view generated from it.

[0045] The spatial recognition function can refer to a deep learning-based algorithm that classifies various objects, including surfaces (roads), parking lines, vehicles, pillars and obstacles observed in an image, into image pixels, or to the process of classifying objects contained in the surrounding image based on light reflection or shadows.

[0046] The driving device 300 can be provided as a device capable of driving a vehicle.

[0047] According to one embodiment, the driving device 300 can have a motor (e.g. an internal combustion engine and / or one or more electric traction motors) and can include various components for driving the motor.

[0048] In particular, the driving device 300 can include a braking device and a steering device. If it is a structure that implements the driving of a vehicle, there is no restriction on the device structure.

[0049] The control unit 200 can perform an operation to detect an object, such as an empty space, a parking line, a column or a vehicle, based on the space detection data.

[0050] The control unit 200 can derive a feature point that corresponds to the parking line (e.g., corresponds to and / or belongs to the parking line) based on the environment image and the room recognition data.

[0051] The feature point can refer to a pixel that forms the parking line.

[0052] The control unit 200 can bundle or cluster feature points (hereinafter referred to as "bundling" – also called "clustering" – derived from the English "clustering" or "to cluster"). When bundling, information about the gradient and direction of each feature point can be used. A detailed description of this will follow.

[0053] Furthermore, the control unit 200 can determine a candidate parking line (e.g., also candidate parking line - e.g., a candidate for a parking line or parking line candidates) based on clustering (bundling, grouping, clustering of feature points).

[0054] The candidate parking line can refer to a parking line that provides a basis for performing autonomous parking by a control device using a feature point, not an image contained in the (camera) image.

[0055] The control device 200 can control the vehicle to perform parking maneuvers in a parking area formed by candidate parking lines. In the present disclosure, the control device 200 can be a computer, a processor (CPU), or an electronic control unit (ECU) that is programmable to control various electronic systems in the vehicle.

[0056] The control unit 200 can remove noise from the image around the vehicle (e.g. the surrounding image) by using a predetermined first filter.

[0057] According to one embodiment, the first filter can be a Gaussian filter. Meanwhile, the control unit can extract an edge based on a gradient of each pixel contained in an image around the vehicle (e.g., apply edge detection or extraction to the surrounding image).

[0058] The gradient can refer to a gradient relationship between the pixels contained in the image.

[0059] If the gradient size exceeds a certain value, the control unit 200 can identify the corresponding section as an edge.

[0060] The control unit 200 can categorize or classify objects contained in the image around the vehicle into at least one category based on spatial recognition data.

[0061] In particular, taking into account the processing efficiency within the algorithm, the control unit 200 can simplify the classification of objects contained in the image around the vehicle into eight or more classes (e.g. three classes, such as space (parking space), parking line, etc.).

[0062] The control unit 200 can additionally perform image interpolation when it generates an environment view using a spatial recognition result from a 4-channel camera.

[0063] The control unit 200 can derive a plurality of feature points corresponding to the parking line from the image around the vehicle and the spatial recognition data using a second predetermined filter that corresponds to the area of ​​the parking line (e.g. depending on the area of ​​the parking line).

[0064] The second filter according to one embodiment can refer to a top-hat filter. The top-hat filter can refer to a real-space or Fourier-space filter technique.

[0065] The top-hat filter outputs a large value on a line with a width proportional to the filter size and shows a low output in other areas.

[0066] Accordingly, the control unit 200 can perform a top-hat filtering with a size similar to the width of the parking line to identify an area with a line fraction similar to the width of the parking line.

[0067] The control unit 200 can determine a maximum value at the output of the top-hat filter and identify the maximum value as a feature point of the parking line.

[0068] To determine the direction (angle) of the line from the feature points of each line, the control unit 200 can determine an orientation based on the gradient of each feature point.

[0069] The control unit 200 can extract line fraction feature points, which are similar to the width of the parking line, by top-hat filtering of the spatial recognition result data.

[0070] The control unit 200 determines the reliability of each of the multiple feature points based on the consistency of direction values ​​from each of the plurality of feature points corresponding to the parking line, and determines the candidate parking line based on the feature point (e.g. the one or more feature points) whose reliability exceeds a predefined value.

[0071] In particular, the control unit 200 can perform top-hat filtering in a predefined direction from the original camera image or the environment view image, extract line feature points (feature points of a line) from an area with a line fraction that has a certain width, and measure the reliability of each feature point.

[0072] The reliability value of the control device 200 can be determined based on the coherence between the output of the top-hat filter at the location of the corresponding feature point and the direction of the feature point.

[0073] The direction consistency value can indicate how consistent the direction dependence (e.g., orientation) of the gradient is within a region with a certain radius around the position of the feature point.

[0074] The control unit 200 can determine that an extracted feature point on a line has a high directional consistency value and a feature point mistakenly extracted from a position other than a line has a low consistency value.

[0075] The control unit 200 can extract line fraction feature points by similar top-hat filtering on the spatial detection result or the result of the environment view type based on the spatial detection result.

[0076] The control unit 200 can determine the reliability of each feature point based on this process. In this case, the reliability of each feature point can be obtained by calculating an average reliability within a range of a specific radius around the location of the corresponding feature point.

[0077] The control unit 200 can determine the reliability for line feature points that are determined by the process described above.

[0078] The control unit 200 can determine a candidate parking line by identifying that only the feature points with a certain threshold or higher have been extracted from the line using the feature points whose reliability exceeds a predetermined value, and can identify the remaining feature points as noise and remove them.

[0079] The sum of the reliability can be determined by the following equation. Total = w1 ⋅ Sorg + w2 ⋅ Ssd

[0080] Referring to equation 1, S denotes total the reliability, and w1 and w2 denote a weight, which each belong to the environmental image and the spatial recognition data.

[0081] Furthermore, S org reliability determined on the basis of an image and can S sd This means reliability determined on the basis of spatial recognition data.

[0082] If the feature point corresponds to the parking line (e.g., it corresponds to and / or belongs to it) which is contained in the vehicle environment image and the space recognition data, then the control unit 200 can determine the parking area based on the feature point belonging to the parking line.

[0083] The parking area can refer to an area or space in which the vehicle is parked using autonomous parking.

[0084] The control unit 200 can determine a parking area based on the feature point and a ratio between the number of pixels corresponding to the parking line and the number of pixels of the feature point.

[0085] The operation of the control unit 200 for determining the ratio between the number of pixels corresponding to the parking line and the number of pixels of the feature point is described in detail below.

[0086] If the overlap ratio of the first feature point, which was determined based on the vehicle environment image, and the second feature point, which was determined based on spatial recognition data, exceeds a predetermined value, the control unit 200 can determine the candidate parking line based on the first feature point and the second feature point.

[0087] The control unit 200 can determine a plurality of candidate parking lines based on the process described above. The control unit 200 can determine an initial area that serves as a boundary around the endpoint of the candidate parking line.

[0088] This means that the first area can represent a boundary point of a candidate parking line derived from the control device.

[0089] The control unit 200 can determine a second area located between the candidate parking lines, where (e.g., adjacent to) the candidate parking lines are provided, and can calculate the parking area based on the ratio of the number of pixels belonging to the first area to the number of pixels in the second area. A more detailed description will follow.

[0090] The control unit 200 can be implemented by a memory (not shown) which stores data about an algorithm for controlling the operation of components in the vehicle or a program that reproduces the algorithm, and a processor (not shown) which carries out the above-described operating sequence using data stored in the memory.

[0091] In this case, the memory and the processor can each be implemented as separate chips. Alternatively, the memory and the processor can be implemented as a single chip.

[0092] At least one component can be selected according to the performance of the components of the system. Fig. The vehicle shown in point 1 may be added to or omitted. Furthermore, the average professional will easily understand that the relative positions of the components may be changed according to the performance or design of the system.

[0093] Each in Fig. 1 Component shown refers to software and / or hardware components, such as a field-programmable gate array (FPGA) and an application-specific integrated circuit (ASIC).

[0094] Fig. Figure 2 is a diagram describing a process for deriving feature points that correspond to parking lines, according to an exemplary embodiment.

[0095] With reference to Fig. 2 shows Fig. 2 a process for determining a feature point P2 based on an image captured by a vehicle.

[0096] Before deriving feature point P2, the control unit can perform filtering to remove noise from the input camera original image or the environment view data, and execute a function (task) to extract edge data.

[0097] The control unit can perform a class simplification function for more efficient algorithm processing of spatial recognition data classified as objects.

[0098] With reference to Fig. 2. The control unit can perform 1D top-hat filtering (e.g., one-dimensional top-hat filtering) in a predefined direction (e.g., vertical, horizontal, etc.) from the original camera image or the environment view image.

[0099] In particular, the top-hat filter can output a large value in a line with a width proportional to an associated filter size, and can output a low value in other areas.

[0100] The control unit can perform a top-hat filter with a size similar to the width of the parking line L2 to find an area with a line fraction similar to the width of the parking line.

[0101] Furthermore, the control unit can determine the maximum value at the output of the top-hat filter as feature point P2 of the line. To determine the direction (angle) of the line at each line feature point P2, a gradient-based direction can be calculated.

[0102] The control unit can perform similar operations on spatial recognition data.

[0103] This means that the control unit can derive a plurality of feature points corresponding to the parking line from the vehicle environment image and the spatial recognition data using a top-hat filter that corresponds to the area of ​​the parking line L2.

[0104] Fig. Figure 3 is a diagram describing a process for removing erroneously recorded feature points according to an exemplary embodiment.

[0105] With reference to Fig. 3. The control unit can project the feature points P31 and P32 onto the domain of the spatial recognition result data.

[0106] The control unit can assign the feature points P31 and P32 to the parking line P3, which is contained in the vehicle environment image and the spatial recognition data.

[0107] Among these feature points, feature points P31 located on objects other than the parking line (vehicles, pillars, obstacles, etc.) are considered feature points that are not necessary to determine a candidate parking line, and the control unit may consider the corresponding feature point P31 as a falsely detected feature point and remove it.

[0108] The control unit can remove the feature point P31, which does not correspond to the parking line, from among the feature points using the procedure described above and can determine the candidate parking line P33 based on the feature point P32 that corresponds to the parking line.

[0109] Fig. Figure 4 is a diagram describing a process for determining a candidate parking line R4 based on a ratio of the number of pixels corresponding to the parking lines L41 and R4 and the number of pixels of the feature point according to an exemplary embodiment.

[0110] The control unit can determine a ratio between the number of pixels corresponding to the parking line and the number of pixels of the feature point.

[0111] Based on these ratios and feature points, the control unit can determine a parking area formed from candidate parking lines.

[0112] The control unit can project the line feature points P41 and P42, which were secured from the environment image and the spatial recognition data, onto the same area D4, i.e. the parking lines L41 and L42, so that they correspond with each other.

[0113] The control device can perform a bundling or clustering or grouping (also called clustering) between feature points that are adjacent to each other and have a similar direction with respect to the line feature points, and can generate a candidate parking line R4.

[0114] The generation of the candidate parking line R4 by the control unit can involve determining information such as width, length, direction and positions of both endpoints of each candidate parking line.

[0115] To check whether the extracted candidate parking line R4 is being falsely detected, the control unit determines the area where each candidate parking line is located, using information about both ends and widths of the candidate parking line(s), and calculates the ratio of pixels based on the spatial detection result within the corresponding candidate parking line area.

[0116] In particular, the control unit can derive the feature point P41 from the parking line L41 contained in the surrounding image.

[0117] Furthermore, the control unit can derive the feature point P42 from the parking line L42, which is provided in the spatial recognition data.

[0118] The control unit can assign these feature points P41 and P42 to the parking lines L41 and L42. This means that the control unit can project each feature point P41 and P42 onto area D4.

[0119] The control unit can determine the ratio of a pixel of a feature point of each projected parking line to a pixel of a corresponding parking line area (e.g., the area of ​​an associated parking line). This ratio can be determined based on Equation 2 below. Sp=FPLP

[0120] Referring to equation 2, F Pdenotes the number of pixels of feature points that correspond to the parking line and can be L P To denote the number of pixels in the parking line area (e.g., the area of ​​an associated parking line). S P can mean a ratio between the number of pixels of the feature point corresponding to the parking line and the number of pixels in the parking line area.

[0121] If the calculated ratio SP is greater than or equal to a predetermined value, the control unit can determine that the corresponding feature point is a parking line (e.g., belongs to a parking line).

[0122] If, however, the ratio is less than or equal to a predetermined value, the control unit can identify the corresponding feature point as a (e.g., as part of a) parking line for which an incorrect measurement has been made and remove the corresponding feature point.

[0123] This means that the control unit can derive a feature point determined on the basis of the environment image and a feature point determined on the basis of the spatial recognition data.

[0124] Furthermore, the derived feature points P41 and P42 can be projected onto the parking line, i.e. the area (domain).

[0125] If a ratio of the number of pixels forming the parking line and the number of pixels of each feature point P41, P42 is determined, and if the ratio exceeds a predetermined value, the control device can determine that the corresponding feature point forms the parking line and can identify the corresponding feature points P41 and P42 as the candidate parking line R4.

[0126] Fig. 4 is only one process for determining a candidate parking line according to one embodiment of the present disclosure, and there is no limitation of the process for determining the candidate parking line.

[0127] Fig. Figure 5 is a diagram to explain an overlap process of a feature point P51 determined based on an environment image and a feature point P52 determined based on spatial recognition data according to an exemplary embodiment.

[0128] If with reference to Fig. 5. If the overlap ratio of the first feature point P51, which was determined based on the image around the vehicle, and the second feature point P52, which was determined based on the spatial recognition data, exceeds a predetermined value, the control unit can determine the candidate parking line based on the first feature point P51 and the second feature point P52.

[0129] For the first feature points extracted from the surrounding area, the control unit can determine the boundary of the parking line in the neighborhood of each feature point.

[0130] The control device can determine the point where the size of the edge component is greatest near each feature point as the parking line boundary point.

[0131] The control unit can determine (e.g., estimate, calculate) the boundary box (B51, B52) using the location information of each feature point and the parking line boundary point.

[0132] Similarly, the control unit can search spatial recognition data for pixels within a predetermined radius next to the extracted second feature point and bundle, cluster, or group the corresponding pixels to determine (e.g., estimate, calculate) the parking line area.

[0133] The control unit can determine a candidate parking line by comparing information such as location (area) and direction with feature points extracted from the environment image and spatial recognition data, and merging the matching lines.

[0134] Based on information B51, which estimates the parking line area based on the first feature point captured from the surrounding image and the second feature point captured from the spatial recognition data (e.g., used to estimate said parking line area), the control device can determine an overlap area of ​​information B52, which is estimated for the parking line area.

[0135] This means that the overlap area O5 of the first feature point and the second feature point can be determined.

[0136] If the corresponding overlap area O5 exceeds a predetermined value, the control device can determine that the corresponding first feature point P51 and the second feature point P52 form a candidate parking line.

[0137] However, if the overlap area O5 is smaller than a predetermined value, the control device can identify the corresponding feature point as noise and remove it.

[0138] If the difference between the directions of the first feature point P51 and the second feature point P52 is less than a predetermined value and the overlap area O5 exceeds a predetermined value, the control device can identify an area formed by the corresponding feature point as a candidate parking line.

[0139] Fig. 6A and Fig. Figure 6B are views explaining a process for determining a parking space and for performing autonomous parking in the corresponding parking space according to an exemplary embodiment of the present disclosure.

[0140] In Fig. 6A the control device can determine a plurality of candidate parking lines L61 and L62, a first area Z61 which is defined by the endpoints P61, P62, P63 and P64 of the parking lines as its boundary, a second area Z62 which lies between the candidate parking lines and at which the candidate parking lines are located, and the parking area based on a ratio of the number of pixels corresponding to the first area Z61 and the number of pixels in the second area Z62.

[0141] The control unit can determine a parking space candidate by pairing candidate parking lines determined on the basis of the feature points.

[0142] More specifically: The parking space allocation candidate could refer to the second area Z62 between candidate parking lines.

[0143] The control unit can determine the second area Z62 using the position of the endpoint of each candidate parking line. This means that the control unit can determine a parking area that has a parking line as its boundary for each parking line, and an area between candidate parking lines as the second area Z62.

[0144] Referring to Fig. 6B, the control unit can identify a found parking space candidate, that is, an empty parking area Z63 among the second areas Z63 and Z64, using spatial recognition data.

[0145] The control unit can determine an empty parking space based on the following equation. PE=1−Ps−PLPTOT,

[0146] Referring to equation P Eto specify the probability of an empty space, and P S can specify the number of pixels in the space. P L can determine the number of pixels in the parking line.

[0147] P TOT This could refer to the total number of pixels in the parking layout area.

[0148] In particular, the control unit calculates a probability (P E ) that a vehicle, obstacle or person exists in the parking area, based on the space detection result, and determines that the parking space is empty (e.g., a free parking space exists) if the probability is less than a predetermined value.

[0149] If it is determined that the relevant area is an empty parking space, the control unit can also control the driving unit (e.g., driving device) 300 to control the vehicle so that it is parked in the relevant area Z63.

[0150] The in the Fig. 6A and Fig. The processes described in 6B are only one embodiment of the present disclosure, and there is no limitation to the process of identifying an empty space in the vehicle and parking it.

[0151] Fig. Figure 7 is a flowchart describing a method for performing autonomous parking according to an exemplary embodiment of the present disclosure.

[0152] With reference to Fig. 7. The vehicle can acquire an environmental image based on the camera (1001). The vehicle can derive spatial recognition data through deep learning based on the acquired environmental images (1002).

[0153] The vehicle can derive feature points from environmental images and spatial recognition data based on the process described above (1003).

[0154] The control device can perform preprocessing to remove noise and extract an edge (1004).

[0155] The vehicle can determine a candidate parking line based on the preprocessed feature points (1005).

[0156] Furthermore, a parking area can be determined based on the derived candidate parking line and parking can take place there (1006).

[0157] On the other hand, the disclosed exemplary embodiments can be implemented in the form of a recording medium for storing instructions executable by a computer. Instructions can be stored in the form of program code and, when executed by a processor, can generate a program module to perform the operations of the disclosed exemplary embodiments. The recording medium can be implemented as a computer-readable recording medium.

[0158] Computer-readable recording media encompasses all types of recording media from which instructions can be decoded by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage devices, and the like.

[0159] As described above, the disclosed exemplary embodiments have been described with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure have been shown and described, the person skilled in the art will recognize that modifications can be made to these embodiments without departing from the principles and essence of the present disclosure, the scope of which is defined by the claims.

[0160] The vehicle and the control method according to an exemplary embodiment can learn an image obtained from a camera and perform an accurate autonomous parking operation using the learned data.

Images