Vehicle for performing automatic parking and control method thereof

Abstract

A vehicle for performing automatic parking includes a camera configured to acquire a surrounding image of the vehicle including a stop line, and a controller configured to derive spatial recognition data based on the surrounding image of the vehicle as an input value, derive a feature point corresponding to the stop line based on the surrounding image and the spatial recognition data, determine a candidate stop line based on clustering of the feature point, and control the vehicle to perform parking in a parking area composed of the candidate stop line.

Description

Technical Field

[0001] This disclosure relates to a vehicle for performing automatic parking and a control method thereof. Background Technology

[0002] Autonomous driving technology is a technology in which a vehicle can learn about road conditions and drive itself even when the driver does not manually control the brakes, steering wheel, or accelerator pedal.

[0003] Autonomous driving technology is the core technology for realizing intelligent vehicles, including systems such as Highway Driving Assist (HDA, a technology that automatically maintains a safe distance from other vehicles), Rear Side Warning (BSD, a technology that detects nearby vehicles and issues an alert when reversing), Automatic Emergency Braking (AEB, a technology that activates the braking system when no vehicle is detected in front), Lane Departure Warning System (LDWS), Lane Keeping Assist System (LKAS, a technology that compensates for leaving the lane without a turn signal), Advanced Smart Cruise Control (ASCC, a technology that maintains a distance between vehicles and drives at a constant speed), Traffic Jam Assist (TJA), Parking Collision Avoidance Assist (PCA), and Automated Parking System (Remote Smart Parking Assist).

[0004] In technologies that identify surrounding objects and parking spaces for automatic parking control of vehicles, ultrasonic signals are used to perform parking.

[0005] In recent years, research has been actively conducted on automatic parking systems that utilize cameras to perform parking. Summary of the Invention

[0006] This disclosure provides a vehicle and its control method capable of performing accurate automatic parking operations by learning from images acquired by a camera and utilizing the learned data.

[0007] According to an aspect of this disclosure, a vehicle for performing automatic parking may include: a camera configured to acquire images of the vehicle’s surroundings, including parking lines; and a controller configured to derive spatial recognition data based on the images of the vehicle’s surroundings as input values, derive feature points corresponding to the parking lines based on the surrounding images and spatial recognition data, determine candidate parking lines based on clustering of the feature points, and control the vehicle to perform parking in a parking area with candidate parking lines.

[0008] The controller can be configured to remove noise from the surrounding image of the vehicle by utilizing a predetermined first filter and to extract edges based on the gradient of each pixel included in the surrounding image of the vehicle.

[0009] The controller can be configured to classify objects included in images surrounding the vehicle into at least one category based on spatial recognition data.

[0010] The controller can be configured to derive multiple feature points corresponding to the parking line from the vehicle’s surrounding image and spatial recognition data using a predetermined second filter corresponding to the width of the parking line.

[0011] The controller can be configured to determine the reliability of each of the plurality of feature points based on the consistency of the direction values ​​of each of the plurality of feature points corresponding to the parking line, and to determine candidate parking lines based on feature points whose reliability exceeds a predetermined value.

[0012] The controller can be configured to determine a parking area based on the feature points corresponding to the parking lines when the feature points correspond to parking lines included in the surrounding images and spatial recognition data of the vehicle.

[0013] The controller can be configured to determine the parking area based on the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature points and the feature points themselves.

[0014] The controller can be configured to determine a candidate parking line based on the first feature point and the second feature point when the overlap rate between the first feature point determined based on the surrounding image of the vehicle and the second feature point determined based on spatial recognition data exceeds a predetermined value.

[0015] The controller can be configured to determine multiple candidate parking lines, determine a first region provided with the endpoints of the candidate parking lines as boundaries, determine a second region between the candidate parking lines where no candidate parking lines are provided, and determine a parking area based on the ratio of the number of pixels in the first region to the number of pixels in the second region.

[0016] A control method for a vehicle performing automatic parking includes: acquiring an image of the vehicle's surroundings, including parking lines; deriving spatial recognition data based on the surrounding images of the vehicle as input values; deriving feature points corresponding to the parking lines based on the surrounding images and spatial recognition data; determining candidate parking lines based on clustering of the feature points; and controlling the vehicle to perform parking in a parking area composed of the candidate parking lines.

[0017] Determining candidate parking lines may include: removing noise from the surrounding image of the vehicle using a predetermined first filter, and extracting edges based on gradients included in the surrounding image of the vehicle.

[0018] Determining candidate parking lines may include classifying objects in images surrounding the vehicle into at least one category based on spatial recognition data.

[0019] Determining candidate parking lines may include deriving multiple feature points corresponding to the parking lines from images of the vehicle's surroundings and spatial recognition data using a predetermined second filter corresponding to the width of the parking lines.

[0020] Determining candidate parking lines may include: determining the reliability of each of the multiple feature points based on the consistency of the direction values ​​of each of the multiple feature points corresponding to the parking line; and determining candidate parking lines based on feature points whose reliability exceeds a predetermined value.

[0021] Determining candidate parking lines can include determining parking areas based on feature points corresponding to parking lines included in images and spatial recognition data surrounding the vehicle.

[0022] Controlling a vehicle to park within a parking area can include determining the parking area based on the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature points, and the feature points themselves.

[0023] Determining candidate parking lines may include determining candidate parking lines based on the first feature point and the second feature point when the overlap rate of the first feature point determined based on the surrounding image of the vehicle and the second feature point determined based on spatial recognition data exceeds a predetermined value.

[0024] Controlling a vehicle to park in a parking area may include: determining multiple candidate parking lines; determining a first area provided with the endpoints of the candidate parking lines as boundaries; determining a second area between the candidate parking lines where no candidate parking lines are provided; and determining a parking area based on the ratio of the number of pixels in the first area to the number of pixels in the second area. Attached Figure Description

[0025] Figure 1 This is a control block diagram according to exemplary embodiments of the present disclosure.

[0026] Figure 2 This is a diagram illustrating the operation of deriving feature points corresponding to parking lines according to exemplary embodiments of the present disclosure.

[0027] Figure 3 This is a diagram illustrating the operation of removing feature points that are incorrectly detected according to exemplary embodiments of the present disclosure.

[0028] Figure 4 This is a diagram illustrating an operation for determining a candidate parking line based on the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature point, according to an exemplary embodiment of the present disclosure.

[0029] Figure 5 This is a diagram illustrating the overlap operation of feature points determined based on surrounding images and feature points determined based on spatial recognition data according to exemplary embodiments of the present disclosure.

[0030] Figure 6A and Figure 6BThis is a view used to illustrate the operation of determining a parking area and performing automatic parking in the corresponding parking area according to an exemplary embodiment of the present disclosure.

[0031] Figure 7 This is a flowchart describing a method for performing automatic parking according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0032] In the following description, the same reference numerals refer to the same elements throughout the specification. This specification does not describe all elements of the embodiments, and general content or embodiments are not repeated within the scope of this disclosure to which exemplary embodiments of the invention pertain. Terms such as “unit,” “module,” “component,” and “block” can be implemented in hardware or software. According to embodiments, a plurality of “units,” “modules,” “components,” and “blocks” can be implemented as a single component, or a single “unit,” “module,” “component,” and “block” can include multiple components.

[0033] It will be understood that when one element is referred to as “connecting” another element, it can be directly or indirectly connected to the other element, where indirect connection includes “connection via a wireless communication network”.

[0034] Furthermore, when a component “comprises” or “includes” an element, the component may further include other elements without excluding them, unless otherwise specified.

[0035] In addition, when a component “includes” a certain component, unless otherwise specified, it means that other components may be included rather than excluded.

[0036] Throughout the specification, when a component is referred to as being "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where the other component is located between the two components.

[0037] Terms such as "first" and "second" are used to distinguish one component from other components, and the component is not limited by the terms mentioned above.

[0038] Singular expressions include plural expressions, unless the context clearly provides an exception.

[0039] In each step, the identifier is for illustrative purposes only. The identifier does not describe the order of each step. Unless the specific order is explicitly specified in the context, the execution order of each step may differ from the specified order.

[0040] The operating principles and embodiments of this disclosure will be described below with reference to the accompanying drawings.

[0041] Figure 1This is a control block diagram of a vehicle according to an exemplary embodiment of the present disclosure.

[0042] Reference Figure 1 According to an exemplary embodiment, vehicle 1 may include camera 100, controller 200 and driver 300.

[0043] Vehicle 1 can be configured to perform automatic parking.

[0044] The camera 100 can have multiple channels and acquire images of the vehicle's surroundings.

[0045] According to an embodiment, the camera 100 can be positioned around the vehicle to perform the operation of a surround view monitor (SVM).

[0046] Camera 100 may include a charge-coupled device (CCD) camera or a CMOS color image sensor installed in vehicle 1.

[0047] Here, both CCD and CMOS refer to sensors that convert light entering through the lens of camera 100 into electrical signals and store those signals.

[0048] Camera 100 can acquire images of the area around the vehicle, including the parking lines.

[0049] The controller 200 can derive spatial recognition data by learning the surrounding image as input.

[0050] Spatial recognition data can refer to data derived by performing deep learning using surrounding images as input data.

[0051] Specifically, spatial recognition data can refer to deep learning-based spatial recognition results for each of the four-channel camera images, or spatial recognition results in the form of a surrounding view generated therefrom.

[0052] Spatial recognition functions can refer to deep learning-based algorithms that classify various objects observed in an image, such as floors (roads), parking lines, vehicles, pillars, and obstacles, into image pixel units, or operations that classify objects included in the surrounding image based on light reflection or shadows.

[0053] The driver 300 can be configured as a device capable of driving a vehicle.

[0054] According to an embodiment, the driver 300 may include an engine and may include various components for driving the engine.

[0055] Specifically, the drive unit 300 may include brakes and steering mechanisms, and if it is a configuration for implementing vehicle drive, there are no restrictions on the configuration of the devices.

[0056] The controller 200 can perform object recognition operations such as free space, parking lines, pillars or vehicles based on spatial recognition data.

[0057] The controller 200 can derive feature points corresponding to the parking lines based on surrounding images and spatial recognition data.

[0058] Feature points can represent the pixels that make up the parking lines.

[0059] The controller 200 can cluster feature points. During clustering, information about the gradient and orientation of each feature point can be utilized. This will be described in detail later.

[0060] In addition, the controller 200 can determine candidate parking lines based on the clustering of feature points.

[0061] Candidate parking lines can be represented as parking lines that serve as the basis for the controller to perform automatic parking by utilizing feature points rather than images included in the image.

[0062] The controller 200 can control the vehicle to perform parking within a parking area consisting of candidate parking lines. In this disclosure, the controller 200 can be a computer, processor (CPU), or electronic control unit (ECU) that is programmable to control various electronic systems in the vehicle.

[0063] The controller 200 can remove noise from the image of the vehicle's surroundings by utilizing a predetermined first filter.

[0064] According to an embodiment, the first filter may represent a Gaussian filter. Meanwhile, the controller may extract edges based on the gradient of each pixel included in the image surrounding the vehicle.

[0065] The gradient can represent the gradient relationship between pixels in an image.

[0066] When the magnitude of the gradient exceeds a predetermined value, the controller 200 can determine the corresponding part as an edge.

[0067] The controller 200 can classify objects included in images surrounding the vehicle into at least one category based on spatial recognition data.

[0068] Specifically, considering the processing efficiency within the algorithm, the controller 200 can simplify the classification of objects included in the image around the vehicle to eight or more categories (e.g., three categories such as space, parking lines, and others).

[0069] When the spatial recognition results from the 4-channel camera are generated in the form of a surround view, the controller 200 may additionally perform image interpolation.

[0070] The controller 200 can use a second predetermined filter corresponding to the width of the parking line to derive multiple feature points corresponding to the parking line from the vehicle's surrounding image and spatial recognition data.

[0071] The second filter according to the embodiment can refer to a top-hat filter. A top-hat filter can represent a practical spatial or Fourier space filtering technique.

[0072] The top-hat filter outputs a high value on a line whose width is proportional to the filter size, and displays a low output in other areas.

[0073] Therefore, the controller 200 can perform top-hat filtering with dimensions similar to the width of the parking line to determine areas with line components similar to the width of the parking line.

[0074] The controller 200 can determine the maximum value of the output of the top cap filter and use that maximum value as the characteristic point of the parking line.

[0075] In order to determine the direction (angle) of a line from the feature points of each line, the controller 200 can determine the direction value based on the gradient of each feature point.

[0076] The controller 200 can extract line component feature points similar to the width of the parking line by using top-hat filtering on the spatial recognition result data.

[0077] The controller 200 determines the reliability of each of the plurality of feature points based on the consistency of the direction values ​​of each of the plurality of feature points corresponding to the parking line, and determines candidate parking lines based on feature points whose reliability exceeds a predetermined value.

[0078] Specifically, the controller 200 can perform top-hat filtering on the original camera image or surround view image in a predetermined direction, extract line feature points from a region with a line component of a certain width, and measure the reliability of each feature point.

[0079] The reliability value of controller 200 can be determined based on the coherence between the output of the top-hat filter at the location of the corresponding feature point and the direction of the feature point.

[0080] The directional consistency value can refer to how consistent the directionality of the gradient is within a region of a predetermined radius centered on the location of the feature point.

[0081] The controller 200 can determine that feature points extracted on the line have high directional consistency values, and feature points incorrectly extracted from locations outside the line have low consistency values.

[0082] The controller 200 can similarly extract line component feature points from spatial recognition results or results in the form of a surround view based on spatial recognition results through top-hat filtering.

[0083] The controller 200 can determine the reliability of each feature point based on this operation. In this case, the reliability of each feature point can be obtained by taking the average reliability of the region within a predetermined radius centered on the location of the corresponding feature point.

[0084] The controller 200 can determine the reliability of the line feature points determined through the above operations.

[0085] The controller 200 can determine candidate parking lines by using feature points with reliability exceeding a predetermined value to extract only feature points above a predetermined threshold from the lines, and identify the remaining feature points as noise and remove them.

[0086] The sum of reliability can be determined by the following equation.

[0087] [Equation 1]

[0088] S total =w1·S org +w2·S sd

[0089] Referring to Equation 1, S total The reliability is represented by w1 and w2, and w1 and w2 represent the weights corresponding to each of the surrounding images and spatial recognition data.

[0090] In addition, S org It can represent the reliability determined based on the image, and S sd It can represent the reliability determined based on spatial identification data.

[0091] When a feature point corresponds to a parking line included in the surrounding image and spatial recognition data of the vehicle, the controller 200 can determine the parking area based on the feature point corresponding to the parking line.

[0092] A parking area can refer to the area where vehicles park themselves using automatic parking systems.

[0093] The controller 200 can determine the parking area based on the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature points and the feature points.

[0094] The following describes in detail the operation by which the controller 200 determines the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature point.

[0095] When the overlap rate between a first feature point determined based on the vehicle's surrounding image and a second feature point determined based on spatial recognition data exceeds a predetermined value, the controller 200 can determine a candidate parking line based on the first and second feature points.

[0096] The controller 200 can determine multiple candidate parking lines based on the above operations. The controller 200 can determine a first region defined by the endpoints of the candidate parking lines.

[0097] That is, the first region can represent the boundary points of the candidate parking lines derived by the controller.

[0098] The controller 200 can determine a second region between candidate parking lines that does not provide candidate parking lines, and can determine the parking area based on the ratio of the number of pixels corresponding to the first region to the number of pixels in the second region. This will be described later.

[0099] The controller 200 may be implemented by a memory (not shown) and a processor (not shown), the memory storing algorithmic data or data of a program for reproducing the algorithm for controlling the operation of components in the vehicle, and the processor using the data stored in the memory to perform the above operations.

[0100] In this scenario, the memory and processor can be implemented as separate chips. Alternatively, the memory and processor can be implemented as a single chip.

[0101] It can correspond to Figure 1 The performance of the components of the vehicle shown can be improved by adding or removing at least one component. Furthermore, those skilled in the art will readily understand that the relative positions of the components can be altered to correspond to the performance or structure of the system.

[0102] Figure 1 Each component shown refers to a software and / or hardware component, such as a field-programmable gate array (FPGA) and an application-specific integrated circuit (ASIC).

[0103] Figure 2 This is a diagram illustrating the operation of deriving feature points corresponding to parking lines according to an exemplary embodiment.

[0104] Reference Figure 2 , Figure 2 The operation of determining feature point P2 based on the image acquired from the vehicle is shown.

[0105] Before exporting feature point P2, the controller can perform filtering to remove noise from the input camera raw image or surround view data and perform edge data extraction.

[0106] The controller can perform category simplification tasks to efficiently process spatial recognition data that has been classified as objects.

[0107] Reference Figure 2 The controller can perform one-dimensional top-hat filtering on the camera's raw image or surround view image in a predetermined direction (vertical, horizontal, etc.).

[0108] Specifically, the top-hat filter can output high values ​​in lines whose width is proportional to the corresponding filter size, and can output low values ​​in other areas.

[0109] The controller can perform top-hat filtering with dimensions similar to the width of the parking line L2 to find regions with line components similar to the width of the parking line.

[0110] Additionally, the controller can determine the maximum value of the top-hat filter's output as the feature point P2 of the line. To estimate the line's direction (angle) at each feature point P2, a gradient-based direction can be determined.

[0111] The controller can perform similar operations on spatial identification data.

[0112] That is, the controller can use a top-hat filter corresponding to the width of the parking line L2 to derive multiple feature points corresponding to the parking line from the vehicle’s surrounding image and spatial recognition data.

[0113] Figure 3 This is a diagram illustrating the operation of removing feature points from erroneous detections according to an exemplary embodiment.

[0114] Reference Figure 3 The controller can project feature points P31 and P32 onto the domain of the spatial recognition result data.

[0115] The controller can map feature points P31 and P32 to parking line P3 included in the vehicle's surrounding image and spatial recognition data.

[0116] Among these feature points, feature point P31 located on objects other than the parking line (vehicles, pillars, obstacles, etc.) is judged as a feature point not needed to determine the candidate parking line. Therefore, the controller can regard the corresponding feature point P31 as an erroneous feature point and remove it.

[0117] The controller can remove feature points P31 that do not correspond to the parking line from the feature points using the above method, and determine the candidate parking line P33 based on feature point P32 that corresponds to the parking line.

[0118] Figure 4This is a diagram illustrating the operation of determining a candidate parking line R4 based on the ratio of the number of pixels corresponding to parking lines L41 and L42 to the number of pixels of feature points, according to an exemplary embodiment.

[0119] The controller can determine the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature point.

[0120] The controller can determine the parking area consisting of candidate parking lines based on ratios and feature points.

[0121] The controller can project the line feature points P41 and P42 obtained from the surrounding images and spatial recognition data onto the same domain D4, that is, onto the parking lines L41 and L42 so that they correspond to each other.

[0122] The controller can perform clustering on feature points that are adjacent to each other and have similar orientations relative to the line feature points, and generate candidate parking lines R4.

[0123] Generating candidate parking lines R4 via the controller can include determining information such as width, length, direction, and the positions of the two endpoints for each candidate parking line.

[0124] To verify whether each extracted candidate parking line R4 was detected incorrectly, the controller uses information about the two endpoints and width of the candidate parking line to estimate the area where each candidate parking line is located, and calculates the pixel ratio based on the spatial recognition results within the corresponding candidate parking line area.

[0125] Specifically, the controller can derive feature point P41 from the parking line L41 included in the surrounding image.

[0126] Additionally, the controller can derive feature point P42 from the parking line L42 set on the spatial recognition data.

[0127] The controller can associate these feature points P41, P42 with parking lines L41, L42. That is, the controller can project each feature point P41, P42 onto domain D4.

[0128] The controller can determine the ratio of the number of pixels at each feature point of the projected parking line to the number of pixels in the corresponding parking line area. This ratio can be determined based on Equation 2 below.

[0129] [Equation 2]

[0130]

[0131] Referring to Equation 2, F P L can represent the number of pixels of the feature point corresponding to the parking line. P This can represent the number of pixels in the parking line area. S PIt can represent the ratio of the number of pixels of the feature point corresponding to the parking line to the number of pixels in the parking line region.

[0132] When the calculated ratio S P When the value is greater than the predetermined value, the controller can determine that the corresponding feature point is a parking line.

[0133] On the other hand, when the ratio is less than or equal to a predetermined value, the controller can identify the corresponding feature point as an incorrectly detected parking line and remove it.

[0134] That is, the controller can derive feature points determined based on surrounding images and feature points determined based on spatial recognition data.

[0135] In addition, the exported feature points P41 and P42 can be projected onto the parking line, i.e., onto the domain.

[0136] When the ratio of the number of pixels constituting the parking line to the number of pixels of each feature point P41, P42 is determined and the ratio exceeds a predetermined value, the controller can determine that the corresponding feature points constitute the parking line and identify the corresponding feature points P41, P42 as candidate parking lines R4.

[0137] Figure 4 The operation of determining candidate parking lines is only an embodiment of the present disclosure and is not limited to that operation.

[0138] Figure 5 This is a diagram illustrating the overlapping operation of feature point P51 determined based on surrounding image and feature point P52 determined based on spatial recognition data according to an exemplary embodiment.

[0139] Reference Figure 5 When the overlap rate of the first feature point P51 determined based on the surrounding image of the vehicle and the second feature point P52 determined based on spatial recognition data exceeds a predetermined value, the controller can determine a candidate parking line based on the first feature point P51 and the second feature point P52.

[0140] For the first feature point extracted from the surrounding area, the controller can determine the boundary of the parking line near each feature point.

[0141] The controller can determine the point with the largest edge component near each feature point as the parking line boundary point.

[0142] The controller can estimate the bounding boxes B51 and B52 by utilizing the location information of each feature point and the parking line boundary point.

[0143] Similarly, in spatial recognition data, the controller can search for pixels within a predetermined radius adjacent to the extracted second feature point and cluster the corresponding pixels to estimate the parking line area.

[0144] The controller can determine candidate parking lines by comparing and fusing information such as the location (region) and orientation of feature points extracted from surrounding images and spatial recognition data, respectively, and matching the lines.

[0145] The controller can determine the overlapping region of the bounding box B51 of the parking line region estimated based on a first feature point obtained from the surrounding image and the bounding box B52 of the parking line region estimated based on a second feature point obtained from spatial recognition data.

[0146] That is, the overlapping region O5 of the first feature point and the second feature point can be determined.

[0147] When the corresponding overlapping area O5 exceeds a predetermined value, the controller can determine that the corresponding first feature point P51 and second feature point P52 form a candidate parking line.

[0148] However, when the overlapping area O5 is less than a predetermined value, the controller can identify the corresponding feature point as noise and remove it.

[0149] When the difference in direction between the first feature point P51 and the second feature point P52 is less than a predetermined value and the overlapping area O5 exceeds a predetermined value, the controller can determine the area formed by the corresponding feature points as a candidate parking line.

[0150] Figure 6A and Figure 6B This is a view used to illustrate the operation of determining a parking area and performing automatic parking in the corresponding parking area according to an exemplary embodiment.

[0151] Reference Figure 6A The controller can determine multiple candidate parking lines L61, L62, determine a first region Z61 with the endpoints P61, P62, P63, P64 of the parking lines as boundaries, determine a second region Z62 between the candidate parking lines that does not provide candidate parking lines, and determine the parking area based on the ratio of the number of pixels corresponding to the first region Z61 to the number of pixels in the second region Z62.

[0152] The controller can determine the parking zone by pairing candidate parking lines determined based on feature points.

[0153] Specifically, the parking zone can be represented as the second zone Z62 between candidate parking lines.

[0154] The controller can determine the second region Z62 by utilizing the position of the endpoints of each candidate parking line. That is, the controller can determine the parking area including the parking lines by the boundaries of each parking line, and define the area between the candidate parking lines as the second region Z62.

[0155] Reference Figure 6B The controller can use spatial recognition data to determine the parking area it finds, namely, the vacant parking area Z63 in the second area Z63 and Z64.

[0156] The controller can determine the available parking area based on the following equation.

[0157] [Equation 3]

[0158]

[0159] Refer to Equation 3, P E P can represent the probability of free space. S It can represent the number of pixels in space. P L P can represent the number of pixels in a parking line. TOT It can represent the total number of pixels in the parking area.

[0160] Specifically, the controller calculates the probability (P0) of the presence of a vehicle, obstacle, or person in the parking area based on the spatial recognition results. E ), and when the probability is less than a predetermined value, it is judged as an empty parking space.

[0161] Additionally, when the corresponding area is determined to be a vacant parking space, the controller can control the driver to park the vehicle in the corresponding area Z63.

[0162] Figure 6A and Figure 6B The operations described herein are merely embodiments of this disclosure and do not limit the operations for determining the available space of a vehicle and parking.

[0163] Figure 7 This is a flowchart describing a method for performing automatic parking according to embodiments of the present disclosure.

[0164] Reference Figure 7 The vehicle can acquire surrounding images based on a camera (1001). The vehicle can then derive spatial recognition data using deep learning based on the acquired surrounding images (1002).

[0165] The vehicle can derive feature points from surrounding images and spatial recognition data based on the above operations (1003).

[0166] The controller can perform preprocessing to remove noise and extract edges (1004).

[0167] The vehicle can determine candidate parking lines (1005) based on preprocessed feature points.

[0168] Alternatively, parking areas can be determined based on the derived candidate parking lines, and parking can be done in the parking areas (1006).

[0169] On the other hand, the disclosed exemplary embodiments can be implemented in the form of a recording medium for storing computer-executable instructions. The instructions can be stored as program code, and when executed by a processor, can generate program modules that perform the operations of the disclosed exemplary embodiments. The recording medium can be implemented as a computer-readable recording medium.

[0170] Computer-readable recording media include all kinds of recording media that store instructions that can be decoded by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0171] As described above, exemplary embodiments of the disclosure have been described with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure have been shown and described, those skilled in the art will understand that changes can be made to these embodiments without departing from the principles and spirit of the present disclosure, the scope of which is limited by the claims and their equivalents.

[0172] The vehicle and its control method according to the exemplary embodiment can learn from images obtained by a camera and use the learned data to perform accurate automatic parking operations.

Claims

1. A vehicle for performing automatic parking, comprising: A camera acquires images of the vehicle's surroundings, including parking lines; as well as The controller is configured as follows: Spatial recognition data is derived based on the surrounding image as input. Based on the surrounding images and the spatial recognition data, feature points corresponding to the parking line are derived. Candidate parking lines are derived based on the clustering of the feature points, and Control the vehicle to park within the parking area comprised of the candidate parking lines. The controller is configured to determine the candidate parking line based on the first feature point and the second feature point when the overlap rate of the first feature point determined based on the surrounding image of the vehicle and the second feature point determined based on the spatial recognition data exceeds a predetermined value.

2. The vehicle according to claim 1, wherein, The controller is configured to remove noise from the surrounding image of the vehicle by utilizing a first filter and to extract edges based on the gradient of each pixel included in the surrounding image of the vehicle.

3. The vehicle according to claim 1, wherein, The controller is configured to classify objects in images surrounding the vehicle into at least one category based on the spatial recognition data.

4. The vehicle according to claim 1, wherein, The controller is configured to derive multiple feature points corresponding to the parking line from the surrounding image of the vehicle and the spatial recognition data using a second filter corresponding to the width of the parking line.

5. The vehicle according to claim 4, wherein, The controller is configured to determine the reliability of each of the plurality of feature points based on the consistency of the direction values ​​of each of the plurality of feature points corresponding to the parking line, and to determine the candidate parking line based on feature points whose reliability exceeds a predetermined value.

6. The vehicle according to claim 1, wherein, The controller is configured to determine the parking area based on the feature point corresponding to the parking line when the feature point corresponds to the parking line included in the surrounding image of the vehicle and the spatial recognition data.

7. The vehicle according to claim 1, wherein, The controller is configured to determine the parking area based on the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature point and the feature point.

8. The vehicle according to claim 1, wherein, The controller is configured to determine a plurality of candidate parking lines, determine a first region provided with the endpoints of the candidate parking lines as boundaries, determine a second region between the candidate parking lines where no candidate parking lines are provided, and determine the parking area based on the ratio of the number of pixels in the first region to the number of pixels in the second region.

9. A control method for a vehicle performing automatic parking, comprising: Acquire an image of the vehicle's surroundings, including the parking lines; Spatial recognition data is derived based on the surrounding image as input value; Based on the surrounding images and the spatial recognition data, feature points corresponding to the parking line are derived. Candidate parking lines are determined based on the clustering of the feature points; as well as Control the vehicle to park within the parking area comprised of the candidate parking lines. The process of determining the candidate parking line includes determining the candidate parking line based on the first feature point and the second feature point when the overlap rate of the first feature point determined based on the surrounding image of the vehicle and the second feature point determined based on the spatial recognition data exceeds a predetermined value.

10. The method according to claim 9, wherein, Determining the candidate parking lines includes: By utilizing a first filter to remove noise from the image surrounding the vehicle, and Edges are extracted based on gradients of each pixel included in the image surrounding the vehicle.

11. The method according to claim 9, wherein, Determining the candidate parking lines includes: Based on the spatial recognition data, objects included in the images surrounding the vehicle are classified into at least one category.

12. The method according to claim 9, wherein, Determining the candidate parking line involves deriving multiple feature points corresponding to the parking line from the surrounding image of the vehicle and the spatial recognition data using a second filter corresponding to the width of the parking line.

13. The method according to claim 12, wherein, Determining the candidate parking lines includes: The reliability of each of the plurality of feature points is determined based on the consistency of the direction values ​​of each of the plurality of feature points corresponding to the parking line; and The candidate parking lines are determined based on feature points whose reliability exceeds a predetermined value.

14. The method according to claim 9, wherein, Determining the candidate parking line includes determining the parking area based on the feature point corresponding to the parking line when the feature point corresponds to the parking line included in the image of the vehicle's surroundings and the spatial recognition data.

15. The method according to claim 9, wherein, Controlling the vehicle to park in the parking area includes determining the parking area based on the ratio of the number of pixels corresponding to the parking line to the number of pixels of the feature point and the feature point.

16. The method according to claim 9, wherein, Controlling the vehicle to park in the parking area includes: Multiple candidate parking lines were identified; Determine a first region bounded by the endpoints of the candidate parking lines; Determine a second area between the candidate parking lines where no candidate parking lines are provided; and The parking area is determined based on the ratio of the number of pixels in the first area to the number of pixels in the second area.

Images