Parking assistance system and method with improved avoidance steering control

Abstract

To provide a parking assist system with improved avoidance steering control and a method thereof.SOLUTION: A parking assist system includes a sensor fusion calculation module that fuses sensing data of an ultrasonic sensor and image data of an SVM camera to calculate position information of an object with respect to a current location of a vehicle and a parking assist module that avoids the object based on the position information and performs steering control of the vehicle for autonomous parking. The parking assist system accurately detects the object on a position departing from a sensing range of the ultrasonic sensor and performs avoidance steering robust to a surrounding object upon autonomous parking.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a parking assistance system and method with significantly improved evasive steering control for autonomously parked vehicles. [Background technology]

[0002] In general, a parking assistance system (RSPA: Remote Smart Parking Assist) is a system that uses ultrasonic sensors to help a vehicle recognize a parking space and control steering, braking, speed, forward and reverse gear shifts, etc. to assist the vehicle in parking.

[0003] As described above, the conventional parking assist system (RSPA) recognizes objects such as surrounding vehicles using ultrasonic sensors, and then performs parking by steering to avoid the recognized objects.

[0004] However, ultrasonic sensors have a limited horizontal detection range, so conventional parking assistance systems based on ultrasonic sensors have had problems with determining the control reference point for obstacles that are detected by the outer ultrasonic sensor but not by the inner ultrasonic sensor, resulting in inaccurate detection of the vertical position.

[0005] In this way, there has been a problem in that, due to inaccurate calculation of information regarding the longitudinal position of an object, avoidance steering control is not performed strongly during autonomous parking by the parking assistance system. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2019-211480 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made in consideration of the above-mentioned conventional technology, and an object of the present invention is to provide a parking assistance system and method with improved avoidance steering control that enables strong avoidance steering to be performed around surrounding obstacles during autonomous parking, by including a sensor fusion calculation module that combines sensing data from a sensor module with video data from an SVM camera to calculate position information of an obstacle based on the current position of the vehicle, and a parking assistance module that avoids the obstacle based on the position information and performs steering control of the vehicle for autonomous parking, thereby making it possible to clearly detect obstacles that are located outside the detection range of the sensor module. [Means for solving the problem]

[0008] In order to achieve the above object, one aspect of the present invention provides a parking assistance system with improved avoidance steering control, which includes a sensor module that detects the distance from the vehicle to an obstacle based on sensing data acquired by searching the area around the vehicle, an SVM (Surround View Monitor) camera that captures images of the area around the vehicle and acquires video data that can detect the position and direction of the obstacle, a sensor fusion calculation module that combines the sensing data from the sensor module and the video data from the SVM camera to calculate position information of the obstacle based on the current position of the vehicle, and a parking assistance module that performs steering control of the vehicle to avoid the obstacle and park autonomously based on the position information.

[0009] The sensor fusion calculation module may include a cell setting unit that divides the area around the vehicle on the image data acquired by the SVM camera into cells at regular intervals and assigns an address to each cell; an obstacle recognition unit that detects the presence or absence of obstacles based on the image data and classifies the types of the detected obstacles to recognize obstacles present in the parking area; and an obstacle map generation unit that matches and stores cell addresses of positions where obstacles are recognized on the image data with obstacle classification information and generates an obstacle map showing the current obstacle distribution status in the parking area.

[0010] Preferably, the sensor fusion calculation module further includes a boundary obstacle identification unit that determines whether a boundary obstacle recognized in a region of interest (ROI) where the front / rear image data and the side image data overlap is the same as any obstacle recognized in at least one of the front / rear image data and the side image data, and identifies the obstacle.

[0011] The boundary obstacle identification unit calculates a first centroid of the boundary obstacle image recognized in the front / rear image data and a second centroid of the boundary obstacle image recognized in the side image data, and may determine that the boundary obstacles having the smallest distance between the first centroid and the second centroid, which are calculated for boundary obstacles matching the same obstacle classification code, are the same obstacle.

[0012] It is preferable that the sensor module is configured with an ultrasonic sensor that acquires sensing data by ultrasonic time of flight (TOF), and the sensor fusion calculation module further includes an obstacle position correction unit that corrects and stores the positions of obstacles recognized by the video data and stored on the obstacle map based on the sensing data of the ultrasonic sensor.

[0013] The obstacle position correction unit may select a point of the obstacle closest to the ultrasonic sensor as a reference coordinate, and then move the reference coordinate on the same cell until the reference coordinate touches a circle having the ultrasonic sensor as its center and a radius equal to a regression path of sensing data (TOF) relative to the reference coordinate, thereby setting the reference coordinate as a correction coordinate of the obstacle, and may similarly move the coordinates of the remaining points of the obstacle parallel to the direction in which the reference coordinate moved to the correction coordinate, thereby correcting the position of the obstacle. In addition, when the obstacle position correction unit receives multiple pieces of sensing data (TOF) for one obstacle, it can correct the position of the obstacle using each piece of sensing data (TOF), calculate the average of the coordinates indicating the corrected position of the obstacle, and correct the calculated average as the final position of the obstacle.

[0014] It is preferable that the parking assistance module includes an avoidance reference point determination unit that selects an obstacle point to be avoided when entering a parking space as an avoidance reference point based on coordinates indicating the obstacle point stored in the obstacle map, and an alignment angle calculation unit that calculates an alignment angle at which a vehicle should be aligned so that the vehicle entering the parking space avoids the obstacle and parks.

[0015] The avoidance reference point determination unit may set a vehicle part where contact with an adjacent obstacle should be avoided when entering a parking space as a vehicle reference point, and may set predetermined areas in the +y-axis direction on one side of the x-axis and the -y-axis direction on the other side of the x-axis in a local coordinate system based on the center of the vehicle as first and second reference point regions of interest, and then select a point of obstacle points in each reference point region of interest that is the shortest distance from the vehicle reference point as an avoidance reference point in each reference point region of interest.

[0016] Preferably, the parking assistance module further includes a weight variable unit that applies a weight in the y-axis direction increasing as the distance from the vehicle to the obstacle decreases, so that when the x-axis distance between the vehicle and the obstacle is greater than a predetermined reference distance, a point at a coordinate close to the vehicle reference point on the x-axis is assigned a high weight and selected as the avoidance reference point, and when the x-axis distance between the vehicle and the obstacle is smaller than the reference distance, a point at a coordinate close to the vehicle reference point on the y-axis is assigned a high weight and selected as the avoidance reference point.

[0017] The alignment angle calculation unit calculates an average of an angle between a vehicle reference point set at the center of the front of the vehicle and the avoidance reference point as an initial alignment angle of the vehicle entering the parking space, calculates an alignment angle change amount that should be increased or decreased for avoidance steering depending on the degree of entry of the vehicle into the parking space, and can provide the calculated amount as data for avoidance steering control of the vehicle.

[0018] The alignment angle calculation unit may use a first avoidance reference line and a second avoidance reference line, which are straight lines that pass through each avoidance reference point parallel to a center reference line that is a straight line indicating the current initial alignment angle, to set an area between the first avoidance reference line and the center reference line as a first avoidance region of interest, and set an area between the second avoidance reference line and the center reference line as a second avoidance region of interest, and may calculate a sum of distances between an obstacle point in the first avoidance region of interest and the first avoidance reference line and a sum of distances between an obstacle point in the second avoidance region of interest and the second avoidance reference line, and then calculate an alignment angle change amount that corrects the initial alignment angle so that the difference between the sums of the two distances is minimized.

[0019] In order to achieve the above object, according to one aspect of the present invention, a parking assistance method with improved avoidance steering control includes: an image data acquisition step of acquiring an image of an obstacle around a parking space using front / rear image data and side image data of a vehicle captured by an SVM (Surround View Monitor) camera; a sensing data acquisition step of detecting the position of the obstacle and the distance to the obstacle using sensing data acquired by a sensor module; an obstacle map generation step of storing coordinates of the obstacle recognized using the front / rear image data and side image data and whose position is corrected using the sensing data in an obstacle map; and a parking assistance step of selecting an avoidance reference point from the coordinates of points constituting the obstacle, at which the vehicle is required to avoid the obstacle, and calculating an alignment angle of the vehicle required for avoidance steering based on the avoidance reference point.

[0020] The method may further include a boundary obstacle identification step of determining whether a boundary obstacle recognized in a region of interest (ROI) where the front / rear image data and the side image data overlap is the same as any obstacle recognized in at least one of the front / rear image data and the side image data, when generating the obstacle map by receiving the front / rear image data, the side image data, and the sensing data (TOF: Time of Flight).

[0021] The boundary obstacle identification step may include a boundary obstacle selection process for selecting an object whose image is recognized in the region of interest (ROI) from the front / rear image data and the side image data as a boundary obstacle; a centroid calculation process for calculating a first centroid and a second centroid, which are plane centers for point coordinates of the boundary obstacle recognized in the front / rear image data and the side image data, respectively; and a boundary obstacle determination process for comparing distances between the first centroid and the second centroid calculated for boundary calculation objects matching the same obstacle classification code and identifying two boundary obstacles having the smallest distance as the same obstacle.

[0022] The generation of the obstacle map may include a reference coordinate correction process in which a point of the obstacle closest to the sensor module is selected as a reference coordinate, and the reference coordinate is moved in a direction toward the installation position of the SVM camera until it touches a circle having the sensor module as its center and a radius equal to the regression path of the sensing data relative to the reference coordinate, thereby generating corrected coordinates of the obstacle; and an obstacle point correction process in which the coordinates of the remaining points of the obstacle are similarly translated in the direction in which the reference coordinate is moved to the corrected coordinates.

[0023] The parking assistance step may include an avoidance reference point determination step of selecting a point of the obstacle that the vehicle should avoid as an avoidance reference point based on the distance relationship between the point coordinates of the obstacle stored in the obstacle map and a vehicle reference point that is set as a vehicle part where contact with the obstacle should be avoided by the vehicle entering the parking space, and an alignment angle calculation step of calculating an alignment angle required to steer and control the vehicle so that the vehicle reference point avoids the avoidance reference point and enters the parking space.

[0024] The avoidance reference point determination step may include a reference point region of interest setting step of setting predetermined regions in a +y-axis direction on one side of an x-axis and a -y-axis direction on the other side of the x-axis on a local coordinate system based on a center of the vehicle as a first reference point region of interest and a second reference point region of interest, and an avoidance reference point selection step of selecting, as an avoidance reference point, coordinates of an obstacle that is the shortest distance from the vehicle reference point among points of obstacles in the first reference point region of interest and the second reference point region of interest.

[0025] The step of determining the avoidance reference point may further include a weight varying step of increasing and applying a weight in a y-axis direction as the distance between the vehicle entering the parking space and the obstacle decreases when selecting a point having the smallest distance from the vehicle reference point among a plurality of obstacle points in the reference point region of interest.

[0026] The alignment angle calculation step may include an initial alignment angle calculation process that calculates an average of angles formed by a vehicle reference point set at the center of the front of the vehicle and an avoidance reference point as an initial alignment angle required for the vehicle to avoid the obstacle and enter the parking space; an avoidance region of interest setting process that uses a first avoidance reference line and a second avoidance reference line, which are straight lines that pass through each avoidance reference point in parallel with a center reference line that is a straight line indicating the initial alignment angle, to set an area between the first avoidance reference line and the center reference line as a first avoidance region of interest and set an area between the second avoidance reference line and the center reference line as a second avoidance region of interest; and an alignment angle change calculation process that calculates an alignment angle change amount that corrects the initial alignment angle so that the difference between the sums of the two distances is minimized after calculating a sum of the distances between a point of an obstacle in the first avoidance region of interest and the first avoidance reference line and a sum of the distances between a point of an obstacle in the second avoidance region of interest and the second avoidance reference line. [Effects of the Invention]

[0027] According to the present invention, the position of an obstacle recognized in the video data generated by the SVM camera with a wide detection range is corrected by the sensing data (TOF) acquired by the sensor module capable of accurate distance measurement, thereby improving the recognition accuracy of surrounding obstacles during autonomous parking, thereby enabling stable avoidance steering.

[0028] In addition, various other effects that can be directly or indirectly grasped by this specification can be provided. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a block diagram of a parking assistance system with improved avoidance steering control according to an embodiment of the present invention; [Figure 2] 10 is an exemplary diagram showing an increase in the sensing range during avoidance steering control according to the present invention; FIG. [Figure 3] 3 is an exemplary diagram showing the sensing range of front, rear and side image data captured by an SVM camera according to the present invention; [Figure 4] 3 is a flowchart illustrating a process of recognizing and clustering obstacles using images shown in video data according to the present invention. [Figure 5] 1 is an exemplary view showing how an obstacle is recognized using images in front, rear, and side image data of a vehicle according to the present invention; [Figure 6] 1 is an exemplary view showing that a boundary obstacle image shown in a region of interest is recognized in front, rear, and side image data according to the present invention; [Figure 7] 10 is an exemplary diagram showing how a boundary obstacle is identified by comparing the distance between the first centroid and the second centroid according to the present invention; FIG. [Figure 8] 10 is an example diagram showing how the coordinates of an obstacle are matched and stored in a vehicle-centered local coordinate system according to the present invention; FIG. [Figure 9] 1 is an exemplary diagram showing how the position of an obstacle is corrected using sensing data (TOF) according to the present invention; [Figure 10]10 is an exemplary diagram showing an obstacle position correction when a plurality of pieces of sensing data for one obstacle are received according to the present invention; [Figure 11] 10 is an exemplary diagram illustrating the selection of a reference point region of interest and an avoidance reference point according to the present invention; [Figure 12] 10 is an exemplary diagram showing a change in weighting coefficient according to a change in the x-axis distance between a vehicle and an obstacle according to the present invention; FIG. [Figure 13] 10 is a graph showing an example of setting tuning parameters for cost calculation according to the present invention. [Figure 14] 4 is a flowchart illustrating a calculation process for calculating an alignment angle according to the present invention. [Figure 15] 10 is an exemplary diagram illustrating how an initial alignment angle and a region of interest to be avoided are set for calculating an alignment angle and how an alignment angle change amount is calculated according to the present invention; [Figure 16] 1 is an exemplary diagram showing how to select target obstacles around a vehicle that has entered a parking space according to the present invention; [Figure 17] 10 is an exemplary diagram showing how an avoidance reference line showing a distribution of points after a space entry avoidance reference point is calculated and parking is performed along the resulting alignment angle according to the present invention; FIG. [Figure 18] 1 is an exemplary view showing alignment target points for guiding a vehicle's entry into a parking space according to the present invention; [Figure 19] FIG. 1 is a block diagram of a parking assistance method with improved avoidance steering control according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] Hereinafter, specific examples of embodiments of the present invention will be described in detail with reference to the drawings. When assigning reference numerals to components in each drawing, the same numerals are assigned to identical components even if they are shown in different drawings. Furthermore, when describing the embodiments of the present invention, if it is determined that a detailed description of related publicly known configurations or functions would hinder understanding of the embodiments of the present invention, such detailed description will be omitted.

[0031] When describing components of embodiments of the present invention, terms such as "first," "second," "A," "B," "(a)," and "(b)" are used to distinguish the component from other components and do not limit the essence, order, or sequence of the components. Furthermore, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention pertains. Terms defined in commonly used dictionaries are to be interpreted as meanings consistent with the meanings they have in the context of the relevant art, and are not to be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0032] Hereinafter, an embodiment of the present invention will be described in detail with reference to FIGS.

[0033] FIG. 1 is a block diagram of a parking assistance system with improved avoidance steering control according to an embodiment of the present invention.

[0034] Referring to FIG. 1, a parking assistance system with improved avoidance steering control according to an embodiment of the present invention includes a sensor module 100 that detects the distance from the vehicle to an obstacle based on sensing data acquired by searching the periphery of the vehicle, an SVM (Surround View Monitor) camera 200 that captures images of the periphery of the vehicle and acquires image data that can be used to detect the position and direction of the obstacle, a sensor fusion calculation module 300 that calculates position information of the obstacle based on the current position of the vehicle by combining the sensing data of the sensor module and the image data of the SVM camera, and a parking assistance module 400 that controls the steering of the vehicle to avoid the obstacle and park autonomously based on the calculated position information.

[0035] In addition, the parking assistance system with improved avoidance steering control according to an embodiment of the present invention may further include an obstacle map 360 that matches the position information of the obstacle calculated by the sensor fusion calculation module 300 with cells that define the periphery of the vehicle, and stores the coordinates of each point indicating the outside of the obstacle.

[0036] In this case, the sensor module 100 is configured with various sensors that can detect the presence or absence of obstacles around the vehicle, as well as the location and distance to the obstacles. Hereinafter, the sensor module 100 will be described as an embodiment configured with an ultrasonic sensor that is widely used in general parking assistance systems, but it goes without saying that the sensor module 100 can be implemented with various sensors other than the ultrasonic sensor. Therefore, the ultrasonic sensor will be described with the same reference numeral as the sensor module.

[0037] The ultrasonic sensor 100 receives the time of flight (TOF) and direction of ultrasonic waves emitted around the vehicle and reflected by surrounding parked vehicles and obstacles, and detects the direction in which the obstacle is located and the distance to the obstacle relative to the vehicle.

[0038] The ultrasonic sensor 100 is also used in a typical parking assistance system to search for a parking space and detect obstacles for avoidance steering. However, because the ultrasonic sensor has a limited detection range, it can be difficult to determine the control reference point for an obstacle that exists beyond the detection range.

[0039] That is, a typical parking assistance system is equipped with an inner sensor for sensing the center of the front of the vehicle and outer sensors for sensing both sides of the front of the vehicle, and determines a control reference point based on the intersection of the sensed positions of both the inner and outer sensors.

[0040] As a result, as shown in (a) of Figure 2, on the left side of the vehicle, the obstacle is detected by both the internal sensor and the external sensor, so the control reference point can be derived, but on the right side of the vehicle, the obstacle is detected only by the external sensor, not by the internal sensor, so the intersection point cannot be derived and therefore the control reference point cannot be determined.

[0041] Therefore, in the present invention, as shown in (b) of FIG. 2, the sensing data sensed by the ultrasonic sensor 100 is combined with the image data acquired by the SVM camera 200, thereby enabling clear detection of obstacles located outside the sensing range of the internal sensor.

[0042] In other words, although the distance accuracy is somewhat inferior to that of ultrasonic sensors, it has a wide detection range and compensates for the limitations of ultrasonic sensors by detecting obstacles using image data from an SVM camera, which provides accurate information on the direction in which obstacles exist.

[0043] For this purpose, the SVM (Surround View Monitor) camera 200 captures images of the front, rear and both sides of the vehicle, and obtains image data that can be used to determine whether an obstacle exists and the direction in which the obstacle is located.

[0044] As a result, even if an obstacle is located outside the detection range of the internal sensor, as shown in Figure 2(b), the control reference point for steering the vehicle to avoid the obstacle can be derived from the position of the obstacle in the video data acquired by the SVM camera and the position and distance information of the obstacle in the sensing data acquired by the ultrasonic sensor.

[0045] The sensor fusion calculation module 300 also includes a cell setting unit 310 that divides the area around the vehicle on the video data acquired by the SVM camera into cells at regular intervals and assigns an address to each cell; an obstacle recognition unit 320 that detects the presence or absence of obstacles based on the video data, classifies the type of the detected obstacle, and recognizes obstacles present in the parking area; and an obstacle map generation unit 340 that matches and stores the cell addresses of positions where obstacles are recognized on the video data with obstacle classification information, and generates an obstacle map that shows the current obstacle distribution status in the parking area.

[0046] 3(a), the cell setting unit 310 divides each of the front and rear areas of the vehicle shown in the video data captured by the SVM camera 200 into a number of cells spaced at equal intervals, and assigns an address to each cell, such as cell 1 to cell 30. At this time, addresses are assigned separately to each of the front and rear areas of the vehicle captured by the SVM camera 200.

[0047] In addition, as shown in (b) of Figure 3, in the case of the areas on both sides of the vehicle photographed by the SVM camera, the cell setting unit 310 divides the area into a number of cells spaced at equal intervals and assigns addresses to the cells, such as cell 1 to cell 60.

[0048] In this case, Figure 3 shows that the anterior and posterior regions are divided into 30 cells each, and the lateral regions are divided into 60 cells each, but it goes without saying that the number of cells into which each region is divided can be set to a different number.

[0049] In addition, the obstacle recognition unit 320 determines whether or not there are obstacles, including parked vehicles shown in the video data acquired by the SVM camera 200, and recognizes the obstacle by comparing the type of the recognized obstacle with the external shape of the obstacle in the obstacle classification table 321 stored in advance.

[0050] For this purpose, the obstacle classification table 321 classifies objects that can be captured as video data in a parking area, including free space where no obstacles are detected, background, cars, pillars, curbs, and pedestrians, and then assigns a specific obstacle classification code (index) to each classified object and stores it.

[0051] The obstacle recognition unit 320 compares the image of the obstacle shown in the video data with the item information stored in the obstacle classification table 321, and then stores the matching obstacle classification code.

[0052] In this case, the obstacle recognition unit 320 determines whether an image recognized as an obstacle exists for each cell divided by the cell setting unit 310 and what kind of obstacle the existing image is, and then stores the coordinates of each point where the obstacle is recognized in a local coordinate system based on the vehicle as position information. Also, the obstacle classification code assigned to the image of the obstacle recognized for each cell is matched with each obstacle point and stored.

[0053] Therefore, the obstacle recognition unit 320 recognizes the area occupied by the obstacle by clustering the cells with matching obstacle classification codes. For this purpose, the obstacle recognition unit 320 recognizes the same obstacle when the same obstacle classification code matches adjacent cells.

[0054] That is, as shown in Fig. 4, after determining whether there is an obstacle classification code matching the i-th cell, the obstacle classification code matching the current cell is compared with the obstacle classification code matching the previous cell to determine whether it matches. If the obstacle classification codes of the two cells are the same, it is determined that they are images of the same obstacle, and if there are no obstacle classification codes or they are different, the same obstacle determination is terminated. In this way, a cluster of points indicating the same obstacle is formed from the obstacle images recognized in each cell.

[0055] After the determination for the i-th cell is completed, the same process is repeated for the next cell, i+1-th cell, to analyze all cells that make up the image data and determine whether or not there is an obstacle recognized in the image data.

[0056] In this case, it goes without saying that the process of determining whether an obstacle classification code exists in the next cell, i+1-th cell, is repeated even if there is no obstacle classification code matching the i-th cell.

[0057] In this way, the obstacle recognition unit 320 recognizes obstacles present around the vehicle by analyzing the image data captured by the SVM camera and determining whether the obstacle classification code matches each cell and whether the obstacle classification codes are the same or different.

[0058] As a result, as shown in (a) of Fig. 5, an obstacle 1 (Obj1) located in front of the vehicle and an obstacle 2 (Obj2) located on the front right side of the vehicle are recognized from the front and rear image data of the vehicle. Also, as shown in (b) of Fig. 5, an obstacle 3 (Obj3) located on the left side of the vehicle and an obstacle 2 (Obj2) located on the front right side of the vehicle are recognized from the side image data of both sides of the vehicle.

[0059] In this case, in the case of obstacle 2 (Obj2), since it is recognized from both the front image data and the side image data, it is necessary to determine whether these recognition results indicate the same obstacle or different obstacles.

[0060] To this end, the sensor fusion calculation module 300 further includes a boundary obstacle identification unit 330 that determines whether a boundary obstacle recognized in a region of interest (ROI) where the front / rear image data and the side image data overlap is the same as any obstacle recognized in at least one of the front / rear image data and the side image data, and identifies the obstacle.

[0061] That is, the boundary obstacle identifying section 330 is configured to identify the same obstacle among the obstacles recognized in both the front / rear image data and the side image data as a single obstacle.

[0062] In this way, to distinguish and explain obstacles recognized in the area of ​​interest from obstacles recognized in the front / rear image data or the side image data, obstacles in the area of ​​interest before identification are referred to as "boundary obstacles," and after identification, they are referred to as "obstacles" like other obstacles.

[0063] The boundary obstacle identification unit 330 calculates the first centroid of the boundary obstacle image recognized in the front and rear image data and the second centroid of the boundary obstacle image recognized in the side image data, and determines that the boundary obstacles having the smallest distance between the first centroid and the second centroid, which are calculated for boundary obstacles matching the same obstacle classification code, are the same obstacle.

[0064] To this end, the boundary obstacle identification unit 330 sets the right and left areas in front of the vehicle and the right and left areas behind the vehicle as regions of interest (ROI) (shown as dotted rectangles in FIG. 6), as shown in (a) and (b) of FIG. 6, and selects obstacles recognized in the ROI as boundary obstacles. FIG. 6 shows that a boundary obstacle with a circular image is selected as a first boundary obstacle, and a boundary obstacle with a rectangular image is selected as a second boundary obstacle.

[0065] After a boundary obstacle in the region of interest is selected, the boundary obstacle identification unit 330 calculates a first centroid using point coordinates of the boundary obstacle recognized in the front and rear image data, and calculates a second centroid using point coordinates of the boundary object recognized in the side image data. Thus, when only a portion of the boundary object is recognized, the centroid of only the recognized point is calculated.

[0066] As a result, as shown in FIG. 6(a) where the region of interest (ROI) is superimposed on the front and rear image data, the first centroid (x c , y c ) is calculated using the following formula 1.

[0067] [Formula 1] TIFF0007739142000001.tif14132

[0068] In Figure 6(a), the position of the first centroid relative to the first boundary obstacle is indicated by the letter X, and then P 1,f The first centroid for the second boundary obstacle is also indicated by the letter X, followed by P 2,f It is written as:

[0069] As shown in FIG. 6(b), where the region of interest (ROI) is superimposed on the side image data, the second centroid (x c , y c ) is calculated using the above formula 1.

[0070] In Figure 6(b), the position of the second centroid relative to the first boundary obstacle is indicated by the letter X, and then P 1,s The second centroid for the second boundary obstacle is also indicated by the letter X, followed by P 2,s It is written as:

[0071] In this way, the first centroid (P 1,f , P 2,f ) and the second centroid (P 1,s , P 2,s ), the boundary obstacle identification unit 330 selects, as a comparison target, boundary obstacles in the front and rear image data and boundary obstacles in the both side image data in each region of interest that have the same obstacle classification code.

[0072] Next, the boundary obstacle identification unit 330 calculates the distance between the first centroid and the second centroid calculated for each boundary obstacle selected as a comparison target using the Euclidean norm, as shown in the following equation 2, and then determines that the two boundary obstacles with the smallest distance are the same obstacle.

[0073] [Formula 2] TIFF0007739142000002.tif9128

[0074] As a result, as shown in FIG. 7, the first centroid P calculated from the front and rear image data for the first boundary obstacle is 1,f and the second centroid P calculated from the lateral image data 1,s Since the distance between the first centroid P and the second boundary obstacle is the smallest, it is determined that they represent the same obstacle. 2,f and the second centroid P 2,s Since the distance between them is the smallest, they are determined to represent the same obstacle.

[0075] In addition, the obstacle map generation unit 340 generates an obstacle map 360 that shows the current obstacle distribution status for the parking area where autonomous parking is to be performed by mapping and storing obstacle recognition information obtained from the video data of the SVM camera together with the coordinates of the position information where each obstacle is recognized, based on the vehicle position.

[0076] As a result, the obstacle map generation unit 340 stores the obstacle information (including the obstacle classification code and the external image) recognized by the obstacle recognition unit 320 based on the front / rear image data and the side image data in the obstacle map 360 based on the current position of the vehicle, as shown in (a) and (b) of FIG. 8.

[0077] In this case, in the case of an obstacle 2 (Obj2) located in a region of interest (ROI) recognized in both the front and rear image data and the side image data, the boundary obstacle identification unit 330 identifies that the boundary obstacle recognized in (a) of Figure 8 and the boundary obstacle recognized in (b) of Figure 8 are the same object, obstacle 2 (Obj2).

[0078] As a result, as shown in (c) of Figure 8, the obstacle map 360 stores information that obstacle 1 (Obj1) acquired only from the front and rear image data is located in front of the vehicle, obstacle 3 (Obj3) acquired only from the side image data is located on the left side of the vehicle, and obstacle 2 (Obj2) recognized in the region of interest is located in front of the right side of the vehicle.

[0079] In this case, in (c) of FIG. 8, the obstacle recognition information acquired from the front and rear image data of the vehicle is displayed as circular dots, and the obstacle recognition information acquired from the image data of both sides of the vehicle is displayed as diamond-shaped dots for easy visual distinction.

[0080] The sensor fusion calculation module 300 further includes an obstacle position correction unit 350 that corrects and stores the positions of obstacles set on the obstacle map based on the sensing data from the ultrasonic sensor.

[0081] The obstacle position correction unit 350 selects the point of the obstacle closest to the ultrasonic sensor as the reference coordinate, then translates the reference coordinate until it touches a circle with the ultrasonic sensor as the center and the radius of the return path of the sensing data (TOF) relative to the reference coordinate, and sets it as the correction coordinate of the obstacle, and similarly translates the coordinates of the remaining points of the obstacle along the direction in which the reference coordinate moved to the correction coordinate, thereby correcting the position of the obstacle.

[0082] In this case, the coordinates of the obstacle closest to the ultrasonic sensor are determined by the coordinate P that matches the obstacle point measured by the ultrasonic sensor with the shortest time of flight (TOF), as shown in FIG. 9 and Equation 3. j,obj These reference coordinates are set as a region that does not deviate from the detection range (FOV: Field of View) of the ultrasonic sensor.

[0083] [Formula 3] TIFF0007739142000003.tif10128

[0084] At this time, P USS indicates the position of an ultrasonic sensor (USS) 100 provided on the vehicle, and P j,obj indicates the reference coordinates of the obstacle recognized in the j-th cell of the image of the obstacle recognized in the video data, and TOF indicates the time of flight of the ultrasonic sensor after being emitted from the ultrasonic sensor and then hitting an obstacle and being reflected.

[0085] The obstacle position correction unit 350 calculates the reference coordinate P j,obj to the position sensed by the ultrasonic sensor and set as the corrected coordinate. USS After setting a circle with a radius equal to the distance from the current coordinate P to the position of the obstacle where the sensing data (TOF) was reflected, j,obj From the installation position P of the SVM camera 200 CAM Heading to By translating along TIFF0007739142000004.tif14128, the corrected coordinate P' j,obj Set.

[0086] Thereafter, the obstacle position correction unit 350 converts the coordinates indicating the obstacle stored in the obstacle map into the reference coordinates P j,obj is the corrected coordinate P' j,obj Moved to The position of the obstacle is corrected by similarly translating it along TIFF0007739142000005.tif14128.

[0087] As a result, the reference coordinates of the obstacle closest to the ultrasonic sensor 100 among the coordinates of the obstacles acquired based on the video data are moved in the direction toward the SVM camera 200 installed in the vehicle, and the remaining coordinates of the obstacles are moved parallel to the movement direction of the reference coordinates, thereby preventing the image of the obstacle from being distorted during the correction process.

[0088] In this case, as shown in FIG. 9, when only one sensing data (TOF) is received for one obstacle (i.e., when the obstacle is detected by the outer ultrasonic sensor but not by the inner ultrasonic sensor), the coordinates of the point with the shortest time of flight (TOF) of the sensing data value are set as the reference coordinates, as described above.

[0089] However, as shown in Figure 10, when multiple pieces of sensing data (TOF) are received for one obstacle (i.e., when sensing data is received from both the outer ultrasonic sensor and the inner ultrasonic sensor), the obstacle position is corrected using each piece of sensing data (TOF), and then the average of the coordinates indicating the corrected obstacle position is calculated as shown in Equation 4 below, and the calculated average is corrected as the final position of the obstacle. In this case, N is the total number of pieces of sensing data (TOF) received for one obstacle.

[0090] [Formula 4] TIFF0007739142000006.tif12128

[0091] In FIG. 10, the outer ultrasonic sensor P USS1 The reference coordinate set by the sensing data (TOF) received by P j,obj1 and the inner ultrasonic sensor P USS2 The reference coordinate set by the sensing data (TOF) received by P j,obj2 is shown.

[0092] In addition, two reference coordinates (P j,obj1 , P j,obj2 ) to the installation position P of the SVM camera CAM In this way, after translating the other coordinates of the obstacle along the vector direction from each reference coordinate to each correction coordinate, the average of the results is set as the final corrected position of the obstacle, as described above.

[0093] The parking assistance module 400 also includes an avoidance reference point determination unit 410 that selects an obstacle point to be avoided when entering a parking space as an avoidance reference point based on coordinates indicating the obstacle point stored in the obstacle map, and an alignment angle calculation unit 430 that calculates an alignment angle at which a vehicle entering the parking space should be aligned so as to avoid the obstacle and park.

[0094] The avoidance reference point determination unit 410 sets a vehicle part where contact with an adjacent obstacle should be avoided when entering a parking space as a vehicle reference point, and selects, from among the obstacle points stored on the obstacle map, the coordinates of the obstacle that is the shortest distance from the vehicle reference point as the avoidance reference point.

[0095] To this end, the avoidance reference point determination unit 410 sets the left edge of the vehicle's bumper as the first vehicle reference point P1, the center edge of the vehicle's bumper as the second vehicle reference point P2, and the right edge of the vehicle's bumper as the third vehicle reference point P3, as shown in Fig. 11. In this case, when parking from the front, the vehicle reference point is set based on the front bumper of the vehicle, but when parking from the rear, the vehicle reference point is set based on the rear bumper of the vehicle.

[0096] The avoidance reference point determination unit 410 also divides and sets first and second reference point regions of interest (ROIs) in the +y-axis direction on one side and the -y-axis direction on the other side around the x-axis of a local coordinate system based on the center of the vehicle. The first and second reference point regions of interest (ROIs) are regions for selecting avoidance reference points, which must be used to avoid contact with obstacles on the left and right sides of the vehicle, from the coordinates of each obstacle.

[0097] The avoidance reference point determination unit 410 selects, from among the obstacle points in each reference point region of interest, a point that is the shortest distance from the vehicle reference point as the avoidance reference point in each reference point region of interest.

[0098] To this end, the avoidance reference point determination unit 410 calculates the distance between the vehicle reference point and the obstacle point in each reference point interest area using a cost function as shown in Equation 5 below, and selects the point with the smallest distance as the avoidance reference point.

[0099] [Formula 5] TIFF0007739142000007.tif14128

[0100] In Equation 5, α is a weighting factor, and the closer α is to 0, the smaller the x-axis coordinate item, so weight is applied to the y-axis direction. The closer α is to 1, the smaller the y-axis coordinate item, so weight is applied to the x-axis direction. Pbase , y Pbase is the coordinate of the vehicle reference point on the local coordinate system based on the center of the vehicle, and x Pi , y Pi are the coordinates of each point relative to the obstacle stored on the obstacle map.

[0101] This allows P base When P1 is the first vehicle reference point, the avoidance reference point selected by Equation 5 is the first avoidance reference point P in the +y-axis direction in the first reference point region of interest. y+ And P base When P3 is the third vehicle reference point, the avoidance reference point selected by Equation 5 is the second avoidance reference point P in the -y-axis direction in the second reference point region of interest. y- In FIG. 11, the avoidance reference point selected in this way is displayed as a hollow circle so that it can be distinguished from the coordinates of other points of the obstacle that do not become avoidance reference points at the current position of the vehicle.

[0102] In addition, the parking assistance module 400 further includes a weight varying unit 420 that applies a weighting coefficient applied in selecting the avoidance reference point by the avoidance reference point determining unit 410 by increasing the weight in the y-axis direction as the distance from the vehicle to the obstacle decreases.

[0103] As a result, as shown in FIG. 12(d), the x-axis distance between the vehicle and the obstacle is a constant reference distance d c If the distance is greater than α, the weighting coefficient α is set to the maximum value α c and the reference distance d c If the distance is closer than , the weighting coefficient α is set to the minimum value α0, and the avoidance reference point is determined by the cost function shown in Equation 5. In this case, the reference distance d c The minimum value of the weighting coefficient α0 and the maximum value of the weighting coefficient α c is a tunable parameter.

[0104] As a result, as shown in (a) to (c) of FIG. 12, the x-axis distance d between the vehicle and the obstacle is equal to the reference distance d c If the x-axis distance d between the vehicle and the obstacle is larger than the reference distance d, the point at the coordinate that is closer to the vehicle reference point will be weighted higher and selected as the avoidance reference point. c If the distance is smaller than the x-axis distance, the point at the coordinate closer to the y-axis distance is weighted higher and selected as the avoidance reference point.

[0105] In this way, the weight variable unit 420 increases the weight in the y-axis direction to control the alignment and separation of the vehicle entering the parking space between the obstacles as the distance between the vehicle and the obstacle decreases, thereby improving the collision avoidance performance against the obstacle.

[0106] In addition, the alignment angle calculation unit 430 calculates the average of the angle between the second vehicle reference point set at the center of the front of the vehicle and the avoidance reference point as an initial alignment angle for the vehicle to avoid the obstacle and enter the parking space, calculates the alignment angle change amount that must be increased or decreased for avoidance steering depending on the degree of vehicle entry into the parking space, and provides it as data for avoidance steering control of the vehicle.

[0107] Before entering the parking space, as shown in FIG. 15(a), the alignment angle calculation unit 430 calculates the average of the angles formed by the second vehicle reference point set at the center of the front of the vehicle, the first avoidance reference point (shown as a hollow circle) selected from the obstacle located on the left side of the vehicle, and the second avoidance reference point (shown as a hollow circle) selected from the obstacle located on the right side of the vehicle as an initial alignment angle δ k It is calculated as follows.

[0108] The initial alignment angle calculated in this manner is a rotation angle from the x-axis on the local coordinate system based on the center of the vehicle, as shown in (a) of Figure 15, and is the current steering angle of the vehicle entering the parking space.

[0109] Furthermore, as shown in FIG. 15(b), the alignment angle calculation unit 430 calculates the current initial alignment angle δ k An avoidance direction ROI is set by a first avoidance reference line and a second avoidance reference line, which are lines l1 and l3 that pass through each avoidance reference point and are parallel to the center reference line, which is a line indicating the center line.

[0110] As a result, the area between the first avoidance reference line that passes through the first avoidance reference point and the center reference line is set as the first avoidance area of ​​interest, and the area between the second avoidance reference line that passes through the second avoidance reference point and the center reference line is set as the second avoidance area of ​​interest.

[0111] In addition, the alignment angle calculation unit 430 calculates the sum of the distances between the points of the obstacles in the first region of interest for avoidance and the first avoidance reference line and the sum of the distances between the points of the obstacles in the second region of interest for avoidance and the second avoidance reference line using the Euclidean norm as shown in Equation 6 below, and then calculates the difference between the sums of the two distances as a cost, and calculates an alignment angle change amount for correcting the initial alignment angle so that this cost is minimized.

[0112] [Formula 6] TIFF0007739142000008.tif12128

[0113] In this case, in Equation 6, P1,i are the coordinates of the obstacle points within the first avoidance region of interest between the first regression reference line and the center reference line, and P 3,j are the coordinates of the obstacle points that lie within the second avoidance region of interest between the second regression reference line and the center reference line.

[0114] Also, the alignment angle calculation unit 430 calculates the alignment angle change amount Δδ using the proportional relationship shown in the graph of FIG. 13 in order to speed up the convergence of the cost calculated by Equation 6. k Calculate.

[0115] Therefore, as the vehicle enters the parking space, the first and second avoidance reference points change, which changes the first and second avoidance regions of interest, and the obstacle points within each avoidance region of interest also change. As the obstacle points within each avoidance region of interest change, the alignment angle change amount calculated by Equation 6 changes, and steering control is performed based on the alignment angle required for obstacle avoidance control at the changed position.

[0116] 13 are tuning parameters, which are set differently depending on the convergence speed and degree of convergence to be achieved when calculating the alignment angle change amount in alignment angle calculation unit 430. That is, when a fast convergence speed is desired, the value of b is increased so that a large alignment angle change amount is calculated even with a small change in cost, and when precise control of the degree of convergence is desired, the value of a is increased so that a precise alignment angle change amount is calculated according to a change in cost.

[0117] After calculating the alignment angle change amount as described above, the alignment angle calculation unit 430 calculates the current alignment angle by reflecting the alignment angle change amount in the previous alignment angle as shown in the following Equation 7. As a result, the current alignment angle is continuously calculated, reflecting the alignment angle change amount calculated while the vehicle is entering the parking space, from the initial alignment angle calculated at the beginning of entering the parking space.

[0118] [Formula 7] TIFF0007739142000009.tif12128

[0119] The alignment angle calculation unit 430 calculates the cost and the alignment angle change amount Δδ k To improve the accuracy of the calculation, the current alignment angle δ k The process of calculating the alignment angle change amount and calculating a new alignment angle is repeated a predetermined number of times (in FIG. 14, the repeated calculation recovery K is set to 10 times), and then the result is output.

[0120] In addition, when the alignment angle calculation unit 430 receives sensing data (TOF) from the side ultrasonic sensor provided on the vehicle a predetermined number of times or more (for example, five times or more) in succession, it determines that the vehicle has entered a parking space between obstacles (parked vehicles or pillars or walls of a parking lot).

[0121] After determining that the vehicle has entered the parking space, the alignment angle calculation unit 430 selects an obstacle to which the avoidance reference point in the +y-axis direction area (i.e., the left side of the vehicle in FIG. 16) and the avoidance reference point in the -y-axis direction area (i.e., the right side of the vehicle in FIG. 16) belong as a target obstacle (Target Object).

[0122] In this case, the target obstacle is selected as the obstacle that is closest in distance in the y-axis direction among the obstacles that exist in the first and second regions of interest for avoidance.

[0123] In addition, after the vehicle enters the parking space, the vehicle must be aligned in the parking space between the obstacles for safe parking. Therefore, the alignment angle calculation unit 430 calculates the avoidance reference point determined when the vehicle enters the parking space as the space entry avoidance reference point (P target1 , P target3 ) is selected.

[0124] At this time, the alignment angle calculation unit 430 calculates the space entry avoidance reference point (P target1 , P target3A new avoidance reference line (l1, l3) expressing the coordinates of a point whose x-axis coordinate value is greater than the x-coordinate of the reference line (l1, l3) is calculated by the least squares method as shown in Equation 8 below.

[0125] [Formula 8] TIFF0007739142000010.tif39128

[0126] 17, avoidance reference lines l1 (y=a1x+b1) and l3 (y=a3x+b3) are calculated using Equation 8. Next, the alignment angle calculation unit 430 uses the average gradient of the two straight lines to calculate a new alignment angle δ that guides the vehicle so as not to come into contact with obstacle points, including the space entry avoidance reference point, as shown in Equation 9 below, and by steering and controlling the vehicle according to this alignment angle, the vehicle is steered to avoid the obstacle and performs autonomous parking.

[0127] In addition, the parking assistance module 400 further includes an alignment target point calculation unit 440 that calculates an alignment target point for guiding a vehicle entering a parking space to complete parking using the following Equation 9, and provides the calculated point as a guidance point for completing parking.

[0128] [Formula 9] TIFF0007739142000011.tif17128

[0129] For this purpose, the alignment target point calculation unit 440 calculates the first avoidance reference point P , which is selected by the alignment angle calculation unit 430 after the vehicle enters the parking space, as shown in FIG. 18 and Equation 9. y+ and the second avoidance reference point P y- The coordinates of the position corresponding to the average of the x and y coordinate values ​​of the vehicle are calculated and presented as the alignment target point where the vehicle should stop.

[0130] Next, a parking assistance method with improved avoidance steering control according to one embodiment of the present invention will be described with reference to FIG.

[0131] FIG. 19 is a block diagram of a parking assistance method with improved avoidance steering control according to an embodiment of the present invention.

[0132] Referring to FIG. 19, a parking assistance method with improved avoidance steering control according to an embodiment of the present invention includes an image data acquisition step (S100) of acquiring an image of an obstacle around a parking space using image data of the front, rear, and side of the vehicle captured by an SVM camera, a sensing data acquisition step (S200) of detecting the position of the obstacle and the distance to the obstacle using sensing data acquired by a sensor module, an obstacle map generation step (S300) of storing the coordinates of the obstacle recognized from the image data and the position corrected using the sensing data in an obstacle map, and a parking assistance step (not shown) of selecting an avoidance reference point from the coordinates of points constituting the obstacle, at which the vehicle is required to avoid the obstacle, and calculating an alignment angle of the vehicle required for avoidance steering based on the avoidance reference point.

[0133] In the video data acquisition step (S100), an SVM (Surround View Monitor) camera installed in the vehicle is used to capture images of the front, rear, and both sides of the vehicle, and video data is acquired that can be used to determine whether there are any obstacles around the parking space the vehicle is entering and the direction in which the obstacles are located relative to the vehicle.

[0134] That is, in the image data acquisition step (S100), image data is acquired using an SVM camera consisting of four wide-angle cameras installed at the front, rear, left and right sides of the vehicle, thereby acquiring image data covering a wider detection range.

[0135] In addition, in the sensing data acquisition step (S200), if the sensor module is configured with an ultrasonic sensor, the time of flight (TOF) and direction of the ultrasonic signal emitted from the ultrasonic sensor installed in the vehicle and reflected back by a surrounding obstacle are recognized, and the presence or absence of the obstacle and the distance to the obstacle are detected.

[0136] In addition, the obstacle map generation step (S300) recognizes the presence and location of obstacles around the parking space based on the video data, and then matches the coordinates of the points where the obstacles exist in a local coordinate system based on the current position of the vehicle with the classification information of the obstacles and stores them to generate an obstacle map showing the current distribution of obstacles in the parking space.

[0137] To this end, in the obstacle map generation step (S300), the front, rear, and both side areas of the vehicle shown in the image data captured by the SVM camera are divided into a number of equally spaced cells, and an address is assigned to each cell.

[0138] In addition, in the obstacle map generation step (S300), the presence or absence of obstacles around the vehicle is determined based on the image shown in the video data, and the recognized image of the obstacle is compared with the obstacle image in the obstacle classification table stored in advance to recognize the type of obstacle.

[0139] In this case, when recognizing an obstacle, the presence or absence of a point showing an obstacle image is determined for each cell, and if the classification code of an obstacle recognized in each cell is the same as the classification code of an obstacle recognized in another adjacent cell, it is determined that the image of the same obstacle is recognized in multiple cells.

[0140] In this way, in the obstacle map generation step (S300), it is determined whether an image of an obstacle can be recognized for each divided cell, and whether the image can be recognized continuously in multiple cells, and the outer shape of the obstacle is grasped.

[0141] In addition, the points on the outer side where obstacles are recognized in each cell are identified, and the coordinates of each point where an obstacle is recognized in a local coordinate system based on the current position of the vehicle are stored as position information to generate an obstacle map.

[0142] When recognizing obstacles from video data to generate an obstacle map, obstacles in the region of interest (ROI), which is an area captured by overlapping front / rear video data and side video data, are recognized in both sets of video data.

[0143] Accordingly, the method further includes a boundary obstacle identification step (S400) for determining whether a boundary obstacle recognized in a region of interest (ROI) where the front / rear image data and the side image data overlap is the same as any obstacle recognized in at least one of the front / rear image data and the side image data, and identifying the obstacle.

[0144] The boundary obstacle identification step (S400) includes a boundary obstacle selection process (S410) for selecting an object whose image is recognized in a region of interest (ROI) from the front / rear image data and the side image data as a boundary obstacle, a centroid calculation process (S420) for calculating a first centroid and a second centroid, which are the plane centers of the point coordinates of the boundary obstacle recognized in the front / rear image data and the side image data, respectively, and a boundary obstacle determination process (S430) for comparing the distances between the first centroid and the second centroid calculated for the boundary calculation object and identifying two boundary obstacles with the smallest distance as the same obstacle.

[0145] In this case, in the boundary obstacle selection process (S410), the front right and left areas of the vehicle and the rear right and left areas of the vehicle are set as regions of interest (ROI), and obstacles recognized in the front / rear image data and the side image data at positions or cells corresponding to the ROI are selected as boundary obstacles.

[0146] In addition, in the centroid calculation process (S420), a first centroid is calculated using the point coordinates of boundary obstacles recognized in the positions or cells corresponding to the area of ​​interest in the front and rear image data, and a second centroid is calculated using the point coordinates of boundary obstacles recognized in the positions or cells corresponding to the area of ​​interest in the side image data.

[0147] At this time, if only a part of the obstacle is recognized in the front / rear image data or the side image data, it goes without saying that the centroid of only the recognized point is calculated.

[0148] In addition, in the boundary obstacle determination step (S430), boundary obstacles in the front and rear image data and boundary obstacles in the side image data within each region of interest that have the same obstacle classification code are selected as targets for comparing the distance of the centroid.

[0149] Next, in the boundary obstacle determination process (S430), the distance between the first and second centroids calculated for each boundary obstacle selected as a comparison target is calculated using the Euclidean norm, and then the two boundary obstacles with the smallest distance are determined to be the same obstacle.

[0150] In this way, when an obstacle recognized by the front / rear image data and the side image data is identified, if the sensor module is configured with an ultrasonic sensor, the method further includes an obstacle position correction step of correcting the position of the obstacle based on sensing data consisting of the time of flight (TOF) of the ultrasonic signal emitted from the ultrasonic sensor, reflected by the obstacle, and returning, and storing the corrected position in an obstacle map.

[0151] To this end, the obstacle position correction step (S500) includes a reference coordinate correction process (S510) in which a point of the obstacle closest to the ultrasonic sensor is selected as a reference coordinate, and the reference coordinate is translated until it touches a circle having the ultrasonic sensor as its center and the return path of the sensing data (TOF) relative to the reference coordinate as its radius, thereby generating a corrected coordinate of the obstacle; and an obstacle point correction process (S520) in which the coordinates of the remaining points of the obstacle are translated in the same direction as the reference coordinate moved to the corrected coordinate.

[0152] At this time, in the reference coordinate correction step (S510), the coordinates that match the point of the obstacle measured by the ultrasonic sensor with the shortest time of flight are set as the reference coordinates.

[0153] In the reference coordinate correction process (S510), in order to move the reference coordinate to the position detected by the ultrasonic sensor, the reference coordinate is calculated by moving the reference coordinate from the current coordinate (the coordinate on the obstacle map calculated based on the image data) in the direction toward the installation position of the SVM camera, thereby calculating the corrected coordinate.

[0154] In addition, in the obstacle point correction step (S520), the remaining coordinates of the obstacle stored in the obstacle map are translated along the vector direction in which the reference coordinates are moved to the correction coordinates, thereby correcting the position of the obstacle.

[0155] This allows the system to check the presence and location of obstacles over a wide area based on video data, then identify the coordinate position of the closest point using the ultrasonic sensor's sensing data (TOF), translate the remaining points of the obstacle, correct the final position of the obstacle, and store it in an obstacle map, thereby obtaining accurate information on the presence or absence of obstacles around the parking space and the distance to each obstacle.

[0156] The parking assistance step also includes an avoidance reference point determination step (S600) for selecting an obstacle point that the vehicle should avoid as an avoidance reference point based on the distance relationship between the obstacle point coordinates stored in the obstacle map and the vehicle reference point entering the parking space, and an alignment angle calculation step (S700) for calculating an alignment angle required to steer the vehicle so that the vehicle reference point avoids the avoidance reference point and enters the parking space.

[0157] In this case, the avoidance reference point determination step (S600) includes a reference point region of interest setting process (S610) for setting predetermined regions in the +y-axis direction on one side and the -y-axis direction on the other side as a first reference point region of interest and a second reference point region of interest, respectively, centered on the x-axis on a local coordinate system based on the center of the vehicle; and an avoidance reference point selection process (S620) for setting a vehicle part where contact with an adjacent obstacle should be avoided when entering the parking space as a vehicle reference point, and selecting, as an avoidance reference point, the coordinates of the obstacle that is the shortest distance from the vehicle reference point among the points of the obstacles in the first reference point region of interest and the second reference point region of interest.

[0158] The first and second reference point regions of interest set in the reference point region of interest setting step (S610) are set as predetermined regions in the +y-axis and -y-axis directions on an arbitrary local coordinate system with the center of the vehicle as the x-axis. That is, they are set as predetermined ranges where collisions of the vehicle reference points set at both ends of the vehicle are expected when the vehicle enters a parking space between obstacles.

[0159] The first and second reference point regions of interest thus set become regions for determining preliminary points to be selected as avoidance reference points.

[0160] In the avoidance reference point selection step (S620), the left edge of the vehicle's bumper is set as the first vehicle reference point, the center edge of the vehicle's bumper is set as the second vehicle reference point, and the right edge of the vehicle's bumper is set as the third vehicle reference point. The first to third vehicle reference points thus set are assigned coordinates on a local coordinate system based on the center of the vehicle.

[0161] Thereafter, in the avoidance reference point selection step (S620), the distance between the vehicle reference point and the obstacle point in each reference point interest area is calculated, and the point with the shortest distance is selected as the avoidance reference point.

[0162] In this case, when selecting the point with the smallest distance from the vehicle reference point among a number of obstacle points in the reference point interest area, the method further includes a weight varying step (S630) of increasing and applying a weight in the y-axis direction as the distance between the vehicle entering the parking space and the obstacle decreases.

[0163] As a result, if the x-axis distance between the vehicle and the obstacle is greater than a preset reference distance, a point at a coordinate that is close to the vehicle reference point on the x-axis is selected with a high weight as the avoidance reference point, and if the x-axis distance between the vehicle and the obstacle is smaller than the reference distance, a point at a coordinate that is close to the vehicle reference point on the y-axis is selected with a high weight as the avoidance reference point.

[0164] In addition, the alignment angle calculation step (S700) includes an initial alignment angle calculation process (S710) for calculating the average of the angles formed by the vehicle reference point set at the center of the front of the vehicle and the avoidance reference point as an initial alignment angle required for the vehicle to avoid the obstacle and enter the parking space; an avoidance interest region setting process (S720) for setting an avoidance interest region using avoidance reference lines that are straight lines passing through each avoidance reference point in parallel with the center reference line that is a straight line indicating the initial alignment angle; and an alignment angle change amount calculation process (S730) for calculating an alignment angle change amount that must be increased or decreased for obstacle avoidance steering depending on the degree of entry of the vehicle into the parking space, and providing the result as data for vehicle avoidance steering control.

[0165] In this case, in the initial alignment angle calculation step (S710), the average of the angles formed by the second vehicle reference point set at the front center of the vehicle, the first avoidance reference point selected from the obstacle located on the left side of the vehicle, and the second avoidance reference point selected from the obstacle located on the right side of the vehicle is calculated as the initial alignment angle. Steering control is performed so that the vehicle enters the parking space along the calculated initial alignment angle.

[0166] In addition, in the step of setting the region of interest for avoidance (S720), the region of interest for avoidance is set by a first avoidance reference line, which is a line that passes through the first avoidance reference point parallel to the center reference line, which is a line that indicates the current initial alignment angle, and a second avoidance reference line, which is a line that passes through the second avoidance reference point parallel to the center reference line.

[0167] As a result, the area between the first avoidance reference line that passes through the first avoidance reference point and the center reference line is set as the first avoidance area of ​​interest, and the area between the second avoidance reference line that passes through the second avoidance reference point and the center reference line is set as the second avoidance area of ​​interest.

[0168] The first and second avoidance regions of interest provide regions from which obstacle points are selected for calculating the vehicle alignment angle change amount, i.e., only points within the first and second avoidance regions of interest are used in cost calculation for calculating the alignment angle change amount.

[0169] In addition, in the alignment angle change amount calculation process (S730), the sum of the distances between the obstacle points in the first avoidance region of interest and the first avoidance reference line and the sum of the distances between the obstacle points in the second avoidance region of interest and the second avoidance reference line are calculated using the Euclidean norm, and then an alignment angle change amount is calculated to correct the initial alignment angle so that the cost, which is the difference between the sums of the two distances, is minimized.

[0170] That is, as the vehicle enters the parking space, the first and second avoidance reference points change, and therefore the first and second avoidance reference lines change, and the first and second avoidance regions of interest also change.

[0171] This causes the obstacle points within each avoidance region of interest to change, and the alignment angle change amount calculated based on the distance between such obstacle points and each avoidance reference line also changes.

[0172] By controlling the steering so that the calculated alignment angle change amount is reflected in the current alignment angle of the vehicle, the vehicle does not collide with an obstacle having the point selected as the avoidance reference point while entering the parking space, and the vehicle is safely parked in a state where the difference in distance from both avoidance reference points or both avoidance reference lines is kept relatively equal (i.e., the cost calculated by Equation 6 is minimized).

[0173] In this case, in the alignment angle change amount calculation step (S730), as shown in FIG. 14, after setting an avoidance region of interest using the current alignment angle of the vehicle, a cost is calculated using the coordinates of points within each avoidance region of interest, and the alignment angle change amount that minimizes the cost is calculated to update the current alignment angle. This process is repeated a predetermined number of times (in FIG. 14, the number of repeated calculations K is set to 10 times, but it is not limited to this specific number and can be increased or decreased), and the result is output as an updated alignment angle.

[0174] In addition, the alignment angle calculation step (S700) further includes an alignment target point calculation step (S740) in which, when it is determined that the vehicle has entered a parking space between obstacles, each obstacle having each avoidance reference point as a point is selected as a target obstacle (Target Object) that is the final avoidance target, and a position corresponding to the average of the coordinate values ​​of the two avoidance reference points is calculated as an alignment target point where the vehicle should stop between the two obstacles.

[0175] For this purpose, in the alignment target point calculation process (S740), when it is determined that the vehicle has entered a parking space between obstacles, each obstacle having an avoidance reference point in the +y-axis direction area and an avoidance reference point in the -y-axis direction area as points, based on the center of the vehicle, is selected as a target obstacle (Target Object) that is the final object to be avoided.

[0176] In this case, in the alignment target point calculation process (S740), the avoidance reference point determined when the vehicle enters the parking space is selected as the space entry avoidance reference point, and a new avoidance reference line expressing the coordinates of the point of the target obstacle whose x-axis coordinate value is greater than the x-coordinate of the space entry avoidance reference point is calculated by the least squares method.

[0177] In addition, in the lining up target point calculation step (S740), the lining up angle of the vehicle is controlled by steering based on the newly calculated avoidance reference line, and the vehicle is guided to the lining up target point to perform parking.

[0178] In addition, the parking assistance step (S700) further includes an avoidance path generation step (not shown) that generates and provides an avoidance path that avoids obstacles and guides the vehicle into a parking space based on the updated alignment angle that reflects the alignment angle change amount calculated in the alignment angle calculation step, and a steering control command generation step (not shown) that generates a steering control command required to travel along the avoidance path and transmits it to the steering device, thereby performing autonomous parking in a parking space while stably avoiding collision with obstacles.

[0179] The above description is merely an illustrative example of the technical concept of the present invention, and various modifications and variations are possible by a person having ordinary knowledge in the technical field to which the present invention pertains, without departing from the essential characteristics of the present invention.

[0180] Therefore, the embodiments disclosed in this specification are for illustrative purposes only and are not intended to limit the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by such embodiments. [Explanation of symbols]

[0181] 100 Sensor Module 200 SVM cameras 300 Sensor Fusion Computing Module 310 Cell setting unit 320 Obstacle Recognition Unit 321 Obstacle Classification Table 330 Boundary Obstacle Identification Department 340 Obstacle map generation unit 350 Obstacle position correction unit 360 Obstacle Map 400 Parking Assist Module 410 Avoidance reference point determination section 420 Weight variable part 430 Alignment angle calculation unit 440 Alignment target point calculation unit

Claims

1. a sensor module that detects the distance from the vehicle to an obstacle based on sensing data acquired by searching the periphery of the vehicle; a Surround View Monitor (SVM) camera that captures images of the vehicle's surroundings and acquires image data that can detect the position and direction of obstacles; a sensor fusion calculation module that combines the sensing data of the sensor module and the image data of the SVM camera to calculate position information of an obstacle based on the current position of the vehicle; a parking assistance module that performs steering control of the vehicle to avoid obstacles and park autonomously based on the position information, The sensor fusion calculation module The vehicle navigation system further includes a boundary obstacle identification unit that identifies a boundary obstacle recognized in a region of interest (ROI) where the front / rear image data and the side image data overlap by determining whether the boundary obstacle is identical to any of obstacles recognized in at least one of the front / rear image data and the side image data, The boundary obstacle identification unit a first centroid of the boundary obstacle image recognized in the front and rear image data and a second centroid of the boundary obstacle image recognized in the side image data, and determines the boundary obstacle having the smallest distance between the first centroid and the second centroid, which are calculated for boundary obstacles having the same obstacle classification code, as the same obstacle.

2. The sensor fusion calculation module a cell setting unit that divides the area around the vehicle on the image data acquired by the SVM camera into cells at regular intervals and assigns an address to each cell; an obstacle recognition unit that detects the presence or absence of an obstacle based on the video data, classifies the detected obstacle into types, and recognizes the obstacles present within the parking area; and an obstacle map generating unit that matches and stores cell addresses of positions where obstacles are recognized in the image data with classification information of the obstacles, and generates an obstacle map showing the current obstacle distribution status for the parking area.

3. The sensor module includes: It is composed of an ultrasonic sensor that acquires sensing data by ultrasonic time of flight (TOF), The sensor fusion calculation module 3. The parking assistance system with improved avoidance steering control according to claim 2, further comprising an obstacle position correction unit that corrects and stores the positions of the obstacles recognized from the video data and stored in the obstacle map based on sensing data from the ultrasonic sensor.

4. The obstacle position correction unit 4. The parking assist system with improved avoidance steering control according to claim 3, wherein a point of the obstacle closest to the ultrasonic sensor is selected as a reference coordinate, and then the reference coordinate is moved on the same cell until it touches a circle having the ultrasonic sensor as its center and a radius equal to a regression path of sensing data (TOF) relative to the reference coordinate, thereby setting the reference coordinate as a corrected coordinate of the obstacle, and the coordinates of the remaining points of the obstacle are similarly moved parallel to the direction in which the reference coordinate has moved to the corrected coordinate, thereby correcting the positions of the obstacle.

5. The obstacle position correction unit 4. The parking assistance system with improved avoidance steering control according to claim 3, wherein, when a plurality of sensing data (TOF) are received for one obstacle, the position of the obstacle is corrected using each of the sensing data (TOF), and then the average of the coordinates indicating the corrected position of the obstacle is calculated, and the calculated average is corrected as the final position of the obstacle.

6. The parking assistance module an avoidance reference point determination unit that selects, as an avoidance reference point, an obstacle point that should be avoided when entering the parking space, based on coordinates indicating the obstacle point stored in the obstacle map; 4. The parking assistance system with improved avoidance steering control according to claim 3, further comprising: an alignment angle calculation unit that calculates an alignment angle at which a vehicle entering a parking space should be aligned so that the vehicle can avoid an obstacle and park.

7. The avoidance reference point determination unit 7. The parking assistance system with improved avoidance steering control according to claim 6, wherein a vehicle portion where contact with an adjacent obstacle should be avoided when entering a parking space is set as a vehicle reference point, and predetermined regions in the +y-axis direction on one side of the x-axis and the −y-axis direction on the other side on a local coordinate system based on the center of the vehicle are set as first and second reference point regions of interest, and then a point of obstacle points in each reference point region of interest that is the shortest distance from the vehicle reference point is selected as the avoidance reference point in each reference point region of interest.

8. The parking assistance module 10. The parking assistance system with improved avoidance steering control according to claim 7, further comprising a weight varying unit that applies a weight in a y-axis direction that increases as the distance from the vehicle to the obstacle decreases, so that when the x-axis distance between the vehicle and the obstacle is greater than a predetermined reference distance, a point at a coordinate that is close to the vehicle reference point on the x-axis is assigned a high weight and selected as the avoidance reference point, and when the x-axis distance between the vehicle and the obstacle is smaller than the reference distance, a point at a coordinate that is close to the vehicle reference point on the y-axis is assigned a high weight and selected as the avoidance reference point.

9. The alignment angle calculation unit 7. The parking assistance system with improved avoidance steering control according to claim 6, wherein the average of the angle formed by the vehicle reference point set at the center of the front of the vehicle and the avoidance reference point is calculated as an initial alignment angle of the vehicle entering the parking space, and an alignment angle change amount that must be increased or decreased for avoidance steering depending on the degree of entry of the vehicle into the parking space is calculated and provided as data for avoidance steering control of the vehicle.

10. The alignment angle calculation unit 7. The parking assist system with improved avoidance steering control according to claim 6, wherein a first avoidance reference line and a second avoidance reference line are straight lines that pass through each avoidance reference point and are parallel to a center reference line that is a straight line indicating a current initial alignment angle, and the area between the first avoidance reference line and the center reference line is set as a first avoidance region of interest, and the area between the second avoidance reference line and the center reference line is set as a second avoidance region of interest, and the sum of the distances between an obstacle point in the first avoidance region of interest and the first avoidance reference line and the sum of the distances between an obstacle point in the second avoidance region of interest and the second avoidance reference line are calculated, and then an alignment angle change amount is calculated to correct the initial alignment angle so that a difference between the sums of the two distances becomes minimum.

11. an image data acquisition step of acquiring an image of an obstacle around the parking space from front / rear image data and side image data of the vehicle captured by an SVM (Surround View Monitor) camera; a sensing data acquisition step of detecting the position of an obstacle and the distance to the obstacle based on sensing data acquired by the sensor module; an obstacle map generating step of storing, in an obstacle map, coordinates of obstacles recognized from the front / rear image data and the side image data and whose positions are corrected by the sensing data; a parking assistance step of selecting an avoidance reference point from among the coordinates of the points constituting the obstacle, at which an avoidance of the vehicle is required, and calculating an alignment angle of the vehicle required for avoidance steering based on the avoidance reference point, and a boundary obstacle identifying step of determining whether a boundary obstacle recognized in a region of interest (ROI) where the front / rear image data and the side image data overlap is the same as any obstacle recognized in at least one of the front / rear image data and the side image data, when generating the obstacle map by receiving the front / rear image data, the side image data, and the sensing data (TOF: Time of Flight), and identifying the obstacle; The boundary obstacle identification step includes: a boundary obstacle selection step of selecting an object whose image is recognized in the region of interest (ROI) from the front / rear image data and the side image data as a boundary obstacle; a centroid calculation step of calculating a first centroid and a second centroid, which are plane centers of point coordinates of the boundary obstacle recognized in the front / rear image data and the side image data, respectively; a boundary obstacle determination process for comparing the distances between the first and second centroids calculated for boundary calculation objects that match the same obstacle classification code, and identifying two boundary obstacles having the smallest distance as the same obstacle.

12. a reference coordinate correction step for generating the obstacle map, in which a point of the obstacle closest to the sensor module is selected as a reference coordinate, and the reference coordinate is moved in a direction toward an installation position of the SVM camera until the reference coordinate is tangent to a circle having the sensor module as a center and a radius equal to a regression path of sensing data relative to the reference coordinate, thereby generating corrected coordinates of the obstacle; and an obstacle point correction step of similarly translating the coordinates of the remaining points of the obstacle along the direction in which the reference coordinates have moved to the corrected coordinates.

13. The parking assistance step includes: an avoidance reference point determination step of selecting, as an avoidance reference point, a point on the obstacle that the vehicle should avoid based on a distance relationship between the point coordinates of the obstacle stored in the obstacle map and a vehicle reference point that is set as a vehicle portion that the vehicle entering the parking space should avoid contact with the obstacle; and calculating an alignment angle required to steer the vehicle so that the vehicle reference point avoids the avoidance reference point and enters the parking space.

14. The avoidance reference point determination step includes: a reference point region of interest setting step of setting a predetermined region in a +y-axis direction on one side of an x-axis and a predetermined region in a −y-axis direction on the other side of the x-axis on a local coordinate system based on the center of the vehicle as a first reference point region of interest and a second reference point region of interest; and selecting, as an avoidance reference point, coordinates of an obstacle that has a minimum distance from the vehicle reference point among points of the obstacles in the first and second reference point regions of interest.

15. The avoidance reference point determination step includes:

14. The parking assistance method with improved avoidance steering control as claimed in claim 13, further comprising a weight varying process for increasing a weight in a y-axis direction as a distance between the vehicle entering the parking space and the obstacle decreases when selecting a point having a minimum distance from the vehicle reference point among a plurality of obstacle points in a reference point interest area.

16. The alignment angle calculation step includes: an initial alignment angle calculation step of calculating an average angle between a vehicle reference point set at the front center of the vehicle and an avoidance reference point as an initial alignment angle required for the vehicle to avoid the obstacle and enter the parking space; an avoidance region of interest setting step of setting a region between the first avoidance reference line and the center reference line as a first avoidance region of interest and a region between the second avoidance reference line and the center reference line as a second avoidance region of interest, using a first avoidance reference line and a second avoidance reference line which are straight lines passing through each avoidance reference point in parallel with the center reference line which is a straight line indicating the initial alignment angle; 14. The parking assist method with improved avoidance steering control according to claim 13, further comprising: calculating a sum of distances between a point of an obstacle in the first region of interest for avoidance and a first avoidance reference line and a sum of distances between a point of an obstacle in the second region of interest for avoidance and a second avoidance reference line, and then calculating an alignment angle change amount for correcting the initial alignment angle so that a difference between the sums of the two distances becomes minimum.

Images