Vehicle position estimation device, vehicle position estimation method, and computer-readable recording medium storing a computer program programmed to execute the method

By matching vehicle cameras with precise maps and using Gaussian probability models, the problem of vehicle position estimation when GPS signals are blocked is solved, achieving high-precision vehicle positioning in complex environments and reducing costs.

CN112166059BActive Publication Date: 2024-11-01SK TELECOM CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN201880093798.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-05-25
Filing Date
2018-11-02
Publication Date
2024-11-01
Estimated Expiration
2038-11-02

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately estimate vehicle location in complex environments such as urban centers filled with high-rise buildings, especially when GPS signals are blocked, which affects the accuracy of advanced driver assistance systems.

Method used

By matching images captured by vehicle cameras with landmarks on a precise map, and combining a Gaussian probability model and a particle filter, accurate location information of the camera and vehicle is obtained. The vehicle's position is then accurately estimated using the camera's initial pose and position information.

Benefits of technology

It improves the accuracy and speed of vehicle position estimation without relying on lidar, reduces manufacturing costs, and maintains high precision in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112166059B_ABST
    Figure CN112166059B_ABST
Patent Text Reader

Abstract

According to one embodiment, a method for estimating the position of a vehicle includes: matching a landmark on a precision map corresponding to the vehicle's GPS-based initial position information with an image captured by a camera of the vehicle to obtain the landmark-based initial position information of the camera; and obtaining the estimated position information of the vehicle by using a comparison result between the landmark on the precision map corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera and the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a vehicle position estimation device, a vehicle position estimation method, and a computer-readable recording medium storing a computer program programmed to execute the method. For reference, this application claims priority based on Korean Patent Application No. 10-2018-0059626 filed on May 25, 2017. The entire contents of the corresponding application serving as the basis for the priority are cited as references in this application. Background Art

[0002] Generally speaking, a vehicle refers to a transportation device that uses fossil fuels, electricity, etc. as a power source to travel on roads or lines.

[0003] With the development of technology, vehicles have been developed to provide drivers with various functions. In particular, with the trend of electrification of vehicles, vehicles equipped with active safety systems (ASS) have emerged, which are activated immediately before an accident or at the moment of an accident to avoid an accident.

[0004] Furthermore, in order to reduce the burden on drivers and improve convenience, research has been actively conducted on vehicles equipped with advanced driver assistance systems (ADAS) that actively provide information on driving environments such as vehicle conditions, driver conditions, and surrounding environments.

[0005] Advanced driver assistance systems operate according to the driving environment determined by the position of the vehicle, so it is necessary to accurately estimate the position of the vehicle. Currently, the method of estimating the position of the vehicle using satellite signals is widely used, but recently, research has been actively conducted on methods that take various information into consideration at the same time in order to improve accuracy. Summary of the invention

[0006] Technical issues

[0007] The problem to be solved by the present invention is to provide a technique for estimating the position of a vehicle more accurately than the prior art.

[0008] Furthermore, a technology is provided that can more accurately determine the position of a vehicle even when a system such as GPS does not function properly due to high buildings or in a complex driving environment such as a city center.

[0009] Furthermore, the technology disclosed in the present invention is applied to various means of transportation, for example, to vehicles equipped with advanced driver assistance systems.

[0010] It should be noted that the technical problems to be solved by the present invention are not limited to the above-mentioned technical problems, and those skilled in the art in the technical field to which the present invention belongs will clearly understand other technical problems to be solved that are not mentioned through the following description.

[0011] Means of solving the problem

[0012] According to one embodiment, a method for estimating the position of a vehicle includes: matching a landmark of a precision map corresponding to the vehicle's GPS-based initial position information with an image captured by a camera of the vehicle to obtain the landmark-based initial position information of the camera; and obtaining the estimated position information of the vehicle by using a comparison result between a landmark of the precision map corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera and the image.

[0013] According to one embodiment, a computer-readable recording medium storing a computer program programmed to execute a method for estimating a position of a vehicle includes: a step of matching a landmark of a precision map corresponding to the vehicle's GPS-based initial position information with an image captured by a camera of the vehicle to obtain the landmark-based initial position information of the camera; and a step of obtaining the estimated position information of the vehicle by using a comparison result between a landmark of the precision map corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera and the image.

[0014] According to one embodiment, a vehicle position estimation device includes: a camera initial position information acquisition unit, which matches the landmark of the precision map corresponding to the vehicle's GPS-based initial position information with the image taken by the camera of the above-mentioned vehicle to obtain the landmark-based initial position information of the above-mentioned camera; a camera estimated position information acquisition unit, which obtains the estimated position information of the above-mentioned camera by using the comparison result between the landmark of the precision map corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the above-mentioned camera and the above-mentioned image; and a vehicle estimated position information acquisition unit, which obtains the estimated position information of the above-mentioned vehicle based on the estimated position information of the above-mentioned camera.

[0015] Effects of the Invention

[0016] According to one embodiment, a vehicle position estimation device, a vehicle position estimation method, and a computer-readable recording medium storing a computer program programmed to execute the method utilize images acquired through a vehicle's camera and a precision map in addition to satellite signals, thereby being able to estimate a more accurate vehicle position.

[0017] Furthermore, the position of the vehicle can be estimated without using expensive devices such as LiDar, so the manufacturing cost can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1a and 1 b is a control block diagram of a position estimation system for a vehicle according to various embodiments.

[0019] Figure 2 is a diagram for explaining a coordinate system of a vehicle and a camera according to an embodiment.

[0020] Figure 3 It is a flow chart of a method for obtaining landmark-based initial position information of a camera through a camera initial position information obtaining unit according to an embodiment.

[0021] Figure 4 FIG. 1 is a diagram for explaining a method for matching a landmark according to an embodiment.

[0022] Figure 5 It is shown based on Figure 3 Figure 3 shows the results of the landmark-based initial position information acquisition method for the camera.

[0023] Figure 6 The invention is a flow chart of a method for acquiring camera estimated position information through a camera estimated position information acquiring unit according to an embodiment.

[0024] Figure 7 1 is a diagram illustrating an extraction result of a first landmark for a lane line and a second landmark other than the lane line in an image captured by a camera according to an embodiment.

[0025] Figure 8 This is an example of combining the landmarks of a precise map with Figure 7 The result of matching the image.

[0026] Figure 9a and 9b is an example based on Figure 8 Plot of the matching error.

[0027] Fig.10 is a diagram illustrating a final result of a method for acquiring position information based on camera estimation according to an embodiment.

[0028] Fig.11 The invention is a flow chart of a method for acquiring vehicle estimated position information by a vehicle estimated position information acquiring unit according to an embodiment.

[0029] Fig.12 is a diagram illustrating a final landmark matching result based on a method for acquiring vehicle estimated position information according to an embodiment. DETAILED DESCRIPTION

[0030] The advantages and features of the present invention and the methods for implementing these methods can be clearly understood by referring to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can also be implemented in various other ways. This embodiment is only provided to fully disclose the present invention and fully inform the technical personnel of the technical field to which the present invention belongs of the protection scope of the present invention. The present invention should be defined by the claims.

[0031] In the process of describing the embodiments of the present invention, when it is judged that the specific description of the known function or structure may confuse the purpose of the present invention, its detailed description is omitted. In addition, the following terms are defined in view of the functions in the embodiments of the present invention, and may be changed according to the wishes or habits of the user or operator. Therefore, they should be defined based on the overall content of this specification.

[0032] Figure 1a and 1b is a control block diagram of a position estimation system for a vehicle according to various embodiments, Figure 2 is a diagram for explaining a coordinate system of a vehicle and a camera according to an embodiment.

[0033] Reference Figure 1a According to an embodiment, a vehicle position estimation system 1 may be composed of a vehicle V and a vehicle position estimation device 100 .

[0034] The vehicle V may represent a means of transport that travels along a road or line and can move people, objects, animals, etc. from one location to another. According to an embodiment, the vehicle V may include a three-wheeled or four-wheeled vehicle, a two-wheeled vehicle such as a motorcycle, an engineering machine, a motorized bicycle, a bicycle, and a train traveling along a line.

[0035] Figure 1a The vehicle V is equipped with a GPS module, which can receive satellite signals including navigation data from at least one GPS (Global Position System) satellite. The vehicle V can obtain the current position of the vehicle V and the direction of travel of the vehicle V based on the satellite signal.

[0036] and, Figure 1a The vehicle V may store a precision map in advance. The precision map may refer to a map that has high accuracy in order to achieve safe and precise control of the vehicle V and includes information on height, slope, curvature, etc. in addition to the planar position of the road. Furthermore, the precision map may refer to a map that also includes information on road facilities such as lane lines, signs, traffic lights, and guardrails.

[0037] and, Figure 1a The vehicle V may be equipped with an advanced driver assistance system (ADAS). The advanced driver assistance system may refer to a system that provides driving environment information such as the vehicle V state, the driver state, and the surrounding environment information or actively controls the vehicle V in order to reduce the burden on the driver and improve convenience.

[0038] The advanced driving assistance system mounted on the vehicle V may include a sensing means for sensing the driving environment of the vehicle V. The sensing means according to an embodiment may include a radar that irradiates pulses around the vehicle V and receives echo pulses reflected from objects located in the corresponding direction to sense the driving environment, a laser radar (LiDAR) that irradiates lasers around the vehicle V and receives lasers reflected from objects located in the corresponding direction, and / or an ultrasonic sensor that irradiates ultrasonic waves around the vehicle V and receives echo ultrasonic waves reflected from objects located in the corresponding direction.

[0039] Furthermore, the advanced driving assistance system may include a camera C as a sensing means. The camera C may be arranged to face the front, side and / or rear of the vehicle V to capture images in the corresponding directions. The captured images may become the basis for acquiring information such as lane lines or signboards in addition to objects around the vehicle V through an image processing process.

[0040] On the other hand, the vehicle V can fuse the image information acquired by the camera C with the wheel rotation information, yaw rate information and other CAN DATA transmitted in accordance with the communication method between the internal modules of the vehicle V, that is, the CAN (Controller Area Network) communication method, and use them for the control of the vehicle V. At this time, the image acquired by the camera C is based on the camera coordinate system, while the CAN DATA can be based on the vehicle coordinate system.

[0041] Figure 2 is a schematic plan view of a vehicle V according to an embodiment, illustrating a coordinate system of the vehicle V and a coordinate system of a camera C disposed on the vehicle V. Figure 2 , the vehicle V may have a V The origin is the X direction of the vehicle V. V Axis, the direction perpendicular to the ground Z V Axis and X V Axis and Z V Axis vertical Y V In contrast, the camera C disposed on the vehicle V may have a vehicle coordinate system consisting of O C The origin is determined by the X position and attitude angle.C Axis, Y C Axis, Z C In order to fuse the two information in different coordinate systems, it is necessary to unify the coordinate system, which is called camera C calibration (CAMERA CALIBRATION).

[0042] To this end, the camera C can be calibrated in advance before the vehicle V is driven. Specifically, the camera C installed on the vehicle V is used to obtain an image of the recognition pattern for calibration, and the angle and position of the camera C can be manually obtained using the image. As another example, during the driving of the vehicle V, the lane line is recognized by the camera C, and the position of the vanishing point is confirmed based on the recognized lane line, so that the attitude angle of the camera C can be obtained.

[0043] Alternatively, the camera C may be calibrated in real time in the vehicle position estimation device 100 in the vehicle position estimation system 1 . This will be described later.

[0044] Refer again Figure 1a According to an embodiment, the vehicle position estimation device 100 can estimate the position information of the vehicle V using the information received from the vehicle V. The position information may include a position and an attitude angle.

[0045] In order to estimate the position, the vehicle position estimation device 100 communicates with the vehicle V according to various known communication methods, so that information can be exchanged. According to one embodiment, the vehicle position estimation device 100 can communicate with the vehicle V via a base station using a known communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, and EPC. In addition, according to another embodiment, the vehicle position estimation device 100 can also communicate with the vehicle V within a predetermined distance using a communication method such as wireless LAN, wireless fidelity (Wi-Fi), Bluetooth, Zigbee, WFD (Wi-Fi Direct), UWB (Ultra wideband), infrared communication (IrDA: Infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), etc. It should be noted that the method for the communication unit to communicate with the vehicle V is not limited to the above-mentioned embodiment.

[0046] The vehicle position estimation device 100 can obtain the landmark-based initial position information of camera C based on the information received from the vehicle V, match the landmark corresponding to the obtained landmark-based initial position information of camera C with the image taken by camera C, thereby obtaining the estimated position information of camera C, and obtain the estimated position information of vehicle V based on the obtained estimated position information of camera C.

[0047] To this end, the vehicle position estimation device 100 may include a camera initial position information acquisition unit 110 for acquiring landmark-based initial position information of camera C of vehicle V, a camera estimated position information acquisition unit 120 for acquiring estimated position information of camera C, and a vehicle estimated position information acquisition unit 130 for acquiring estimated position information of vehicle V.

[0048] On the other hand, Figure 1a The example in which the vehicle position estimating device 100 and the vehicle V are independently configured to constitute the vehicle position estimating system 1 is shown. However, the vehicle position estimating device 100 may be provided inside the vehicle V.

[0049] Reference Figure 1b According to another embodiment, the vehicle position estimation system 1 may also be composed of a vehicle V including a vehicle position estimation device 100. It should be noted that, in addition to the configuration of the vehicle position estimation device 100, Figure 1a The vehicle position estimation system 1 and Figure 1b The vehicle position estimation system 1 operates in the same manner.

[0050] Below, refer to Figures 3 to 12 , each component of the vehicle position estimation device 100 is described in detail.

[0051] According to an embodiment, the vehicle position estimation device 100 can match the image captured by the camera C with the landmarks on the precision map, thereby estimating the position information of the vehicle V. In order to match the image and the map, accurate position information of the camera C is required. As mentioned above, sometimes the coordinate system of the vehicle V and the coordinate system of the camera C are set to be different, so the coordinate system conversion is required. In addition, the initial position information of the vehicle V based on GPS may be inaccurate, such as in areas with dense high-rise buildings, so before estimating the position of the vehicle V, the camera C needs to be calibrated first.

[0052] The camera initial position information acquisition unit 110 can calibrate the camera C using the initial position information of the vehicle V based on GPS. Figures 3 to 5 , the operation of the camera initial position information acquisition unit 110 is described in detail.

[0053] Figure 3is a flow chart of a method for obtaining landmark-based initial position information of a camera through a camera initial position information obtaining unit according to an embodiment, Figure 4 is a diagram for explaining a landmark matching method according to an embodiment, Figure 5 It is shown based on Figure 3 Figure 3 shows the results of the landmark-based initial position information acquisition method for the camera.

[0054] Reference Figure 3 First, the camera initial position information acquisition unit 110 may acquire the GPS-based initial position information of the vehicle V determined based on the satellite signal (S100). According to one embodiment, the camera initial position information acquisition unit 110 may acquire the GPS-based initial position information including the initial position and initial attitude angle of the vehicle V from the vehicle V. Differently, according to another embodiment, the camera initial position information acquisition unit 110 may also receive only the GPS-based initial position of the vehicle from the vehicle V, and use the initial position to acquire the GPS-based initial position information including the initial attitude angle of the vehicle V.

[0055] In the case of receiving only the initial position of the vehicle V from the vehicle V, the camera initial position information acquisition unit 110 may use the received initial position of the vehicle V to acquire the initial attitude angle of the vehicle V. Specifically, the camera initial position information acquisition unit 110 may acquire the X indicating the traveling direction of the vehicle V based on the continuously received initial positions of the vehicle V. V Axis, get the Z direction perpendicular to the ground area determined by the lane lines on the precision map V Axis, then relative to Z V Axis and X V The cross product of the axes to obtain the Y V axis. And considering the X axis obtained above due to the error in the satellite signal V Axis and Y V The axis may not be vertical, and the camera initial position information acquisition unit 110 is relative to the Z axis. V Axis and Y V The cross product of the axis can correct the X V axis.

[0056] Thus, the camera initial position information acquisition unit 110 can acquire the three-dimensional coordinate value as the initial position of the camera C and the three-dimensional coordinate axis as the initial posture angle.

[0057] Next, the camera initial position information acquisition unit 110 may determine a first area of ​​interest in the precise map according to the GPS-based initial position information of the vehicle V (S110). Specifically, the camera initial position information acquisition unit 110 may determine an area within a first radius as the first area of ​​interest based on the GPS-based initial position information of the vehicle V. For example, the first radius may be less than several meters.

[0058] After determining the first region of interest, the camera initial position information acquisition unit 110 may match the first landmark of the lane line existing in the first region of interest with the image captured by the camera C, thereby acquiring the initial posture angle of the camera C (S120). Specifically, the camera initial position information acquisition unit 110 may acquire the rotation matrix R based on the initial posture angle of the camera C according to Mathematical Formula 1.

[0059]

Mathematical formula 1

[0060]

[0061] Among them, S*, which is the solution of mathematical formula 1, can represent the initial position information based on the landmark composed of the rotation matrix R for the initial posture angle of the camera C and the movement matrix T for the initial position, Z k represents the coordinate value of the lane line detected from the image, P k It corresponds to Z k The coordinates of the landmarks on the precise map, C zk and C pk Respectively show that for Z k and P k The covariance of the error, H represents the partial differential (Jacobian) of the function h(). And, the function h() represents the function of projecting the coordinate values ​​of the landmarks on the precision map to the image, which can be defined according to Mathematical Formula 2.

[0062]

Mathematical formula 2

[0063] h(T, R, P)-J(R×P+T)

[0064] Among them, T can represent the movement matrix for the initial position of camera C, R represents the rotation matrix for the initial posture angle of camera C, P represents the coordinate value of the landmark on the precision map, and K represents the internal parameter matrix of camera C used to project the coordinate value based on the camera C coordinate system to the image taken by camera C.

[0065] In order to obtain the solution of Mathematical Formula 1, the camera initial position information acquisition unit 110 may select at least one of the well-known algorithms, for example, the Gauss-Newton algorithm or the Levenberg-Marquardt algorithm.

[0066] Figure 4 This is an example of a feature point P in a two-dimensional space. I1 , P I2 , P I3 Point Cloud Set L I And the characteristic point P M1 , P M2 , P M3 , P M11 , P M12 , P M13 The point cloud set L M The dotted area in the figure can represent the covariance of the internal feature points.

[0067] Reference Figure 4 , for a landmark such as a lane line extending in one direction, the covariance of the feature points of the point cloud set can form an ellipse in the extension direction. Based on this, the camera initial position information acquisition unit 110 takes the feature points closest to each other as the corresponding relationship and repeatedly calculates the solution of mathematical formula 1, so that the first landmark can be matched with the image. Figure 4 The feature points connected by dotted lines in the figure can represent the corresponding relationship between them.

[0068] When mathematical formula 1 is calculated to match the first landmark of a lane line extending in one direction with the image, the error is larger in the longitudinal direction in which the lane line extends, whereas the error is likely to be smaller in the lateral direction. As a result, the rotation matrix R of the initial posture angle of the camera C can have a reliable degree of accuracy.

[0069] On the contrary, the movement matrix T for the initial position of camera C may be less accurate than the rotation matrix R. Therefore, the camera initial position information acquisition unit 110 matches the second landmark other than the lane line with the image, so as to obtain the movement matrix T for the initial position of camera C. Specifically, the camera initial position information acquisition unit 110 can determine the second area of ​​interest (S130) based on the GPS-based initial position information of vehicle V in the precision map. Specifically, the camera initial position information acquisition unit 110 can determine the area within the second radius above the first radius and within the second radius as the second area of ​​interest based on the GPS-based initial position information of vehicle V. This is because the initial attitude angle in the landmark-based initial position information of camera C has been obtained, so by using landmarks in a wider range, the initial position of camera C can be obtained more accurately. At this time, the second radius can be less than a few hundred m (meters).

[0070] Next, the camera initial position information acquisition unit 110 matches the second landmarks other than the lane lines in the second focus area with the image based on the initial posture angle of the camera C, thereby acquiring the initial position of the camera C (S140). Specifically, the camera initial position information acquisition unit 110 substitutes the previously acquired rotation matrix R for the initial posture angle of the camera C into the mathematical formula 1 for calculation, thereby acquiring the movement matrix T for the initial position of the camera C.

[0071] At this time, the movement matrix T of the initial positions of multiple cameras C corresponding to the rotation matrix R of the initial posture angle of one camera C can be obtained.

[0072] The landmark-based initial position information including the initial position and initial attitude angle of the camera C obtained in this way can be used as an input value for estimating the position information of the camera C described later.

[0073] Compared with the situation where only the GPS-based initial position information of vehicle V is used as the position information of camera C to estimate the position information of camera C, according to the above method, more accurate landmark-based initial position information of camera C is used as the input value, so the accuracy and processing speed of estimating the position information of camera C can be improved.

[0074] Figure 5 is a diagram showing a plurality of candidate groups obtained when the initial position information of the camera C based on the landmark obtained by the landmark matching is used as an input value on a precise map. Figure 5 In the case of including multiple landmarks L M The precision map shows multiple candidate locations P estimated according to the above method. C . Set the initial position of the vehicle to P G, the actual position of the vehicle is set to P R When the initial position P of the vehicle V is confirmed G Compared with the landmark matching result P C Closer to the actual position P of the vehicle V R .

[0075] After acquiring the initial position information of the camera C based on the landmark, the camera estimated position information acquisition unit 120 may use the initial position of the camera C as an input value to acquire the estimated position information of the camera C. Figures 6 to 10 , a method for obtaining the estimated position information of camera C is described in detail.

[0076] Figure 6 is a flow chart of a method for obtaining camera estimated position information by a camera estimated position information obtaining unit according to an embodiment, Figure 7 is a diagram illustrating an extraction result of a first landmark for a lane line and a second landmark other than the lane line in an image captured by a camera according to an embodiment, Figure 8 This is an example of combining the landmarks of a precise map with Figure 7 The image is matched with the result of Figure 9a and 9b is an example based on Figure 8 The matching error graph is Fig.10 is a diagram illustrating a final result of a method for acquiring position information based on camera estimation according to an embodiment.

[0077] according to Figure 6 , the camera estimated position information acquisition unit 120 can sample multiple candidate position information around the initial position information of camera C based on the landmark (S200). Specifically, the camera estimated position information acquisition unit 120 can take the initial position information of camera C based on the landmark as the average and perform sampling using a Gaussian probability model modeled by a predetermined covariance setting error. At this time, the Gaussian probability model can be defined as at least six dimensions with three degrees of freedom for the attitude angle and three degrees of freedom for the position.

[0078] If there are multiple landmark-based initial position information of the camera C, the camera estimated position information acquisition unit 120 may perform sampling on each landmark-based initial position information of the camera C according to the Gaussian probability model.

[0079] Next, the camera estimated position information acquisition unit 120 may use a particle filter to acquire the estimated position information of the camera C. Specifically, the camera estimated position information acquisition unit 120 may reflect the driving information of the vehicle V in the plurality of candidate position information (S210). At this time, the camera estimated position information acquisition unit 120 may be based on Mathematical Formula 3.

[0080]

Mathematical formula 3

[0081]

[0082] The matrix [x(k), y(k), θ(k)] can represent the position and direction of the vehicle V at time point k, S r represents the moving distance based on the right wheel speed of vehicle V, S l Indicates the moving distance based on the left wheel speed of the vehicle V.

[0083] To this end, the camera estimated position information acquisition unit 120 may receive driving information including wheel speed information and yaw rate information from the vehicle V.

[0084] Next, the camera estimated position information acquisition unit 120 may assign a weighted value to each of the plurality of candidate position information based on a matching error between a landmark of the precise map corresponding to each of the plurality of candidate position information and the image ( S220 ).

[0085] To this end, the camera estimated position information acquisition unit 120 may use an image from which a landmark is extracted. Figure 7 In the image captured by the camera C, it can be confirmed that the first landmark L for the lane line is extracted from the ground as a white line. I1 and a second landmark L for the sign extracted as a black rectangular shape in front I2 .

[0086] At this time, candidate position information that is off the road as a result of reflecting the driving information or candidate position information that is on a road in the opposite direction to the traveling direction of the vehicle V may be excluded.

[0087] Extract the first landmark L from the image I1 And the second landmark L I2 After that, the camera estimated position information acquisition unit 120 can match the landmarks of the precise map corresponding to each of the plurality of candidate position information with the image. At this time, in order to match the landmarks, the camera estimated position information acquisition unit 120 uses Figure 2 The mathematical formula of .

[0088] Reference Figure 8 It can be confirmed that the first landmark L for the vehicle V extracted as a line on the precise map corresponding to the specific candidate position information M1 and a second landmark L extracted as a cross in front of the lane line M2 and Figure 7 At this time, there may be various first landmarks L I1 , L M1and each second landmark L I2 , L M2 The matching result error.

[0089] Reference Figure 9a , matching the second landmark L I2 , L M2 As a result, it can be confirmed that there is an error corresponding to the distance d2. Figure 9b , matching the first landmark L I1 , L M1 As a result, it can be confirmed that there is an error corresponding to the distance d1.

[0090] After confirming the matching error, the camera estimated position information acquisition unit 120 may acquire a weighted value corresponding to the matching error. This process may be performed according to Equation 4.

[0091]

Mathematical formula 4

[0092]

[0093] Among them, G σ Δx, Δy may represent a weighted value, (Δx, Δy) represents the error for x and y in the image, and σ represents the standard deviation.

[0094] Secondly, the camera estimated position information acquisition unit 120 assigns corresponding weighted values ​​to corresponding candidate position information, so as to reflect the matching error.

[0095] After assigning weighted values, the camera estimated position information acquisition unit 120 can resample multiple candidate position information using the multiple candidate position information assigned weighted values ​​(S230). Since the resampling is performed based on the result of assigning weighted values, the multiple candidate position information can converge around the candidate position information with a smaller matching error.

[0096] After the sampling is completed, the camera estimated position information acquisition unit 120 may confirm whether the standard deviation of the resampled multiple candidate position information is below the reference standard deviation (S240). The reference standard deviation may represent the maximum standard deviation of the camera C estimated position information that can be acquired using multiple candidate positions.

[0097] If the standard deviation of the resampled plurality of candidate position information is below the reference standard deviation, the camera estimated position information acquisition unit 120 may acquire an average value of the resampled plurality of candidate position information as the estimated position information of the camera C ( S250 ).

[0098] On the contrary, if the standard deviation of the resampled multiple candidate position information is greater than the reference standard deviation, the camera estimated position information acquisition unit 120 reflects the driving information of the vehicle V in the resampled multiple candidate position information (S260), and then can enter step S220 again.

[0099] Fig.10 The result of repeatedly performing the above method until the standard deviation of multiple candidate position information is below the reference standard deviation is shown on the precision map. By assigning a high weight to the candidate position information with a smaller matching error, the resampled candidate position P can be confirmed. C Converge around candidate location information with smaller matching errors.

[0100] If the estimated position information of the camera C is acquired, the estimated position information acquisition unit of the vehicle V verifies the validity of the estimated position information of the camera C, and then acquires the estimated position information of the vehicle V based on the valid estimated position information of the camera C. Fig.11 and 12, a method for obtaining estimated position information of vehicle V is described in detail.

[0101] Fig.11 is a flow chart of a method for obtaining vehicle estimated position information by a vehicle estimated position information obtaining unit according to an embodiment, Fig.12 is a diagram illustrating a final landmark matching result based on a method for acquiring vehicle estimated position information according to an embodiment.

[0102] Reference Fig.11 The vehicle estimated position information acquisition unit 130 may first confirm a second landmark other than the lane line among the landmarks of the precision map corresponding to the estimated position information of the camera C (S300).

[0103] Next, the vehicle estimated position information acquisition unit 130 may acquire a weighted value of the camera C estimated position information corresponding to the matching error of each second landmark ( S310 ). The weighted value of the camera C estimated position information may be based on the above-mentioned mathematical formula 4.

[0104] After obtaining the weighted value corresponding to the matching error of each second landmark, the vehicle estimated position information acquisition unit 130 may confirm whether the number of weighted values ​​above the reference value is above the reference number (S320). The reference value may represent the minimum weighted value based on the matching error when the position information of the camera C can be determined as valid, and the reference number may represent the minimum number of weighted values ​​above the reference value when the position information of the camera C can be determined as valid.

[0105] At this time, the reference number may be determined according to the number of confirmed second landmarks. According to an embodiment, the reference number may be determined according to any ratio of the number of confirmed second landmarks.

[0106] If the number of weighted values ​​above the reference value is greater than the reference number, the vehicle estimated position information acquisition unit 130 may determine that the estimated position information of the camera C is valid (S330). As a result, the vehicle estimated position information acquisition unit 130 may use the estimated position information of the camera C to acquire the estimated position information of the vehicle V (S340). At this time, the vehicle estimated position information acquisition unit 130 may use the movement matrix T and the rotation matrix R to acquire the vehicle V estimated position information.

[0107] On the contrary, if the number of weighted values ​​above the reference value is less than the reference number, the vehicle estimated position information acquisition unit 130 may determine the estimated position information of the camera C as invalid (S350). As a result, the vehicle estimated position information acquisition unit 130 reacquires the landmark-based initial position information of the camera C (S360), so that the vehicle V estimated position acquisition process can be performed again.

[0108] exist Fig.11 In the example, a single image is used as a reference to verify the validity of the position information of the camera C estimated based on whether the number of weighted values ​​above the reference value is above the reference number. Different from this, the vehicle estimated position information acquisition unit 130 can also perform the above-mentioned validity verification for each frame of image, and compare the execution result with a predetermined reference, so as to verify the validity of the estimated position information of the camera C. Alternatively, the vehicle estimated position information acquisition unit 130 can also repeatedly perform the verification while the vehicle V travels a predetermined reference distance, and if the estimated position information of the camera C is invalid, the initial position information of the camera C based on the landmark is re-acquired.

[0109] In this way, the vehicle estimated position information acquisition unit 130 may repeatedly perform validity verification based on each frame image, a predetermined cycle, or a predetermined driving distance, thereby monitoring the estimated camera C position information.

[0110] Fig.12 Shown based on Fig.11 The estimated position information of the vehicle V is obtained, and the result of matching the landmark with the image. It can be confirmed that the position of the vehicle V is accurately estimated, so for the first landmark L of the lane line I1 , L M1 And the second landmark L for the sign I2 , L M2 Matches the image exactly.

[0111] According to the above-mentioned embodiments, the vehicle position estimation device, the vehicle position estimation method, and the computer-readable recording medium storing a computer program programmed to execute the method utilize images acquired through the vehicle's camera and a precision map in addition to satellite signals, thereby being able to estimate a more accurate vehicle position. Furthermore, the vehicle position can be estimated without using expensive devices such as LiDar, thereby being able to reduce manufacturing costs.

[0112] On the other hand, the various steps included in the above-described method for estimating the position of a vehicle according to an embodiment may be implemented in a computer-readable recording medium storing a computer program programmed to execute such steps.

[0113] The above description is only an example of the technical idea of ​​the present invention. A technician in the technical field of the present invention can bring various modifications and deformations without departing from the essential qualities of the present invention. Therefore, the multiple embodiments disclosed in the present invention are used to illustrate the present invention, not to limit the technical idea of ​​the present invention, and the protection scope of the present invention is not limited to these embodiments. The protection scope of the present invention should be interpreted based on the claims, and all technical ideas within the equivalent scope are included in the protection scope of the present invention.

[0114] Industrial Applicability

[0115] According to one embodiment, the above-mentioned vehicle position estimation device, vehicle position estimation method and computer-readable recording medium storing a computer program programmed to execute the method can be used in various fields such as home or industrial sites, so they have industrial applicability.

Claims

1. A method for estimating the position of a vehicle, include: The step of matching a landmark on a precision map corresponding to the vehicle's initial position information based on GPS with an image captured by a camera of the vehicle to obtain the camera's initial position information based on the landmark; as well as a step of obtaining estimated position information of the vehicle by using a comparison result between the landmark of the precise map and the image corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera, The step of obtaining the landmark-based initial position information of the camera includes: In the precise map, a step of matching a first landmark for a lane line with the image to obtain an initial attitude angle in the landmark-based initial position information of the camera, wherein the first landmark exists within a first area of ​​interest determined by the GPS-based initial position information of the vehicle, The position estimation method further includes: Based on the initial attitude angle of the camera obtained above, in the above-mentioned precision map, a second landmark other than the lane line is matched with the above-mentioned image to obtain the initial position of the above-mentioned camera in the above-mentioned landmark-based initial position information, wherein the above-mentioned second landmark exists in a second area of ​​interest determined by the above-mentioned vehicle's GPS-based initial position information, and the above-mentioned second area of ​​interest includes the above-mentioned first area of ​​interest.

2. The method for estimating the position of a vehicle according to claim 1, in, The steps of obtaining the estimated position information of the above vehicle include: A step of obtaining estimated position information of the camera according to a matching error between a landmark of the precise map and the image corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera and the driving information of the vehicle; and The step of obtaining the estimated position information of the vehicle based on the estimated position information of the camera.

3. The method for estimating the position of a vehicle according to claim 2, in, The steps of obtaining the estimated position information of the above camera include: The step of assigning a weighted value to each of the plurality of candidate location information based on a matching error between a landmark of the precise map corresponding to each of the plurality of candidate location information and the image; Based on the plurality of candidate position information assigned with the weighted values, a step of changing the plurality of candidate position information; and When the standard deviation of the changed plurality of candidate position information is less than the reference standard deviation, an average value of the changed plurality of candidate position information is obtained as the estimated position information of the camera.

4. The method for estimating the position of a vehicle according to claim 3, in, The step of obtaining the estimated position information of the camera further includes: When the standard deviation of the above-mentioned changed multiple candidate position information is greater than the benchmark standard deviation, the step of obtaining the estimated position information of the camera based on the matching error between the landmark of the precision map corresponding to each of the above-mentioned changed multiple candidate position information and the above-mentioned image.

5. The method for estimating the position of a vehicle according to claim 2, in, The steps of obtaining the estimated position information of the above vehicle include: A step of verifying the estimated position information of the camera based on a matching error between a second landmark other than a first landmark for a lane line among the landmarks of the precise map corresponding to the estimated position information of the camera and the image; and When it is determined according to the verification that the estimated position information of the camera is valid, a step of acquiring the estimated position information of the vehicle from the estimated position information of the camera.

6. The method for estimating the position of a vehicle according to claim 5, in, The steps of verifying the estimated position information of the above camera include: A step of obtaining a weighted value of the estimated position information of the camera corresponding to each of the second landmarks; and A step of determining the estimated position information of the camera as valid when the number of weighted values ​​greater than a reference value among the acquired weighted values ​​is greater than a reference number.

7. The method for estimating the position of a vehicle according to claim 5, in, In the step of verifying the estimated position information of the camera, For each frame, the estimated position information of the camera is verified based on a matching error between the image acquired by the camera and the second landmark.

8. The method for estimating the position of a vehicle according to claim 5, in, The position estimation method further includes: When it is determined through the above verification that the estimated position information of the above camera is invalid, a step of reacquiring the above landmark-based initial position information of the above camera.

9. A computer-readable recording medium storing a computer program programmed to execute a position estimation method for a vehicle, in, The above vehicle position estimation method includes: A step of matching a landmark on a precision map corresponding to the vehicle's initial location information based on GPS with an image captured by a camera of the vehicle to obtain the camera's initial location information based on the landmark; and a step of obtaining estimated position information of the vehicle by using a comparison result between the landmark of the precise map and the image corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera, The step of obtaining the landmark-based initial position information of the camera includes: In the precise map, a step of matching a first landmark for a lane line with the image to obtain an initial attitude angle in the landmark-based initial position information of the camera, wherein the first landmark exists within a first area of ​​interest determined by the GPS-based initial position information of the vehicle, The vehicle position estimation method further includes: Based on the initial attitude angle of the camera obtained above, in the above-mentioned precision map, a second landmark other than the lane line is matched with the above-mentioned image to obtain the initial position of the above-mentioned camera in the above-mentioned landmark-based initial position information, wherein the above-mentioned second landmark exists in a second area of ​​interest determined by the above-mentioned vehicle's GPS-based initial position information, and the above-mentioned second area of ​​interest includes the above-mentioned first area of ​​interest.

10. The computer-readable recording medium storing a computer program programmed to execute the position estimation method of a vehicle according to claim 9, in, The steps of obtaining the estimated position information of the above vehicle include: A step of obtaining estimated position information of the camera according to a matching error between a landmark of the precise map and the image corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera and the driving information of the vehicle; and The step of obtaining the estimated position information of the vehicle based on the estimated position information of the camera.

11. The computer-readable recording medium storing a computer program programmed to execute the position estimation method of a vehicle according to claim 10, in, The steps of obtaining the estimated position information of the above camera include: The step of assigning a weighted value to each of the plurality of candidate location information based on a matching error between a landmark of the precise map corresponding to each of the plurality of candidate location information and the image; Based on the plurality of candidate position information assigned with the weighted values, a step of changing the plurality of candidate position information; and When the standard deviation of the changed plurality of candidate position information is less than the reference standard deviation, an average value of the changed plurality of candidate position information is obtained as the estimated position information of the camera.

12. The computer-readable recording medium storing a computer program programmed to execute the position estimation method of a vehicle according to claim 10, in, The steps of obtaining the estimated position information of the above vehicle include: A step of verifying the estimated position information of the camera based on a matching error between a second landmark other than a first landmark for a lane line among the landmarks of the precise map corresponding to the estimated position information of the camera and the image; and When it is determined according to the verification that the estimated position information of the camera is valid, a step of acquiring the estimated position information of the vehicle from the estimated position information of the camera.

13. The computer-readable recording medium storing a computer program programmed to execute the position estimation method of a vehicle according to claim 12, in, The steps of verifying the estimated position information of the above camera include: A step of obtaining a weighted value of the estimated position information of the camera corresponding to each of the second landmarks; and A step of determining the estimated position information of the camera as valid when the number of weighted values ​​greater than a reference value among the acquired weighted values ​​is greater than a reference number.

14. A vehicle position estimation device, include: A camera initial position information acquisition unit matches a landmark of a precision map corresponding to the vehicle's initial position information based on GPS with an image captured by the camera of the vehicle to acquire the camera's initial position information based on the landmark; a camera estimated position information acquisition unit, which acquires the estimated position information of the camera by using a comparison result between the landmark of the precise map and the image corresponding to each of a plurality of candidate position information sampled based on the landmark-based initial position information of the camera; as well as The vehicle estimated position information acquisition unit acquires the estimated position information of the vehicle based on the estimated position information of the camera. When the camera initial position information acquisition unit acquires the landmark-based initial position information of the camera, In the precise map, a first landmark for the lane line is matched with the image to obtain an initial attitude angle in the landmark-based initial position information of the camera, wherein the first landmark exists within a first area of ​​interest determined by the GPS-based initial position information of the vehicle, Based on the initial attitude angle of the camera obtained above, in the above-mentioned precision map, a second landmark other than the lane line is matched with the above-mentioned image to obtain the initial position of the above-mentioned camera in the above-mentioned landmark-based initial position information, wherein the above-mentioned second landmark exists in a second area of ​​interest determined by the above-mentioned vehicle's GPS-based initial position information, and the above-mentioned second area of ​​interest includes the above-mentioned first area of ​​interest.

Citation Information

Patent Citations

  • Knuckle for vehicle

    KR1020180059626A

  • Lane line detection and GIS map information development-based vision navigation method

    CN103954275A

  • System and method for reversely educing position of automobile by means of coordinates of traffic signboards

    CN105021201A

  • Road sign drawing and vehicle positioning method and system for automatic drive

    CN105783936A

  • Vehicle-mounted camera calibrating method, device and vehicle-mounted device

    CN106981082A