Vehicle side positioning method and device

Abstract

To provide a vehicle localization method and a device.SOLUTION: A vehicle localization method includes the steps of: determining an image-based reference position of a vehicle based on image information of the vehicle captured using a camera of the vehicle; setting an acceptance range regarding the image-based reference position based on a travel situation of the vehicle; and comparing an error level of the image-based reference position with the acceptance range to estimate a current position of the vehicle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The following embodiments relate to a vehicle side positioning method and device. [Background technology]

[0002] There are navigation systems that provide drivers with various visual information through augmented reality (AR) to assist them in driving vehicles and other transportation means. Such navigation systems receive GPS (global positioning system) signals from satellites via a GPS sensor and estimate the current location of the vehicle based on the received GPS signals. The absolute position values ​​of the vehicle in terms of latitude and longitude can be derived from the GPS signals. Summary of the Invention [Problem to be solved by the invention]

[0003] The purpose of the present embodiment is to provide a vehicle side positioning method and apparatus. [Means for solving the problem]

[0004] According to one embodiment, a vehicle positioning method includes the steps of determining a first reference position based on vehicle position information measured using a vehicle position sensor, determining a second reference position of the vehicle based on image information of the vehicle captured using a vehicle camera, setting a range for the second reference position based on the vehicle's driving conditions, comparing the second reference position with the first reference position to predict an error level of the second reference position, and comparing the error level of the second reference position with the range to estimate the current position of the vehicle.

[0005] The step of determining the second reference position may include a step of determining the second reference position based on geometric information of the lane shown in the image information. The second reference position may include at least one of information about a lane on which the vehicle is located and information about a detailed position of the vehicle within the lane on which the vehicle is located.

[0006] Setting the accommodation range for the second reference position based on the driving conditions of the vehicle includes determining the driving conditions from a plurality of driving conditions.

[0007] The plurality of driving situations may include at least one of a first driving situation in which the vehicle travels through an intersection and a second driving situation in which the vehicle travels around a corner.

[0008] The step of setting the accommodation range may include a step of selectively adjusting the size of the accommodation range according to the driving situation. The step of setting the accommodation range may include a step of setting a wider accommodation range for a second driving situation in which the vehicle turns a corner compared to a accommodation range for a first driving situation in which the vehicle travels through an intersection.

[0009] The step of estimating the current position may include a step of estimating the current position without considering the second reference position if the error level of the second reference position does not fall within the range of tolerance. The step of estimating the current position may include a step of estimating the first reference position as the current position of the vehicle if the error level of the second reference position does not fall within the range of tolerance, and a step of estimating a new position estimated based on a weighted sum of the second reference position and the first reference position as the current position if the error level of the second reference position falls within the range of tolerance.

[0010] The step of determining the error level may include a step of determining an error level of the second reference position based on a distance between the second reference position and the first reference position. The vehicle positioning method further includes a step of setting another accommodation range for another second reference position in a next time step, and the step of setting the other accommodation range may include a step of setting a width of the other accommodation range wider than the width of the accommodation range if the second reference position is not within the accommodation range.

[0011] The vehicle positioning method may further include a step of determining a change in a reference heading angle corresponding to the second reference position based on the second reference position and a plurality of second reference positions corresponding to a plurality of previous time steps, and the step of estimating the current position may include a step of estimating the current position by further considering a comparison result between the output of the gyroscope and the determined change in the reference heading angle.

[0012] According to one embodiment, the navigation device includes a processor that determines a first reference position based on vehicle position information measured using a vehicle position sensor, determines a second reference position of the vehicle based on image information of the vehicle captured using a vehicle camera, sets a coverage area for the second reference position based on the vehicle's driving conditions, compares the second reference position with the first reference position to predict an error level of the second reference position, and compares the error level of the second reference position with the coverage area to estimate the vehicle's current position.

[0013] According to one embodiment, the vehicle control device includes a processor that determines a first reference position based on vehicle position information measured using the vehicle's position sensor, determines a second reference position of the vehicle based on image information of the vehicle captured using the vehicle's camera, sets a range for the second reference position based on the vehicle's driving conditions, compares the second reference position with the first reference position to predict an error level of the second reference position, compares the error level of the second reference position with the range for estimating the vehicle's current position, and generates control commands for the vehicle's driving based on the current position; and a control system that controls the vehicle based on the control commands.

[0014] According to one embodiment, a vehicle positioning method includes the steps of: determining a first reference position based on position information of the vehicle measured using a vehicle position sensor; determining a second reference position of the vehicle based on image information of the vehicle captured using a vehicle camera; selecting, based on a determination of whether the second reference position is an erroneous position, either estimating a current position of the vehicle based on the first reference position and the second reference position or estimating the current position of the vehicle based on the first reference position; and estimating the current position based on a result of the selection, wherein the determination of whether the second reference position is an erroneous position is based on consideration of an error regarding the second reference position, which depends on the current driving situation of the vehicle.

[0015] The determination of whether the second reference position is an erroneous position may be based on an tolerance range that is dependent on the determination of the current driving situation and on whether the estimated error of the second reference position is within the tolerance range. The tolerance range may be set to be wider for cornering situations than for intersection situations, and may be set to be different for at least two different driving situations. The determination of whether the second reference position is an erroneous position may include consideration of at least one or both of a previous time step in which the determined second reference position was determined to be erroneous and a comparison between the output of the vehicle's gyroscope and a determined change in the reference heading angle corresponding to the second reference position. [Effects of the Invention]

[0016] According to the present invention, a vehicle side positioning method and device can be provided. [Brief explanation of the drawings]

[0017] [Figure 1] 3 illustrates the operation of a navigation device according to an embodiment. [Figure 2] 1 illustrates a location estimation process according to an embodiment. [Figure 3A] 10 illustrates a process for estimating a location through error detection using coverage according to an embodiment. [Figure 3B] 10 illustrates a process for estimating a location through error detection using coverage according to an embodiment. [Figure 3C] 10 illustrates a process for estimating a location through error detection using coverage according to an embodiment. [Figure 4A] An example of an error in video-based reference positioning is shown below. [Figure 4B] An example of an error in video-based reference positioning is shown below. [Figure 5] 10 illustrates a comparison between a reference position and a coverage area according to one embodiment. [Figure 6] 10 illustrates setting of a storage range based on a driving situation according to one embodiment. [Figure 7] 1 illustrates detection of a rotation situation according to one embodiment. [Figure 8] 1 illustrates cross-road driving detection according to one embodiment. [Figure 9] 1 illustrates an expanded coverage area according to one embodiment. [Figure 10] 10 illustrates a process for detecting an error using a heading angle according to an embodiment. [Figure 11] 1 illustrates a vehicle side positioning method according to one embodiment. [Figure 12] 1 shows the configuration of a navigation device according to an embodiment. [Figure 13] 1 illustrates a configuration of an electronic device according to an embodiment. Specific details for implementing the invention

[0018] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. However, the specific structural or functional descriptions disclosed in this specification are merely examples for the purpose of describing the embodiments, and the embodiments may be implemented in various different forms, and the present invention is not limited to the embodiments described in this specification. It should be understood that all modifications, equivalents, or alternatives to the embodiments are included in the scope of the claims.

[0019] The terms used in the embodiments are merely used for the purpose of explanation and are not to be construed as being limiting. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this specification, the terms "comprise" or "have" indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0020] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention pertains. Commonly used predefined terms should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as having an ideal or overly formal meaning unless expressly defined herein.

[0021] In addition, when describing the present invention with reference to the drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant descriptions thereof will be omitted. In the description of the embodiments, if a detailed description of related known technology is determined to unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

[0022] Furthermore, in describing components of the embodiments, terms such as first, second, A, B, (a), (b), etc. may be used. Such terms are used to distinguish the component from other components, and do not limit the nature, order, or sequence of the components. When a component is referred to as being "coupled" or "connected" to another component, it should be understood that components that are directly coupled or connected to the other component but that are further different from each other may also be "coupled," "coupled," or "connected."

[0023] Additionally, the term "vehicle" refers to any type of transportation means having a driving engine and used to transport people or goods, such as a car, bus, motorcycle, or truck. The term "road" refers to a road on which vehicles travel, including various types of roads such as expressways, national highways, local roads, express national highways, and expressways. The term "lane" refers to a road space separated by lane boundaries marked on the road surface. The term "current driving lane" refers to the lane on which the current vehicle is traveling among various lanes, and refers to the lane space occupied and currently being used by the current vehicle, and may also be referred to as an "ego lane." The term "lane boundary" refers to a solid or dotted line marked on the road surface to distinguish between lanes. The term "lane" may also be referred to as "lane marking."

[0024] Components having common functions with components included in one embodiment will be described using the same names in other embodiments. Unless otherwise specified, the description of one embodiment will be applied to other embodiments, and detailed description will be omitted to the extent that they overlap.

[0025] 1 illustrates the operation of a navigation apparatus according to an embodiment. Referring to FIG. 1, a navigation apparatus 100 may provide navigation parameters based on at least one of vehicle image data, sensor data, and map data. For example, the navigation parameters may include information regarding at least one of the vehicle's pose, velocity, and position.

[0026] A vehicle can generate navigation information through navigation parameters and provide the navigation information to a user (e.g., a driver) and / or other vehicle (e.g., an autonomous vehicle). The vehicle can provide the navigation information using an augmented reality (AR) technique through a three-dimensional head-up display (HUD). The navigation device 100 can overlay a virtual image on a real background by taking the navigation parameters into consideration. To realize an error-free AR environment, it is necessary to accurately measure the vehicle's state.

[0027] The vehicle includes one or more cameras that capture images in one or more directions, including forward, side, rearward, upward, and downward directions, and the navigation device 100 can receive image information of the vehicle from such one or more cameras. The vehicle also includes position sensors for measuring the position of the vehicle, such as an inertial measurement unit (IMU), a global positioning system (GPS), and on-board diagnostics (OBD), and the navigation device 100 receives vehicle position information from such position sensors. Here, the IMU may include an acceleration sensor and a gyroscope.

[0028] The navigation device 100 can use a high definition map (HD map) as map data. The high definition map may include information about various map elements (e.g., lanes, centerlines, guide signs, etc.) generated using various sensors. The various elements of the high definition map are represented as a point cloud, and each point in the point cloud corresponds to a three-dimensional position. The three-dimensional position is represented as latitude, longitude, and altitude.

[0029] 2 shows a location estimation process according to an embodiment. Referring to FIG. 2, the navigation device performs sensor-based localization in step S210, performs image-based localization in step S220, and performs fusion based on the results of steps S210 and S220 to determine the final location in step S240.

[0030] More specifically, in step S210, the navigation system estimates a sensor-based position based on vehicle position information measured via the vehicle's position sensor. The sensor-based positioning may include sensor fusion. Sensor fusion is an estimation technique that fuses various information and corresponds to a Kalman filter. Here, the information to be fused may include values ​​estimated using an estimation model based on Kalman filter theory and values ​​estimated using sensor information. For example, the navigation system may use an IMU, GPS, and OBD for the sensor fusion in step S210. In this case, the navigation system may estimate the vehicle's position, speed, and attitude using the outputs of the GPS and OBD as sensor information and the output of the IMU as input for the estimation model.

[0031] In step S220, the navigation device performs image-based positioning using image information of the vehicle. The image information may include geometric information of lanes visible from the front of the vehicle. For example, the geometric information of lanes may include information on at least one of lane type, direction, and arrangement. The navigation device may determine the image-based position by combining other sensor information with the image information. For example, the navigation device may determine the approximate position of the vehicle using vehicle position information measured through a position sensor such as a GPS sensor. The navigation device may then determine the lane on which the vehicle is located and / or the vehicle's detailed position within the lane through the image information, thereby estimating the image-based position. The estimated image-based position may be provided by mapping it to a high-definition map (HD map).

[0032] Sensor-based positioning has a fast output cycle and can accurately reflect changes. Therefore, relative position can be determined relatively accurately through the sensor-based estimated position. Image-based positioning is suitable for determining absolute position. In step S240, the navigation system performs fusion based on the sensor-based estimated position and the image-based estimated position to derive a final position. In this case, the navigation system performs fusion through sensor fusion. For example, the navigation system can use the sensor-based estimated vehicle position in step S210 and the image-based estimated vehicle position in step S220 as sensor information and the estimated vehicle speed in step S210 as a model input to finally estimate the vehicle position.

[0033] Generally, images can contain information suitable for estimating absolute position. However, if image information is lost due to occlusion or saturation, or if changes in the driving environment occur, such as an increase or decrease in the number of lanes, the reliability of the image-based estimated position can be significantly reduced. Therefore, deriving the final position using the image-based estimated position can result in a decrease in the accuracy of the final position. In this regard, the navigation system determines whether the image-based position corresponds to a fault in step S230. If the fault is found, the navigation system can maintain high accuracy by deriving the final position while excluding the fault in step S240.

[0034] According to one embodiment, the navigation system can predict an error level of a video-based estimated position, set an acceptance range appropriate for the vehicle's driving conditions, and compare the error level of the video-based position with the acceptance range. The navigation system can predict an error level of the video-based position using the sensor-based position. If the error level exceeds the acceptance range, the navigation system classifies the video-based position as erroneous. In this case, the navigation system can derive a final position while excluding the video-based estimated position.

[0035] In another embodiment, the navigation system calculates a heading angle using an image-based estimated position at successive time steps and compares the change in the calculated heading angle with the output of a gyroscope. If the difference between the calculated heading angles exceeds a threshold, the navigation system classifies the associated estimated position (e.g., the estimated position at the last time step) as erroneous. In another embodiment, the navigation system can use both the coverage method and the heading angle method. In this case, the two methods may be performed sequentially or in parallel. When both conditions of the two methods are satisfied, the image-based estimated position can be reflected in the process of deriving the final position.

[0036] FIG. 3 illustrates a process for estimating a position through error detection using coverage according to an embodiment. Referring to FIG. 3A, a navigation system performs sensor-based positioning in step S310. The navigation system may estimate the position, speed, and attitude of a vehicle based on sensor fusion. The navigation system also performs image-based positioning in step S330. The navigation system may estimate the lane on which the vehicle is located and / or the vehicle's precise position within the lane based on image information and sensor information. In this case, the navigation system may use the vehicle's position and attitude estimated through sensor fusion in step S310 based on sensor information. Because the image-based positioning result is based on image information, it may also be referred to as an image-based reference position.

[0037] The navigation device determines the vehicle's driving situation in step S320. The vehicle's driving situation is determined based on the type of road and / or driving direction. For example, the driving situation may include driving through an intersection, driving around a corner, and other driving situations. Other driving situations include, for example, driving straight through a section that is not an intersection without turning.

[0038] The navigation system performs error detection for the image-based reference position in step S340. To perform error detection, the navigation system sets a coverage area in step S341 and compares the error level of the image-based reference position with the coverage area in step S342. The navigation system can predict the error level of the image-based reference position based on the sensor-based positioning results. For example, the navigation system can perform a first estimation in step S351 based on the sensor-based positioning results to determine a model estimation-based reference position. The model estimation-based reference position is used as a reference for predicting the error level of the image-based reference position. The first estimation process is explained as follows: The navigation system can predict the error level of the image-based reference position based on the difference between the image-based reference position and the model estimation-based reference position. If the difference between the two is large, the error level of the image-based reference position is considered to be high.

[0039] The navigation system may set a coverage range according to the driving conditions. According to an embodiment, the navigation system may set different coverage ranges depending on the driving conditions. As a result, image-based reference positions with the same error level may be accommodated or not accommodated depending on the driving conditions. For example, the coverage range may be set small for intersection driving, and large for turning driving. When intersection driving is performed, there is a high possibility that noise may be included in the image information, such as when lanes disappear, lane patterns change, or road closures occur. Therefore, strict criteria must be applied to the image-based reference position. When turning driving is performed, there is a possibility that the accuracy of lateral estimation may drop sharply. Therefore, it is advantageous to accommodate as much image information as possible in order to improve the accuracy of position estimation. The setting of the coverage range will be described in more detail below with reference to FIGS. 6 to 9. The navigation system classifies image-based reference positions that fall within the coverage range as no fault, and classifies image-based reference positions that do not fall within the coverage range as faulty.

[0040] In step S350, the navigation system performs fusion based on the sensor-based positioning results and the image-based positioning results (image-based reference position). The fusion includes a primary estimation in step S351 and a secondary estimation in step S352. The navigation system can estimate the position of the current time step by performing a primary estimation using the final position estimated in the previous time step and the sensor-based positioning results. The primary estimation result corresponds to a model estimate based on sensor information (e.g., vehicle position data measured by a position sensor such as GPS), and is therefore referred to as a model-estimated reference position to distinguish it from the image-based reference position. The navigation system can set the model-estimated reference position to the center of the coverage area.

[0041] The navigation system performs secondary estimation by correcting the model-based reference position using the image-based reference position. The secondary estimation can be selectively performed depending on the error detection result of step S340. If the image-based reference position is incorrect, secondary estimation is not performed and the model-based reference position is output as the final position. If the image-based reference position is error-free, secondary estimation is performed and a new position resulting from the position correction is output as the final position. The new position is estimated based on the image-based reference position and a weighted sum of the image-based reference position. The fusion operation of step S350 will be described in more detail below with reference to FIGS. 3B and 3C.

[0042] FIG. 3B shows an example of sensor fusion. Referring to FIG. 3B, the sensor fusion can include model propagation, measurement update, and time update in steps S361, S362, and S363. Model propagation uses information from a previous time step ( TIFF0007827422000001.tif1015 or This is an operation that calculates the information of the current time step based on the model from the TIFF0007827422000002.tif815). The calculation result is It is displayed as TIFF0007827422000003.tif910. k means the current time step, and + / - indicates whether the sensor information is reflected or not. If a measurement update was performed at the previous time step, the information of the previous time step is If a measurement update has not been performed in the previous time step, the information in the previous time step is The file is TIFF0007827422000005.tif915.

[0043] Time update is the process of "model-based estimation" of the information of the current time step using Kalman filter theory. The estimation result is The measurement update is shown in TIFF0007827422000006.tif1113. The hat indicates an estimate. TIFF0007827422000007.tif911 and This is an operation to perform a weighted sum of TIFF0007827422000008.tif810. TIFF0007827422000009.tif810 is the sensor information input for the current time step. The weighting value is TIFF0007827422000010.tif911 and The result is determined based on the accuracy or covariance of TIFF0007827422000011.tif911. It is displayed in TIFF0007827422000012.tif1313. When TIFF0007827422000013.tif1113 is input and measurement update is performed, the estimated result for the current time step is The file is TIFF0007827422000014.tif1111. If TIFF0007827422000015.tif1013 is not entered and measurement update is not performed, the current time step estimation result is TIFF0007827422000016.tif1113. At the next time step, TIFF0007827422000017.tif1212 or Model propagation is performed through TIFF0007827422000018.tif1214.

[0044] FIG. 3C shows an embodiment of the sensor fusion for the first estimation in step S351 and step S352 shown in FIG. 3A. The navigation system of FIG. 3C performs model propagation, measurement update, and time update in steps S353, S354, and S355, which correspond to steps S361, S362, and S363 of FIG. 3B. The navigation system can calculate the position of the current time step by applying the velocity of the current time step estimated through the sensor-based position in the model propagation operation to the position of the previous time step. For example, Like TIFF0007827422000019.tif963 TIFF0007827422000020.tif1010 may be calculated, where: TIFF0007827422000021.tif1010 is the current time step position, TIFF0007827422000022.tif917 is the position of the previous time step, TIFF0007827422000023.tif1010 is the current time step speed, TIFF0007827422000024.tif1012 is the time interval between two consecutive time steps. The calculated result is Corresponds to TIFF0007827422000025.tif1112.

[0045] The navigation device can generate a first-order estimation result through a time update operation. TIFF0007827422000026.tif1213, which may be referred to as a model-estimated reference position, as described above. The navigation system may also perform a weighted sum of the sensor information and the primary estimation result to generate a secondary estimation result. The secondary estimation result is This corresponds to TIFF0007827422000027.tif1313. The sensor information includes a position of the current time step estimated through a sensor-based positioning system and a position of the current time step estimated through an image-based positioning system. As described above, the position of the current time step estimated through an image-based positioning system may be referred to as an image-based reference position. The navigation device generates a first estimation result and a second estimation result through steps S353, S354, and S355, and can select one of the first estimation result and the second estimation result as a final position based on the error detection result. FIG. 4 shows an example of an error in the image-based reference position. In FIGS. 4A and 4B, the solid lines represent the actual driving path, and the dots represent the image-based estimated position. FIG. 4A shows a case where a vehicle is driving in the direction indicated by arrow 410 and a blockage occurs in section 412. For example, a blockage may occur if a vehicle ahead blocks the camera. Due to the blockage in section 412, it is difficult for the image information captured in section 412 to contain adequate information for determining the absolute position. Therefore, position 411 differs from the actual driving path.

[0046] FIG. 4B illustrates a case in which a vehicle is traveling in the direction indicated by arrow 420 and the number of lanes increases at boundary 422. If all lanes are shown in the image information, it is relatively easy to identify the lane number of the vehicle's traveling lane. However, if the image information shows only some lanes (e.g., if only two lanes are captured on a four-lane road), it is difficult to identify the lane number of the traveling lane. In such a situation, if lanes increase or decrease, the accuracy of the location information may be further reduced. For example, if a left-turn lane appears on lane 1, a lane change is required, such as changing from lane 2 to lane 3. However, information regarding this lane change may not be immediately reflected in the location information through the image information. Therefore, even if a vehicle is traveling on the same lane, an error in the image-based positioning may occur, resulting in the lane being recognized as having changed, as shown at position 421.

[0047] Referring to position 411 shown in FIG. 4A and position 421 shown in FIG. 4B, errors due to image-based side positioning are likely to involve lane jumps, which result in the vehicle being mistakenly recognized as a change in lane. This is because image-based side positioning results are provided by mapping them onto a high-precision map, which includes position information for each lane, and side positioning results are typically mapped onto the high-precision map based on these lanes. Furthermore, in situations such as road closures and lane changes, while image information may include information for estimating a detailed position within the current lane, it is highly likely that it does not include information for estimating the lane number of the current lane from the total number of lanes. Therefore, the navigation system can detect errors in the position estimated by image-based side positioning using such lane jumps as a reference.

[0048] 5 illustrates a comparison between a reference position and an acceptance region according to an embodiment. Referring to FIG. 5, a first acceptance region 511 for a first image-based reference position 510 and a second acceptance region 521 for a second image-based reference position 520 are illustrated. Each acceptance region is a visual representation of each acceptance region to aid in understanding each acceptance region.

[0049] More specifically, the first accommodation area 511 is a circle having a radius equal to the first accommodation range set for the first image-based reference position 510, and the center of the first accommodation area 511 is a first model-estimation-based reference position 512 corresponding to the first image-based reference position 510. Similarly, the second accommodation area 521 is a circle having a radius equal to the second accommodation range set for the second image-based reference position 520, and the center of the second accommodation area 521 is a second model-estimation-based reference position 522 corresponding to the second image-based reference position 520. Thus, when any image-based reference position belongs to the accommodation area, it means that the difference between the corresponding image-based reference position and the corresponding model-estimation-based reference position belongs to the accommodation range. In FIG. 5, the first accommodation range and the second accommodation range are assumed to be the same. For example, each accommodation range may have a width based on the width of each lane.

[0050] The navigation system may determine the distance between the first image-based reference position 510 and the first model-estimated reference position 512 as the error level of the first image-based reference position 510, and the distance between the second image-based reference position 520 and the second model-estimated reference position 522 as the error level of the second image-based reference position 520. The navigation system may also compare the error level of the first image-based reference position 510 with the first accommodation range, and compare the error level of the second image-based reference position 520 with the second accommodation range. In this case, since the first image-based reference position 510 belongs to the first accommodation range 511, the navigation system classifies the first image-based reference position 510 as no error. Furthermore, since the second image-based reference position 520 does not belong to the second accommodation range 521, the navigation system classifies the second image-based reference position 520 as an error.

[0051] Since accuracy in the lateral direction (Y) is more important than accuracy in the longitudinal direction (X) when a vehicle is traveling, the navigation system may determine the error level of each image-based reference position based on the lateral direction (Y). For example, the navigation system may determine the error level of the first image-based reference position 511 based on the difference between the y-coordinate of the first image-based reference position 510 and the y-coordinate of the first model-estimated reference position 512 in the coordinate system shown in FIG. 5, set a corresponding accommodation range, and compare it with the error level.

[0052] 6 shows how to set a range based on a driving situation according to an embodiment. Referring to FIG. 6, the navigation device determines the current driving situation in steps S611, S621, and S631, and sets a range based on the current driving situation in steps S612, S622, S632, and S633. In steps S612, S622, S632, and S633, TIFF0007827422000028.tif1011 shows the current time step range. TIFF0007827422000029.tif1212 indicates the first range, TIFF0007827422000030.tif1011 indicates the second storage range. Compared to TIFF0007827422000031.tif1011 TIFF0007827422000032.tif1112 will have a wider coverage range. For example, if the current driving situation is a cornering, the navigation system will If the current driving situation is crossroad driving, the navigation system will set the current coverage range. It can be set to TIFF0007827422000034.tif1313.

[0053] Even if the current driving situation does not involve both a corner turn and an intersection driving situation, the navigation system can adjust the size of the coverage area depending on whether the previous image-based reference position was classified as an error. For example, if the previous image-based reference position was classified as an error, the navigation system TIFF0007827422000035.tif1112 Compared to TIFF0007827422000036.tif923 If the previous image-based reference position is classified as error-free, the navigation system TIFF0007827422000038.tif1314 You can also set it to TIFF0007827422000039.tif1020. TIFF0007827422000040.tif1021 indicates the initial width, and has a value based on the width of the road, for example. If the error persists and the image-based reference position is continuously excluded in the position estimation, the error will spread. Therefore, if the error persists, the navigation device can gradually increase the width of the coverage area to prevent the error from spreading.

[0054] In step S640, the navigation device TIFF0007827422000041.tif1114 and Compare TIFF0007827422000042.tif1211. TIFF0007827422000043.tif1115 is the difference between the image-based reference position and the model-estimated reference position. If the difference is large, the error level of the image-based reference position is considered high. For example, TIFF0007827422000044.tif1318 shows the lateral difference between the image-based reference position and the model-estimated reference position. TIFF0007827422000045.tif1413 indicates the set storage range, where i is one of L, S, or k. If TIFF0007827422000046.tif1216 belongs to the range, for example, TIFF0007827422000047.tif1216 If it is less than TIFF0007827422000048.tif1514, the navigation system classifies the corresponding image-based reference position as error-free. If TIFF0007827422000049.tif1216 does not belong to the range, for example, TIFF0007827422000050.tif1419 If it is greater than TIFF0007827422000051.tif1615, the navigation system classifies the corresponding image-based reference position as erroneous.

[0055] FIG. 7 illustrates detection of a turning situation according to an embodiment. Because road-related information is generally distributed widely in the horizontal direction, longitudinal estimation tends to be less accurate than lateral estimation. When a vehicle turns around a corner, longitudinal inaccuracy affects lateral estimation for a certain period of time before and after the turn. Lateral inaccuracy can lead to fatal consequences, such as estimating that the vehicle is outside the roadway or is positioned on the wrong road. Therefore, such lateral inaccuracy needs to be corrected urgently.

[0056] A navigation system can use image information to expand its coverage range to reduce lateral inaccuracies. Referring to FIG. 7, it can be seen that the coverage range 720 after rotation is wider than the coverage range 710 before rotation. This increases the likelihood that the image-based reference position can be used to estimate the final position, even if the error level of the image-based reference position is slightly high. The navigation system accumulates gyroscope output for a certain period of time, and if the accumulated angle during this period exceeds a threshold, it can determine that the driving situation corresponds to rotational driving. If the accumulated angle subsequently falls below the threshold, the navigation system determines that the rotation has ended, and from that moment on, it can expand its coverage range and converge the vehicle's lateral position to the image-based reference position.

[0057] FIG. 8 illustrates intersection detection according to an embodiment. Referring to FIG. 8, a current estimated position 810 of a vehicle is displayed on a high-precision map 800. Each point on the high-precision map 800 may include information about various elements (e.g., lanes, centerlines, guide signs, etc.). Therefore, it is possible to determine what map elements are present around the current estimated position 810 through the points on the high-precision map 800. The navigation system checks whether lanes exist in a surrounding area 820 of the current estimated position 810. If no lanes exist, it determines that the current estimated position 810 is an intersection. Image-based positioning algorithms that recognize road environments are likely to encounter errors in intersection sections where lanes disappear or new lanes appear. To minimize position estimation errors at intersections, the navigation system can reduce its coverage area for a certain period of time after passing through an intersection.

[0058] FIG. 9 illustrates an expansion of the accommodation range according to an embodiment. Referring to FIG. 9, the image-based reference position begins to leave the accommodation area at time step t2, and this state is maintained until time step t4. Therefore, the image-based reference position from time steps t2 to t4 is not used to estimate the final position. However, if the image-based reference position is continuously ignored in such a situation where an error persists, the error may diverge, resulting in a lower accuracy of estimation compared to using the image-based reference position. Therefore, if the error persists, the navigation system gradually increases the size of the accommodation range to prevent the error from diverging. The navigation system expands the accommodation area starting from time step t2, when the image-based reference position begins to leave the accommodation area, and eventually, the image-based reference position enters the accommodation area at time step t5. Therefore, the navigation system can use the image-based reference position to estimate the final position from time step t5.

[0059] 10 illustrates a process for detecting an error using a heading angle according to an embodiment. A navigation system can classify an image-based reference position using other methods in addition to the method using the error level and coverage range described above. FIG. 10 illustrates one such method, which uses a heading angle.

[0060] The navigation system can determine the change in reference heading angle corresponding to the image-based reference position of the current time step based on multiple image-based reference positions corresponding to multiple time steps, including the current time step. The navigation system compares this with the output of the gyroscope to detect an error related to the image-based reference position of the current time step. If the difference is greater than a threshold, the navigation system classifies the image-based reference position as erroneous; if the difference is smaller, the navigation system classifies the image-based reference position as non-error. If the error level is within the acceptable range and the difference in heading angle is less than the threshold—in other words, if both validity conditions are met—the navigation system classifies the image-based reference position as non-error and can use it to estimate the current position.

[0061] Referring to FIG. 10, image-based reference positions for three consecutive time steps (k-2 to k) are JPEG0007827422000052.jpg1145 is shown. In this case, the average heading angle can be calculated using Equation (1).

number

[0062] In formula (1), TIFF0007827422000054.tif1212 shows the latitude of the current time step, TIFF0007827422000055.tif1310 shows the longitude of the current time step, TIFF0007827422000056.tif1121 shows the latitude of the previous time step, TIFF0007827422000057.tif1321 shows the longitude of the previous time step. The latitude and longitude are based on the image-based reference position. It can be grasped through the coordinates of JPEG0007827422000058.jpg1145. The average heading angle between the current time step and the previous time step is calculated using Equation (1). TIFF0007827422000059.tif1011 is calculated. As shown in FIG. 10, if three or more time steps are given, the average heading angle during the given time steps can be calculated.

[0063] Since a gyroscope is a sensor that detects rotation through angular velocity, the output of the gyroscope is stored at each time step in which the image-based reference position is measured, and the average heading change during the time step is calculated using the stored output. The navigation device compares the average heading change calculated through the image-based reference position with the average heading change calculated through the gyroscope output, and if the difference is greater than a threshold, the navigation device can classify the associated image-based reference position (e.g., the image-based reference position of the most recent time step) as erroneous.

[0064] 11 shows a vehicle positioning method according to one embodiment. Referring to FIG. 11, in step S1110, the navigation device determines a first reference position based on vehicle position information measured via a vehicle position sensor, in step S1120, determines a second reference position of the vehicle based on image information of the vehicle captured by a vehicle camera, in step S1130, sets a range for the second reference position based on the vehicle's driving conditions, in step S1140, compares the second reference position with the first reference position to predict an error level of the second reference position, and in step S1150, compares the error level of the second reference position with the range to estimate the vehicle's current position. The vehicle positioning method is similar to the descriptions of FIGS. 1 to 10, 12, and 13.

[0065] 12 shows the configuration of a navigation device according to one embodiment. Referring to FIG. 12, the navigation device 1200 includes a processor 1210 and a memory 1220. The memory 1220 is connected to the processor 1210 and can store instructions executable by the processor 1210, data operated by the processor 1210, or data processed by the processor 1210. The memory 1220 can include a non-transitory computer-readable medium, such as a high-speed random access memory and / or a non-volatile computer-readable storage medium (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0066] The processor 1210 executes commands to perform one or more operations described with reference to Figures 1 to 11 and 13. For example, the processor 1210 may determine a first reference position based on vehicle position information measured via a vehicle position sensor, determine a second reference position of the vehicle based on image information of the vehicle captured by a vehicle camera, set an accommodation range for the second reference position based on the vehicle's driving conditions, compare the second reference position with the first reference position to predict an error level of the second reference position, and compare the error level of the second reference position with the accommodation range to estimate the vehicle's current position. In addition, the descriptions with reference to Figures 1 to 11 and 13 may be applied to the navigation device 1200.

[0067] 13 shows the configuration of an electronic device according to one embodiment. Referring to FIG. 13, the electronic device 1300 includes a processor 1310, a memory 1320, a camera 1330, a sensor 1340, a control system 1350, a storage device 1360, an input device 1370, an output device 1380, and a network interface 1390. These can communicate with each other via a communication bus 1305.

[0068] The electronic device 1300 estimates the position of a vehicle through sensor-based and image-based measurements and performs subsequent operations using the estimated position. For example, the electronic device 1300 corresponds to a vehicle control device and performs various subsequent operations related to the vehicle (e.g., driving function control, additional function control, etc.). For example, the electronic device 1300 may structurally and / or functionally include the navigation device 100 shown in FIG. 1 and be implemented as a part of a vehicle (e.g., an autonomous vehicle).

[0069] The processor 1310 executes functions and instructions to be executed within the electronic device 1300. For example, the processor 1310 processes instructions stored in the memory 1320 and / or the storage device 1360. According to one embodiment, the processor 1310 estimates the vehicle's position through sensor-based and image-based measurements, and generates control commands for driving the vehicle based on the estimated position. In addition, the processor 1310 may perform one or more of the operations described with reference to FIGS. 1-12.

[0070] The memory 1320 stores data for the vehicle side system. The memory 1320 may include a computer-readable storage medium or a computer-readable storage device. The memory 1320 may store instructions to be executed by the processor 1310 and may store information related to the execution of software and / or applications by the electronic device 1300.

[0071] The camera 1330 captures video and / or still images (photos). For example, the camera 1330 may be installed in a vehicle and capture images of the vehicle's surroundings in predetermined directions, such as in front, to the side, behind, above, or below the vehicle, to generate image information related to the vehicle's movement. According to one embodiment, the camera 1330 may provide 3D images including depth information related to objects.

[0072] The sensors 1340 can sense information such as visual, auditory, and tactile information related to the electronic device 1300. For example, the sensors 1340 may include position sensors, ultrasonic sensors, radar sensors, and lidar sensors. The control system 1350 controls the vehicle based on control instructions from the processor 1310. For example, the control system 1350 can physically control various functions related to the vehicle, including driving functions such as acceleration, steering, and braking of the vehicle, and additional functions such as opening and closing doors, closing and activating airbags.

[0073] Storage device 1360 may include a computer-readable storage medium or computer-readable storage device. According to one embodiment, storage device 1360 stores a larger amount of information than memory 1320 and is capable of storing information for a longer period of time. For example, storage device 1360 may include a magnetic hard disk, an optical disk, flash memory, a floppy disk, or different forms of non-volatile memory known in the art.

[0074] Input device(s) 1370 receive input from a user through traditional input methods such as a keyboard and mouse, and newer input methods such as touch input, voice input, and image input. For example, input device(s) 1370 may include a keyboard, mouse, touchscreen, microphone, or any other device capable of detecting input from a user and communicating the detected input to electronic device 1300.

[0075] The output device(s) 1380 provide output of the electronic device 1300 to a user through visual, auditory, or tactile channels. The output device(s) 1380 may include, for example, a display, a touchscreen, a speaker, a vibration generator, or any other device capable of providing output to a user. The network interface 1390 can communicate with external devices through a wired or wireless network.

[0076] The methods of the present invention may be embodied in the form of program instructions that can be executed by various computer means and stored on a computer-readable storage medium. The storage medium may include program instructions, data files, data structures, and the like, alone or in combination. The storage medium and program instructions may be specially designed and constructed for the purposes of the present invention, or they may be well-known and available to those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code that is executed by a computer using an interpreter, for example. A hardware device may be configured to operate as one or more software modules to perform the operations described in the present invention, or vice versa.

[0077] Software includes computer programs, codes, instructions, or a combination of one or more thereof, which can configure a processing device to operate as desired or can independently or in combination instruct the processing device. The software and / or data can be permanently or temporarily embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave to be interpreted by the processing device or to provide instructions or data to the processing device. The software can be distributed across computer systems coupled to a network and stored and executed in a distributed manner. The software and data can be stored on one or more computer-readable recording media.

[0078] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted with other components or equivalents, and still achieve suitable results.

[0079] Therefore, other implementations, other embodiments, and equivalents of the claims are also within the scope of the following claims. [Explanation of symbols]

[0080] 100 Navigational Equipment 410 Arrow 411 position 412 sections 420 Arrow 421 position 422 Boundary 510 First Image Base Reference Position 511 Containment Area 1 512 First model estimated base reference position 520 Second Image Base Reference Position 521 Second Containment Area 522 Second model estimated base reference position 710 Containment Area 720 Containment Area 800 Map 810 Estimated position 820 Surrounding Area 1200 Navigation Equipment 1210 processor 1220 memory 1300 Electronic equipment 1305 Communication Bus 1310 processor 1320 memory 1330 Camera 1340 Sensor 1350 Control System 1360 Enclosure 1370 Input Device 1380 output device 1390 network interface

Claims

1. determining a first reference position based on position information of the vehicle measured using a position sensor of the vehicle; determining a second reference position of the vehicle based on image information of the vehicle captured using a camera of the vehicle; setting an accommodation range for the second reference position based on a running state of the vehicle; comparing the second reference position to the first reference position to estimate an error level of the second reference position; comparing the error level of the second reference position with the coverage area to estimate a current position of the vehicle; the step of setting an accommodation range for the second reference position based on a driving situation of the vehicle includes a step of determining the driving situation from a plurality of driving situations; The plurality of driving situations include at least one of a first driving situation in which the vehicle travels through an intersection and a second driving situation in which the vehicle travels around a corner, Vehicle side position method.

2. The vehicle positioning method according to claim 1 , wherein the step of determining the second reference position includes a step of determining the second reference position based on geometric information of a lane shown in the image information.

3. 3. The vehicle side positioning method according to claim 1, wherein the second reference position includes at least one of information about a road to which the vehicle belongs and information about a detailed position of the vehicle within the road to which the vehicle belongs.

4. The vehicle positioning method according to claim 1 , wherein the step of setting the accommodation range includes a step of selectively adjusting the size of the accommodation range in accordance with the driving situation.

5. 5. The vehicle positioning method according to claim 1, wherein the step of setting the accommodation range includes a step of setting a wider accommodation range for the second driving condition than the accommodation range for the first driving condition.

6. 6. The vehicle positioning method according to claim 1, wherein the step of estimating the current position includes a step of estimating the current position without taking the second reference position into consideration if the error level of the second reference position does not fall within the accommodation range.

7. The step of estimating the current location includes: if the error level of the second reference position does not fall within the range, estimating the first reference position as the current position of the vehicle; and estimating a new position estimated based on a weighted sum of the second reference position and the first reference position as the current position if the error level of the second reference position falls within the accommodation range. The vehicle side positioning method according to any one of claims 1 to 6.

8. 8. The vehicle side positioning method according to claim 1, wherein the step of estimating the error level comprises a step of determining the error level of the second reference position based on a distance between the second reference position and the first reference position.

9. The vehicle side positioning method further includes a step of setting another accommodation range of another second reference position in a next time step; the step of setting the other accommodation range includes a step of setting the size of the other accommodation range to be larger than the size of the accommodation range when the second reference position is not within the accommodation range. The vehicle side positioning method according to any one of claims 1 to 8.

10. The vehicle positioning method further includes determining a change in a reference heading angle corresponding to the second reference position based on the second reference position and a plurality of second reference positions corresponding to a plurality of previous time steps; the step of estimating the current position includes estimating the current position by further considering a comparison result between a gyroscope output and the determined change in the reference heading angle. The vehicle side positioning method according to any one of claims 1 to 9.

11. A computer program stored on a recording medium for executing the method according to any one of claims 1 to 10 in combination with hardware.

12. determining a first reference position based on position information of the vehicle measured using a position sensor of the vehicle; determining a second reference position of the vehicle based on image information of the vehicle captured using a camera of the vehicle; setting an accommodation range for the second reference position based on a running state of the vehicle; comparing the second reference position to the first reference position to estimate an error level of the second reference position; comparing the error level of the second reference position with the coverage area to estimate a current position of the vehicle; a processor; The processor determines the driving situation from a plurality of driving situations; The plurality of driving situations include at least one of a first driving situation in which the vehicle travels through an intersection and a second driving situation in which the vehicle travels around a corner, Navigation equipment.

13. 13. The navigation system of claim 12, wherein the processor determines the error level of the second reference position based on a distance between the second reference position and the first reference position.

14. The processor: If the error level of the second reference position does not fall within the range, estimating the first reference position as the current position of the vehicle; If the error level of the second reference position falls within the range, a new position estimated based on a weighted sum of the second reference position and the first reference position is estimated as the current position. A navigation system according to claim 12 or 13.

15. The navigation device according to claim 12 , wherein the navigation device is a vehicle further including a control system for controlling the vehicle based on a current position, the position sensor, and the camera.

16. 16. The navigation device according to claim 12, wherein the processor sets a wider accommodation range for the second driving situation than a accommodation range for the first driving situation.

17. A navigation device as described in any one of claims 12 to 16, wherein the processor sets another accommodation range for another second reference position in the next time step, but if the second reference position is not within the accommodation range, sets the width of the other accommodation range to be wider than the width of the accommodation range.

18. The navigation device is further comprising a memory for storing an instruction word; When the instruction is executed by the processor, the processor determines the first reference position, determines the second reference position, sets the accommodation range, compares the second reference position with the first reference position to predict the error level, and compares the error level of the second reference position with the accommodation range to estimate the current position. A navigation system according to any one of claims 12 to 17.

19. determining a first reference position based on position information of the vehicle measured using a position sensor of the vehicle; determining a second reference position of the vehicle based on image information of the vehicle captured using a camera of the vehicle; setting an accommodation range for the second reference position based on a running state of the vehicle; comparing the second reference position to the first reference position to estimate an error level of the second reference position; comparing the error level of the second reference position with the coverage area to estimate a current position of the vehicle; generating a control command for driving the vehicle based on the current position; a processor; a control system that controls the vehicle based on the control command, The processor determines the driving situation from a plurality of driving situations; The plurality of driving situations include at least one of a first driving situation in which the vehicle travels through an intersection and a second driving situation in which the vehicle travels around a corner, Vehicle control device.

20. The vehicle control device according to claim 19, wherein the vehicle control device is a vehicle further including the camera.

21. 21. The vehicle control device according to claim 19, wherein the processor determines the error level of the second reference position based on a distance between the second reference position and the first reference position.

22. The processor: If the error level of the second reference position does not fall within the range, estimating the first reference position as the current position of the vehicle; If the error level of the second reference position falls within the range, a new position estimated based on a weighted sum of the second reference position and the first reference position is estimated as the current position.

22. A vehicle control device according to any one of claims 19 to 21.

23. determining a first reference position based on position information of the vehicle measured using a position sensor of the vehicle; determining a second reference position of the vehicle based on image information of the vehicle captured using a camera of the vehicle; selecting, based on a determination of whether the second reference position is an erroneous position, either to estimate a current position of the vehicle based on the first reference position and the second reference position, or to estimate the current position of the vehicle based on the first reference position; and estimating the current location based on the result of the selection; the determination of whether the second reference position is the erroneous position is based on an error consideration for the second reference position that is dependent on a current driving situation of the vehicle; The determination of whether the second reference location is the error location comprises: and considering at least one or both of a previous time step in which the determined second reference position was determined to be erroneous and a comparison between the output of the vehicle's gyroscope and a determined change in reference heading angle corresponding to the second reference position. Vehicle side position method.

24. The determination of whether the second reference location is the error location comprises: Based on the coverage area dependent on the determination of the current driving situation, based on whether the estimated error of the second reference position is within the accommodation range; 24. The vehicle side positioning method according to claim 23.

25. 25. The vehicle positioning method according to claim 24, wherein the coverage range is set differently for at least two different driving situations, including being set wider in a cornering situation than in an intersection situation.

Images