Map generation device

Abstract

The map generation device includes: an outside detecting section that detects an outside situation around the host vehicle; a map generating section that generates a map including position information of a road marking that defines a travel lane of the host vehicle, based on information of the detected outside situation; a road marking estimating section that estimates a position of a road marking predicted to be detected at a second time point, based on information of the outside situation detected at a first time point; and a determining section that determines whether or not there is a lane change of the host vehicle, based on a difference between the estimated position of the road marking and a position of the road marking detected by the outside detecting section at the second time point, and the map generating section generates the map based on a result of the determination.

Description

Technical Field

[0001] The present invention relates to a map generation apparatus that generates a map based on data showing external conditions obtained during driving. Background Technology

[0002] As such a device, there are previously known devices that use images captured by a camera mounted on a vehicle to identify road markings and detect lane changes, thereby deducing that the vehicle is not traveling in a lane and updating map data. Such a device is described, for example, in Patent Document 1. In the device described in Patent Document 1, lane changes are detected when the area within the camera image where the road markings are located gradually moves to the left or right.

[0003] However, in the device described in Patent Document 1, continuous detection of road markings is required to detect lane changes. This is because lane changes cannot be detected if road markings are interrupted midway, making it difficult to accurately generate a map including the location information of the road markings.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent document 1: Japanese Patent Application Publication No. 2007-241470 (JP2007-241470A). Summary of the Invention

[0007] A map generation apparatus according to one embodiment of the present invention comprises: an external environment detection unit that detects the external environment surrounding a vehicle; a map generation unit that generates a map, including the position information of road markings defining the vehicle's driving lane, based on the external environment information detected by the external environment detection unit; a road marking estimation unit that estimates the position of a road marking detected by the external environment detection unit at a second time point predicted to be after the first time point, based on the external environment information detected by the external environment detection unit at a first time point; and a determination unit that determines whether the vehicle has changed lanes based on the difference between the position of the road markings estimated by the road marking estimation unit and the position of the road markings detected by the external environment detection unit at the second time point. The map generation unit generates a map based on the determination result of the determination unit. Attached Figure Description

[0008] The objectives, features, and advantages of the present invention are further illustrated by the following description of embodiments in conjunction with the accompanying drawings.

[0009] Figure 1 This is a block diagram that schematically illustrates the overall configuration of a vehicle control system having a map generation apparatus according to an embodiment of the present invention.

[0010] Figure 2This is a diagram illustrating an example of a driving scenario using a map generation device according to an embodiment of the present invention.

[0011] Figure 3 This is a block diagram illustrating the main components of a map generation apparatus according to an embodiment of the present invention.

[0012] Figure 4 This is a diagram showing an example of an approximate curve calculated by the map generation apparatus according to an embodiment of the present invention.

[0013] Figure 5A This diagram illustrates an example of the action related to the determination of lane change performed by the map generation apparatus according to an embodiment of the present invention.

[0014] Figure 5B This is a diagram illustrating another example of the action related to the determination of lane change performed by the map generation apparatus according to an embodiment of the present invention.

[0015] Figure 5C This is another example of the action related to the determination of lane change performed by the map generation apparatus according to an embodiment of the present invention.

[0016] Figure 6 It is shown by Figure 3 A flowchart of an example of the processing performed by the controller. Detailed Implementation

[0017] The following is for reference Figures 1-6 Embodiments of the present invention will now be described. The map generation apparatus of the present invention is configured to generate a map (described later) used when, for example, a vehicle with autonomous driving capabilities (autonomous driving vehicle) is in motion. It should be noted that, sometimes, the vehicle equipped with the map generation apparatus of this embodiment is referred to as "this vehicle," distinguishing it from other vehicles.

[0018] The map generation performed by the map generation device is executed when the driver manually drives the vehicle. Therefore, the map generation device can be installed in vehicles without autonomous driving capabilities (manually driven vehicles). It should be noted that the map generation device can be installed not only in manually driven vehicles but also in autonomous vehicles, which can switch from an autonomous driving mode that does not require driver operation to a manual driving mode that requires driver operation. The following description focuses on the case where the map generation device is installed in an autonomous vehicle.

[0019] First, the composition of an autonomous vehicle will be explained. This vehicle can be any one of the following: an engine vehicle with an internal combustion engine as the driving source, an electric vehicle with a motor as the driving source, or a hybrid vehicle with both an engine and a motor as driving sources. Figure 1 This is a block diagram that schematically illustrates the overall configuration of a vehicle control system 100 having a map generation apparatus according to an embodiment of the present invention.

[0020] like Figure 1 As shown, the vehicle control system 100 mainly includes a controller 10 and external sensor group 1, internal sensor group 2, input / output device 3, positioning unit 4, map database 5, navigation device 6, communication unit 7, and driving actuator AC, which are communicatively connected to the controller 10 via CAN communication line, etc.

[0021] External sensor group 1 is a collective term for multiple sensors (external sensors) that detect information about the vehicle's surroundings, i.e., external conditions. For example, external sensor group 1 includes: a lidar that detects the position (distance and direction from the vehicle) of objects around the vehicle by irradiating laser light and detecting the reflected light; a radar that detects the position of objects around the vehicle by irradiating electromagnetic waves and detecting the reflected waves; and cameras with imaging elements such as CCD (charge-coupled device) and CMOS (complementary metal-oxide-semiconductor) to capture images of the vehicle's surroundings (front, rear, and sides).

[0022] Internal sensor group 2 is a collective term for multiple sensors (internal sensors) that detect the driving status of the vehicle. For example, internal sensor group 2 includes: a vehicle speed sensor to detect the vehicle's speed, an acceleration sensor to detect the vehicle's acceleration in the forward and backward and left and right directions respectively, and a speed sensor to detect the rotational speed of the driving engine. Sensors that detect the driver's driving operations in manual driving mode, such as operation of the accelerator pedal, brake pedal, and steering wheel, are also included in internal sensor group 2.

[0023] Input / output device 3 is a general term for devices that input commands to the driver and output information to the driver. For example, input / output device 3 includes: various switches for the driver to input various commands by operating the control components, a microphone for the driver to input commands by voice, a display that provides information to the driver by displaying images, and a speaker that provides information to the driver by voice, etc.

[0024] The positioning unit (GNSS unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning sensor can also be included in the internal sensor group 2. Positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 4 uses the positioning information received by the positioning sensor to determine the current position (latitude, longitude, and altitude) of the vehicle.

[0025] Map database 5 is a device that stores general map information used in navigation device 6, and is composed of, for example, a hard disk and semiconductor components. The map information includes: road location information, road shape (curvature, etc.) information, and the location information of intersections and forks in the road. It should be noted that the map information stored in map database 5 is different from the high-precision map information stored in storage unit 12 of controller 10.

[0026] The navigation device 6 is a device that searches for a target path on the road leading to the destination input by the driver and guides the user along the target path. The destination is input and the user is guided along the target path via the input / output device 3. The target path is calculated based on the vehicle's current position determined by the positioning unit 4 and map information stored in the map database 5. Alternatively, the vehicle's current position can be determined using the detection values ​​from the external sensor group 1, and the target path can be calculated based on this current position and high-precision map information stored in the storage unit 12.

[0027] Communication unit 7 communicates with various servers (not shown) via a network including wireless communication networks such as the Internet and mobile phone networks, and periodically or at any time retrieves map information, driving record information, and traffic information from the servers. The network includes not only public wireless communication networks but also closed communication networks set up in each designated management area, such as wireless LAN, Wi-Fi (registered trademark), and Bluetooth (registered trademark). The retrieved map information is output to map database 5 and storage unit 12, and the map information is updated. Communication unit 7 can also communicate with other vehicles.

[0028] An actuator (AC) is a driving actuator used to control the movement of the vehicle. When the driving source is an engine, the actuator AC includes a throttle actuator for adjusting the opening of the engine's throttle valve (throttle opening). When the driving source is a drive motor, the drive motor is included in the actuator AC. Braking actuators that operate the vehicle's braking system and steering actuators that drive the steering system are also included in the actuator AC.

[0029] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 is configured as a computer including an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as ROM (read-only memory) and RAM (random access memory), and other peripheral circuits (not shown) such as I / O (input / output) interfaces. It should be noted that multiple ECUs with different functions, such as an engine control ECU, a drive motor control ECU, and a braking system ECU, can be set up separately, but for convenience, they are shown separately in the following diagram. Figure 1 The controller 10 is shown as a collection of these ECUs.

[0030] High-precision road map information is stored in storage unit 12. This road map information includes: road location information, road shape (curvature, etc.) information, road slope information, intersection and fork location information, number of lanes information, lane width and position information of each lane (center position of the lane, lane boundary information), position information of landmarks (traffic lights, signs, buildings, etc.) used as markers on the map, and road surface features such as unevenness. The map information stored in storage unit 12 includes: map information obtained from outside the vehicle via communication unit 7 and map information generated by the vehicle itself using detection values ​​from external sensor group 1 or detection values ​​from external sensor group 1 and internal sensor group 2. Storage unit 12 also stores driving record information composed of detection values ​​from external sensor group 1 and internal sensor group 2 in a correspondence with the map information.

[0031] The computing unit 11 has a functional structure including a vehicle position recognition unit 13, an external recognition unit 14, an action plan generation unit 15, a driving control unit 16, and a map generation unit 17.

[0032] The vehicle position recognition unit 13 identifies the vehicle's position on the map (vehicle position) based on the vehicle's position information acquired by the positioning unit 4 and the map information in the map database 5. It can also use map information stored in the storage unit 12 and surrounding information detected by the external sensor group 1 to identify the vehicle's position, thereby enabling high-precision identification of the vehicle's position. It should be noted that when the vehicle's position can be determined using external sensors installed on or beside the road, the vehicle's position can also be identified by communicating with those sensors via the communication unit 7.

[0033] The external identification unit 14 identifies the external conditions around the vehicle based on signals from the external sensor group 1, such as lidar, radar, and cameras. For example, it identifies the position, speed, and acceleration of surrounding vehicles (vehicles in front and behind) traveling around the vehicle, the position of surrounding vehicles parked or stationary around the vehicle, and the position and state of other objects. Other objects include: signs, traffic lights, road markings or stop lines, buildings, guardrails, utility poles, billboards, pedestrians, bicycles, etc. The state of other objects includes: the color of traffic lights (red, green, yellow), the speed and direction of pedestrians and bicycles, etc.

[0034] The action plan generation unit 15 generates the vehicle's driving trajectory (target trajectory) from the current time to a predetermined time based on, for example, the target path calculated by the navigation device 6, map information stored in the storage unit 12, the vehicle's position identified by the vehicle position recognition unit 13, and the external conditions identified by the external environment recognition unit 14. When multiple trajectories exist as candidate target trajectories on the target path, the action plan generation unit 15 selects the optimal trajectory that complies with laws and meets criteria such as efficient and safe driving, and designates the selected trajectory as the target trajectory. Then, the action plan generation unit 15 generates an action plan corresponding to the generated target trajectory. The action plan generation unit 15 generates various action plans corresponding to overtaking, lane changing, following, maintaining lane position, deceleration, or acceleration. When generating the target trajectory, the action plan generation unit 15 first determines the driving mode and generates the target trajectory based on the driving mode.

[0035] In autonomous driving mode, the driving control unit 16 controls each actuator AC to make the vehicle travel along the target trajectory generated by the action plan generation unit 15. More specifically, in autonomous driving mode, the driving control unit 16 considers driving resistance determined by factors such as road gradient and calculates the required driving force to obtain the target acceleration per unit time calculated by the action plan generation unit 15. Then, for example, feedback control is performed on the actuator AC to make the actual acceleration detected by the internal sensor group 2 the target acceleration. That is, the actuator AC is controlled to make the vehicle travel at the target speed and target acceleration. It should be noted that when the driving mode is manual driving mode, the driving control unit 16 controls each actuator AC according to driving commands (steering operations, etc.) obtained from the driver by the internal sensor group 2.

[0036] While driving in manual driving mode, the map generation unit 17 generates an environmental map composed of three-dimensional point cloud data using detection values ​​detected by the external sensor group 1. Specifically, from images captured by the camera, the edges that show the contours of objects are extracted based on the brightness and color information of each pixel, and feature points are extracted using this edge information. Feature points include, for example, points on edges, intersections of edges, and correspond to markings on the road surface, corners of buildings, corners of road signs, etc. The map generation unit 17 calculates the distance to the extracted feature points and plots the feature points sequentially on the environmental map, thereby generating an environmental map of the area surrounding the road through which the vehicle has traveled. Alternatively, instead of a camera, feature points of objects around the vehicle can be extracted using data obtained by radar or lidar to generate an environmental map.

[0037] The vehicle position recognition unit 13 and the map generation unit 17 perform vehicle position estimation processing in parallel. That is, the vehicle's position is estimated based on the changes in the positions of feature points over time. Map creation and position estimation are performed simultaneously using signals from, for example, cameras or LiDAR, according to the SLAM (Simultaneous Localization and Mapping) algorithm. The map generation unit 17 can generate environment maps not only when driving in manual mode but also when driving in automatic mode. If an environment map has already been generated and stored in the storage unit 12, the map generation unit 17 can also update the environment map based on newly obtained feature points.

[0038] Next, the configuration of the map generation apparatus of this embodiment, that is, the map generation apparatus of the vehicle control system 100, will be described. Figure 2 This diagram illustrates an example of a road 200 using the map generation apparatus of this embodiment. Figure 2 The diagram illustrates an example of vehicle 101 changing lanes. Specifically, vehicle 101 changes lanes from the first lane LN1, defined by left and right road markings 201 and 202, to the second lane LN2, defined by left and right road markings 202 and 203, as shown by arrow A. Vehicle 101 travels in the first lane LN1 at a first time point t1 (state B1), and then travels in the second lane LN2 at a second time point t2 (state B2).

[0039] A camera 1a is mounted on the front of the vehicle 101. The camera 1a is capable of capturing a roughly fan-shaped shooting area AR centered on the camera 1a, determined by a predetermined field of view θ. The shooting area AR of the vehicle 101 at a first time point t1 includes road markings 201 and 202, and the shooting area AR at a second time point t2 includes road markings 202 and 203. Therefore, by extracting edge points from the camera image, road markings 201 and 202 can be detected at the first time point t1 and the second time point t2, respectively. Thus, by sequentially connecting the road markings (strictly speaking, feature points corresponding to the road markings) detected from the camera image in a temporal sequence, a map including road markings 201 to 203 can be generated.

[0040] However, when changing lanes from lane 1 (LN1) to lane 2 (LN2), when... Figure 2 When the road markings 202 shown are interrupted, they sometimes cannot be detected continuously. Even if the road markings 202 are not interrupted, they sometimes cannot be detected due to obstacles (such as other vehicles) obstructing the view of camera 1a, or due to decreased detection accuracy of camera 1a caused by weather conditions. As a result, the road markings 201 detected at the first time point t1 and the road markings 202 detected at the second time point t2, as well as the road markings 202 detected at the first time point t1 and the road markings 203 detected at the second time point t2, are connected to generate incorrect maps. Therefore, this embodiment is configured as follows to generate maps correctly even when lane changes occur.

[0041] Figure 3 This is a block diagram showing the main components of the map generation apparatus 20 according to this embodiment. The map generation apparatus 20 includes... Figure 1 In the vehicle control system 100. For example... Figure 3 As shown, the map generation device 20 includes a camera 1a, a sensor 2a, and a controller 10.

[0042] Camera 1a is a single-lens reflex camera with imaging elements (image sensors) such as CCD and CMOS, constituting... Figure 1 It is part of the external sensor group 1. Camera 1a can also be a stereo camera. For example... Figure 2 As shown, camera 1a is installed at a predetermined position at the front of the vehicle 101, and continuously captures images of the space in front of the vehicle 101 to obtain images of objects (camera images). It should be noted that lidar or other similar devices can also be used to replace camera 1a or detect objects together with camera 1a.

[0043] Sensor 2a is a detector used to calculate the amount and direction of movement of the vehicle 101. Sensor 2a is part of the internal sensor group 2, and may consist of, for example, a vehicle speed sensor and a yaw rate sensor. Specifically, the controller 10 (vehicle position recognition unit 13) integrates the vehicle speed detected by the vehicle speed sensor to calculate the amount of movement of the vehicle 101, and integrates the yaw rate detected by the yaw rate sensor to calculate the yaw angle. When creating a map, the position of the vehicle 101 is estimated using a range estimation method. It should be noted that the configuration of sensor 2a is not limited to this; information from other sensors can also be used to estimate the vehicle's position.

[0044] Figure 3 The controller 10 serves as the arithmetic unit 11. Figure 1 In addition to the storage unit 12 and the map generation unit 17, the functional structure of the map generation unit 17 also includes a road marking estimation unit 21 and a determination unit 22. It should be noted that the road marking estimation unit 21 and the determination unit 22 also have map generation functions, so they can also be included in the map generation unit 17.

[0045] The storage unit 12 stores map information. This stored map information includes map information obtained from outside the vehicle 101 via the communication unit 7 (referred to as external map information) and map information created by the vehicle itself (referred to as internal map information). External map information is information from maps obtained via, for example, a cloud server (referred to as cloud maps), while internal map information is information from map data generated by mapping using technologies such as SLAM (referred to as environmental maps). External map information is shared by the vehicle 101 and other vehicles; in contrast, internal map information is map information unique to the vehicle 101 (e.g., map information owned solely by the vehicle). The storage unit 12 also stores information about various control programs, thresholds used in those programs, etc.

[0046] The road marking estimation unit 21 estimates the position of the road markings predicted to be detected by camera 1a at the current time point (or future time point) based on camera images acquired at past time points (or the current time point). Specifically, from past time points (e.g., ... Figure 2 In the camera image acquired at the first time point t1, feature points corresponding to the road markings 201 and 202 contained in the camera image are extracted, and an approximate curve passing through the feature points is calculated. Not only past time points, but also feature points before past time points can be used to calculate approximate curves, such as feature points from several frames before past time points. Figure 4This diagram illustrates an example of approximate curves L1 and L2. In the diagram, the horizontal and vertical axes represent the X and Y coordinates of a feature point P with any location as the origin. The X and Y coordinates are taken on the road surface. Feature point P is divided into a group P1 (called the left feature point group) corresponding to one side of a pair of road markings (e.g., the left side) and a group P2 (called the right feature point group) corresponding to the other side (e.g., the right side).

[0047] The approximate curves L1 and L2 are represented by polynomials, such as polynomials derived from multiple feature points P detected at the first time point t1, i.e., polynomials where the Y-coordinate value is a function of the X-coordinate value. It should be noted that the approximate curves L1 and L2 can also be calculated by smoothly extending the curve obtained by sequentially and smoothly connecting the feature points along the vehicle's direction of travel along the same direction of travel. Within the dashed region Ra including feature point P, the approximate curves L1 and L2 are identical or approximately identical to the road markings 201 and 202 detected by the camera image. The calculated approximate curves L1 and L2 are stored in the storage unit 12.

[0048] As the vehicle 101 moves, when a new feature point P corresponding to the road markings is detected, the road marking estimation unit 21 uses the new feature point P to calculate approximate curves L1 and L2. Thus, the approximate curves L1 and L2 are updated continuously over time. Alternatively, by averaging the approximate curves calculated at multiple consecutive time points (tn, tn-1, tn-2, ...), a representative approximate curve at time point tn can be obtained and stored in the storage unit 12.

[0049] When no new feature points corresponding to the road markings are detected due to reasons such as interruption of road markings, the approximate curves L1 and L2 are not updated. Therefore, if... Figure 2 As shown, when vehicle 101 changes lanes from lane 1 (LN1) to lane 2 (LN2) at a location where road markings 202 are not detected, the approximate curves L1 and L2 (approximate curves corresponding to road markings 201 and 202) calculated immediately after the lane change (second time point t2) and just before the lane change (first time point t1) are stored unchanged in storage unit 12. Approximate curves L1 and L2 calculated a predetermined time before the current time point can also be stored in storage unit 12. Approximate curves L1 and L2 calculated at a predetermined distance before the current position of vehicle 101 can also be stored in storage unit 12. In other words, the approximate curves L1 and L2 stored in storage unit 12 can be updated after a predetermined time has elapsed from the current time point (e.g., after a few seconds), or after a predetermined distance has been traveled from the current position (e.g., after traveling a few meters).

[0050] The determination unit 22 calculates the difference between the positions of the approximate curves L1 and L2 estimated by the road marking estimation unit 21 and stored in the storage unit 12, and the positions of the feature points corresponding to the road markings detected by the camera image at the current time point. Then, based on the magnitude of the difference, it determines whether the vehicle 101 has changed lanes. Figures 5A to 5C These are graphs illustrating an example of the relationship between approximate curves L1 and L2 calculated at past time points and feature point P detected at the current time point. That is, as shown... Figure 4 As shown, after calculating approximate curves L1 and L2 based on feature point P at a past time point, no feature point corresponding to the road marking was detected temporarily. The storage unit 12 stores an example of the case where feature point P is detected again at the current time point while the approximate curves L1 and L2 calculated at the past time point remain unchanged.

[0051] exist Figure 5A As shown in the enlarged view of section a, the difference between the approximate curve L1 corresponding to the left road marking and the left feature point group P1 (e.g., the difference in Y-coordinate Δy1) is less than or equal to a predetermined value Δy1a. It should be noted that the difference Δy between the approximate curve L2 corresponding to the right road marking and the right feature point group P2 is also less than or equal to a predetermined value Δy1a. The predetermined value Δy1a is set to a value less than half the lane width (e.g., approximately 1 meter). Thus, when the difference Δy1 between the approximate curve L1 and the left feature point group P1 on the same side in both left and right directions is less than or equal to the predetermined value Δy1a, the determination unit 22 determines that the vehicle 101 has not changed lanes between the past time point and the current time point, i.e., lane keeping.

[0052] exist Figure 5B In the enlarged view of section a, the difference Δy2 between the approximate curve L2 corresponding to the road marking on the right and the feature point group P1 on the left is less than or equal to the specified value Δy2a. The specified value Δy2a is a value that is different from or the same as the specified value Δy1a, and is set to be less than or equal to half the lane width. Thus, when the difference Δy2 between the approximate curve L2 on the right and the feature point group P1 on the left is less than or equal to the specified value Δy2a, the determination unit 22 determines that the vehicle 101 has changed lanes to the right lane.

[0053] On the other hand, Figure 5C In the enlarged view of section a, the difference Δy3 between the approximate curve L1 corresponding to the road marking on the left and the feature point group P2 on the right is less than or equal to a predetermined value Δy3a. The predetermined value Δy3a is, for example, the same as the predetermined value Δy2a, and is set to be less than or equal to half the lane width. Thus, when the difference Δy3 between the approximate curve L1 on the left and the feature point group P2 on the right is less than or equal to the predetermined value Δy3a, the determination unit 22 determines that the vehicle 101 has changed lanes to the left lane.

[0054] It should be noted that, in Figures 5A-5C In this process, the determination unit 22 can also replace the determination unit 22 in determining whether the differences Δy1, Δy2, and Δy3 in the Y-coordinates of the approximate curves L1 and L2 and the feature point group P1 and P2 are below the specified values ​​Δy1a, Δy2a, and Δy3a, respectively, and whether the shortest distance from the approximate curves L1 and L2 to the feature point group P1 and P2 is below the specified value. Then, based on the determination result, it can be determined whether lane keeping was implemented, lane change to the right lane or lane change to the left lane was performed.

[0055] The map generation unit 17 generates an environmental map of locations where road markings are not identified, based on the determination result of the determination unit 22. That is, when the determination unit 22 determines that the vehicle 101 has not changed lanes ( Figure 5A The map generation unit 17 connects the left and right road lines of the vehicle 101 identified from the camera image at a past time point with the left and right road lines of the vehicle 101 identified from the camera image at the current time point.

[0056] On the other hand, when the determination unit 22 determines that the vehicle 101 has changed lanes to the right ( Figure 5B The map generation unit 17 connects the road line on the right side of the vehicle 101 as identified from the camera image at a past time point with the road line on the left side of the vehicle 101 as identified from the camera image at the current time point. When the determination unit 22 determines that the vehicle 101 has changed lanes to the left... Figure 5C The map generation unit 17 connects the road lines on the left side of the vehicle 101 as identified from camera images at a past time point with the road lines on the right side of the vehicle 101 as identified from camera images at the current time point. This allows the lane position to be determined even if the lane is interrupted midway. The lane position information (the lane position information defined by the road line position information) is stored in the storage unit 12 as part of the map information.

[0057] Figure 6 It was done according to a pre-determined procedure. Figure 3 The flowchart illustrates an example of the processing performed by the controller 10. This processing, as shown in the flowchart, begins, for example, when the vehicle 101 is being driven in manual mode to generate an environment map, and is repeatedly executed at predetermined intervals.

[0058] like Figure 6As shown, firstly, in step S1 (S: processing step), signals from camera 1a and sensor 2a are read in. Next, in step S2, it is determined whether a pair of left and right road markings defining the driving lane (this lane) of the vehicle 101 are detected based on the camera image. When S2 is negative (S2: No), the process proceeds to step S9, where a marker is set to 1. It should be noted that the marker is initially 0, and is set to 1 if at least one of the left or right road markings is not detected. Next, in step S8, an environmental map is generated and stored in the storage unit 12. In this case, if the current lane is not determined, the environmental map information is stored.

[0059] When S2 is positive (S2: Yes), proceed to S3 to determine if the flag is 1. That is, determine if the road markings have just been detected again. When S3 is negative (S3: No), proceed to S10. Based on the road markings detected in S2, or more precisely, multiple feature points corresponding to the road markings, calculate a pair of approximate curves L1 and L2 passing through these feature points. Next, in S8, generate an environment map and store it in storage unit 12. In this state, left and right road markings are continuously detected, and map information including the position information of this lane is stored. While left and right road markings are continuously detected, by determining whether this vehicle 101 has crossed the road markings, it is possible to determine whether this vehicle 101 has changed lanes. Thus, the position of this lane after the lane change can be determined. In S8, the approximate curves L1 and L2 calculated in S10 are also stored.

[0060] When S3 is affirmative (S3: Yes), meaning it is determined that the road markings have just been detected again, proceed to S4, where it is determined whether the difference Δy1 between the approximate curve L1 stored on the left side of the storage unit 12 and the left feature point group P1 detected in S2 is less than or equal to a predetermined value Δy1a. When S4 is affirmative (S4: Yes), proceed to S11, where it is determined that lane keeping is in place, and proceed to S8. In S8, a map is generated that connects the left and right road markings detected at past time points, i.e., the road markings stored in the processing from S10 to S8, and the left and right road markings detected at the current time point (S2), and stored in the storage unit 12. That is, in areas where road markings have not been detected, a map is generated that connects the road markings before and after it. Thus, an environmental map including lane position information can be generated.

[0061] On the other hand, when S4 is negative (S4: No), proceed to S5, and determine whether the difference Δy2 between the approximate curve L2 stored on the right side of the storage unit 12 and the left feature point group P1 detected in S2 is less than or equal to the predetermined value Δy2a. When S5 is positive (S5: Yes), proceed to S12, determine that the vehicle 101 has changed lanes to the right, and proceed to S8. In S8, generate a map that connects the right-side road markings detected at past time points (i.e., the right-side road markings stored in the processing from S10 to S8) with the left-side road markings detected at the current time point (S2), and store it in the storage unit 12. Thus, even if road markings are not explicitly detected during lane changes, an environmental map including lane position information can be generated.

[0062] On the other hand, when S5 is negative (S5: No), proceed to S6, and determine whether the difference Δy3 between the approximate curve L1 on the left side stored in the storage unit 12 and the right feature point group P2 detected in S2 is less than or equal to the predetermined value Δy3a. When S6 is positive (S6: Yes), proceed to S7, determine that the vehicle 101 has changed lanes to the left, and proceed to S8. In S8, generate a map that connects the left-side road markings detected at a past time point, i.e., the left-side road markings stored in the processing from S10 to S8, and the right-side road markings detected at the current time point (S2), and store it in the storage unit 12. Thus, even if changing lanes to either the left or right, an environmental map including lane position information can be generated.

[0063] When S6 is negative (S6: No), proceed to S8. In this case, the presence or absence of lane changes is not determined. Therefore, a map is generated and stored in storage unit 12 without connecting road lines in areas where road lines have not been detected. Subsequently, when the presence or absence of lane changes is determined during repeated processing, road lines are connected in areas where road lines have been detected, and the map information is updated.

[0064] The operation of the map generation apparatus 50 in this embodiment will be explained in more detail. In this embodiment, while the vehicle 101 is driven in manual driving mode, an environmental map of the surrounding environment of the vehicle 101 is generated based on camera images. This environmental map includes the position information of a pair of road lines defining the left and right lanes (lane position information). Figure 2As shown, when vehicle 101 changes lanes from the first lane LN1 to the second lane LN2, assuming that the left and right road markings 201 and 202 before the lane change (first time point t1) and the left and right road markings 202 and 203 after the lane change (second time point t2) are continuously detected, and the camera image detects that vehicle 101 has crossed road marking 202, it can be determined that vehicle 101 has changed lanes. As a result, map information including lane position information can be generated. At this time, using the latest feature points detected based on the camera image, approximate curves L1 and L2 along the road markings are calculated at any time and stored in the storage unit 12 (S10→S8).

[0065] In contrast, such as Figure 2 As shown, when vehicle 101 changes lanes from the first lane LN1 to the second lane LN2, if no crossing of road marking 202 is detected, an approximate curve is used to determine whether a lane change has occurred. That is, in this case, if the difference Δy2 between the approximate curve L2 calculated along the right side of road marking 202 at the first time point t1 and the left feature point P1 detected at the second time point t2 is less than a predetermined value Δy2a, a lane change to the right lane is determined (S12). As a result, it is possible to prevent the right-side road marking 202 before the lane change and the right-side road marking 203 after the lane change from being incorrectly connected. Therefore, it is possible to generate a correct environmental map that connects the same road markings to each other.

[0066] The following effects can be achieved by adopting this implementation method.

[0067] (1) The map generation device 20 includes: a camera 1a that detects the external conditions surrounding the vehicle 101; a map generation unit 17 that generates a map including the position information of road markings 201-203 defining the driving lane of the vehicle 101 based on the information of the external conditions detected by the camera 1a; a road marking estimation unit 21 that estimates the position of road markings 201-203 detected by the camera 1a at a second time point t2, which is predicted to be after the first time point t1, based on the information of the external conditions detected by the camera 1a at a first time point t1, i.e., estimates approximate curves L1 and L2; and a determination unit 22 that determines whether the vehicle 101 has changed lanes based on the difference between the position of the road markings estimated by the road marking estimation unit 21 and the position of the road markings detected by the camera at the second time point t2. Figure 3 The map generation unit 17 generates a map based on the determination result of the determination unit 22.

[0068] Therefore, even when driving on roads without map information, the vehicle 101 can accurately determine whether a lane change has occurred, even if the road markings are interrupted midway. Thus, a map including the location information of the road markings can be accurately generated. As a result, it is possible to set the optimal target path, for example, in autonomous driving mode, and to provide driver assistance technology that further improves traffic safety and convenience, thereby contributing to the development of sustainable transportation systems. Because lane changes are determined solely based on camera images, the map generation device 20 can be configured inexpensively. In environmental map generation utilizing recognition information, it is necessary to connect sequentially identified road markings to form continuous lanes. Therefore, in order to correctly connect the same road markings to form lanes, it is necessary to accurately determine whether the vehicle is maintaining the same lane or has changed lanes. In this embodiment, the presence or absence of lane changes can be accurately determined, thus enabling the accurate generation of a map including the location information of the road markings.

[0069] (2) The road marking estimation unit 21 calculates approximate curves L1 and L2 passing through the road markings 201 and 202 detected by the camera 1a at the first time point t1 based on the position information of the road markings 201 and 202 detected by the camera 1a at the first time point t1, and estimates the position of the road markings 201 and 202 predicted to be detected by the camera 1a at the second time point t2 based on the approximate curves L1 and L2, that is, the position of the road markings 201 and 202 assuming lane keeping. Figure 4 Therefore, even in locations where road markings 201 and 202 cannot be detected, the location of the road markings can be accurately estimated, thus improving robustness.

[0070] (3) Camera 1a is configured to detect the road markings 201 (first road markings) on one side of the driving lane of the vehicle 101 in the left and right directions and the road markings 202 (second road markings) on the other side in the left and right directions. Figure 2 The determination unit 22 determines whether there is a lane change to the right lane based on the difference Δy2 between the position of the road marking 202 (approximate curve L2) estimated by the road marking estimation unit 21 and the position of the road marking 202 detected by the camera 1a. Figure 6 Therefore, it can suppress the judgment of changes in vehicle width direction as lane changes when traveling in the same lane, and can accurately determine whether a lane change has occurred.

[0071] (4) The second time point t2 is the current time point. The map generation unit 17 generates a map including the position information of the road markings detected by the camera 1a while the vehicle 101 is moving. As a result, the vehicle 101 can immediately generate a high-precision environmental map corresponding to the shape of the road while driving on a road without map information.

[0072] The above-described embodiments can be modified in various ways. In the above embodiments, the external conditions around the vehicle 101 are detected by the external sensor group 1, such as camera 1a, but the external conditions can also be detected using LiDAR or the like, and the configuration of the external detection unit is not limited to those described above. In the above embodiments, the map generation unit 25 generates an environment map while driving in manual driving mode, but it can also generate an environment map while driving in automatic driving mode. In the above embodiments, the environment map is generated based on camera images, but instead of camera 1a, feature points of objects around the vehicle 101 can be extracted using data obtained from radar or LiDAR, and an environment map can be generated. Therefore, the configuration of the map generation unit is not limited to those described above.

[0073] In the above embodiment, based on the position information of the road marking detected by camera 1a at the first time point t1, approximate curves L1 and L2 passing through the road marking are calculated, and based on these approximate curves L1 and L2, the position of the road marking predicted to be detected by the camera at the second time point t2 while the vehicle 101 is maintaining its lane is estimated. However, the configuration of the road marking estimation unit 21 is not limited to the above description. That is, based on information about the external conditions detected at the first time point, the position of the road marking predicted to be detected at the second time point can be estimated without using approximate curves, and the configuration of the position estimation unit is not limited to the above description. Therefore, the configuration of the determination unit 22, which determines whether the vehicle 101 has changed lanes based on the difference between the position of the road marking estimated by the road marking estimation unit 21 and the position of the road marking detected at the second time point t2, is also not limited to the above description.

[0074] In the above embodiment, the environment map is generated by the map generation unit 17 while the vehicle 101 is moving. However, data acquired by camera images during the movement of the vehicle 101 can also be stored in the storage unit 12, and the environment map can be generated using the stored data after the vehicle 101 has finished moving. Therefore, it is also possible to generate the map without moving the vehicle.

[0075] In the above embodiments, an example of the vehicle 101 having autonomous driving capabilities functioning as a map generation device 20 was described. However, the vehicle 101 without autonomous driving capabilities can also function as a map generation device. In this case, it can also share the map information generated by the map generation device 20 with other vehicles and use the map information to assist the driving of other vehicles (e.g., autonomous vehicles). That is, the vehicle 101 only has the function of being a map generation device 20.

[0076] It is possible to combine one or more of the above-described embodiments and variations, and to combine the variations with each other.

[0077] Using this invention, even in cases where road markings are interrupted midway, it is possible to accurately generate maps including the location information of the road markings.

[0078] The present invention has been described above in conjunction with preferred embodiments, but those skilled in the art should understand that various modifications and alterations can be made without departing from the scope of the following claims.

Claims

1. A map generation device, characterized in that, have: External detection unit (1a) detects the external conditions surrounding the vehicle (101); The map generation unit (17) generates a map including the location information of the road markings that define the driving lane of the vehicle (101) based on the information of the external conditions detected by the external detection unit (1a). The road marking estimation unit (21) estimates the position of the road markings predicted to be detected by the external detection unit (1a) at a second time point after the first time point, based on information about the external conditions detected by the external detection unit (1a) at a first time point; and The determination unit (22) determines whether the vehicle (101) has changed lanes based on the difference between the position of the road markings estimated by the road marking estimation unit (21) and the position of the road markings detected by the external detection unit (1a) at the second time point. The map generation unit (17) generates a map based on the determination result of the determination unit (22).

2. The map generation apparatus according to claim 1, characterized in that, The road marking estimation unit (21) calculates an approximate curve passing through the road marking based on the position information of the road marking detected by the external detection unit (1a) at the first time point, and estimates the position of the road marking predicted to be detected by the external detection unit (1a) at the second time point based on the approximate curve.

3. The map generation apparatus according to claim 1 or 2, characterized in that, The external detection unit (1a) is configured to detect a first road marking on one side of the driving lane of the vehicle (101) in the left and right directions and a second road marking on the other side in the left and right directions. The determination unit (22) determines whether there is a lane change based on the difference between the position of the first road line estimated by the road line estimation unit (21) and the position of the second road line detected by the external detection unit (1a).

4. The map generation apparatus according to claim 3, characterized in that, When the difference is below a specified value, the determination unit (22) determines that a lane change has been performed.

5. The map generation apparatus according to claim 4, characterized in that, The specified value is less than 1 / 2 of the lane width.

6. The map generation apparatus according to claim 1 or 2, characterized in that, The second point in time is the current point in time. The map generation unit (17) generates a map, including the location information of road markings detected by the external detection unit (1a), while the vehicle (101) is moving.

Images