Map Generator

Abstract

To accurately generate a map that includes position information for a division line even when the division line is discontinuous in the middle.SOLUTION: A map generation device 20 comprises: a camera 1a that detects an environment situation around a self vehicle; a map generation unit 17 that generates a map including position information for a division line defining a traveling traffic lane of the self vehicle on the basis of the detected environment situation; a division line estimation unit 21 that estimates a position of the division line which is predicted to be detected at a second point of time, on the basis of information on the environment situation detected at a first point of time; and a determination unit 22 that determines the presence / absence of a traffic lane change by the self vehicle on the basis of a difference between the estimated position of the division line and the position of the division line detected by the camera 1a at the second point of time. The map generation unit 17 generates a map on the basis of a result of this determination.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a map generating device that generates a map based on data indicating an external environment obtained while traveling. [Background technology]

[0002] A known example of this type of device is one that uses images captured by a camera mounted on a vehicle to recognize road dividing lines and detect lane changes, thereby estimating lanes in which the vehicle is not traveling and updating map data (see, for example, Patent Document 1 above). The device described in Patent Document 1 detects a lane change when the area in the camera image where the road dividing lines are located gradually moves left or right. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-241470 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the device described in Patent Document 1 requires continuous detection of road dividing lines to detect lane changes. Therefore, if the dividing lines are interrupted, lane changes cannot be detected, making it difficult to accurately generate a map that includes position information of the dividing lines. [Means for solving the problem]

[0005] A map generating device according to one aspect of the present invention includes an external environment detection unit that detects the external environment surrounding the host vehicle, a map generating unit that generates a map including position information of marking lines that define the lane the host vehicle is traveling in based on information about the external environment detected by the external environment detection unit, a marking line estimation unit that estimates the positions of marking lines that are predicted to be detected by the external environment detection unit at a second time point that is later than the first time point based on information about the external environment detected by the external environment detection unit at a first time point, and a determination unit that determines whether the host vehicle is changing lanes based on the difference between the positions of the marking lines estimated by the marking line estimation unit and the positions of the marking lines detected by the external environment detection unit at the second time point. The map generating unit generates the map based on the determination result by the determination unit. [Effects of the Invention]

[0006] According to the present invention, even when a lane marking is interrupted midway, a map including position information of the lane marking can be generated with high accuracy. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing an outline of the overall configuration of a vehicle control system having a map generating device according to an embodiment of the present invention; [Figure 2] 1 is a diagram showing an example of a driving scene to which a map generating device according to an embodiment of the present invention is applied; [Figure 3] 1 is a block diagram showing the configuration of a main part of a map generating device according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing an example of an approximate curve calculated by the map generating device according to the embodiment of the present invention. [Figure 5A] FIG. 4 is a diagram showing an example of an operation related to lane change determination by the map generating device according to the embodiment of the present invention. [Figure 5B] FIG. 10 is a diagram showing another example of the operation related to the determination of a lane change by the map generating device according to the embodiment of the present invention. [Figure 5C] FIG. 10 is a diagram showing yet another example of the operation related to the determination of a lane change by the map generating device according to the embodiment of the present invention. [Figure 6]4 is a flowchart showing an example of processing executed by the controller of FIG. 3; DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to Figures 1 to 6. A map generating device according to an embodiment of the present invention is configured to generate a map (an environmental map to be described later) used when a vehicle having an automatic driving function (an automatic driving vehicle) is traveling, for example. Note that the vehicle on which the map generating device according to the present embodiment is installed may be referred to as the host vehicle to distinguish it from other vehicles.

[0009] The map generation device generates a map when a driver manually drives the vehicle. Therefore, the map generation device can be installed in a vehicle that does not have an automatic driving function (a manually driven vehicle). Note that the map generation device can be installed not only in manually driven vehicles, but also in automatically driven vehicles that can switch from an automatic driving mode that does not require driver operation to a manual driving mode that does require driver operation. The following description of the map generation device will be given assuming that the map generation device is installed in an automatically driven vehicle.

[0010] First, the configuration of an autonomous vehicle will be described. The vehicle may be an engine vehicle having an internal combustion engine (engine) as a driving source, an electric vehicle having a traction motor as a driving source, or a hybrid vehicle having an engine and a traction motor as driving sources. Fig. 1 is a block diagram showing the overall configuration of a vehicle control system 100 having a map generation device according to an embodiment of the present invention.

[0011] As shown in FIG. 1, the vehicle control system 100 mainly includes a controller 10, a group of external sensors 1 each communicatively connected to the controller 10 via a CAN communication line or the like, a group of internal sensors 2, an input / output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7, and a driving actuator AC.

[0012] The external sensor group 1 is a collective term for a plurality of sensors (external sensors) that detect the external situation, which is information about the surroundings of the vehicle. For example, the external sensor group 1 includes a lidar that detects the position (distance and direction from the vehicle) of objects around the vehicle by emitting laser light and detecting reflected light, a radar that detects the position of objects around the vehicle by emitting electromagnetic waves and detecting reflected waves, and a camera that has an imaging element such as a CCD or CMOS and captures images of the surroundings (front, rear, and sides) of the vehicle.

[0013] The internal sensor group 2 is a collective term for a plurality of sensors (internal sensors) that detect the driving state of the host vehicle. For example, the internal sensor group 2 includes a vehicle speed sensor that detects the vehicle speed of the host vehicle, an acceleration sensor that detects the acceleration in the forward / backward and left / right directions of the host vehicle, a rotation speed sensor that detects the rotation speed of the driving source, etc. The internal sensor group 2 also includes sensors that detect the driving operations of the driver in manual driving mode, such as operation of the accelerator pedal, operation of the brake pedal, operation of the steering wheel, etc.

[0014] The input / output device 3 is a general term for devices that input commands from the driver and output information to the driver. For example, the input / output device 3 includes various switches through which the driver inputs various commands by operating operating members, a microphone through which the driver inputs commands by voice, a display that provides information to the driver via displayed images, and a speaker that provides information to the driver by voice.

[0015] The positioning unit (GNSS unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning sensor can also be included in the internal sensor group 2. The positioning satellite is an artificial satellite such as a GPS satellite or a quasi-zenith satellite. The positioning unit 4 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

[0016] The map database 5 is a device that stores general map information used in the navigation device 6, and is configured with, for example, a hard disk or semiconductor elements. The map information includes road position information, road shape information (curvature, etc.), and position information of intersections and branch points. Note that the map information stored in the map database 5 is different from the highly accurate map information stored in the memory unit 12 of the controller 10.

[0017] The navigation device 6 is a device that searches for a target route on roads to a destination input by the driver and provides guidance along the target route. The input of the destination and guidance along the target route are performed via the input / output device 3. The target route is calculated based on the current position of the vehicle measured by the positioning unit 4 and map information stored in the map database 5. The current position of the vehicle can also be measured using detection values ​​from the external sensor group 1, and the target route can be calculated based on this current position and high-precision map information stored in the memory unit 12.

[0018] The communication unit 7 communicates with various servers (not shown) via networks including wireless communication networks such as the Internet and mobile phone networks, and acquires map information, driving history information, traffic information, and the like from the servers periodically or at any timing. Networks include not only public wireless communication networks but also closed communication networks established for each predetermined management area, such as wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like. The acquired map information is output to the map database 5 and the storage unit 12, where the map information is updated. Communication with other vehicles is also possible via the communication unit 7.

[0019] Actuators AC are driving actuators for controlling the driving of the host vehicle. When the driving source is an engine, actuators AC include a throttle actuator that adjusts the opening of the engine's throttle valve (throttle opening). When the driving source is a driving motor, actuators AC include the driving motor. Actuators AC also include a brake actuator that operates the host vehicle's braking device and a steering actuator that drives the steering device.

[0020] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 includes a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM and RAM, and other peripheral circuits (not shown) such as an I / O interface. Note that although multiple ECUs with different functions, such as an engine control ECU, a traction motor control ECU, and a braking device ECU, can be provided separately, for convenience, the controller 10 is shown in FIG. 1 as a collection of these ECUs.

[0021] High-precision road map information is stored in the memory unit 12. This road map information includes road position information, road shape information (such as curvature), road gradient information, intersection and branch point position information, number of lanes information, lane width and lane position information (information on lane center positions and lane boundary lines), position information of landmarks (traffic lights, signs, buildings, etc.) as map markers, and road surface profile information such as road surface irregularities. The map information stored in the memory unit 12 includes map information acquired from outside the vehicle via the communication unit 7 and map information created by the vehicle itself using detection values ​​from the external sensor group 1 or detection values ​​from the external sensor group 1 and the internal sensor group 2. The memory unit 12 also stores driving history information consisting of detection values ​​from the external sensor group 1 and the internal sensor group 2 in association with the map information.

[0022] The calculation unit 11 has, as functional components, a vehicle position recognition unit 13, an external environment recognition unit 14, a behavior plan generation unit 15, a driving control unit 16, and a map generation unit 17.

[0023] The vehicle position recognition unit 13 recognizes the position of the vehicle on the map (own vehicle position) based on the vehicle position information obtained by the positioning unit 4 and the map information in the map database 5. The vehicle position may be recognized using the map information stored in the storage unit 12 and information about the surroundings of the vehicle detected by the external sensor group 1, thereby enabling the vehicle position to be recognized with high accuracy. Note that when the vehicle position can be measured by an external sensor installed on or beside the road, the vehicle position can also be recognized by communicating with the sensor via the communication unit 7.

[0024] The external environment recognition unit 14 recognizes the external situation around the vehicle based on signals from the external sensor group 1, such as a lidar, radar, and camera. For example, it recognizes the positions, speeds, and accelerations of surrounding vehicles (vehicles ahead and vehicles behind) traveling around the vehicle, the positions of surrounding vehicles stopped or parked around the vehicle, and the positions and states of other objects. Examples of other objects include signs, traffic lights, markings such as road dividing lines and stop lines, buildings, guardrails, utility poles, signs, pedestrians, bicycles, etc. Examples of the states of other objects include the color of traffic lights (red, green, yellow), the moving speed and direction of pedestrians and bicycles, etc.

[0025] The behavior plan generation unit 15 generates a driving trajectory (target trajectory) of the host vehicle from the current time to a predetermined time ahead based on, for example, a target route calculated by the navigation device 6, map information stored in the memory unit 12, the host vehicle position recognized by the host vehicle position recognition unit 13, and external conditions recognized by the external environment recognition unit 14. When there are multiple trajectories that are candidates for the target trajectory on the target route, the behavior plan generation unit 15 selects an optimal trajectory from among them that satisfies criteria such as compliance with laws and regulations and efficient and safe driving, and sets the selected trajectory as the target trajectory. The behavior plan generation unit 15 then generates a behavior plan according to the generated target trajectory. The behavior plan generation unit 15 generates various behavior plans corresponding to overtaking driving to overtake a preceding vehicle, lane-changing driving to change lanes, following driving to follow a preceding vehicle, lane-keeping driving to maintain the vehicle in its lane without deviating from the lane, decelerating driving, accelerating driving, etc. When generating the target trajectory, the behavior plan generation unit 15 first determines a driving mode and generates the target trajectory based on the driving mode.

[0026] In the autonomous driving mode, the driving control unit 16 controls each actuator AC so that the host vehicle travels along the target trajectory generated by the behavior plan generation unit 15. More specifically, in the autonomous driving mode, the driving control unit 16 calculates a required driving force for achieving the target acceleration per unit time calculated by the behavior plan generation unit 15, taking into account the driving resistance determined by the road gradient, etc. Then, for example, the driving control unit 16 feedback-controls the actuators AC so that the actual acceleration detected by the internal sensor group 2 becomes the target acceleration. In other words, the driving control unit 16 controls the actuators AC so that the host vehicle travels at the target vehicle speed and target acceleration. Note that when the driving mode is the manual driving mode, the driving control unit 16 controls each actuator AC in accordance with a driving command (such as a steering operation) from the driver acquired by the internal sensor group 2.

[0027] While driving in manual driving mode, the map generation unit 17 generates an environmental map consisting of three-dimensional point cloud data using detection values ​​detected by the external sensor group 1. Specifically, edges indicating the contours of objects are extracted from camera images acquired by a camera based on brightness and color information for each pixel, and feature points are extracted using the edge information. Feature points are, for example, points on edges or intersections of edges, and correspond to road markings on the road surface, corners of buildings, corners of road signs, etc. The map generation unit 17 calculates the distance to the extracted feature points and sequentially plots the feature points on the environmental map, thereby generating an environmental map of the area around the road on which the vehicle has traveled. Instead of using a camera, data acquired by radar or lidar may be used to extract feature points of objects around the vehicle and generate an environmental map.

[0028] The vehicle position recognition unit 13 performs a process of estimating the position of the vehicle in parallel with the map generation process by the map generation unit 17. That is, the vehicle position is estimated based on changes in the positions of feature points over time. The map generation process and the position estimation process are performed simultaneously according to a SLAM (Simultaneous Localization and Mapping) algorithm using signals from a camera or a lidar, for example. The map generation unit 17 can generate an environmental map not only when driving in manual driving mode, but also when driving in automatic driving mode. If an environmental map has already been generated and stored in the memory unit 12, the map generation unit 17 may update the environmental map with newly obtained feature points.

[0029] Next, a map generating device according to this embodiment, that is, the configuration of the vehicle control system 100 as a map generating device, will be described. FIG. 2 is a diagram showing an example of a road 200 to which the map generating device according to this embodiment is applied. FIG. 2 shows an example in which the host vehicle 101 changes lanes. That is, the host vehicle 101 changes lanes from a first lane LN1 defined by left and right dividing lines 201, 202 to a second lane LN2 defined by left and right dividing lines 202, 203, as indicated by arrow A. The host vehicle 101 travels in the first lane LN1 at a first time point t1 (state B1), and then travels in the second lane LN2 at a second time point t2 (state B2).

[0030] A camera 1a is mounted on the front of the host vehicle 101. The camera 1a is capable of capturing an approximately sector-shaped image capture area AR, which is determined by a predetermined viewing angle θ and has the camera 1a at its center. The image capture area AR of the host vehicle 101 at a first time point t1 includes lane markings 201 and 202, and the image capture area AR at a second time point t2 includes lane markings 202 and 203. Therefore, by extracting edge points from the camera image, it is possible to detect lane markings 201 and 202 at the first time point t1 and lane markings 202 and 203 at the second time point t2. By sequentially connecting the lane markings (strictly speaking, feature points corresponding to the lane markings) detected by the camera image in this way in chronological order, it is possible to generate a map that includes lane markings 201 to 203.

[0031] However, when changing lanes from the first lane LN1 to the second lane LN2, if the lane markings 202 are interrupted as shown in FIG. 2 , the lane markings 202 may not be detected continuously. Even if the lane markings 202 are not interrupted, the lane markings 202 may not be detected if an obstacle (e.g., another vehicle) obstructs the field of view of the camera 1a or if the detection accuracy of the camera 1a is reduced due to weather conditions or other factors. As a result, the lane markings 201 detected at the first time point t1 and the lane markings 202 detected at the second time point t2 may be connected, and the lane markings 202 detected at the first time point t1 and the lane markings 203 detected at the second time point t2 may be connected, resulting in the generation of an incorrect map. Therefore, in this embodiment, the map generation device is configured as follows so that an accurate map can be generated even when a lane change occurs.

[0032] Fig. 3 is a block diagram showing the configuration of the main parts of a map generating device 20 according to this embodiment. The map generating device 20 is included in the vehicle control system 100 of Fig. 1. As shown in Fig. 3, the map generating device 20 has a camera 1a, a sensor 2a, and a controller 10.

[0033] Camera 1a is a monocular camera having an imaging element (image sensor) such as a CCD or CMOS, and constitutes part of the external sensor group 1 in FIG. 1. Camera 1a may be a stereo camera. Camera 1a is attached to a predetermined position in front of vehicle 101 as shown in FIG. 2, and continuously captures images of the space ahead of vehicle 101 to obtain images of objects (camera images). Objects include lane markings 201 to 203 on the road. Note that instead of camera 1a, or together with camera 1a, objects may be detected by a lidar or the like.

[0034] The sensor 2a is a detector used to calculate the amount of movement and the direction of movement of the host vehicle 101. The sensor 2a is part of the internal sensor group 2 and is configured by, for example, a vehicle speed sensor and a yaw rate sensor. That is, the controller 10 (host vehicle position recognition unit 13) calculates the amount of movement of the host vehicle 101 by integrating the vehicle speed detected by the vehicle speed sensor, and calculates the yaw angle by integrating the yaw rate detected by the yaw rate sensor, and estimates the position of the host vehicle 101 by odometry when creating a map. Note that the configuration of the sensor 2a is not limited to this, and the host vehicle position may be estimated using information from other sensors.

[0035] The controller 10 in Fig. 3 has, as functional components performed by the calculation unit 11 (Fig. 1), a lane line estimation unit 21 and a determination unit 22 in addition to the memory unit 12 and the map generation unit 17. Note that the lane line estimation unit 21 and the determination unit 22 also have a map generation function, and therefore can also be included in the map generation unit 17.

[0036] The memory unit 12 stores map information. The stored map information includes map information acquired from outside the vehicle 101 via the communication unit 7 (referred to as external map information) and map information created by the vehicle itself (referred to as internal map information). The external map information is, for example, information on a map acquired via a cloud server (referred to as a cloud map), and the internal map information is, for example, information on a map (referred to as an environmental map) made up of point cloud data generated by mapping using a technology such as SLAM. The external map information is shared between the vehicle 101 and other vehicles, whereas the internal map information is map information unique to the vehicle 101 (for example, map information that is solely owned by the vehicle). The memory unit 12 also stores information on various control programs, thresholds used in the programs, and the like.

[0037] The lane marking estimation unit 21 estimates the position of lane marks predicted to be detected by the camera 1a at the present time (or a future time) based on camera images acquired at a past time (or the present time). Specifically, feature points corresponding to lane markings 201 and 202 included in the camera images acquired at a past time (e.g., the first time point t1 in FIG. 2) are extracted from the camera images, and an approximate curve passing through the feature points is calculated. The approximate curve may be calculated not only from feature points acquired at past times but also from feature points acquired at previous times, for example, from feature points from several frames prior to the past time. FIG. 4 shows examples of approximate curves L1 and L2. In the figure, the horizontal and vertical axes represent the X and Y coordinates of feature point P when an arbitrary point is set as the coordinate origin. The X and Y coordinates are coordinates on the road surface. The feature point P is divided into a feature point group P1 (referred to as a left feature point group) corresponding to one of the pair of left and right lane marks (e.g., the left side) and a feature point group P2 (referred to as a right feature point group) corresponding to the other (e.g., the right side).

[0038] The approximate curves L1 and L2 are represented by polynomials that pass through the multiple characteristic points P detected at the first time point t1, i.e., polynomials in which the Y coordinate value is a function of the X coordinate value. The approximate curves L1 and L2 can also be calculated by smoothly extending a curve obtained by smoothly connecting the characteristic points in the direction of vehicle travel along the direction of vehicle travel. Within the dotted-line region Ra that includes the characteristic points P, the approximate curves L1 and L2 match or nearly match the lane markings 201 and 202 detected from the camera image. The calculated approximate curves L1 and L2 are stored in the memory unit 12.

[0039] When a new characteristic point P corresponding to a lane marking is detected as the vehicle 101 moves, the lane marking estimation unit 21 calculates the approximate curves L1 and L2 using the new characteristic point P. As a result, the approximate curves L1 and L2 are updated as time passes. A representative approximate curve at time tn may be determined by averaging the approximate curves calculated at multiple consecutive time points (tn, tn-1, tn-2, . . . ), and this may be stored in the memory unit 12.

[0040] If a new feature point corresponding to the lane marking is not detected due to reasons such as the lane marking being discontinued, the approximate curves L1 and L2 are not updated. Therefore, as shown in FIG. 2, when the host vehicle 101 changes lanes from the first lane LN1 to the second lane LN2 at a point where the lane marking 202 is not detected, the approximate curves L1 and L2 (approximate curves corresponding to the lane markings 201 and 202) calculated immediately before the lane change (first time point t1) remain stored in the memory unit 12 immediately after the lane change (second time point t2). The approximate curves L1 and L2 calculated a predetermined time before the current time point may also be stored in the memory unit 12. The approximate curves L1 and L2 calculated at a point a predetermined distance before the current position of the host vehicle 101 may also be stored in the memory unit 12. In other words, the approximate curves L1 and L2 stored in the memory unit 12 may be updated after a predetermined time has elapsed from the current point in time (for example, after a few seconds have elapsed), or after moving a predetermined distance from the current position (for example, after moving a few meters).

[0041] The determination unit 22 calculates the difference between the positions of the approximate curves L1 and L2 estimated by the lane marking estimation unit 21 and stored in the memory unit 12 and the positions of the feature points corresponding to the lane markings currently detected in the camera image. Then, based on the magnitude of the difference, it determines whether the host vehicle 101 has changed lanes. FIGS. 5A to 5C are diagrams showing an example of the relationship between the approximate curves L1 and L2 calculated in the past and the feature point P detected in the present. That is, after the approximate curves L1 and L2 were calculated in the past based on the feature point P as shown in FIG. 4, the feature point corresponding to the lane markings was temporarily not detected, and the feature point P was detected again in the present while the approximate curves L1 and L2 calculated in the past were still stored in the memory unit 12.

[0042] As shown in the enlarged view of part a in FIG. 5A, the difference (e.g., the difference Δy1 in the Y coordinate) between the approximate curve L1 corresponding to the left lane marking and the left feature point group P1 is equal to or less than a predetermined value Δy1a. The difference Δy between the approximate curve L2 corresponding to the right lane marking and the right feature point group P2 is also equal to or less than a predetermined value Δy1a. The predetermined value Δy1a is set to a value equal to or less than half the lane width (e.g., approximately 1 m). When the difference Δy1 between the approximate curve L1 and the left feature point group P1, both of which are on the same lateral side, is equal to or less than the predetermined value Δy1a, the determination unit 22 determines that the host vehicle 101 has not changed lanes between the past and present times, i.e., that the host vehicle 101 is maintaining its lane.

[0043] In FIG. 5B, as shown in the enlarged view of part a, the difference Δy2 between the approximate curve L2 corresponding to the right-side lane marking and the left feature point group P1 is equal to or less than a predetermined value Δy2a. The predetermined value Δy2a may be different from or equal to the predetermined value Δy1a and is set to a value equal to or less than half the lane width. When the difference Δy2 between the right-side approximate curve L2 and the left feature point group P1 is equal to or less than the predetermined value Δy2a, the determination unit 22 determines that the host vehicle 101 has changed lanes to the right lane. On the other hand, in FIG. 5C, as shown in the enlarged view of part a, the difference Δy3 between the approximate curve L1 corresponding to the left-side lane marking and the right feature point group P2 is equal to or less than a predetermined value Δy3a. The predetermined value Δy3a may be the same as the predetermined value Δy2a and is set to a value equal to or less than half the lane width. When the difference Δy3 between the left approximate curve L1 and the right feature point group P2 is equal to or smaller than the predetermined value Δy3a, the determination unit 22 determines that the vehicle 101 has changed lanes to the left lane.

[0044] 5A to 5C, instead of determining whether the differences Δy1, Δy2, Δy3 in the Y coordinates between the approximate curves L1, L2 and the feature point groups P1, P2 are equal to or smaller than predetermined values ​​Δy1a, Δy2a, Δy3a, respectively, the determination unit 22 may determine whether the shortest distance from the approximate curves L1, L2 to the feature point groups P1, P2 is equal to or smaller than a predetermined value. Then, depending on the determination result, it may be determined that the vehicle has kept in the lane, changed to the right lane, or changed to the left lane.

[0045] The map generation unit 17 generates an environmental map of the point where no lane markings were recognized, based on the determination result by the determination unit 22. That is, when the determination unit 22 determines that the host vehicle 101 has not changed lanes (FIG. 5A), the map generation unit 17 connects the lane markings on the left and right sides of the host vehicle 101 recognized in the camera image at a previous time point with the lane markings on the left and right sides of the host vehicle 101 recognized in the camera image at the current time point.

[0046] On the other hand, if the determination unit 22 determines that the vehicle 101 has changed lanes to the right (FIG. 5B), the lane marking on the right side of the vehicle 101 recognized in the camera image at a previous time point is connected to the lane marking on the left side of the vehicle 101 recognized in the camera image at the current time point. If the determination unit 22 determines that the vehicle 101 has changed lanes to the left (FIG. 5C), the lane marking on the left side of the vehicle 101 recognized in the camera image at a previous time point is connected to the lane marking on the right side of the vehicle 101 recognized in the camera image at the current time point. This makes it possible to identify the position of the lane. The identified lane position is stored in the memory unit 12 as part of the map information.

[0047] Fig. 6 is a flowchart showing an example of processing executed by the controller 10 of Fig. 3 in accordance with a predetermined program. The processing shown in this flowchart is started when the host vehicle 101 is traveling in manual driving mode, for example, to generate an environmental map, and is repeated at predetermined intervals.

[0048] As shown in FIG. 6, first, in step S1, signals are read from the camera 1a and the sensor 2a. Next, in step S2, it is determined based on the camera image whether a pair of left and right dividing lines defining the driving lane (own lane) of the host vehicle 101 have been detected. If the result in step S2 is negative, the process proceeds to step S9, where a flag is set to 1. The flag is initially set to 0, and is set to 1 when at least one of the left and right dividing lines is not detected. Next, in step S8, an environmental map is generated and stored in the memory unit 12. In this case, information on the environmental map in a state where the host lane has not been identified is stored.

[0049] If the result of step S2 is affirmative, the process proceeds to step S3, where it is determined whether the flag is 1. That is, it is determined whether the lane markings have just been detected again. If the result of step S3 is negative, the process proceeds to step S10, where a pair of left and right approximate curves L1 and L2 that pass through the lane markings detected in step S2, or more specifically, the plurality of characteristic points corresponding to the lane markings, are calculated. Next, in step S8, an environmental map is generated and stored in the memory unit 12. In this state, the left and right lane markings continue to be detected, and map information including the position information of the vehicle's lane is stored. In this state in which the left and right lane markings continue to be detected, it is possible to determine whether the vehicle 101 has changed lanes by determining whether the vehicle 101 has crossed the lane markings. In step S8, the approximate curves L1 and L2 calculated in step S10 are also stored.

[0050] If step S3 is positive, i.e., if it is determined that a lane marking has just been detected again, the process proceeds to step S4, where it is determined whether the difference Δy1 between the left-side approximate curve L1 stored in memory unit 12 and the left feature point group P1 detected in step S2 is equal to or less than a predetermined value Δy1a. If step S4 is positive, the process proceeds to step S11, where it is determined that the vehicle is in lane keeping, and then the process proceeds to step S8. In step S8, a map is generated that connects the left and right lane markings detected in the past, i.e., the lane markings stored in the processing from step S10 to step S8, with the left and right lane markings detected at the current time (step S2), and the map is stored in memory unit 12. In other words, in areas where lane markings have not yet been detected, a map is generated that connects the lane markings before and after them. This makes it possible to generate an environmental map that includes lane position information.

[0051] On the other hand, if the result in step S4 is negative, the process proceeds to step S5, where it is determined whether the difference Δy2 between the right-side approximate curve L2 stored in the memory unit 12 and the left feature point group P1 detected in step S2 is equal to or less than a predetermined value Δy2a. If the result in step S5 is positive, the process proceeds to step S12, where it is determined that the vehicle 101 has changed lanes to the right, and the process proceeds to step S8. In step S8, a map is generated that connects the right-side lane marking detected in the past, i.e., the right-side lane marking of the left and right lane marks stored in the processing from step S10 to step S8, with the left-side lane marking detected at the current time (step S2), and the map is stored in the memory unit 12. This makes it possible to generate an environmental map that includes lane position information, even if lane markings are not clearly detected when changing lanes.

[0052] On the other hand, if the result in step S5 is negative, the process proceeds to step S6, where it is determined whether the difference Δy3 between the left-side approximate curve L1 stored in the memory unit 12 and the right-side feature point group P2 detected in step S2 is equal to or less than a predetermined value Δy3a. If the result in step S6 is positive, the process proceeds to step S7, where it is determined that the vehicle 101 has changed lanes to the left, and the process proceeds to step S8. In step S8, a map is generated that connects the left-side lane line detected in the past, i.e., the left-side lane line of the left and right lane lines stored in the processing from step S10 to step S8, with the right-side lane line detected at the current time (step S2), and the map is stored in the memory unit 12. This makes it possible to generate an environmental map that includes lane position information regardless of whether the lane change was to the left or right.

[0053] If the result of step S6 is negative, the process proceeds to step S8. In this case, it is not yet clear whether a lane change has occurred. Therefore, a map is generated without connecting lane lines in areas where lane lines have not been detected, and the map is stored in the memory unit 12. After that, when it is determined whether a lane change has occurred through repeated processing, lane lines are connected in areas where lane lines have not been detected, and the map information is updated.

[0054] The operation of the map generating device 50 according to this embodiment will be described in more detail. While the host vehicle 101 is traveling in manual driving mode, an environmental map of the surroundings of the host vehicle 101 is generated based on camera images. This environmental map includes position information of a pair of left and right lane markings that define the host vehicle's lane. As shown in FIG. 2, when the host vehicle 101 changes lanes from the first lane LN1 to the second lane LN2, if the left and right lane markings 201 and 202 before the lane change (first time point t1) and the left and right lane markings 202 and 203 after the lane change (second time point t2) are continuously detected, it is possible to determine that the host vehicle 101 has changed lanes by detecting that the host vehicle 101 has crossed the lane marking 202 based on the camera images. As a result, map information including lane position information can be generated. At this time, approximate curves L1 and L2 along the lane markings are calculated as needed using the most recent feature points detected from the camera images and stored in the memory unit 12 (step S10 → step S8).

[0055] In contrast, as shown in FIG. 2, when the vehicle 101 changes lanes from the first lane LN1 to the second lane LN2, if it is not detected that the vehicle has crossed the lane marking 202, the presence or absence of a lane change is determined using an approximate curve. That is, in this case, the difference Δy2 between the right-side approximate curve L2 along the lane marking 202 calculated at the first time point t1 and the left feature point P1 detected at the second time point t2 is equal to or less than a predetermined value Δy2a, and this determines that the vehicle has changed to the right lane (step S12). As a result, it is possible to prevent the right-side lane marking 202 before the lane change from being erroneously connected to the right-side lane marking 203 after the lane change. This makes it possible to generate an accurate environmental map in which the same lane marks are connected to each other.

[0056] According to this embodiment, the following effects can be achieved. (1) The map generating device 20 includes a camera 1a that detects the external environment surrounding the host vehicle 101, a map generating unit 17 that generates a map including position information of lane markings 201-203 that define the lane the host vehicle 101 is traveling in based on information about the external environment detected by the camera 1a, a lane marking estimation unit 21 that estimates the positions of the lane markings 201-203 that are predicted to be detected by the camera 1a at a second time point t2 that is later than the first time point t1, i.e., approximate curves L1 and L2, based on the information about the external environment detected by the camera 1a at a first time point t1, and a determination unit 22 that determines whether the host vehicle 101 is changing lanes based on the difference between the positions of the lane markings estimated by the lane marking estimation unit 21 and the positions of the lane markings detected by the camera at the second time point t2 (FIG. 3). The map generating unit 17 generates the map based on the determination result by the determination unit 22.

[0057] This allows for accurate determination of whether or not a lane change has occurred while the vehicle 101 is traveling on a road for which no map information is available, even if the lane markings are interrupted. This allows for accurate generation of a map including lane marking position information. As a result, for example, an optimal target route can be set in autonomous driving mode, and driving assistance technology can be provided that further improves traffic safety and convenience and contributes to the development of a sustainable transportation system. Because lane changes are determined based solely on camera images, the map generation device 20 can be configured inexpensively. In environmental map generation using recognition information, it is necessary to connect sequentially recognized lane marks to form continuous lanes. Therefore, in order to correctly connect identical lane marks to form lanes, it is necessary to correctly determine whether the vehicle is traveling in the same lane or has changed lanes. In this regard, the present embodiment allows for accurate determination of whether or not a lane change has occurred, thereby allowing for accurate generation of a map including lane marking position information.

[0058] (2) Based on the position information of the lane markings 201, 202 detected by the camera 1a at the first time point t1, the lane marking estimation unit 21 calculates approximate curves L1, L2 that pass through the lane markings 201, 202, and estimates the positions of the lane markings 201, 202 that are predicted to be detected by the camera 1a at the second time point t2, i.e., the positions of the lane markings 201, 202 assuming lane keeping, based on the approximate curves L1, L2 (Figure 4). This allows the positions of the lane markings to be accurately estimated even if there are points where the lane markings 201, 202 cannot be detected, improving robustness.

[0059] (3) Camera 1a is installed to detect a lane marking 201 (first lane marking) on ​​one side in the left-right direction that defines the lane in which vehicle 101 is traveling, and a lane marking 202 (second lane marking) on ​​the other side in the left-right direction (FIG. 2). Determination unit 22 determines whether or not the vehicle is changing lanes to the right lane based on the difference Δy2 between the position of lane marking 202 estimated by lane marking estimation unit 21 (approximate curve L2) and the position of lane marking 202 detected by camera 1a (FIG. 6). This makes it possible to prevent positional fluctuations in the vehicle width direction while traveling within the same lane from being erroneously determined as a lane change, thereby enabling accurate determination of whether or not the vehicle is changing lanes.

[0060] (4) The second time point t2 is the current time point, and the map generation unit 17 generates a map including position information of the lane markings detected by the camera 1a while the vehicle 101 is traveling. This allows the vehicle 101 to instantly generate a highly accurate environmental map according to the road shape while traveling on a road for which there is no map information.

[0061] The above embodiment can be modified in various ways. In the above embodiment, the external environment surrounding the vehicle 101 is detected by the external sensor group 1, such as the camera 1a. However, the external environment may be detected using a lidar or the like, and the configuration of the external environment detection unit is not limited to the above. In the above embodiment, the map generation unit 25 generates the environmental map while driving in manual driving mode. However, the environmental map may be generated while driving in automatic driving mode. In the above embodiment, the environmental map is generated based on camera images. However, instead of the camera 1a, data acquired by radar or lidar may be used to extract feature points of objects around the vehicle 101, and the environmental map may be generated. Therefore, the configuration of the map generation unit is not limited to the above.

[0062] In the above embodiment, approximate curves L1 and L2 that pass through the lane markings detected by the camera 1a at the first time point t1 are calculated based on position information of the lane markings. Based on these approximate curves L1 and L2, the position of the lane markings predicted to be detected by the camera at the second time point t2 when the host vehicle 101 is traveling in its lane is estimated. However, the configuration of the lane marking estimation unit 21 is not limited to the above. That is, the position of the lane markings predicted to be detected at the second time point may be estimated based on information about the external environment detected at the first time point without using the approximate curves. The configuration of the position estimation unit is not limited to the above. Therefore, the configuration of the determination unit 22 that determines whether the host vehicle 101 is changing lanes based on the difference between the position of the lane markings estimated by the lane marking estimation unit 21 and the position of the lane markings detected at the second time point t2 is also not limited to the above.

[0063] In the above embodiment, the map generation unit 17 generates an environmental map while the host vehicle 101 is traveling, but data obtained by camera images while the host vehicle 101 is traveling may be stored in the storage unit 12, and the environmental map may be generated using the stored data after the host vehicle 101 has completed traveling. Therefore, it is not necessary to generate a map while traveling.

[0064] In the above embodiment, an example has been described in which the host vehicle 101 having an automatic driving function functions as the map generating device 20. However, the host vehicle 101 without an automatic driving function may function as the map generating device. In this case, map information generated by the map generating device 20 may be shared with other vehicles, and the driving of the other vehicles (e.g., automatic driving vehicles) may be assisted using the map information. In other words, the host vehicle 101 may only have the function of the map generating device 20.

[0065] The above description is merely an example, and the present invention is not limited to the above-described embodiment and modifications as long as the features of the present invention are not impaired. One or more of the above-described embodiment and modifications can be arbitrarily combined, and modifications can also be combined with each other. [Explanation of symbols]

[0066] 1a camera, 10 controller, 17 map generation unit, 20 map generation device, 21 lane line estimation unit, 22 determination unit, 201, 202 lane line, L1, L2 approximate curve

Claims

1. an external environment detection unit that detects an external environment around the vehicle; a map generation unit that generates a map including position information of lane markings that define the lane in which the host vehicle is traveling, based on information about the external environment detected by the external environment detection unit; a lane marking estimation unit that estimates a position of a lane marking that is predicted to be detected by the external environment detection unit at a second time point that is later than the first time point, based on information about an external environment situation detected by the external environment detection unit at a first time point; a determination unit that determines whether or not the host vehicle is changing lanes based on a difference between the position of the lane marking estimated by the lane marking estimation unit and the position of the lane marking detected by the external environment detection unit at the second time point, The map generating device is characterized in that the map generating unit generates a map based on the determination result by the determining unit.

2. 2. The map generating device according to claim 1, The map generation device is characterized in that the lane line estimation unit calculates an approximate curve that passes through the lane line based on position information of the lane line detected by the external environment detection unit at the first time point, and estimates the position of the lane line that is predicted to be detected by the external environment detection unit at the second time point based on the approximate curve.

3. 3. The map generating device according to claim 1, the external environment detection unit is configured to detect a first dividing line on one side in the left-right direction and a second dividing line on the other side in the left-right direction that define a driving lane of the host vehicle, The map generating device is characterized in that the determination unit determines whether or not a lane change has occurred based on the difference between the position of the first lane line estimated by the lane line estimation unit and the position of the second lane line detected by the external environment detection unit.

4. The map generating device according to any one of claims 1 to 3, the second time point is the current time point; The map generating device is characterized in that the map generating unit generates a map including position information of the lane markings detected by the external environment detecting unit while the host vehicle is traveling.

Images