Position estimation device and vehicle control system

Abstract

To restrain the amount of data storage and estimate a position of one's own vehicle precisely.SOLUTION: A position estimation device 50 includes: a feature point extraction section 112 for extracting a feature point from a camera image of a camera 1a; a storage section 12 for storing an environment map including a feature point corresponding to the outside situation in advance; a position estimation section 113 for estimating a position of one's own vehicle on the basis of a feature point extracted by the feature point extraction section 112 and an environment map stored in the storage section 12; a reliability determination section 114 for determining whether the reliability of a position of one's own vehicle estimated by the position estimation section 113 is less than a predetermined degree; an environment map generation section 115 for generating an environment map by using a feature point extracted by the feature point extraction section 112 when the reliability determination section 114 determines that the reliability is less than a predetermined degree; and a sub-position estimation section 116 for estimating a position of one's own vehicle on the basis of a feature point extracted by the feature point extraction section 112 and an environment map generated by the environment map generation section 115.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a position estimation device for estimating the position of a host vehicle and a vehicle control system.

Background Art

[0002] Conventionally, as this type of device, there is known a device configured to estimate the current position of a terminal device by collating feature points extracted from a captured image of a camera mounted on a terminal device such as a smartphone with a feature point map (see, for example, Patent Document 1). The device described in Patent Document 1 selects a feature point map corresponding to the imaging conditions at the time when the captured image is acquired from among a plurality of feature point maps corresponding to a plurality of imaging conditions defined by season, time, weather, etc., and estimates the current position of the terminal device using the feature point map.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, if feature point maps are prepared for each imaging condition as in the device described in Patent Document 1, the storage amount of data increases. On the other hand, if only a feature point map corresponding to specific imaging conditions is prepared in order to suppress the increase in the storage amount of data, it is difficult to accurately estimate the position of the host vehicle.

Means for Solving the Problems

[0005] A position estimation device according to one aspect of the present invention includes a detector that detects the external situation of a vehicle, a feature point extraction unit that extracts feature points from the detection data of the detector, a storage unit that stores first map information including feature points corresponding to the external situation in advance, a first estimation unit that estimates the position of the vehicle based on the feature points extracted by the feature point extraction unit and the first map information stored in the storage unit, a reliability determination unit that determines whether the reliability of the position of the vehicle estimated by the first estimation unit is less than a predetermined level, a map generation unit that generates second map information using the feature points extracted by the feature point extraction unit when the reliability determination unit determines that the reliability is less than the predetermined level, and a second estimation unit that estimates the position of the vehicle based on the feature points extracted by the feature point extraction unit and the second map information generated by the map generation unit.

[0006] A vehicle control system according to another aspect of the present invention includes the above-described position estimation device, a traveling actuator, and a traveling control unit that controls the traveling actuator based on the position of the vehicle estimated by the first estimation unit or the second estimation unit. When the traveling control unit controls the traveling actuator based on the position of the vehicle estimated by the first estimation unit and the reliability determination unit determines that the reliability is less than a predetermined level, the traveling control unit stops the control of the traveling actuator based on the position of the vehicle estimated by the first estimation unit and starts the control of the traveling actuator based on the position of the vehicle estimated by the second estimation unit.

Advantages of the Invention

[0007] According to the present invention, the position of the host vehicle is accurately estimated while suppressing an increase in the data storage amount.

Brief Description of the Drawings

[0008]

Fig. 1

Fig. 2A

Fig. 2B

Fig. 3

Fig. 4

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the invention will be described with reference to the drawings. The position estimation device according to an embodiment of the present invention can be applied to a vehicle having an automatic driving function, that is, an autonomous vehicle. Note that the vehicle to which the position estimation device according to the present embodiment is applied may be referred to as the host vehicle to distinguish it from other vehicles. The host vehicle may be any of an engine vehicle having an internal combustion engine (engine) as a driving source for traveling, an electric vehicle having a driving motor as a driving source for traveling, and a hybrid vehicle having an engine and a driving motor as driving sources for traveling. The host vehicle can travel not only in the automatic driving mode that does not require a driving operation by the driver but also in the manual driving mode by the driver's driving operation.

[0010] First, a schematic configuration of the host vehicle related to automatic driving will be described. FIG. 1 is a block diagram schematically showing the overall configuration of a vehicle control system 100 of a host vehicle having the position estimation device according to the present embodiment. As shown in FIG. 1, the vehicle control system 100 mainly includes a controller 10, an external sensor group 1, an internal sensor group 2, an input / output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7, and a traveling actuator AC, which are communicably connected to the controller 10, respectively.

[0011] The external sensor group 1 is a general term for a plurality of sensors (external sensors) that detect the external situation, which is the surrounding information of the host vehicle. For example, the external sensor group 1 includes a lidar that measures the scattered light of the omnidirectional irradiation light of the host vehicle to measure the distance from the host vehicle to surrounding obstacles, a radar that irradiates electromagnetic waves and detects reflected waves to detect other vehicles and obstacles around the host vehicle, and a camera mounted on the host vehicle and having an imaging element (image sensor) such as a CCD or CMOS to image the surrounding (front, rear, and sides) of the host vehicle.

[0012] The internal sensor group 2 is a general term for a plurality of sensors (internal sensors) that detect the driving state of the host vehicle. For example, the internal sensor group 2 includes an inertial measurement unit (IMU) that detects the rotational angular velocity around three axes and the acceleration in three axial directions of the vertical direction, the front-rear direction (travel direction), and the left-right direction (vehicle width direction) of the center of gravity of the host vehicle. Sensors that detect the driving operations of the driver in the manual driving mode, such as the operation of the accelerator pedal, the brake pedal, and the steering wheel, are also included in the internal sensor group 2.

[0013] The input / output device 3 is a general term for a device through which commands are input from the driver or information is output to the driver. For example, the input / output device 3 includes various switches through which the driver inputs various commands by operating operation members, a microphone through which the driver inputs commands by voice, a display that provides information to the driver via a display image, a speaker that provides information to the driver by voice, and the like.

[0014] The positioning unit (GNSS unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 4 measures the current position (latitude, longitude, altitude) of the host vehicle using the positioning information received by the positioning sensor.

[0015] The map database 5 is a device that stores general map information used in the navigation device 6, and is composed of, for example, a hard disk or a semiconductor element. The map information includes road position information, road shape (such as curvature) information, and intersection and branch point position information. Note that the map information stored in the map database 5 is different from the high-precision map information stored in the storage unit 12 of the controller 10.

[0016] The navigation device 6 is a device that searches for a target route on the road to the destination input by the driver and provides guidance along the target route. The input of the destination and the guidance along the target route are performed via the input / output device 3. The target route is calculated based on the current position of the host vehicle measured by the positioning unit 4 and the map information stored in the map database 5. It is also possible to measure the current position of the host vehicle using the detection values of the external sensor group 1, and calculate the target route based on this current position and the high-precision map information stored in the storage unit 12.

[0017] The communication unit 7 communicates with various servers (not shown) via a network including a wireless communication network typified by the Internet or a mobile phone network, and periodically or at an arbitrary timing acquires map information, driving history information, traffic information, etc. from the server. Not only can it acquire driving history information, but it is also possible to transmit the driving history information of the host vehicle to the server via the communication unit 7. The network includes not only a public wireless communication network but also a closed communication network provided for each predetermined management area, such as a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc. The acquired map information is output to the map database 5 and the storage unit 12, and the map information is updated.

[0018] Actuator AC is a driving actuator for controlling the running of the host vehicle. When the driving power source is an engine, actuator AC includes a throttle actuator for adjusting the opening degree of the throttle valve of the engine (throttle opening degree). When the driving power source is a driving motor, the driving motor is included in actuator AC. Actuator AC also includes a brake actuator for operating the braking device of the host vehicle and a steering actuator for driving the steering device.

[0019] Controller 10 is constituted by an electronic control unit (ECU). More specifically, controller 10 includes a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM and a RAM, and other peripheral circuits (not shown) such as an I / O interface. Although a plurality of ECUs with different functions such as an engine control ECU, a driving motor control ECU, and a braking device ECU can be provided separately, in FIG. 1, for convenience, controller 10 is shown as an aggregation of these ECUs.

[0020] The storage unit 12 stores highly accurate and detailed map information (referred to as highly accurate map information). The highly accurate map information includes road position information, road shape (such as curvature) information, road gradient information, intersection and branch point position information, types and position information of road demarcation lines such as white lines, number of lanes information, lane width and position information for each lane (information on the center position of the lane and the boundary lines of the lane position), position information of landmarks (buildings, traffic lights, signs, etc.) as marks on the map, and road surface profile information such as road surface unevenness. In the embodiment, center lines, lane boundary lines, outer lane lines, etc. are collectively referred to as road demarcation lines. The highly accurate map information stored in the storage unit 12 includes map information acquired from outside the host vehicle via the communication unit 7 (referred to as external map information) and a map created by the host vehicle itself using detection values from the external sensor group 1 or detection values from the external sensor group 1 and the internal sensor group 2 (referred to as internal map information).

[0021] The external map information is information of a map (referred to as a cloud map) obtained via, for example, a cloud server, and the internal map information is information of a map (referred to as an environmental map) composed of three-dimensional point cloud data generated by mapping using a technology such as SLAM (Simultaneous Localization and Mapping). The external map information is shared between the host vehicle and other vehicles, while the internal map information is unique map information of the host vehicle (for example, map information uniquely possessed by the host vehicle). In the case of a road not traveled by the host vehicle, a newly constructed road, etc., an environmental map is created by the host vehicle itself. Note that the internal map information may be provided to a server device or other vehicles via the communication unit 7. The storage unit 12 stores, in addition to the above-described high-precision map information, information such as the travel trajectory information of the host vehicle, various control programs, and thresholds used in the programs.

[0022] As a functional configuration, the arithmetic unit 11 includes a host vehicle position recognition unit 13, an external world recognition unit 14, a behavior plan generation unit 15, a travel control unit 16, and a map generation unit 17.

[0023] The host vehicle position recognition unit 13 recognizes (which may be referred to as estimating) the position of the host vehicle on the map (host vehicle position) based on the position information of the host vehicle obtained by the positioning unit 4 and the map information of the map database 5. The host vehicle position may be recognized (estimated) using the high-precision map information stored in the storage unit 12 and the peripheral information of the host vehicle detected by the external sensor group 1, whereby the host vehicle position can be recognized with high precision. The movement information (movement direction, movement distance) of the host vehicle can also be calculated based on the detection values of the internal sensor group 2, and thereby the host vehicle position can be recognized. Note that when the host vehicle position can be measured by a sensor installed outside the road or beside the road, the host vehicle position can also be recognized by communicating with the sensor via the communication unit 7.

[0024] The external situation recognition unit 14 recognizes the external situation around the host vehicle based on signals from an external sensor group 1 such as a lidar, radar, camera, etc. For example, it recognizes the positions, speeds, and accelerations of surrounding vehicles (front vehicles and rear vehicles) traveling around the host vehicle, the positions of surrounding vehicles parked or stopped around the host vehicle, and the positions and states of other objects. Other objects include signs, traffic lights, markings such as road lane lines and stop lines, buildings, guardrails, utility poles, billboards, pedestrians, bicycles, etc. The states of other objects include the colors of traffic lights (red, blue, yellow), the moving speeds and directions of pedestrians and bicycles, etc. A part of the stationary objects among other objects constitutes landmarks that are indicators of positions on the map, and the external situation recognition unit 14 also recognizes the positions and types of landmarks.

[0025] The action plan generation unit 15 generates a driving trajectory (target trajectory) of the host vehicle from the current time to a predetermined time ahead based on, for example, the target route calculated by the navigation device 6, the high-precision map information stored in the storage unit 12, the host vehicle position recognized by the host vehicle position recognition unit 13, and the external situation recognized by the external situation recognition unit 14. When there are a plurality of trajectories that are candidates for the target trajectory on the target route, the action plan generation unit 15 selects the optimal trajectory that complies with the laws and regulations and meets criteria such as efficient and safe driving from among them, and sets the selected trajectory as the target trajectory. Then, the action plan generation unit 15 generates an action plan corresponding to the generated target trajectory. The action plan generation unit 15 generates various action plans corresponding to overtaking driving for overtaking a preceding vehicle, lane change driving for changing the driving lane, following driving for following a preceding vehicle, lane keep driving for maintaining the lane so as not to deviate from the driving lane, deceleration driving, or acceleration driving, etc. When generating the target trajectory, the action plan generation unit 15 first determines the driving mode and generates the target trajectory based on the driving mode.

[0026] The travel control unit 16 controls each actuator AC so that the host vehicle travels along the target trajectory generated by the action plan generation unit 15 in the automatic driving mode. More specifically, in the automatic driving mode, the travel control unit 16 calculates a required driving force for obtaining the target acceleration per unit time calculated by the action plan generation unit 15 in consideration of the travel resistance determined by the road gradient or the like. Then, for example, the actuator AC is feedback-controlled so that the actual acceleration detected by the internal sensor group 2 becomes the target acceleration. That is, the actuator AC is controlled so that the host vehicle travels at the target vehicle speed and the target acceleration. In the manual driving mode, the travel control unit 16 controls each actuator AC according to the travel command (such as a steering operation) from the driver acquired by the internal sensor group 2.

[0027] The map generation unit 17 generates an environmental map around the road on which the host vehicle has traveled as internal map information using the detection values detected by the external sensor group 1 while traveling in the manual driving mode. For example, from a plurality of frames of camera images acquired by a camera, edges indicating the contours of objects are extracted based on the luminance and color information for each pixel, and feature points are extracted using the edge information. The feature points are, for example, intersections of edges and correspond to corners of buildings, corners of road signs, etc. The map generation unit 17 estimates the position and orientation of the camera while calculating the three-dimensional position of the feature points so that the same feature point converges to one point among a plurality of frames of camera images according to the algorithm of the SLAM technique. By performing this calculation process for each of a plurality of feature points, an environmental map composed of three-dimensional point cloud data is generated. Note that, instead of the camera, data acquired by a radar or a lidar may be used to extract feature points of objects around the host vehicle and generate an environmental map.

[0028] The own-vehicle position recognition unit 13 may perform own-vehicle position recognition processing based on the environmental map generated by the map generation unit 17 and the feature points extracted from the camera image. Further, the own-vehicle position recognition unit 13 may perform own-vehicle position recognition processing in parallel with the map creation processing by the map generation unit 17. The map creation processing and the position recognition (estimation) processing are performed simultaneously according to the algorithm of the SLAM technology. The map generation unit 17 can generate an environmental map in the same manner not only when traveling in the manual driving mode but also when traveling in the automatic driving mode. When an environmental map has already been generated and stored in the storage unit 12, the map generation unit 17 may update (update) the environmental map based on the feature points (which may be referred to as new feature points) newly extracted from the newly acquired camera image.

[0029] FIGS. 2A and 2B are diagrams for explaining the position recognition processing in the own-vehicle position recognition unit 13. FIG. 2A shows a state in which the own-vehicle position is recognized based on the environmental map generated by the map generation unit 17 and the feature points extracted from the camera image. Further, FIG. 2A schematically shows a part (seven feature points FP) of the point cloud data included in the environmental map. The camera images IM t-3 , IM t-2 , IM t-1 , IM t are frame image data (camera images) acquired by the camera 1a at each of the times t-3, t-2, t-1, and t. The black circles fp in each camera image schematically represent the feature points extracted from each camera image. The own-vehicle position recognition unit 13 estimates the position and orientation of the own vehicle (specifically, the camera 1a) on the environmental map by finding the feature points fp (hereinafter referred to as corresponding points) corresponding to the feature points FP on the environmental map from among the feature point group in the camera image. The broken line connecting the feature point FP and the feature point fp indicates that the feature point FP and the feature point fp correspond to each other. The own-vehicle position recognition unit 13 adjusts (corrects) the estimated own-vehicle position and orientation based on the camera images of a plurality of frames so as to improve the position estimation accuracy. For example, the position and orientation of the own vehicle at time t estimated based on the camera image IM t are adjusted based on the camera images IM t-3 , IMt-2 , IM t-1 is corrected using this. The host vehicle position recognition unit 13 recognizes the position of the host vehicle on the environmental map during traveling by tracking the position and attitude of the host vehicle at each time thus estimated.

[0030] Incidentally, when the time zone when the environmental map was generated, the brightness around the host vehicle, conditions such as weather (climate) (hereinafter referred to as environmental conditions) are different from when the camera image was acquired, the feature point FP corresponding to the feature point fp extracted from the camera image may not exist on the environmental map, or the corresponding point of the feature point FP on the environmental map may not exist in the camera image. FIG. 2B shows an example of a camera image acquired under environmental conditions different from when the environmental map was generated. For example, when the time zone when the environmental map was generated is daytime and the time zone when the camera image was acquired is nighttime, the appearance of subjects such as structures around the vehicle is different between daytime and nighttime. Therefore, as in the camera image IM t-2 of FIG. 2B, there may be a case where the feature point corresponding to the feature point FP on the environmental map is not extracted. The broken-line circles in the camera image IM t-2 in FIG. 2B schematically show a state where the feature point corresponding to the feature point FP on the environmental map is not extracted from the camera image IM t-2 . In such a case, the matching accuracy of the feature points between the environmental map and the camera image decreases, and the position of the vehicle cannot be accurately recognized. Therefore, in order to address such a problem, in this embodiment, the position estimation device is configured as follows.

[0031] FIG. 3 is a block diagram showing a main configuration of a position estimation device 50 according to this embodiment. This position estimation device 50 constitutes a part of the vehicle control system 100 in FIG. 1. As shown in FIG. 3, the position estimation device 50 includes a controller 10, a camera 1a, and an illuminance sensor 1b.

[0032] Camera 1a is a monocular camera having an imaging device (image sensor) such as a CCD or CMOS, and constitutes part of the external sensor group 1 in FIG. 1. Camera 1a is attached, for example, at a predetermined position in the front part of the host vehicle, continuously images the front space of the host vehicle at a predetermined frame rate, and sequentially outputs frame image data (camera image) as detection information to the sequencer 10. Note that camera 1a may be a stereo camera.

[0033] Illuminance sensor 1b has a light receiving element and detects the brightness (illuminance) of light incident on the light receiving element. Illuminance sensor 1b is installed outside the host vehicle (e.g., on the roof) or inside the host vehicle (on the dashboard) so as to be able to detect the illuminance (brightness) around the host vehicle. Illuminance sensor 1b outputs a detection value (detection data) to the controller 10. Note that since the light passing through the windshield is attenuated to some extent by the glass, when illuminance sensor 1b is installed inside the host vehicle, the sensor value may be corrected in consideration of this attenuation amount.

[0034] Controller 10 includes an arithmetic unit 11 and a storage unit 12. Arithmetic unit 11 functionally includes an information acquisition unit 111, a feature point extraction unit 112, a position estimation unit 113, a reliability determination unit 114, an environmental map generation unit 115, and a secondary position estimation unit 116. Storage unit 12 stores map information (environmental map) of the roads on which the host vehicle has traveled in the past.

[0035] Feature point extraction unit 112 and environmental map generation unit 115 are included, for example, in map generation unit 17 in FIG. 1. Position estimation unit 113, reliability determination unit 114, and secondary position estimation unit 116 are included, for example, in host vehicle position recognition unit 13 in FIG. 1.

[0036] The information acquisition unit 111 acquires information regarding the past and current weather, time, and ambient brightness of the road section on which the host vehicle has traveled. More specifically, the information acquisition unit 111 acquires the past or current weather information of the road section on which the host vehicle has traveled from an external server (not shown) that provides weather information of the past or current time via the communication unit 7. At this time, the information acquisition unit 111 acquires the past weather information of the road section on which the host vehicle has traveled from the external server based on the time information stored in the storage unit 12 in association with the environmental map. In the time information, information indicating the time (imaging time) when the camera image in which the feature point was extracted was acquired by the camera 1a is recorded for each feature point included in the environmental map.

[0037] Also, the information acquisition unit 111 acquires the detection data of the illuminance sensor 1b as the current brightness information of the road section on which the host vehicle has traveled. The information acquisition unit 111 acquires the past brightness information of the road section on which the host vehicle has traveled from the storage unit 12. In the storage unit 12, brightness information is stored in association with the time information in the environmental map. In the brightness information, the detection data acquired by the illuminance sensor 1b at the imaging time of the camera image in which the feature point was extracted is recorded for each feature point included in the environmental map.

[0038] The feature point extraction unit 112 extracts feature points from the camera image acquired by the camera 1a while the host vehicle is traveling on the road. The position estimation unit 113 estimates the host vehicle position based on the feature points extracted by the feature point extraction unit 112 and the environmental map stored in the storage unit 12. The position estimation unit 113 estimates the current position of the host vehicle on the road on which the host vehicle is traveling based on the feature points extracted by the feature point extraction unit 112 from the camera image acquired by the camera 1a while the host vehicle is traveling on the road and the environmental map stored in the storage unit 12. The environmental map stored in the storage unit 12 includes a group of feature points (three-dimensional point cloud data) extracted by the feature point extraction unit 112 when the host vehicle has traveled on that road in the past.

[0039] The reliability determination unit 114 determines whether the reliability of the host vehicle position estimated by the position estimation unit 113 is less than a predetermined level. The reliability determination unit 114 determines that the reliability is less than a predetermined level when the difference between the number of feature points corresponding to a predetermined area in front of the traveling direction of the host vehicle extracted by the feature point extraction unit 112 from the current camera image and the number of feature points corresponding to the predetermined area among the feature points included in the environmental map stored in the storage unit 12 is equal to or greater than a predetermined threshold value. The predetermined threshold value may be changed according to the number of feature points corresponding to the predetermined area included in the environmental map stored in the storage unit 12. For example, the larger the number of feature points corresponding to the predetermined area included in the environmental map stored in the storage unit 12, the larger the value may be set for the predetermined threshold value. The predetermined area is the current imaging range of the camera 1a.

[0040] In addition, the reliability determination unit 114 compares the environmental conditions at the time when the environmental map stored in the storage unit 12 was generated (hereinafter referred to as the map generation time point) with the current environmental conditions. The environmental conditions include at least one of the travel time, weather, and brightness around the host vehicle. The reliability determination unit 114 recognizes the time (time zone) when the host vehicle traveled the current position in the past based on the time information associated with the point cloud data corresponding to the predetermined area included in the environmental map stored in the storage unit 12. At this time, the weather information corresponding to the time (time zone) when the host vehicle traveled the current position in the past is acquired from an external server as past weather information by the information acquisition unit 111. In addition, the brightness information corresponding to the time (time zone) when the host vehicle traveled the current position in the past is acquired from the storage unit 12 as past brightness information by the information acquisition unit 111. The comparison of the environmental conditions is executed each time a camera image is acquired by the camera 1a, that is, at predetermined intervals based on the frame rate of the camera 1a. Note that the comparison of the environmental conditions may be executed for each frame, or may be executed each time a predetermined number of camera images are acquired. In addition, the comparison of the environmental conditions may be executed at other timings. For example, it may be executed each time the host vehicle travels a predetermined distance or a predetermined time. The reliability determination unit 114 determines that the reliability of the host vehicle position estimated by the position estimation unit 113 is less than a predetermined level when there is a predetermined difference between the mutual environmental conditions as a result of the comparison.

[0041] For example, when the weather at the time of map generation is clear and the current weather is bad weather (such as rain, fog, snow, etc.) that deteriorates the visibility in front of the host vehicle, as in the example of FIG. 2B, there is a possibility that corresponding points may not be extracted from the camera image. Also, for example, when the time of map generation is during the day and the current time is a time zone or brightness where it is difficult to extract the edges of objects included in the imaging range of camera 1a from the camera image, there is also a possibility that corresponding points may not be extracted from the camera image. In cases like the above, the accuracy of the host vehicle position estimated based on the feature points extracted from the camera image and the environmental map stored in the storage unit 12 may decrease. Therefore, when there are the above differences between the environmental conditions at the time of map generation and the current environmental conditions, it is determined that the reliability of the host vehicle position is less than a predetermined level. Note that the time zone where it is difficult to extract the edges of objects is, for example, at night. The brightness where it is difficult to extract the edges of objects is, for example, an illuminance equal to or less than a predetermined value.

[0042] In addition, even in a backlight scene where sunlight or light from lighting (such as the headlights of an oncoming vehicle) enters camera 1a from behind the subject (hereinafter referred to as a backlight scene), it becomes difficult to extract the edges of objects included in the imaging range of camera 1a from the camera image. Therefore, even when the imaging environment at the time of map generation is not a backlight scene and the current imaging environment is a backlight scene, it may be determined that there is a predetermined difference between the environmental conditions at the time of map generation and the current environmental conditions. Whether the imaging environment is a backlight scene or not may be determined based on the brightness of the camera image (the brightness of the entire image), or may be determined based on the position and orientation of the host vehicle (camera 1a) and the position of the sun or lighting, or may be determined by other methods.

[0043] When the reliability determination unit 114 determines that the reliability is less than a predetermined level, the environmental map generation unit 115 starts generating an environmental map using the feature points extracted by the feature point extraction unit 112. The environmental map generation unit 115 continues generating the environmental map while the reliability determination unit 114 determines that the reliability is less than a predetermined level. The sub-position estimation unit 116 estimates the position of the host vehicle on the environmental map based on the feature points extracted by the feature point extraction unit 112 and the environmental map generated by the environmental map generation unit 115. The generation of the environmental map by the environmental map generation unit 115 and the estimation of the host vehicle position by the sub-position estimation unit 116 are performed simultaneously according to the algorithm of the SLAM technology. Note that the environmental map (3D point cloud data) generated by the environmental map generation unit 115 may be used not only for estimating the position of the host vehicle by the sub-position estimation unit 116 but also for updating the environmental map stored in the storage unit 12.

[0044] The travel control unit 16 controls the actuator AC based on the position of the host vehicle estimated by the position estimation unit 113 or the sub-position estimation unit 116. More specifically, the travel control unit 16 controls the actuator AC so that the host vehicle travels along the target trajectory generated by the action plan generation unit 15 based on the position of the host vehicle estimated by the position estimation unit 113 or the sub-position estimation unit 116.

[0045] While the reliability determination unit 114 determines that the reliability is equal to or greater than a predetermined level, the travel control unit 16 controls the actuator AC based on the position of the host vehicle estimated by the position estimation unit 113. Thereafter, when the reliability determination unit 114 determines that the reliability is less than a predetermined level and the generation of the environmental map by the environmental map generation unit 115 and the estimation of the position of the host vehicle by the sub-position estimation unit 116 are started, the travel control unit 16 starts controlling the actuator AC based on the position of the host vehicle estimated by the sub-position estimation unit 116.

[0046] When the control of the actuator AC based on the vehicle position estimated by the sub-position estimation unit 116 is being executed, if the reliability determination unit 114 determines that the reliability is equal to or higher than a predetermined level, the travel control unit 16 resumes the control of the actuator AC based on the vehicle position estimated by the position estimation unit 113. At this time, the generation of the environmental map by the environmental map generation unit 115 and the estimation of the vehicle position by the sub-position estimation unit 116 are stopped.

[0047] Figure 4 is a flowchart showing an example of the processing executed by the CPU of the controller 10 in FIG. 3 according to a predetermined program. The processing shown in this flowchart is executed, for example, at a predetermined cycle while the host vehicle is traveling in the automatic driving mode.

[0048] First, in step S1, the controller 10 acquires a camera image from the camera 1a. In step S2, the controller 10 extracts feature points from the camera image by the feature point extraction unit 112. In step S3, the controller 10 determines whether there is a difference between the number of feature points extracted from the camera image in step S2 and the number of feature points of the point cloud data (hereinafter referred to as target point cloud data) corresponding to a predetermined area (imaging range of the camera 1a) included in the environmental map stored in the storage unit 12. Specifically, it is determined whether the difference between the numbers of the respective feature points is equal to or greater than a predetermined threshold.

[0049] If it is denied in step S3, in step S4, it is determined whether there is a difference in the external environment. Specifically, it is determined whether there is a predetermined difference between the environmental conditions at the time when the target point cloud data was generated (map generation time) and the environmental conditions at the time when the camera image was acquired in step S1, that is, the current time.

[0050] If it is negated in step S4, in step S5, based on the feature points extracted in step S2 and the environmental map stored in the storage unit 12, the position of the host vehicle on the environmental map is estimated. On the other hand, if it is affirmed in step S3 or step S4, in step S61, an environmental map is generated using the feature points extracted in step S2. In step S62, based on the feature points extracted in step S2 and the environmental map generated in step S61, the position of the host vehicle on the environmental map is estimated. The generation of the environmental map in step S61 and the estimation of the position of the host vehicle in step S62 are performed simultaneously according to the algorithm of the SLAM technology.

[0051] Summarizing the operation of the position estimation device 50 according to this embodiment, it is as follows. First, when a camera image is acquired by the camera 1a while the host vehicle is traveling on a road (S1), feature points are extracted from the camera image (S2). Then, it is determined whether there is a difference between the number of feature points extracted from the camera image and the number of feature points of the target point group data (point group data corresponding to the imaging range of the camera 1a) included in the environmental map stored in the storage unit 12 (S3). When there is no difference in the number of mutual feature points, further, it is determined whether there is a difference between the external environment at the time of map generation and the current external environment (S4). When there is no difference in the mutual external environments, it is determined that the reliability of the position of the host vehicle estimated based on the feature points extracted from the camera image and the environmental map stored in the storage unit 12 is equal to or higher than a predetermined level. Then, until it is determined that the reliability is less than the predetermined level, the position of the host vehicle is estimated based on the feature points extracted from the camera image and the environmental map stored in the storage unit 12 (S1 to S5).

[0052] After that, if the weather deteriorates and the feature points (corresponding points) corresponding to the target point cloud data cannot be extracted from the camera image, and a difference occurs between the number of feature points extracted from the camera image and the number of feature points of the target point cloud data, it is determined that the above reliability is less than a predetermined level (S1, S3). In addition, if a difference occurs between the external environment at the time of map generation and the current external environment due to the deterioration of the weather, even if there is no difference between the number of feature points extracted from the camera image and the number of feature points of the target point cloud data, it is determined that the above reliability is less than a predetermined level (S1, S3, S4). When it is determined that the above reliability is less than a predetermined level, the estimation of the vehicle position based on the feature points extracted from the camera image and the environmental map stored in the storage unit 12 is stopped, and the generation of the environmental map using the feature points extracted from the camera image and the estimation of the vehicle position based on the environmental map are started (S61, S62). After that, when the weather recovers and the feature points (corresponding points) corresponding to the target point cloud data can be accurately extracted from the camera image, it is determined that the above reliability has reached a predetermined level or more, and the estimation of the vehicle position based on the feature points extracted from the camera image and the environmental map stored in the storage unit 12 is restarted (S1 to S5).

[0053] According to the embodiments described above, the following operational effects can be obtained. (1) The position estimation device 50 includes a camera 1a as a detector for detecting the external situation of the host vehicle, a feature point extraction unit 112 that extracts feature points from the detection data (camera image) of the camera 1a, a storage unit 12 that stores an environmental map as first map information including feature points corresponding to the external situation in advance, a position estimation unit 113 that estimates the position of the host vehicle based on the feature points extracted by the feature point extraction unit 112 and the environmental map stored in the storage unit 12, a reliability determination unit 114 that determines whether the reliability of the position of the host vehicle estimated by the position estimation unit 113 is less than a predetermined level, and an environmental map generation unit 115 that generates an environmental map as second map information using the feature points extracted by the feature point extraction unit 112 when the reliability determination unit 114 determines that the reliability is less than a predetermined level, and a sub-position estimation unit 116 that estimates the position of the vehicle based on the feature points extracted by the feature point extraction unit 112 and the environmental map generated by the environmental map generation unit 115. The environmental map stored in the storage unit 12 is map information of a road section where the host vehicle has traveled in the past, and includes feature points extracted by the feature point extraction unit 112 from a camera image acquired by the camera 1a while the host vehicle was traveling on that road section. The position estimation unit 113 estimates the current position of the host vehicle traveling on that road section based on the feature points extracted by the feature point extraction unit 112 from the current camera image acquired by the camera 1a while the host vehicle was traveling on that road section and the environmental map stored in the storage unit 12. Thereby, even when, due to deterioration of the weather or the like, feature points cannot be accurately extracted from the camera image temporarily, or when there is a difference between the external environment at the time of map generation and the current external environment, the position of the host vehicle can be continuously estimated accurately. Also, since it is not necessary to prepare a plurality of environmental maps corresponding to imaging conditions and the like, an increase in the data storage amount can be suppressed. Further, through such a driving support technology, traffic safety and convenience can be further improved. Furthermore, it can contribute to the development of a sustainable transportation system.

[0054] (2) When the difference between the number of feature points corresponding to a predetermined area in front of the traveling direction of the host vehicle extracted by the feature point extraction unit 112 from the current camera image and the number of feature points corresponding to the predetermined area among the feature points included in the environmental map stored in the storage unit 12 is equal to or greater than a predetermined threshold, the reliability determination unit 114 determines that the reliability of the position of the host vehicle estimated by the position estimation unit 113 is less than a predetermined level. The predetermined threshold is set to a larger value as the number of feature points corresponding to the predetermined area included in the environmental map stored in the storage unit 12 is larger. Thereby, changes in the external environment that have occurred between the map generation time and the current time can be accurately recognized.

[0055] (3) The position estimation device 50 further includes an illuminance sensor 1b that detects the brightness around the host vehicle, and an information acquisition unit that acquires at least one of past environmental information and current environmental information including weather information, time information, and brightness information including detection data of the illuminance sensor 1b during traveling on the above road section in the past and at the current time. The reliability determination unit 114 determines whether the reliability is less than a predetermined level based on the past environmental information and the current environmental information acquired by the information acquisition unit 111. Thereby, even when there is a possibility that the matching accuracy of feature points between the environmental map and the camera image may decrease due to deterioration of the weather, a glare phenomenon caused by light sources entering the camera, an evaporation phenomenon caused by the headlight irradiation of an oncoming vehicle, etc., the position of the host vehicle can be accurately estimated.

[0056] (4) The vehicle control system 100 further includes a position estimation device 50, an actuator AC for traveling, and a travel control unit 16 that controls the actuator AC based on the position of the host vehicle estimated by the position estimation unit 113 or the sub-position estimation unit 116. When the travel control unit 16 controls the actuator AC based on the position of the host vehicle estimated by the position estimation unit 113 and the reliability determination unit 114 determines that the reliability is less than a predetermined level, the travel control unit 16 stops the control of the actuator AC based on the position of the host vehicle estimated by the position estimation unit 113 and starts the control of the actuator AC based on the position of the host vehicle estimated by the sub-position estimation unit 116. Thereby, the host vehicle can travel well in the automatic driving mode. Also, when the travel control unit 16 controls the actuator AC based on the position of the host vehicle estimated by the sub-position estimation unit 116 and the reliability determination unit 114 determines that the reliability is equal to or higher than a predetermined level, the travel control unit 16 stops the control of the actuator AC based on the position of the host vehicle estimated by the sub-position estimation unit 116 and resumes the control of the actuator AC based on the position of the host vehicle estimated by the position estimation unit 113. Thereby, the host vehicle can travel well in the automatic driving mode.

[0057] The above embodiment can be modified into various forms. Hereinafter, modification examples will be described. In the above embodiment, when the reliability determination unit 114 determines that the reliability is less than a predetermined level while the host vehicle is traveling on a road, the controller 10 stops the estimation of the host vehicle position by the position estimation unit 113 and starts the generation of the environmental map by the environmental map generation unit 115 and the estimation of the host vehicle position by the sub-position estimation unit 116. However, when the number of times the reliability determination unit 114 determines that the reliability is less than a predetermined level exceeds a predetermined number, the controller 10 may output a stop instruction to stop the estimation of the position of the host vehicle to the position estimation unit 113 as the first estimation unit and the sub-position estimation unit 116 as the second estimation unit. In this way, when the loss of the host vehicle position is continuous, the controller 10 as the stop control unit can reduce the processing load of the position estimation device 50 by interrupting the estimation of the host vehicle position.

[0058] Incidentally, the stop control unit may output a stop instruction to the position estimation unit 113 and the sub-position estimation unit 116 based on the driving state of the host vehicle. In this case, the controller 10 also functions as a state acquisition unit that acquires vehicle state information indicating the state of the host vehicle. The stop control unit determines whether the host vehicle can continue to drive based on the vehicle state information acquired by the state acquisition unit. When the stop control unit determines that continued driving is not possible, it outputs a stop instruction to the position estimation unit 113 and the sub-position estimation unit 116. The vehicle state information includes information indicating the presence or absence of a flat tire of a wheel (tire), acceleration information indicating the degree of shaking of the vehicle body (vertical and lateral shaking), and the like. For example, when the stop control unit determines based on the vehicle state information that a wheel is flat, it determines that continued driving is not possible. Also, when the acceleration in the vertical or lateral direction of the vehicle body indicated by the vehicle state information (acceleration information) is equal to or greater than a predetermined value, it is determined that the road surface condition has deteriorated, and it is determined that continued driving is not possible.

[0059] Incidentally, depending on the degree of deterioration of the road surface condition, it may be possible to continue driving. However, even if it is possible to continue driving but the road surface condition is deteriorating, the position and orientation of the camera 1a may change due to the shaking of the vehicle body, and there is a possibility that the feature points (corresponding points) corresponding to the target point cloud data may not be extracted from the camera image. Therefore, the road surface condition of the road on which the host vehicle is driving may be included in the environmental conditions. In this case, when the current road surface condition has deteriorated and is different from the road surface condition at the time of map generation, in the determination of step S4, it is determined that there is a difference between the environmental conditions at the time of map generation and the current environmental conditions.

[0060] Also, in the above embodiment, as in the example of FIG. 2B, when there is a possibility that feature points corresponding to the feature points included in the environmental map stored in the storage unit 12 are not extracted from the camera image, the reliability determination unit 114 determines that the reliability of the host vehicle position estimated by the position estimation unit 113 is less than a predetermined level. However, when the weather at the time of map generation is bad weather such as rain and the current weather is sunny, or when the time of map generation is at night and the current time is during the day, there is a possibility that the feature points corresponding to the feature points extracted from the camera image do not exist in the environmental map stored in the storage unit 12. In such a case, the accuracy of the host vehicle position estimated by the position estimation unit 113 may decrease. Therefore, even in such a case, it may be determined that there is a predetermined difference between the environmental conditions at the time of map generation and the current environmental conditions, and it may be determined that the reliability of the host vehicle position estimated by the position estimation unit 113 is less than a predetermined level.

[0061] Also, in the above embodiment, after determining whether there is a difference between the number of feature points extracted from the camera image and the number of feature points of the target point group data included in the environmental map stored in the storage unit 12 (S3), it is determined whether there is a difference between the external environment at the time of map generation and the current external environment (S4). However, the determination in step S3 may be performed after the determination in step S4. Also, in the above embodiment, an example is shown in which information indicating the imaging time of the camera image from which the feature point is extracted is recorded in the time information for each feature point included in the environmental map. However, each time the host vehicle travels a predetermined distance or a predetermined time, the imaging time of the camera image acquired at that time may be recorded in the time information. Also, in the above embodiment, an example is shown in which the detection data acquired by the illuminance sensor 1b at the imaging time of the camera image from which the feature point is extracted is recorded in the brightness information for each feature point included in the environmental map. However, similar to the time information, each time the host vehicle travels a predetermined distance or a predetermined time, the detection data acquired by the illuminance sensor 1b at that time may be recorded in the brightness information.

[0062] Also, in the above embodiment, the camera 1a as the first detector is configured to detect the external situation of the host vehicle. However, the first detector may be other than a camera, such as a radar or a lidar. Also, in the above embodiment, the illuminance sensor 1b as the second detector is configured to detect the brightness around the host vehicle. However, the camera 1a may be used as the second detector to detect the brightness around the host vehicle based on the camera image.

[0063] Furthermore, in the above embodiment, the position estimation device 50 is applied to an autonomous vehicle, but the position estimation device 50 can also be applied to vehicles other than autonomous vehicles. For example, the position estimation device 50 can also be applied to a manually driven vehicle equipped with ADAS (Advanced driver-assistance systems).

[0064] The above description is merely an example, and the present invention is not limited to the above-described embodiments and modifications as long as the features of the present invention are not impaired. It is also possible to arbitrarily combine one or more of the above embodiments and modifications, and it is also possible to combine the modifications with each other.

Explanation of reference numerals

[0065] 1a Camera, 1b Illuminance sensor, 10 Controller, 11 Arithmetic unit, 12 Storage unit, 16 Travel control unit, 50 Position estimation device, 111 Information acquisition unit, 112 Feature point extraction unit, 113 Position estimation unit, 114 Reliability determination unit, 115 Environment map generation unit, 116 Sub-position estimation unit

Claims

1. A detector for detecting the external situation of a vehicle, A feature point extraction unit for extracting feature points from the detection data of the detector, A storage unit for storing first map information including feature points corresponding to the external situation in advance, A first estimation unit for estimating the position of the vehicle based on the feature points extracted by the feature point extraction unit and the first map information stored in the storage unit, A reliability determination unit for determining whether the reliability of the position of the vehicle estimated by the first estimation unit is less than a predetermined level, When the reliability determination unit determines that the reliability is less than the predetermined level, a map generation unit that generates second map information using the feature points extracted by the feature point extraction unit, A second estimation unit for estimating the position of the vehicle based on the feature points extracted by the feature point extraction unit and the second map information generated by the map generation unit, wherein the position estimation device is characterized by comprising the above components.

2. In the position estimation device according to Claim 1, The first map information is map information of a road section that the vehicle has traveled in the past, and includes the feature points extracted by the feature point extraction unit from the detection data acquired by the detector while the vehicle is traveling in the road section, The first estimation unit estimates the current position of the vehicle traveling in the road section based on the feature points extracted by the feature point extraction unit from the current detection data acquired by the detector while the vehicle is traveling in the road section and the first map information. The position estimation device is characterized by this.

3. In the position estimation device according to Claim 2, When the difference between the number of feature points corresponding to a predetermined area in front of the traveling direction of the vehicle, which are extracted by the feature point extraction unit from the current detection data, and the number of feature points corresponding to the predetermined area among the feature points included in the first map information stored in the storage unit is equal to or greater than a predetermined threshold, the reliability determination unit determines that the reliability is less than the predetermined level. The position estimation device is characterized by this.

4. In the position estimation device according to Claim 3, The predetermined threshold is set to a larger value as the number of feature points corresponding to the predetermined area included in the first map information is larger. The position estimation device is characterized by this.

5. In the position estimation device according to Claim 2, The detector is a first detector, A second detector for detecting the brightness around the vehicle, An information acquisition unit that acquires past environmental information and current environmental information, including at least one of weather information, time information, and brightness information including detection data of the second detector, when driving in the past and current road sections. The reliability determination unit determines whether the reliability is less than the predetermined level based on the past environmental information and the current environmental information acquired by the information acquisition unit. A position estimation device characterized by this.

6. In the position estimation device according to claim 2, When the number of times the reliability determination unit determines that the reliability is less than the predetermined level exceeds a predetermined number while the vehicle is traveling on the road section, a stop instruction to stop estimating the position of the vehicle is sent to the first estimation unit And a stop control unit that outputs to the second estimation unit. A position estimation device characterized by this.

7. In the position estimation device according to claim 1, A state acquisition unit that acquires the vehicle state of the vehicle; Based on the vehicle state acquired by the state acquisition unit, a stop control unit that outputs a stop instruction to stop estimating the position of the vehicle to the first estimation unit and the second estimation unit. A position estimation device characterized by this.

8. The position estimation device according to any one of claims 1 to 7; A traveling actuator; A travel control unit that controls the travel actuator based on the position of the vehicle estimated by the first estimation unit or the second estimation unit. When the travel control unit controls the travel actuator based on the position of the vehicle estimated by the first estimation unit, and the reliability determination unit determines that the reliability is less than the predetermined level, the vehicle estimated by the first estimation unit stops controlling the travel actuator based on the position, and starts controlling the travel actuator based on the position of the vehicle estimated by the second estimation unit. A vehicle control system characterized by this.

9. In the vehicle control system according to claim 8, When the traveling control unit controls the traveling actuator based on the position of the vehicle estimated by the second estimation unit, if the reliability determination unit determines that the reliability is equal to or higher than the predetermined level, the traveling control unit stops controlling the traveling actuator based on the position of the vehicle estimated by the second estimation unit and resumes controlling the traveling actuator based on the position of the vehicle estimated by the first estimation unit. A vehicle control system characterized by this is provided.

Images